⬅️ Buttondown with 11ty on Netlify with GitHub 🧭 Tip of the Apache Iceberg ➡️

Interoperability and Convergence Ahead

January 14, 2024 by Jay Cuthrell parquet open-source mlops dataops pipelines ai ml aws pandas polars cerebras gartner vast weka fhe airbyte airflow astronomer starburst milvus momento tecton plg com devx otel gcp azure oci tackleio

This week is a potpourri of topics. Buckle up and let’s go down the rabbit hole(s).

Getting Informed

Last year, Gartner published a blog listicle of predictions around data and analytics for the next 5 years in large enterprise organizations. Within Gartner’s listicle are ten (10!) crystal ball insights with deeper links to past Gartner publications.

https://www.gartner.com/en/articles/gartner-strategic-data-and-analytics-predictions-through-2028

Listicles, like commercial musicians’ throwaways B-sides might become more popular than imagined. A few predictions caught my eye and I’ll paraphrase each one.

2026: Tick-tock goes the product-led-growth (PLG) clock

Within two years, one in four chief data and analytics officers will have fully instrumented a product at the companies they serve.
That is a short timeline to go from gut feelings to data-driven insights to arrive at a product science that can explain and predict.

2026: Tick-tock goes the cloud operating model (COM) clock

Within two years, three in four operations leaders will use cloud platform foundations as the underpinning for digital transformation.
That begs the question of clinging to the legacy IT way as a deferral or an indication that those organizations will perish.

2026: Tick-tock goes cloud service provider marketplace (CSPM) clock

Within two years, the exhaustive interoperable and interchangeable composition of SaaS on PaaS/IaaS will enable organizations to rapidly and securely pick from any of the top twenty (20) preferred independent software vendors (ISV) on an array of cloud service providers.
That vision of complete ISV saturation across all cloud service provider marketplaces will require the launching of many Tackle.io initiatives to go from “Call for Pricing” towards “Click here to deploy”.

2027: Tick-tock goes the developer experience (DevX) for the knowledge worker clock

Within three years, the organizations that embrace DevX patterns and apply similar investments to knowledge workers will unlock a digital employee experience that contributes to revenue growth.
That embrace of DevX patterns or the deferral of those investments would seem to be an embrace of revenue growth or a deferral of revenue growth.

But how to get there?

In short, we will need to see interoperability and convergence.

Let’s consider the product leader in one of the organizations that will not just survive but thrive in the coming years. Arguably, the product leader that embraces powerful data analysis will survive but the product leader that converges that analysis for wider team contribution and effective collaboration in a secure policy consistent manner will thrive.

First, consider a past Fudge Sunday that explored fully homomorphic encryption (FHE). As such, there are going to be multiple parts and players for the solutions to enable and empower product leaders.

Second, consider the background of the team at Polars that is providing services for product leaders focused on feature engineering. Also, after a dozen years towards the simplification of Pandas, those who saw value in Apache projects with commercial support experience considerations are a growing community.

Third, consider the background of the team at Cerebras that is working with VAST to provide a new definition for the value of data. Indeed, a pre-engineered approach is an attribute of convergence.

Fourth, consider the trends observed by the team at WEKA and a steady focus on performance with density. Further, the notion of deployment flexibility is an essential proof point of interoperability.

Fifth, consider what essential cloud service providers are (frankly) missing from the Gartner Magic Quadrant for Distributed Hybrid Infrastructure. Fundamentally, any current provider with a “cloud only” strategy will require an adjustment that contemplates edge to core to cloud in their roadmaps, M&A strategy, and corporate development.

Gartner Magic Quadrant for Distributed Hybrid Infrastructure