As more companies ingest and leverage ever-increasing amounts of data, a new problem has come to light: it turns out that monolithic data platforms built and managed by centralized teams are rarely the best engine with which to create value.

By challenging this predominant paradigm, the new framework of data mesh has taken the analytics world by storm. It reallocates ownership and accountability to the people closest to the business use case for data. Just a few years after Zhamak Deghani introduced the concept, data mesh was cited as a top analytics trend for 2021 and became a more popular search term than “data lakehouse”. And Forrester predicts more organizations will find value in the use case-driven philosophy of data mesh as regulations and AI investments increase. 

At the same time, the practice of building data products has also grown in popularity. Businesses are finding value in building intentional tools and dashboards that allow multiple teams to access and use data from a shared product, reducing the error and confusion inherent in less scalable approaches like ad-hoc reporting. 

While data product thinking has proven incredibly useful, it is not the same thing as data mesh — even though the two are related, and sometimes confused. The good news is that if your organization is already building and deploying data products, and continues to grow an appetite for more value-driven results from data, then adopting data mesh may be around the corner. 

So let’s recap exactly how data-as-a-product thinking stacks up against data mesh, what the key differences are, and how you can tell if your organization is ready to graduate from data product thinking to full data mesh.

Data-as-a-Product vs Data Mesh

Data product thinking and data mesh are frameworks and approaches that have to be adopted holistically by the organization, not simply platforms or tools to be bought or implemented. Both can unlock immense value for teams that take the time to understand how they create leverage and impact the ways companies use data.

The Basics of Data-as-a-Product Thinking

The data-as-a-product mindset adopts principles of product management and applies them to data sets. Usually, the goal is to create value and monetize data in a way that can benefit the wider organization. We delve into this concept in our articles Introducing Data Products to Deliver Better Value from Data and Benefits of the Data Product Approach. But in essence, when companies apply product thinking to data, it typically involves:

  • Partnering with key stakeholders to gather requirements for the data product use cases that meet specific business needs
  • Defining data sets as the customer wants to consume them
  • Aligning formats, cadence, destinations, and semantics with consumer needs
  • Prototyping early versions of data products and iterating over time 
  • Developing key metrics to quantify success, including reliability, integrity, impact, and customer adoption and satisfaction

Fundamentals of the Data Mesh

Data mesh, on the other hand, is an approach to defining a data architecture that layers distributed ownership and data modeling techniques on top of the data product mindset. The data mesh framework helps organizations build shareable data sets securely, and at scale.

Data mesh has four fundamental concepts, including treating data as a product, as we just described. The others include: 

  • Business data domains: This principle pertains to the distribution of data ownership to individual domains, or units, within an organization, leading to more responsive and context-aware management of data.
  • Self-service infrastructure: This refers to a system design that allows individual domains to independently create and maintain their data products, fostering efficiency by centralizing infrastructure issues and eliminating development bottlenecks.
  • Federated governance: This is the practice of decentralizing data governance to individual domains, while maintaining a set of overarching organizational rules and tools, facilitating both autonomy and coherence in data management.

You can explore more of the complexity and nuance in our guide to data mesh.

Differences Between Data-as-a-Product and the Data Mesh

Now that we’ve established the basics, let’s dive into what makes data as a product and data mesh different. 

As we mentioned before, some teams are quick to conflate data-as-a-product thinking with data mesh. The confusion is understandable — after all, product thinking is a core component of data mesh. However, without domain ownership, self-service infrastructure, and federated governance, your data mesh will be incomplete. 

When organizations apply data-as-a-product thinking without these other three fundamental concepts, we typically see the organization struggle with poorly defined data domains, or revert to centralized ownership and governance. This can lead to business SMEs becoming siloed from data engineering teams, and a lack of self-serve infrastructure to facilitate the agile development of data products outside of the centralized data engineering function. 

Graduating from Data-as-a-Product to a Full Data Mesh

If you’re already on board with data product thinking, you’ve already taken the first step on your journey to data mesh. Along the way, challenges will arise that can serve as signals that data mesh adoption should be on your roadmap.

Red Flags Signaling the Limitations of Data-as-a-Product

Without the complementary elements of data mesh, product thinking alone tends to  result in lower levels of trust in — and adoption of — the data products created by the centralized IT organization. This effect happens because data teams often build products without the continual input of business SMEs, or because different parts of the organization are publishing conflicting data products. 

The sheer force of will required to reconcile these problems after the fact can become insurmountable, leading to neglect and disuse of the data products. Another effect is that the redundant implementation and maintenance of conflicting data products runs up the cloud infrastructure bills, leading to cutbacks and  reduced adoption of data to drive the business. 

Signs You May Be Ready for a Data Mesh

Some organizations can identify these key limitations of the data-as-a-product approach, and recognize that the data mesh may be a logical next step. These hints may signal you’re already inching along the path to data mesh:

  • Data products are already becoming the responsibility of the business unit closest to the data, unofficially segmenting the organization into “data domains”
  • Business SMEs are already taking proactive roles in documenting requirements and validating the data products relevant to their business unit
  • Data is already being shared across a common, shared self-serve platform

Tips for a Successful Transition to the Data Mesh

Experience from leading practitioners is showing that transitioning to data mesh doesn’t happen overnight, and it cannot happen in a silo. It’s a complex process, but the following tips will help you chart a smoother course to your data mesh destination. 

Maintain Flexibility

The data mesh architecture is not a piece of technology — it’s an organizational framework that requires buy-in by several stakeholders and results in behavior change. In other words, it takes time to adopt. So as you move along your data mesh journey, stay flexible. 

At first, domain teams may need the technical support of a central data engineering group to develop successful data products. Shared automation tools can make it significantly easier for teams to build and operate the data pipelines that produce data products, without centralizing data product development. And data products can be cached and updated at the point-of-use to avoid becoming data silos. 

Over time, domain teams will become more proficient at building, maintaining, and governing data products. But don’t let a rigid adherence to data mesh principles get in the way of progress while skill sets and competencies are built out.

Measure Success with Business Impact

Adopting a data mesh takes continual buy-in and collaboration across the organization, so be ready to measure and demonstrate the positive business impact of your efforts. 

Consider how you can measure and report on positive outcomes as the data mesh takes hold:

  • Increased SME accountability for data products
  • Higher confidence in (and adoption of) data products across domains
  • Increased use of self-service and automation in data pipelines
  • Reduced impact of data outages — localized to a specific domain, versus organization-wide
  • Higher efficiency and lower running costs for data products
  • Reduced data redundancy and wasted resources (including storage, technology, and engineering costs)

Developing buy-in across the organization is one of the key elements to successfully implementing data mesh, so look for opportunities to create value, celebrate wins, and prove the outcomes are worth the effort.

Is data mesh the holy grail?

For larger companies with fairly autonomous business units, adopting a data mesh can unlock profound business value. When domain teams can easily build and share data products across such an organization, crucial data that would otherwise remain siloed can be used to deliver a competitive advantage and win in the marketplace. 

But, it isn’t a one-size-fits-all solution. Smaller companies or startups that are centered entirely on data may be hampered by the complexity a data mesh can bring. If the organization isn’t large enough to warrant distinct domains that would create and use data products in different ways, a data mesh may not make sense. 

And some businesses may lack an understanding of what a data mesh or even data products actually means. Those organizations will need champions to advocate, educate, and build buy-in for the concept before teams can start to adopt a data mesh framework. 

In the meantime, data-as-a-product thinking can already provide immense value for organizations of all sizes and maturity levels. Focus on areas where the benefits are obvious and clear, such as a data product that consolidates a single source of truth for a shared — but often misinterpreted — metric. Master those steps first, then explore other ways to introduce data products across the organization. 

Additional Reading and Resources