The modern data stack is intended to harness the latent power of a company’s data and unlock a wealth of opportunities. So how come it hasn’t?
It turns out that instead of rapidly unlocking insights for the business, your data engineers are most likely bogged down in technical details, concerned about software code, QA, reliability, cloud security, and compute costs. They’re worried about IT problems — not business problems, and certainly not usability.
By the time our team at Ascend meets these teams, they are often operating in a dangerous environment. Stakeholders have grown frustrated with how long it takes to build data pipelines. And when they are finally ready, it’s unclear where the outputs are coming from and what they’re composed of. Business users are questioning the accuracy and data reliability of the data pipelines and often have shifted back to operating on hunches rather than facts.
The antidote to this harmful situation? Data products.
Data products shift the focus from building a modern data stack to actually using it to achieve business outcomes. A data product mindset borrows principles from product management to focus on eliminating friction and getting decision-makers the data they need. This approach is key to achieving and sustaining a competitive advantage in a data-first world.
But what are data products, how do you adopt a data product mindset, and what are the tangible benefits of doing so? Read on to find out.
The Data as a Product Mindset
At Ascend, we define data products as assets specifically created to help businesses and consumers make better decisions, improve processes, gain insight, and generate value.
Unlike data projects which declare success by completing activities that follow a set of standard steps, data products require an entirely different mindset. Rather than becoming distracted by the complexity of data infrastructure, data product-driven companies have a much more impactful focus: the internal and external consumers of data and their goals.
That means data engineers consider functionality (usability, accessibility, relevance) and format (dashboards, reports, datasets) of the required data products before they start building. They gather requirements, demo early versions of data products, and iterate on them over time — just like a conventional product squad would. Then they build reliability, quality, and timeliness into their products and measure their impact by customer adoption and satisfaction.
Over time, I have frequently witnessed how a data-as-a-product mindset gets business users to view data as the foundation for how they create value in the business — not just a measure of their output. I’ve also had the pleasure of helping data engineers fuel a culture shift, concentrating on value creation as opposed to just busy work and maintenance. The results have, at times, been dramatic.
Related Reading: Introducing Data Products to Deliver Better Value from Data
Why is it important to treat data as a product?
The idea behind traditional data lakes was that if you put all of your data in one place, you’d find cool stuff. But an interesting trend doesn’t move the needle in and of itself. Without the compass of intended business value, the data came to lack the necessary context to realize the latent value it held.
That’s why treating data as a product is so important: it allows consumers to participate in the creation of data with which they could act on meaningful insights.
For example, say you’re working at a trucking company. You see a correlation between the weather, the number of miles driven, and the collisions in your fleet. That’s great, but what can you do about it?
If you’re treating data as a product, you can align the data team on the purpose of such analytics, for example, to forecast optimal times for maintenance checks, crew substitutions, and alternative routes. Then, you can push those insights directly into your fleet systems, improving efficiency and safety.
Or, maybe you work at a credit reporting company. You notice a high correlation between age, social media usage, interest in video games, and location, but it is the business context that reveals that certain companies will want to target that specific audience. So you work with your data team to create a dynamic data product formatted for simple upload into Facebook, Instagram, and TikTok’s lookalike audience features. You test it, then sell it externally.
Extrapolating the above small examples to other industries — or even just other areas of each business — I have seen data products create significant competitive advantages in terms of:
- Limiting risk
- Increasing cost savings
- Improving quality
- Promoting reusability
- Launching in a new market
5 Benefits of the Data as a Product Approach
Let me summarize some key benefits that the data as a product approach provides:
1. Differentiation
The same data can be operationalized in very different ways. For example, a weather dataset could be leveraged for retail, but it could also be used for emergency readiness or self-driving car analyses. A data product mindset forces engineers to get clear on their internal customer’s requirements and create pipelines that will serve those exact use cases. That doesn’t always mean they have to start from scratch. Using a tool like Ascend, engineers can identify pipelines that already contain 75% of the information their customer needs. Skipping straight to customization saves them time and effort.
2. Transparency and Accessibility
When leadership asks data engineers what they’ve accomplished, they don’t want to hear about the nuts and bolts of data lakes or the intricacies of Python code. They want to hear how data is helping the business grow. Data products make that transparent by showing the logic behind each pipeline, how it contributes to the company strategy, and who can access it.
3. Repeatability and Reliability
Data pipelines are always being tweaked — not just for optimization purposes but also to fix bugs, update data sources, and change formats. Without automation, propagating those adjustments through the entire data stack can be a big pain and cause downstream issues that people may not even realize until it’s a huge problem. But because data products are actually being used, even slight variations in output will immediately catch business analysts’ attention. With well-defined lineage (based on the customer’s original requirements) visible in a single pane of glass, problems are far less frequent, and root cause analysis becomes much faster.
4. Discoverability and Stewardship
In any company, but especially large enterprises, it’s easy for two engineers to spend time working on the same project without knowing it. The discoverability of data products eliminates this wastefulness, making the purpose of each product known to the business via data catalog. Because stakeholders participate in the creation of data products to meet their specific requirements, they are more inclined to believe in and steward their data. They lean in to hold their teams accountable, and stand behind their decisions and results.
5. Measurability and Defining Success
Data products can include metrics like time to market, product adoption, customer satisfaction, brand sentiment, market share, and everything in between. Regardless of where data products are used in the business, they lend themselves to tracking progress over time, helping organizations monitor existing KPIs, uncover new leading indicators, and commit to realistic goals. And because data products often stem from the same or similar data streams, executives can visualize how pulling a lever on one side of the business affects another — all in real time.
Build Your Data Product Factory
When the success of engineers’ jobs is defined by code releases, data outages, broken integrations, and managing a multitude of tool vendors, they don’t have the time or motivation to be strategic. Before taking on the success of data products as their mission, the teams I have encountered are fractured: observability specialists here, Snowflake specialists there, and business analysts on the other side of a fence with no cross-communication. As a result, exceptional talent — and precious data — goes to waste.
A data product mindset changes all that, giving engineers purpose, converting data and code into tangible business outcomes, and evangelizing data across and outside of the business. At the same time, customers and stakeholders benefit from timely, accurate, explainable data that is designed from the ground up to move the business forward.
While it’s possible to develop data products without a unified platform by instituting draconian rules of the road, in fact individual point solutions often lack overarching cohesion, acting as band-aids and causing more problems down the line. Instead of getting bogged down by building and maintaining a generic data stack by integrating many tools that solve pin-point problems, engineers can create actionable data products much faster with an end-to-end solution specifically designed for data product delivery.
Want to learn more about data products? Read our introduction to the term and our approach to data products at Ascend.
