Don’t Call It a “Data Product” Unless It Meets These 5 Requirements
In this special feature, Barr Moses, co-founder and CEO of Monte Carlo, believes that data products can transform an organization’s ability to be data-driven as long as they meet 5 key requirements. Monte Carlo, the data reliability company, is the creator of the industry’s first end-to-end data observability platform. Monte Carlo works with data-driven companies such as Fox, Affirm, Vimeo, ThredUp, PagerDuty and other leading companies to help them build trust in data. Prior to starting Monte Carlo, Barr was a vice president at Gainsight, where she helped grow the company’s revenue 10x and build the analytics and customer data team. Barr has also been a consultant at Bain & Company, worked in the statistics department at Stanford, and served in the Israeli Air Force as commander of an intelligence data analysis unit. She graduated from Stanford with a B.Sc. in Mathematical and Computational Sciences.
One of the hottest moves in the data world right now is to treat “data as a commodity”. Almost overnight, data platforms were rebranded as data products in the presentations and architecture diagrams of data leaders everywhere.
Data products are a hugely important, and frankly overdue, concept that states that data teams should treat how they present data across the business with the same rigor that software teams use to develop modern apps.
This rigor is essential given the elevated role of data, as it has been freed from dashboards to power machine learning modules, be part of customer-facing applications, and much more. But as data has become more valuable, the cost of bad data has also increased. According to Gartner, bad data costs the average business $12.9 million per year.
Not only does the data product concept raise the bar for the availability, reliability and usability of enterprise data, but it also represents a profound shift in perspective.
Instead of thinking from the inside out, with data teams huddled and virtually tinkering with data before occasionally getting up to relay nuggets of useful information, data products foster business thinking. ‘outside in, data teams focusing on how to maximize the usability of data for internal customers so that these stakeholders can make data-driven decisions.
It is powerful and scalable. It’s also a lot of work and, like many trends, not as well-defined as it could be.
So, in the spirit of product development, here are 5 essential requirements for data products that can help organizations move from buzz to production:
- Documentation is easily accessible. No, your Apache Spark management console DAG does not matter (although that is also important). User-focused documentation detailing what the product is, how to use it, and why it works is an effective first step that data teams can take on their data product journey. It forces you to answer basic but elusive questions, “what exactly is the product, is this dashboard part of it?” It also helps kick-start the key shift your team will need to go through, from “this makes sense to me” to “will this be useful to my client?” »
- Feedback and iteration take center stage. Don’t show up to your next business review saying, “We developed this new data product for you, what do you think?” Start with the very basic, but tragically overlooked step of making sure there is a problem your data product needs to solve. Collect customer requirements by asking many questions such as “When do you report on this data?” “How often should it be updated?” “What business process is this data used to inform?” » Find out if the team is operating in a system that can display your data in their workflow or if they have the ability to automatically optimize based on your data (reverse ETL can be your friend here). Ship the minimum viable product and continue to act on feedback. Don’t forget your internal customers once the product is successfully deployed! NPS surveys can be a useful formal feedback mechanism for roadmaps (your data product Is do you have a roadmap, right?) in addition to less formal meetings on a cadence of at least a month.
- There is specialized support and ownership. Virtually every app has a product manager, the person responsible for setting the vision and coordinating development. Data products should be no different. While data products and a decentralized data mesh architecture go together like peanut butter and jelly, a decentralized organizational structure is not necessarily a requirement. However, the data product model should be supported by dedicated engineers and analysts with specialized knowledge of the product and the business needs it serves.
- Service Level Agreements (SLAs) have been established. Just as SLAs play a crucial role in setting application performance expectations, it is becoming increasingly common to set performance expectations for data products as part of a data contract or ALS. Data observability solutions that proactively monitor, alert, and measure end-to-end data quality play an important role in helping data teams define, report, and meet these SLAs.
- Internal customers actually use it. Products without customers are just ideas. Adoption is the true measure of the value of your data product. Be sure to track your monthly active users and find creative ways to evangelize within your organization. If you are successful, make sure your product is designed to scale! While a data warehouse backbone isn’t necessarily a requirement for every use case, it will eliminate any scaling issues.
Congratulations, it’s a data product
Not all data systems or initiatives need or should be a data product.
With a common understanding and standards for data products, the label can be reserved and revered as the awe-inspiring, high-impact achievement that it is.
Sign up for the free insideBIGDATA newsletter.
Join us on Twitter: @InsideBigData1 – https://twitter.com/InsideBigData1