Is Big Data the Tail Wagging the Data Economy Dog?

Segmenting the overall IT market horizontally typically results in five sub-markets: Semiconductors, hardware, software, telecommunications, and professional services. But an anomaly buried in the usual segmentation has existed for several decades, glossed over because it was such a slender slice of IT (e.g., the "MRM" market?!). That hidden slice has widened considerably post-2000 however, and the time has come to give those IT suppliers, for want of a better term we will call them “data providers,” their fair due—recognition of their own market space which I refer to as the “Data Economy.”

Even though many data providers are not-for-profit, if one aggregates the revenues of all the data providers the “Data Economy” market now runs in excess of $100 billion in annual revenues. By comparison, ESG estimates the software and core services revenues associated with the BI/analytics platform market at around $20 billion. Even if you add all the adjunct products and services required for big data, such as servers, storage, networking, point software solutions, and professional services, it probably still slightly trails the Data Economy in terms of market size. And ESG believes the Data Economy is growing even faster than big data. Who are these data providers? Let’s barely scratch the surface of some of the Data Economy players.

You are probably familiar with some of the world’s largest data providers like the multi-billion dollar Acxiom and Lexis Nexis. Unless you pay close attention to the securities arm of the financial services industry, you may not have heard of Interactive Data, a nearly $1b firm, and similarly if you are quite interested in channel data, you might have heard of Zyme. Not all data providers focus on a particular industry or role, for example DataLab USA offers data spanning insurance, credit, healthcare, and real estate. If you have ever been wondering about how best to classify industries to optimize search, you might try WAND, and of course the U.S. Department of Labor’s Bureau of Labor Statistics will help you with understanding job taxonomies and data thereof. And if you really want to span the globe in terms of data, you might want to start at which acts as a portal for governmentally sourced data from 39 states, 41 other countries, and a host of other governmental organizations, as part of the movement to “democratize data.” The Open Archives Initiative is another data democratization example. While not-for-profits are important participants in the Data Economy, the United States Postal Service offers data products for a price to help try to offset its rather notorious non-profitability.

Most data providers don’t simply provide data. Particularly commercial providers like Lexis Nexis offer a variety of products for understanding the data they offer, in terms of data attributes, how to best ingest and use the data, and even tools to perform data analysis a la big data. Almost all data providers offer information about the metadata, or at least how to interpolate the metadata, for the data they distribute. Data providers generally gather, aggregate, qualify, refine, and distribute (preferably with value-added ease) data. Data has been referred to as “the new oil,” and while I might extend the metaphor to all kinds of mining and agricultural activities as well, the basic idea that data increasingly acts as the caloric source for an increasing number of modern pursuits, business, governmental, and consumer, is the fundamental driver behind the Data Economy.

If you are a business professional, or a data analyst, or a CIO, why should you care about the Data Economy? First, your competitors may have already jumped ahead of you by tapping into the Data Economy. For example, if you are highly dependent on channel partners, but have little visibility as to their performance in terms of reselling your products other than some simple monthly reports and word-of-mouth, you may be over or under-investing in various channels. Your competitor, however, working with the aforementioned Zyme, might have a far clearer grasp of what is and isn’t working in the channel, and is making workflow and investment decisions accordingly. In that example, if you are not plugged into the Data Economy, what you don’t know may indeed be hurting you.

As a data analyst, you, with help from your IT department, may have done a fantastic job culling all the internal data available for business intelligence and analytical purposes. However, that internal data may lack context, or perhaps could be further enriched with third-party data. Some big data BI/analytics platform vendors, like Alteryx, make it really easy to tap into data providers but offering relevant built-in data services from those providers. To the data analyst using such a feature, the third-party data looks just like an internal data source—except you may have to pay for the data. Regardless, the data analyst or scientist who regularly scans and potentially uses external data for business intelligence and analytical model development is using a best practice. Data analysts who only look to internal data sources are potentially overlooking major insight opportunities.

CIOs should think of the Data Economy as another external resource, like public cloud or professional services, which may be brought to bear to help the IT department deliver the best possible information technology for the business. If the CIO has a CDO, Chief Data Officer, the CDO should certainly track potential external sources for the data needs of the business. If the business has a CDO, Chief Digital Officer, that CDO should likewise be tapping into, as applicable, third-party data, and perhaps should consider using the company’s internal data as a revenue-generating asset; perhaps your company could be a data provider in the Data Economy, too.

The big data movement has largely been technology based. But the innovation for much of the core technology for big data, like Hadoop, was originally developed by Web 2.0 companies, like Yahoo and Google, for business purposes. BI/analytics platforms offer the tools to gain deeper insights into business performance, market opportunities, research and development, and customer understanding. But they are merely the tools, like an automobile. The fuel they need to run on is data, and increasing that data will come from outside of the firewall. For IT departments, being your organization’s steward of technology is no longer enough. IT and its data professional partners in the lines of business also increasingly carry the responsibility for ensuring that the company has ALL the right data, from inside and outside, to help guide the business from daily tactical operations through strategic decision-making.

Topics: Data Platforms, Analytics, & AI