ESG Research publishes Reference Research briefs for our subscription clients that include a data-centric perspective on certain aspects of the market. In the case of data management, we were in search of a number for big data - how big is big when it comes to databases? Many who support their enterprise data warehouses already believe they have big data - they just call them VLDBs (very large databases). We wanted to quantify how many production databases organizations have deployed and the total amount of data stored in those databases.
Our rock star research team and I just published two briefs that provide our analysis of total database data as well as the total number of production databases supported by large midmarket and enterprise organizations. Each brief includes analysis of production databases by number of employees, total amount of database data, total annual revenue, and industry.
At a high level, large mid-market peaked (28%) around 1-4 TB of total database data while enterprises peaked (21%) between 250-499TB. That's quite a spread. The same distance exists between total number of production databases: the bulk of large mid-market companies (29%) stated between 5-10 production databases versus more than 200 production databases took the bulk of enterprises (20%).
When we correlated this data with the adoption of big data analytics platforms in our research report The Impact of Big Data on Data Analytics, we found a strong relationship between these two data points and a tendency to either deploy or plan to deploy MapReduce-based platforms such as Hadoop. It would seem that organizations with a plethora of information recognize how challenging it is to process, analyze, and leverage big data volumes when they know the limitations of their own database systems.
For more details on factoids around sizes and quantity of databases, check out these reports and more on our site.