So you have a decision to make: behind door number 1 is a huge, costly on-prem infrastructure that requires expertise across a bunch of different tools involved in the data pipeline for support, and ongoing management and maintenance. Behind door number 2, you have a no-upfront-cost cloud infrastructure that requires minimal expertise, and a single point of support and management. Which do you pick? There are a lot of considerations, such as performance, availability, and arguably most importantly, long-term cost. What is my overall data footprint and growth over the next 3-5 years? How much data does my organization process on a monthly basis? Is the cloud a final destination for my data sets? And that’s just the start. Now with that said, for organizations that are looking into becoming more data-driven as quickly as possible without a potentially massive upfront investment, I would like to be the first to say…welcome to the cloud my friend.
ESG’s 2018 spending intentions research is out, and while the entire dataset is ripe with interesting and compelling data across all of IT, one of the areas that jumped out to me was the relationship between cloud and analytics/BI. We asked about the purposes for which cloud infrastructure services (think IaaS and/or PaaS) were used, and of the 386 respondents, 43% are running business intelligence queries (so that’s big data analytics and processing, data warehousing, data mining, etc.) in some way in the cloud. Initially, that number seemed high to me, but as I’ve talked with more customers, it’s becoming clear: They don’t want to deal with the potential headaches that come with an on-prem solution.
I recently spoke with an IT administrator who went through the door number 1/door number 2 conundrum. While laughing, they informed me they initially went with door number 1, purchasing $1mil+ of hardware to support their data streaming and analytics workflows. Guess what happened? They never even deployed it. They looked at the pile of gear and considered the man power it would take to deploy, configure, test, innovate, iterate, manage, maintain, and support, and just returned it all. No thanks. They went with the cloud, and couldn’t be happier. The admin said, “I was supposed to run a 20+ person team of folks to support and maintain that on-prem solution. Now, I don’t even have a team at all. I can do most of it myself, with a little help from one other person. It’s that easy. And it came in under my budget.”In terms of available technology in the space or, I should say, services because it’s all cloud-based, there are a number of different options…you have direct cloud offerings from the likes of Amazon, Google, Microsoft, Oracle, and IBM; cloud-partnered services that leverage the aforementioned clouds and complement their on-prem solutions, like HPE, SAP, Teradata, Hortonworks, MapR, and Cloudera (to name a few...there are a ton of vendors that fall in this category ); and some cloud-first/cloud-only solutions from vendors like Snowflake. I should point out that I’m well aware these vendors don’t all do the same thing, but the trend of leveraging the cloud for big data streaming, processing, analytics, and BI is undeniable and only going up from here. Stay tuned for a brief I’ll be putting together with more ESG research about BI/analytics in the cloud.