In mid-2012, ESG found only about 10% of organizations working on “big data” projects using public cloud services and infrastructure in the context of the project in some fashion. Few companies ran production big data instances in the cloud and usage was experimental or for initial discovery purposes. Much has been written about security being an adoption hurdle for public cloud for enterprises, in fact security was cited as the #1 concern for big data projects too, a double whammy for cloud plus big data. The next greatest challenge cited for big data was integration, and SaaS apps typically ran in silos for the first decade of the 2000s; in the last few years, cloud demand for integration exploded and many SaaS providers were caught napping. Looking at the evidence from six months ago, it sure seemed bleak for big data on cloud.
But, due to the appeal of quick provisioning enabled by cloud providers offerings Hadoop-as-a-Service, and augmented by several analytics databases made available as-a-service, the latter half of 2012 saw a rapid changing of fortune for big data on the cloud. Mind you that full-scale cloud-based big data implementations remain few and far between—security, data movement, and integration all remain key concerns. But let’s face it, the notion of SaaS big data is every bit as appealing, in terms of provisioning and ease of access/distribution, as SaaS apps. Also, at least in some cases, IT departments found that the type of infrastructure required for Hadoop and parallelized analytics fit nicely into the notion of cloud elasticity.