This is the week! Between the Teradata Partner Conference, Strata/Hadoop World, and IBM Information on Demand, with some little analyst-firm hosted event down in Orlando, the marketing and communications teams of big data vendors must feel like they are spinning out of control. The flood of press releases announcing the next big thing in big data have already begun, but two key themes already seem to be emerging:
- Platform: Vendors and companies have figured out that big data isn't a project, but an on-going practice or facility; it has its own continuous lifecycle. Until now, for the most part, we have all been trying to describe the big data elephant by hunting around with our hands in a pitch black room, led around by pilots and projects. But enough light is now shining that we can increasingly see the whole beast; it is now clear big data is quite complex, and dealing with the "3 Vs" completely understates the challenges most enterprises face in establishing a fully functioning big data facility. How to reduce the complexity? The answer and the word of the week is "platform."
Fastest out with a platform/architecture approach is Teradata, who announced the Teradata Aster Big Analytics Appliance which contains the Aster database and visualization for query/discovery, the Teradata data warehouse, plus the Hortonworks Hadoop distribution. The appliance unveils Teradata's notion of a "Unified Data Architecture," managed through a common console known as Teradata Viewpoint. The development model is pulled together through SQL-H. This counts as a coup for Hortonworks who has not shown up as a Hadoop distribution partner with mega-BI and Cloud players as often as Cloudera and MapR have respectively. The combination of warehouse, Hadoop and advanced analytics platform all in a box with integrated management and development should prove compelling for existing customers climbing the big data curve.
- Lack of Skills: It seems that everyone has grown aware that data scientists are not only rare but expensive, and waiting for the universities to churn out enough applicably trained brains may not move most companies towards big data fast enough. The other option, then, are productivity tools. Just as visual development environments in the 1990s increased the productivity of developers of all stripes, expect all kinds of tools to help everyone in the big data ecosystem to become more productive.
The coolest idea on the productivity front thus far comes from Datameer, who rolled-out the first big data apps market, a la iOS or Android. Datameer took steps to make development easier too, for example by expanding the number of data connectors. Furthermore, to show the way, they put up their own three apps: one for Twitter-data social analysis, one for Facebook, and one for Linkedin. In total there are already about 30 apps in the store, check out the list. Datameer takes a different approach than other data scientist community sites by simply creating an app store—no competitions. If you believe that building data scientist community around a platform is as important as the platform itself, Datameer just jumped the competition.
The week is still young, but Teradata and Datameer have it off to a big start.