In this 3-part ESG Video Capsule Series, ESG's Nik Rouda talk about his expectations for Big Data in 2017. Part 3 of 3 covers streaming data and pipelines.
Hi, I'm Nik Rouda, ESG senior analyst. I want to talk to you about some predictions for 2017. This one's about streaming data in pipelines. What we're seeing in the industry is really moving from batch activities and processes, analytics on old data to real realtime. We're able to read and capture data as it flows from multiple sources at speed. We can apply definitions, transform, join data in the moment. We can leverage in memory plus very fast sequential read write capabilities. We can store and analyze data simultaneously now, not as sequential tasks. Put the data in the context of historical info and pre-built models. You can respond in the moment. We can serve it up to dashboards, to interactive BI, have other interfaces available to applications. Now we can do all of this robustly for the enterprise. These tools can now handle anomalies, recover from interruptions and failures. It was very tricky before around pipelines and streaming. Now with Kafka, Flink and other open source or vendor proprietary tools it's becoming very easy. High performance at scale is now relatively achievable due to the better economics and maturing technology. See more on streaming and data pipelines coming soon. Check out other ESG prediction videos on the website. Thank you.