AI & Machine Learning Take Center Stage


Live from New York, if just a bit Off Broadway, were the O'Reilly AI conference and Strata & Hadoop World shows last week. These events were fascinating as character studies of overlapping technology stars. AI is now the fresh faced celebrity, if a bit immature, raring to get a big break, while Hadoop has become recognized as always delivering a solid performance, but may be fading a bit in glamour.

Starting with the AI conference then. This was a stunningly cool event. The stuff being done with machine learning, deep learning, neural networks, etc., is incredible. A well-trained AI can accurately process rough inputs, analyze to build complex models, and predict all manner of outcomes. In an instant. Except it's all still rather hard to use and could benefit from some better packaging and integration with other IT technologies. If there is a shortage of general data scientists, I wonder how many people can really master machine learning as it's presented today. Can we say "user interface?" That aside, we are on the cusp of some truly remarkable things. Though AI may still struggle to do some tasks my 10-year old tackles effortlessly, at the same time it can process phenomenally more info and find hidden patterns. And it's fascinating how an AI thinks, perceiving the world in fundamentally different ways than us meat. This space is ripe for massive growth, if we can get the tech accessible to more than a handful of companies, universities, and government labs.

Topics: Hadoop Data Management O'Reilly Strata+Hadoop World artificial intelligence O'Reilly AI Conference

"Big data, huh, what is it good for?" at the Hadoop Summit

The mood of this week’s Hadoop Summit has felt wonderfully diverse. There is a cognitive disconnect between the incremental progress of dot release feature sets and the revolutionary new business and societal applications of the technology.

In the same keynote session the topics can swerve from optimizing cluster utilization to optimizing marketing yields to finding a cure for cancer. The technical lectures were packed, while the expo floor was focused. A loud rock and roll string trio in (unnecessarily short) black dresses exits the stage to be replaced by serious talk of open-source projects and community. One day a presenter will explain how clever they are to be able to apply pervasive surveillance of drivers for more profits, the next day a keynote is focused on rallying the audience to develop their ethics and fight the forces of ignorance.

Topics: Big Data Hadoop Data Management

Psycho query: qu'est-ce que c’est?

Any Talking Heads fans reading this blog? Take any French classes in high school? No? Nevermind then.

I get asked a lot about SQL on Hadoop, and I know what you're thinking: "this guy must have the coolest friends and the go to all best parties." And you're right, I do. Lenny Kravitz by a rooftop pool in Vegas. Fitz and the Tantrums. Duran Duran. The Astoria Middle School Marching Band on Loyalty Day. (10-year old daughter with a shiny new flute...)

Topics: Hadoop Data Management SQL

The Delta-V Awards: Hadoop & Spark

ESG's Delta-V awards recognize the top 20 companies that made an impact in big data and analytics in 2015. Here's the set of awards celebrating organizations for their contributions to Hadoop and Spark. These are listed in alphabetical order, so consider rebranding your company as "Aardvark Advanced Analytics" if you really want to be "first" next year.

To learn a little more about the awards, check out this overview.

Topics: Analytics Big Data Hadoop spark

The Delta-V Awards: 20 Companies That Changed Big Data in 2015

Avid readers will have no doubt noticed I've been unusually quiet lately. It's been a busy end to the year, wrapping up a lot of holiday gifts project deliverables for clients. Thankfully, I've also had some extra time to reflect on 2015 and all the big developments in the wide, wide world of big data and analytics. I'd like to share my lasting impressions of the year that was, and to make it even more special, I'm introducing the ESG "Delta-V" awards to give credit where credit is due.

Topics: Analytics Hadoop databases spark delta-v awards

Pentaho's Three-Legged Race to Big Data (with HDS)

Pentaho World 2015 was held in sunny Orlando this year, with over 500 attendees, and was by all accounts a friendly and informative affair. About the only question no one could answer is why the company is called Pentaho, but a rose by any other name is still very nice. One thing that was quite clear is that the team is hitting its stride with HDS as a powerful running mate.

Topics: Analytics Big Data Hadoop HDS business intelligence data integration

The Latest Data on What's Big in Big Data

ESG's research crack team has been working hard on getting the insider's view of the big data and analytics marketplace, including Hadoop and Spark trends. While we're not quite ready to release our findings yet, I thought I'd share a little teaser of what is coming. Enjoy the overview video and let me know if you'd like to schedule a personal briefing to discuss how this will impact your business and opportunities.

Topics: Analytics Big Data Hadoop spark

Schrodinger's Cat and Analytics Accessibility

Everyone loves the concept of Schrodinger's cat, with the possible exception of a few serious PETA members. The metaphor that an entrapped feline can be both poisoned and/or not poisoned until directly observed is a catchy way to understand uncertainty around various possible states and outcomes.

Topics: Big Data Hadoop Data Management SQL

A More Focused Hadoop Summit

Last week 4,000 fine people gathered in San Jose for the Hadoop Summit. Hosted by Hortonworks, but largely non-denominational, the event showcased the broader ecosystem of Hadoop, big data, and analytics. Despite the wide variety of approaches, the attendees were clearly focused on finding the value and learning practical techniques to be successful. Less hype, more tech.

Watch the short recap video for highlights and then read on below for a roundup of selected announcements.

Topics: Analytics Big Data Hadoop

Teradata Broadens the Base

In recent years, many were thinking that Teradata was losing its edge in the greater analytics marketplace. Long a favorite for large enterprise customers seeking to do world-class data warehouses, the company realized the market was rapidly shifting around it. Two major trends seem to have the biggest impact here: big data and cloud. While Teradata's core install base isn't at significant risk of going away, they may be looking at complementary alternatives. For the company to remain a dominant player, some new approaches would be required. The recent Teradata third party influencers event in Del Mar, California, showcased just how far their efforts at transformation have progressed.

Topics: Analytics Big Data Hadoop Teradata data warehouse