The Future of Big Data and Analytics Will Repeat History

Technology cycles come and go in waves, sometimes with ebb tides, sometimes neap, and every now and then we see a rare but stunning tsunami. In past cycles we’ve surfed a few famous patterns, many times over:

  • Swings between centralized or distributed computing models
  • Battles between dominant giants and myriad disruptive startups
  • Pushes between IT control against business unit demands
  • Heaves between focus on cost or on competitive advantage
  • Sweeps between cutting edge versus most proven tech

The broad market space of big data and analytics is now a perfect storm which has all of the debates above raging at once, bringing the “Humonga Cowabunga from Down Under” crashing upon the shores. How do you avoid getting washed out?

Well, the first thing to do is choose your targets. Survey the skies for new business requests and new operational threats. Set your sights on the most important clay pigeons but also the ones you can actually hit. Consider how different weapons in your arsenal, new and old, can help you pick off these big data birds most quickly and accurately.

Some questions to help you think about the best strategy for surfing the big waves while shooting the fast moving skeet:

  • Is it more efficient for you to scale up or scale out, and where are the users?
  • Have you been well-served by your current suppliers?
  • Can you balance operational safety versus new functionality?
  • What do you risk by not embracing changes in your industry?
  • How much would new capabilities really accelerate your business?

Recognize that as IT professionals, we’ve had to answer these questions before, many times over, and our value comes from being able to meet seemingly conflicting needs. Think about the successes and failures your organization has had in adoption of cloud, of virtualization, of thin clients, and you should see there are many patterns that suggest the right approach for the quirks of your particular organization. New techologies will both complement and displace existing approaches, with the right blend of speed, scale, simplicity, safety, and all the other characteristics you need. It's just a matter of time and maturity.

And for those who say this whole idea is ridiculous, well, I’d like to provide photographic evidence that skeet surfin’ has been done as far back as 1984… new waves, better firepower, but with many past lessons learned and best practices to adapt.

Image courtesy of the amazingly still relevant and entertaining film "Top Secret!" (Paramount Pictures)

Topics: Data Platforms, Analytics, & AI