The Discouragement of Data

Imagine being omniscient. Knowing everything that has happened, everything that is happening now, even being able to see into the many possible futures. Sounds kinda slick, no? Certainly it would help in the stock market or casino, if perhaps be a little disconcerting in personal life.

The recent boom of interest in (big) data and analytics is in some ways long overdue. Everyone says they want more insights, and that ready access to the facts will help steer decision-making for the better. But is this actually true?

Psychology, organizational behavior, related disciplines, and even casual observation of office politics suggest that more information isn’t always welcome. People ask me a lot about how big data projects can be (un)successful, solutions choices, lessons learned, best practices, and the like. They seem surprised when I talk more about the human elements than the technology, but here’s what I see as more common issues:

  1. “A case of bad management.” Overpromise and underdeliver is still alive, and no doubt many big data projects are too ambitious to start. The Utopian promises of vendors may help get the green light, but they also can set expectations too high. ESG research found over two thirds (67%) of respondents want a significant business impact from big data in less than a year. Disappointed or underwhelmed staff will be vocal, tainting the actual results and adoption by users.
  2. “Don’t confuse me with the facts.” Our study found most often executive staff in IT (52%) or lines of business (46%) are responsible for starting new big data initiatives. This is good, any large effort needs sponsorship. The challenge can be when a visionary finds the data doesn’t support the official story line. All too often people prefer a compelling anecdote over a lot of statistics. “My customer said…” is used as a magic trump card in too many disputes. Data analysis doesn’t exist just to support an agenda but also to change minds, assuming the minds are open to change.
  3. “Good data is a big hassle.” In my past life as a marketing director, we were data starved. Getting any information on campaign performance, sales forecasts, customer satisfaction was a painstaking, arduous task. Most records were incomplete, IT took months to build reports, the various data sources and applications weren’t integrated, and no one could agree what any number meant. Quarterly business reviews were fairly random and would have been hilarious if we weren’t trying to drive results. Part of this stems from the previous, but the actual data definition, quality, and “wrangling” were equally serious problems. Our surveys show the biggest challenge is “data integration is complex.”

So if you really want your business to be “data-driven,” you need to set expectations clearly, work on cultural change management, and be prepared for a lot of plumbing discussions. The technology generally does what it should, but the human factors will determine the ultimate success of the initiative.

Figure 1: Senior executive when confronted with data that contradicts the established viewpoint.

My data referenced in this blog is from the soon-to-be-published ESG research report, Enterprise Data Analytics Trends: Market Drivers, Organizational Dynamics, and Customer Expectations.

Topics: Data Platforms, Analytics, & AI