Is bigger data better? Helping “HiPPOs” make big data decisions

When are data-driven decisions better than those coming from HiPPOs (Highest-Paid Person’s Opinions)?

At the annual Thought Leadership Forum in Melbourne, Ross Dawson delivered an insightful keynote speech on this topic. He says that in domains where sufficient data is available and the decisions are definable and tractable, algorithms will surpass human judgement.

What is new compared with longstanding analytic practices is the rapidly growing volume of data and the diversity of sources. In a study by Andrew McAfee and Erik Brynjolfsson of MIT, companies using data-driven decisions and predictions were shown to be 6% more profitable than their competitors using human intuition and HiPPOs.

Yet Dawson doubts the effectiveness of algorithms in situations which cannot be clearly defined. “A prominent example is an organization’s strategic positioning. Any data-driven model of industry structure and potential decisions will be based on implicit assumptions by the creator of the algorithm.” And McAfee and Brynjolfsson emphasize that human insight is still very much needed. Humans see the opportunities and the challenges. And they ask the right questions, which will eventually be answered by data and algorithms.

Data can show the way consumers navigate through a website, which products they look at and which ones they buy. The products on the Amazon site are placed and shown accordingly, optimized for each customer. Often the many small decisions, following a well-defined model, are the ones done perfectly by algorithms. And those millions of good decisions on which books to show, for which customer, add up to a substantial profit. Even if the algorithm is occasionally wrong, it rarely matters.

However, it’s a different story if there is only one significant decision to make, especially one which is hard to define. Then humans want to get involved.

Combining both worlds

Dawson points out that big data needs interpretation and communication in order to be able to support human decisions. And while many jobs will be replaced by algorithms, a growing number of jobs will be needed to mediate between data, algorithms, and human decision-makers.

This appears to be in line with managers’ views. In a study of C-level executives by the Economist Intelligence Unit in September 2014, nearly half believe big data to be a useful tool, while only less than a quarter believe it will revolutionize the way businesses are managed. The biggest named obstacle to using data more extensively is the lack of managers’ understanding of how to apply data in their functions, while the most named solution to this is to create enterprise-wide teams to assess and propose approaches.

“The true professional is one who can communicate data in a way that changes the thinking of the highest-paid person,” Dawson says. “This is a human art requiring relationship skills that far transcend those of computers.” These people need to understand human cognition and empathy as much as they understand the data. Let the HiPPOs and trained professionals decide—and keep them well informed with qualified data.

For more on this topic read this post by Ross Dawson.

Image source: Caitlin