Business intelligence needs boost from decision intelligence

Limitations of BI

But BI has limitations, according to Wayne Eckerson, founder and principal consultant of Eckerson Group, who spoke during a recent webinar hosted by analytics vendor Sisu.

"The promise of BI was to turn data into insights and action, and it gets us part of the way there," he said. "But we've discovered over the years that it doesn't get us all the way there. It doesn't go that last mile to go from data to insights and action. There's nothing wrong with BI, but it has built-in limitations."

Eckerson noted that BI's limitations include the following:

• It's not actionable because it's based on historical data.
• It's too generalized, showing summarized trends and patterns that can miss subtle nuances that drive potential changes.
• It's too manual, requiring humans to sift through copious data to find relevant information.
• It's unable to predict future outcomes or prescribe solutions for those outcomes.
• It's not automated.

BI's constraints, meanwhile, are being exposed by the expontential growth in the amount of data organizations now collect and the increasing complexity of that data as it flows in from an increasing number of sources.

The result of BI's limitations is a decision-making bottleneck. The data is too much for humans to manage, and its complexity is beyond the scope of self-service users.

Just like how data teams building reports and dashboards was a slow process before the rise of self-service analytics -- with data consumers waiting weeks or even months for data teams to complete a given project -- data teams are again being overwhelmed by questions about data, and projects are again getting stalled.

Addressing the bottleneck

A few weeks or a couple of months to produce a report or dashboard may have been good enough a decade or two ago. Now that more of an organization's peers are as data driven as they are, it isn't. Additionally, fast-changing economic conditions due to worldwide events such as the pandemic and the war in Ukraine require organizations to act and react quickly.

"Solving the bottleneck -- other than data quality -- is the biggest issue facing the analytics industry," Eckerson said. "Business users just can't get the insights they need to take action."

And neither hiring more data analysts and data scientists nor developing better self-service analytics tools are the way to loosen the bottleneck, he continued.

As data volume and complexity increase, there will never be enough data workers to keep up nor enough money to pay them. Adding more self-service users and technology requires expensive data literacy training and stringent data governance to control potential chaos.

Instead, data teams need technology that makes them more efficient. That technology is decision intelligence.

"Decision intelligence platforms are like hiring an army of data analysts without spending any extra money, other than on a software license," Eckerson said.

Joel McKelvey, vice president of product marketing at Sisu -- a vendor that, like Pyramid Analytics and Tellius, specializes in decision intelligence tools -- likewise noted that BI needs decision intelligence capabilities to improve efficiency and meet organizations' modern needs.

"BI has been incredibly successful," he said. "But the data has grown -- and the complexity of that data has grown -- beyond our ability to serve it to just anybody in the company. I don't think BI is bad, but the implementation of a reporting and dashboarding tool has been outgrown. What we need now is a tool that automates much of what BI does."

The need for decision intelligence

Decision intelligence essentially uses AI and machine learning to monitor data.

Every organization has a set of business metrics that are most important to its success. Their data teams can program a decision intelligence platform to perform around-the-clock surveillance on those metrics and the data that drives them.

Decision intelligence platforms are like hiring an army of data analysts without spending any extra money, other than on a software license.

Any time there's a change in those metrics, the decision intelligence tool automatically alerts key stakeholders.

But they don't simply monitor what is happening. Because they can sift through millions of data point combinations in seconds, they can reveal why metrics are changing and suggest the most likely causes, saving analysts time otherwise spent performing root cause analysis.

The result is data workers becoming 10 to 100 times more productive than with a BI tool alone, according to Eckerson.

"Decision intelligence is about empowering data analysts and data scientists with a powerful engine that will rifle through millions of records in sub-second time and surface relevant issues that you need to deal with," he said. "You could never analyze your data that way as even an army of analysts, but these tools do that. They lift to the surface things that are worth looking at by the analysts that you do have."

That said, decision intelligence tools should not replace BI platforms, according to McKelvey.

Instead, decision intelligence should complement BI by surfacing insights that otherwise take months to discover and offering suggested subsequent actions. Despite its automation capabilities, decision intelligence should only be used to automate certain basic and repeatable actions resulting from automatically surfaced insights.

"Decision intelligence is about getting to a decision," McKelvey said. "It's not correct for us to assume decisions will be automated through this process. It really is about getting people to the point where decisions are data-driven and clear. It takes machine-scale data and winnows it down to a point where humans can take action in a meaningful way."

Human vs. machine

Humans possess critical intuitive knowledge, McKelvey continued.

They understand certain distinctions that machines may not. Therefore, while decision intelligence can automate certain straightforward processes and decisions -- "if X happens, then always do Y" -- more nuanced decisions require some human interpretation, according to McKelvey.

"We use decision intelligence to augment the human, not replace them," he said.

Humans also are better than machines at distinguishing if something seems a little off.

While machines can sift through data and surface insights exponentially faster than humans, they aren't necessarily better at recognizing when an insight might be based on bad data and, therefore, not correct. An action taken based on bad data can prove disastrous.

"Decision intelligence does not solve data wrangling and data preparation issues," Eckerson said. "You still need good data. It's garbage in, garbage out -- you want to make sure you're delivering high quality data, because the better-quality data you have, the better-quality results you're going to get."

Eventually, decision intelligence will let organizations automate more actions. But the tools are still in their infancy, according to Eckerson.

Ultimately, the goal of decision intelligence is to detect what happened, analyze why it happened, forecast what will happen as a result, prescribe potential responses and act on that prescription.

"We're not quite there yet," Eckerson said. "But I think in three to five years, this is where the vast majority of the technology will be. They're going to evolve very quickly in the next several years. The really cool features are coming down the road, such as the ability to do forecasting, prescription and even taking action, if needed."