Using data to make better decisions, faster.

Usually when I ask a client why they need me to improve and enhance the data they measure we end up with an answer like:

“Because better data allows us to make better decisions.”

For a long time I agreed with this blanket statement. But like everything in our complex world, there is a “it depends” part attached to it. In other words, the title of this post is partly clickbait ;-).

We’re measuring the least important things

I started out my career simply measuring data and improving the quality of it. In the ‘early days’ this already provided so many easy insights to fix that you hardly needed to do any actual analysis to fill the backlog of “things to improve”. By the time these were fixed, the client simply commissioned a new website to be built and threw away all improvements in favor of a new homepage image slider.

These insights are hardly ever the ones that truly differentiate a business from its competitors though. This “low hanging fruit” is essential to fix if you don’t want to leave money on the table, but they are rarely the tweaks that determine bottom line business success.

The things that do really matter tend to be the things that are hard to measure, especially without any customization. Even with customization, we’re often measuring a “proxy” of what we truly want to measure about our customers and their interactions with our businesses.

Things like “how excited are you about your purchase” and “how easy was it to find & decide what to buy” and “how likely is this person to return to us in the future” are all extremely hard to get quantitative unbiased answers to via the data we measure.

The More Data = Better Decisions Fallacy

Somewhere along the ride we’ve collectively gotten addicted to data. I’m not sure what the reasons are, but I think I’ve identified at least one and that is that staring at data makes us look smart and gives us something to blame when decisions go bad.

There are 2 problems with using data to improve decision making:

1. Humans are biased
2. Data is always “imperfect” at best*

*I focus on user interaction data with digital platforms but I think this statement is true for almost all data. Usually, there are technical and human issues in collecting, storing and enriching it.

That is not to say that data can’t improve your decision making, but I would argue that there is a point of diminishing returns where getting more data will not make your decision any better. So where is this point?

Classifying Decisions

Almost every action we take could be drilled down to a “decision”. Do you get out of bed with your left foot first or the right? Decision. Fortunately we’re really good at programming ourselves to eliminate most decisions from our rational brain to free up our decision making capacity for important questions like “What Netflix serie should I watch next?”.

In business, however, it pays to sometimes zoom out and think about what you’re trying to achieve and why. I wrote about that here.

Inevitably when you start to rationally think about what you’re trying to achieve and how to achieve it you’ll run into challenges that require you to make decisions worth thinking about. I usually classify them with the following attributes.

Risk Involved: One & Two-Way Doors

I’ll let my buddy Jeff explain this one:

Most decisions should probably be made with somewhere around 70% of the information you wish you had. If you wait for 90%, in most cases, you’re probably being slow. Many decisions are reversible, two-way doors, so what if you’re wrong?

Jeff Bezos

So what Jeff and I are saying here is that a lot of decisions that look really risky and impactful turn out to be reversible. If that is the case, the real risk and impact suddenly go down. What you’re left with is the required investment to test the idea.

Speed + Accuracy = Velocity

The main levers you can pull when it comes to decisions are the Speed with which you want to make them and the Accuracy that the decisions have the results that you intended them to have.

I would argue that in most areas of uncertainty (like introducing new products to new markets) your ability to be accurate drops significantly and you’re better off testing more things at a higher speed.

You want to end up at the right velocity of making decisions at the highest speed possible with an attainable portion of accuracy mixed in (keeping in mind that accuracy based on data is highly flawed in itself.)

Using data for Speed, not Accuracy

So do I think data is useless? Not at all! But what I do think is that besides using data to fix the “obvious flaws” and “low hanging fruit” we should be more aware of the fact that we’re likely wasting time trying to make our decisions more accurate.

Although all data is flawed, it is perfectly suited to create a baseline on which we are then able to create a framework to improve. It actually allows us to speed up decision making by testing their impact instead of researching their accuracy.

So in the end, data can help us to make better decisions faster, once we embrace the fact that it’s not a tool for improving our accuracy before making the decision but rather a tool that allows us to validate more decisions at a higher speed.