Not Everything that Matters can be Measured

Slide 189 from the Internet Trends Report 2018

Shown above is a well-known slide from the always interesting Internet Trend Report by Mary Meekers that shows the explosion of data being collected and to illustrate we’re just at the beginning.

It’s a nice conversation starter, the big exponential growth projected is largely due to Internet of Things appliances gathering data in the future.

The problem that arises because of slides like these and articles like “Big Data is The New Oil” it that they lack context and meaning. (I’m guilty as well!)

Yes, we are collecting more data each year, but does more actually include what matters?

What matters?

When guiding businesses to figure out what truly matters, or zooming out before diving in, you will likely end up at some kind of proven or unproven value proposition. A proposition of value the company makes that appeals to the customer hopefully resulting in a transaction.

This can be something tangible like “easier” “cheaper” or “faster” but often times there are also intangible values at play like brand association etc and usually, it’s a combination.

In order to deliver on this value proposition, there are multiple processes at work to realize it. Some of them are lower down the value chain and have no direct relation to the end consumer. For instance the purchase and logistics of ordering base materials for product fabrication. Others directly interact with the end consumer, like the digital platforms a user interacts with to gather information or complete a purchase.

It is my current opinion that for the latter, the digital platforms that our end users interact directly with (including advertising) we measure a lot but most of what we’re measuring is irrelevant and we’re missing the most important bits.

What do we measure?

When talking about the realm of digital analytics what we measure is digital events tied to user identifiers. This is often direct or indirect user behavior. For example:

User 12345 clicked on Add To Cart on page product_name.html is direct behavior.

User 12345 triggered event_name_10_seconds on page abc.html is indirect behavior because we have configured the website to send a custom event after 10 seconds for which the user does not have to do anything.

All of this data is super interesting if you’re a website developer looking to figure out what people click, how they navigate your website and where they drop off. Obviously, this matters and as a company you should utilize this data to make your digital experience better. But unless people are truly unable to purchase via your website or submit the lead generation form I would argue it’s not the most important factor in conversion.

Measuring the Unmeasurable

Until we stick electrodes into every consumer’s brain to measure their emotional response we’re stuck here measuring how they bash their keyboards and click their mouses.

When we see users are not completing their purchases, we really want to know why they are doing that. Are they not ready right now? Are they evaluating other options? Are they not sure about some aspects of the product?

Although a lot can be inferred when tying qualitative research results to quantitative user behavior, exercises like these quickly become expensive. Also, for most companies, the sheer volume of data required to segment specific behavior traits is just too low.

When companies try to implement a data-driven way of working, what we usually end up with are proxies. Measurable events that have a high correlation to the overlying objective we want to achieve and seem to signal movement in the right direction.

Unfortunately, most of the explosion in data is skewed towards observational data of behavior and usage (like we already have). Hopefully, we will be able to derive better control metrics from this additional data but it’s likely that most will be “more of the same”.

So more ≠ better?

I think there is a lot of value in the growing volume of data that is being collected waiting to be unlocked. But I don’t think the benefits apply equally to all businesses and types of problems. For most of the challenges at the top of the value chain where companies interact with the end user, I would argue the increase in data will not have a big impact because we’re still not measuring what matters.

Like I wrote before, I think that the amount of risk involved in the decisions you make should dictate your data-strategy. The companies and problems operating at higher volumes with more certainty are likely to benefit the most from this surge in additional data. But because of those often being more mature and scaled already, I suspect there is less additional value to unlock to begin with.

Not everything that counts, can be counted. Not everything that can be counted, counts.

Albert Einstein or William Bruce Cameron

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.