Big Data, Small Results. How We Measure More and Know Less

Lately, I’ve been looking back on the 2010s, my first decade in the online business: The news stories, the conversations, the sense of break-neck speed.

• Monday: “Voice assistants, can you believe it?”
• Wednesday: “Yay, autonomous cars by 2020!”
• Friday: “What if the machines become too smart and kill us?”

At the same time I still receive ads for a fridge I already bought. Drones don’t deliver packages. And analytics systems aren’t smart enough to tell me whether today’s numbers are cause for concern or not. What’s the hold-up?

As opposed to what your Agile Guru might have you believe the pace of innovation has been slowing down over the last two decades.

Keep in mind: Innovation is not what you write on a whiteboard, it’s output.

Sure, Smartphones and Social Media are a huge deal. It’s just that one generation invented the Automobile and another invented Uber. Those two aren’t quite the same.

My personal theory goes like this:

We think that innovation is everywhere because we talk so much about it. And we talk so much about it because it is missing.

This has been on my mind because of the innovation I’ve been trying to push all these years: Big Data for Startups.

Big Data

So, let’s quickly get a working definition of ‘Big Data’, that is broad enough to allow me to lump a few phenomena together. Queue, Wikipedia:

“Big data is a field that treats ways to analyze, systematically extract information from, or otherwise deal with data sets that are too large or complex to be dealt with by traditional data-processing application software.”

In short: handling datasets that you can’t really store in Excel sheets and that you don’t usually analyze making a pivot table and then squinting at it really hard to find numbers that stand out to you.

Let’s call this “Excel-and-squint”-approach “small data”. The weapons managers use to survive in the “Small Data” world are anecdotes, common sense, and gut feeling. All of these are fine traits which mother nature gave us for good reasons. It’s just that sometimes you need to promise some numbers to your investors or decide whether to hire a new team. Less information won’t make that any easier to decide.

A Tale of Two Managers

Let me tell you about two managers:

• Manny is a business school graduate. He sees his strengths in his instincts and his dogged focus. He is great at networking and loves to talk business when he socializes. His home is in the “Small Data World”. Even under great uncertainty he delivers results.
• Linda has a science background. She is not very comfortable with uncertainty and not-knowing things. She also knows by her own training what data can accomplish. She likes to say: “If we CAN find out, we SHOULD find out.”

One day Manny and Linda have a fight: He is concerned about “analysis paralysis” and wasting money and time. She replies: “You just don’t get Big Data”, and is really smug about it.

But here’s the kicker: Any one of those two might be totally right or totally wrong depending on the business they are in.

Sometimes Big Data is the only salvation, often times it isn’t. Manny might need more data and not want it, while Linda might want more data when it’s really time to make decisions.

The two of them end up assembling a Data Team (maybe Linda won the fight or some investor pushed Manny to do it) but they really only ever come up with:

1. A self-made recommendation engine: Sure. Amazon is making bank with this, but is it going to save your struggling candle shop?
2. A customer lifetime value prediction model: Wouldn’t it be great if you could know how much you could spend on any given customer? Sure. But sometimes all you need to know is that your CLV is between 0 and 5€ and you can’t get your acquisition cost down below 200€. Fix this!

Based on these misunderstandings the Data team gets disbanded a year in. Never to be rebuilt. Why did this Big Data revolution fail?

The secret lies in how to handle innovation in general.

Inventions Versus Innovations

Touch screens are an invention, the iPhone is an innovation. The world wasn’t changed by the touch screen. Carrying around a Lithium-Ion battery by itself also isn’t appealing. Innovation means that a bunch of inventions (technological, organizational, institutional) come together to a coherent whole and that with time most people come around to adopting it.

Big Data is really just a collection of inventions, most of which make sense to the practitioners. There is no end to the discussions within data teams whether they want to combine Segment, Snowflake, and Tableau, or Snowplow, Redshift, and Looker. That kind of talk constitutes “fun” for those people. Believe me. I am one of them.

This is exactly how you fill up all the whiteboards in your office without any output.

Making It Stick

I have had my fair share of data projects that took months to introduce into an organization just to be abandoned soon after. Failed projects like this lead to a loss of trust in data in general. When failure strikes Linda might say: “People are too stupid to understand my great solution,” and Manny might respond: “It’s not that I don’t get it, it’s just that I don’t care.”

This is where the “Big Data Revolution” is getting stuck. And by ‘getting stuck’ I don’t mean that Google isn’t increasing their bottom line the more data they collect. I mean that Manny’s and Linda’s startup never catches up.

While those two were fighting over Attribution models, LTV Predictions, or recommendation engines, they never nailed down the things EVERY company needs to know.

For example:

• Cohort Reports: How are the users I acquired in January affecting my business in March? Does my retention enable me to continuously build up a user base?
• ROI: What is the difference between my user acquisition costs and my revenues? Is the growth target in my business plan going to lead to bankruptcy?
• Unit Economics: What’s the relationship between costs, revenues, and prices? Am I going to become more or less profitable when I scale my business up or down?

Answering questions like these is plenty of work for a data team and offers countless avenues to add projects and responsibilities as you go. And every time another relevant question is sorted out, the Big Data Revolution moves on an inch or two instead of rolling back one mile.