CPG Product Innovation Should Be A Science, Not An Art

John Wanamaker, the American retail magnate, once said: “half the money I spend on advertising is wasted; the trouble is, I don’t know which half.”

Today, every advertising dollar is tracked, attributed, and rigorously analyzed. We’ve entered the age of predictable revenue, in which fluffy campaigns without clear goals and measurements are no longer tolerated. Instead, tech CEOs like me are interested in building a systematic, reliable demand engine to fuel business growth. And while marketing remains a deeply creative profession, it’s now a science as much as it is an art.

Why has this shift not yet infiltrated the world of new product innovation?

The Problem with Innovation

Most CPGs have come to accept high innovation failure rates (as high as 85% according to Nielsen data). Failure, and the exorbitant cost associated with it, are not only tolerated – but expected. Why? Well, because the common mindset is that all innovation is hard. And, like marketing, it’s an inherently creative pursuit.

But, just as the marketing industry has evolved to prove Wanamaker wrong, CPG leaders must also strive to build a predictable innovation model to fuel reliable, consistent business growth. It can be done, but it requires a significant mindset change, alongside strategic investments in data, process and technology.

First, let’s distinguish between the different types of innovation that most CPGs employ: incremental and breakthrough.

• Incremental innovation refers to refreshing and restaging products. This could be a new flavor, new format or repositioning an existing product against a new benefit or usage occasion. The consumer ‘job to be done’ is established and therefore the risk of getting it fundamentally wrong is minimal and carries a lower risk.
• Breakthrough innovation on the other hand is disruptive by nature. Brands innovate to enter or create whitespace territories to serve new or unmet consumer needs. These are risky, uncertain, and expensive undertakings, but the rewards can be game changing.

From my experience, less than 10% of CPG innovation is truly breakthrough. And when you’re breaking new ground, in unknown territory, some degree of failure is inevitable. But the incremental innovation projects shouldn’t fail. We know what success looks like (it’s been done before), so the risk to rigor equation can and should be adjusted to prioritize speed to market and operational efficiency.

Let’s say you’re responsible for launching a new energy drink flavor. Do you really need a slow, multi-methodology, expert-led research process? Or do you need a continuous data source that allows you to tap into consumer needs and preferences in real time? The answer, at least in my mind, is clearly the latter. With the right consumer data inputs, incremental innovation can be systematized and standardized, ensuring consumers needs are reliably and repeatedly met.

Product Innovation as a Science

What’s clear is that, to achieve predictable innovation results, we’re going to need to shake up the traditional market research space and inject more science and objectivity into the innovation process. That’s where big data comes in. More specifically, big social data.

We know that primary research (consumer surveys, focus groups, etc.) is subjective and error prone. There are a couple of problems with it. Firstly, people lie. When asked to predict what they would do in the future, or recount what they’ve done in the past, consumers are much more likely to tell you what you want to hear (or what they wish was true). There’s a well-documented “say-do” gap, but no way to fix it. Secondly, most consumer surveys or focus groups rely on a handful of participants. It’s difficult to put together a truly representative sample. Finally, and perhaps most importantly, we don’t know what we don’t know. The questions asked of consumers, and the pre-coded lists of answers available, can never be free of the researcher’s assumptions and biases.

I believe that we’re living through a revolution in market research. Question-based methods are being replaced by observational, behavioral methods which yield much more accurate and insightful results. We don’t need to involve countless research agencies and consultants to over-engineer and bloat the innovation process. We just need better data.

Every day, millions of consumers jump online to share product reviews, opinions, and requests. By leveraging artificial intelligence, brands can pool, parse, and analyze these data points in real time. Why is this better? Let’s take the problems with primary research one by one.

Firstly, by analyzing big social data, you’re listening to what consumers are saying when you’re not in the room. Say goodbye to the say-do gap. Secondly, you’ve got the benefit of scale. Millions of data points are far more reliable than dozens or hundreds. Lastly, you can train AI models to unearth every trend consumers associate with a topic or need-space, and scientifically forecast which will grow in relevance in the future.

It means that instead of relying on a consumer expert or strategist to join the dots, the data can do it for you. Giving you accurate, objective data on the most relevant flavor, ingredient, benefit or claim to base your incremental innovation on.

The Market Research Revolution

Behavioral consumer research methodologies are far superior to their question-based counterparts. That’s been true for a while, and c ompanies like Conagra Brands and Pepsi Co have been vocal about their decisions to move away from primary research.

But recent advancements in technology, specifically within the realm of artificial intelligence, are changing the game. Big data models that leverage AI and behavioral consumer inputs provide far more rigorous, reliable, and forward-looking outputs. It’s the perfect blend of qualitative and quantitative data – and it’s the first step to a more scientific innovation process.

Innovation, like marketing, is part art and part science. Of course, we need creative minds and mad scientists to produce truly great products. But, in my view, the CPGs that pursue a standardized, predictable model for incremental innovation will gain an invaluable competitive advantage.