S2 Ep 30: Forty percent fail: Why it's not about money or talent

In this episode of A Splice of Life Science Marketing, Matt Wilkinson and Jasmine Gruia-Gray unpack a deceptively simple question: why do so many launched products fail commercially, and what do top performers do differently? They start with Coca-Cola's 1985 reformulation disaster—a case study in how rigorous research with the wrong question can fail just as catastrophically as no research at all—and move into the data that actually holds up: the 40% commercial failure rate that separates process discipline from market luck.

The Coca-Cola paradox: 190,000 taste tests, one catastrophe

Jasmine (00:02)

Hey Matt.

Matt Wilkinson (00:04)

Hey Jasmine, how you doing?

Jasmine (00:06)

I'm excited for today's chat. It gave me an opportunity to get outside the life sciences for a few minutes and read about an interesting case study on Coca-Cola.

Matt Wilkinson (00:23)

Nice. Tell me more.

Jasmine (00:25)

Okay. So in 1985, when both of us were very, very young pups, Coca-Cola ran roughly 190,000-person blind taste tests before it changed the formula of its flagship product. Little risky, but it seems like they did a lot of great market research beforehand. That's not cutting corners on research, that's one of the largest consumer studies in commercial history. The new formula beat Pepsi, it also beat the original Coke. 79 days after launch, drumroll, the old formula was back on the shelves under a different name, and the reformulation had become the most quoted failure in marketing.

The lesson people usually take from that story is listen to your customers. But in this particular case, that's the wrong lesson. Coke listened to 190,000 of their customers. What the tests never asked was how anyone would feel if the new formula replaced the old one. The data was real, the sample was enormous, and the question was pointed slightly away from the decision being made.

Nobody in the building knew that until the phones started ringing. This obviously 1985, this is way before smartphones, etc. Now, hold that against a number that has nothing to do with a soda or soft drink. Around two in five products that reach a market fail commercially. Not two in five ideas. Two in five of the products a company chooses to fund to staff. It approves through the whole new product development process, and those fail. Every decision in the process was a yes along the way, along those stage gates. The people in those rooms were not careless. They were unanimous. And still, those two things were not the same. So, Matt. You wrote about this last week. What put you in this chair to write about it?

Untangling the myths: 90% failure vs. 40% vs. 3,000 ideas

Matt Wilkinson (02:54)

Well it's because I was wrong. I've been quoting an article from researchers at Cass Business School, now Bayes Business School, for nearly 15 years. And there was an article that was published that states 90% of technological innovations fail. Now the researchers argue it's usually because their developers don't communicate early enough. Now they're talking about technological innovations here. We're talking about the forty, you know, 90% of those. 40%, you know, is you know, two and five is new product introductions. So there's a bit of a challenge here in are we really you know comparing apples to apples? And and what the researchers don't do is actually define what they mean by technical and technological innovation.

But it seems that that article that I've been quoting for 15 years was quoting the same source that a pair of researchers called Castellan and Markham decided to go hunting for. And then they quite publicly published that they couldn't find the the actual data to support that ninety percent claim. So they then described it as an urgent urban legend.

So I thought it was time to put the record straight because there is, if you go back up in the blog archive enough, actually a blog that I've written that starts off with 90% of new products fail. So I thought I'd better put the record straight. There's another idea though that people quote, and that says that for every successful idea, there are 3,000 unsuccessful ones. So that means for us to be successful, Jasmine, we need to be having thousands of ideas, not just a few. So that's that that's something for us to to you know to mull on.

But the idea of those 3,000 is that that figure comes from researchers called Stevens and Burley and it counts raw ideas. Now, most of those are the sorts of ideas that come up when people sit around a table brainstorming, and they're the ones that are discarded before anybody even considers putting together a business case. So, you know, it's again, it's not you know, it's not a fair comparison. Obviously, if one in three thousand products failed, clearly RD wouldn't be happening in the way it does. But it's it, I thought it was a really interesting kind of study to look at and go, right, maybe we need to do something about helping people to not fall into that 40% of products that fail, because there are interesting trends across different industries and between different companies. That would indicate there are things that you can do to improve your likelihood of success.

The 40% that actually holds up, and the 76% that separates top performers

Jasmine (05:48)

All right, so tell us more. What what's the number that actually does hold up and what can you do to improve the likelihood of success?

Matt Wilkinson (05:59)

So the number that does hold up is the 40%. I mean the 3,000 ideas as well is is it seems to be true, but I don't think it's very helpful that we you're just just discarding ideas. What really we're looking at is that is that 40%, but there's a split underneath it. You know, top performing companies fail at only 24% at the time. So that means that 76% of their new RD budget, or you know, new new product programs actually are meet their commercial targets. Everyone else sits at sort of 46%. So those are across the same sectors, same buyers, same market conditions. So that sort of shows that the failure rate is really not to do with market conditions, it's to do with a process variable. And it's something that an organization can change because it's a process that they own. And so that's that's what was really exciting about.

Why failure is a process variable, not a market condition

Jasmine (07:04)

So what in the process are you advocating can change? I mean you clearly have said that money and talent are are not the answer.

Matt Wilkinson (07:16)

Yeah, so what separates those two sets of company is who is down to sort of product definition and really clear and clearly defining who the product is designed for and how precisely that that is being done. Now in life science tools, you know, that failure sort of has a signature. The customer never gets pinned down. So the ordering chemist, the lab head and the budget holder all get addressed as if they are one person with one motivation.

Things like workflow friction get underestimated because the people modeling it have not run the assay in six years. Evidence gets generated to satisfy internal reviewers or friendlies rather than to move a purchase decision. You know, none of those are scientific failures, they're commercial failures that really go to show that that companies haven't done a good enough job of really testing the water of the market that they're going to be launching into.

How approval cycles pull the message away from the buyer

Jasmine (08:16)

So if that split comes down to process rather than circumstance, what does that say about where the responsibility actually sits? Is is somebody choosing that outcome? I doubt it.

Matt Wilkinson (08:30)

No, so nobody chooses it. It it's down to the process. You know, the the the original you know product description or target product profile gets written when you're pretty close to the buyer, when you pretty much understand what needs to happen. But then as it goes through approvals, each cycle pulls the message a little closer to the room it's in, a little closer to RD, a little closer to what we believe to be true rather than what is true. And the people with the most approval authority are usually the furthest from the buyer. So what happens is that things get massaged to appease the people that are in the room, the stakeholders in the organization, rather than to really make sure that things are a truly commercial success. And I think that's one of the really interesting things that the research found.

Grounded synthetic customers and the echo chamber problem

Jasmine (09:26)

Okay, so let's test what you propose to put in the room. You argue for a grounded synthetic customer, and I'll give you a few minutes to talk about what that means, built from a company's own transcripts and win-loss calls, etc., and you'll go into more detail. The pushback I hear from great researchers is that this is an echo chamber with a better interface. If it's built from what you already recorded, how can it ever tell you the thing you failed to ask?

Matt Wilkinson (10:00)

Well it can't. Clearly it can't. But what it can do is it can tell you the things that you've missed or the things that you've forgotten. I don't know how many times you know, you've you've been in this position, but you go out and you keep you know, you you collect voice of customer research and you've got a ton of you know, you've got a ton of documents that that are transcripts from calls, you've got research from you know that you've done online.

And you know, if you're in a product review, you know, or a stage gate review, you're probably going to grab ten or fifteen of those, but you never quite know which ones are going to unlock the thing that is going to help you think differently about your product or about a particular feature or a particular decision. So what a grounded synthetic customer does is it creates a mechanism where you're able to actually go in and ask it, you know, specific questions around, around different features or different parts of a product or different aspects of a decision.

And you can then iterate with that synthetic customer rather than having to go back, for example, to your research team, or in the life science context, having to go back and run more customer interviews. It's much faster than going through that whole process again. I think there's a real risk here and it's it's quite interesting actually. The risk is that if you're, you know, if you're not thinking carefully about what questions you're asking, and you're just sort of asking the synthetic customer, you know, lots of random things, then yeah, you could end up with an echo chamber effect. But if you're asking it really good questions, really, really well structured questions, I think you can actually, you know, you can definitely beat the echo chamber.

Why Coca-Cola asked a preference question when it needed to ask a purchase question

Jasmine (14:06)

So I think, you know, going back to the beginning opener on the Coca-Cola story, the Coca-Cola decision was grounded in reasonable data. You nobody could argue that 190,000 interviews isn't reasonable data. But the no model imagining anything. Let me start again. So I I think I handed you a problem in my own opener. So let me put it back on the table. Coke, Coke's data was grounded. 190,000 real people, real responses, no model imagining anything. Is it still pointing at the wrong question? Does grounding in the case of synths help at all with the aim?

Matt Wilkinson (15:06)

Well, I I think you almost answered your own question there. In terms of the question that Coke were asking was a blind taste test, which of these do you, which of these tastes do you prefer? So there we were asking a preference question. The question wasn't which of these would make you want to buy, you know, select Coca-Cola over Pepsi, which was why they were going through this because at the time they were losing market share to Pepsi. So the question really was that, you know, when somebody buys a Coke, they're buying a brand and they're buying a taste. They might prefer another taste, but actually, if they're going to go and buy a Coke, they want to keep, you know, they want it to taste the same. And so that that's the that's the challenge. That's where we've got to get smarter about the questions that we ask, the way that we ask them, where we ask them. We've got to try and understand our customers better and better.

And in this age of AI, one of the things that I think that I truly believe is going to set apart, you know, customer focused companies from from those that are sort of following the old build it and you know, the field of dreams model, you know, build it and they will come approach, is that if you're really taking a customer focused approach, then you're really trying to understand not just what does somebody prefer, but what does somebody expect. What's the emotional journey that they go go through? How are they going to feel if the next time they pick up a you know a red can of Coca-Cola that it tastes different? And that was the problem. It was the fact that they they never really tested that you know the test didn't involve somebody opening a can of Coke and you know having that taste from that can or from that bottle. And that's where the that's where the research fell down.

What to do at a stage gate review without synthetic customers

Jasmine (17:05)

Yeah, I I agree that we need to be better in life sciences in general and specifically in life sciences tools at asking some of those emotionally based questions because there are a lot of unmet needs in that emotion that could certainly be addressed when defining the the new product you're developing. So it when we have a product manager or a marketing manager coming up to a stage gate review and they don't have any synthetic buyers at their disposal, what what would you recommend they do to try and and get around this situation?

Matt Wilkinson (17:57)

I think the biggest thing you have to look at is what assumptions are you making and clearly state the assumptions that you're making. You know, in the Coke example, the assumption was was that people would prefer, you know, if somebody preferred the taste of something, they would prefer that to be the taste in the can. Those two, you know, that's a c you know, they were trying to make a correlation between the preference of a taste versus the preference for what somebody would buy or expect when they were buying a can of coke. That's a big that's a big difference. So we have to be better at asking questions. We're better having better at articulating the assumptions and testing those assumptions. And I think that then it's about being able to go to market with with small pilots rather than switching your entire product line to to make a new taste and then find out that the market, you know, is in rebellion.

Pre-mortems: stress-testing assumptions through failure scenarios

Jasmine (18:53)

Yeah, I completely agree. I I would add to that that this this technique called pre-mortem that Gary Klein has written about is another good way of putting customers in the situation of of a failure or in the situation of something not working and asking them why why didn't it work? Why if if this new formula fails right now, what would be your reason for it failing? And through those kinds of questions, uncovering the some of the the the pains that would prevent you from launching a product and having it fail months later.

Matt Wilkinson (19:40)

I I'd agree. It's it it's a very tricky thing to do right. And and one of the things I've experienced with pre mortems is that not you know, even if you do a great job of a pre mortem, internal politics can often undo that good work. All that that'll never happen here. And I think that that that's one of the biggest learnings I've had from, you know, from projects in the past that I've led that, you know, we really should have kept some of those some of those items that we flagged as big risks on the risk register rather than taking them off because they weren't a risk that we needed to deal with.

Why pilots matter more than large-scale rollouts

Jasmine (20:24)

Mm-hmm. Mm-hmm. Yeah. As well as what you were saying earlier, okay, if if you so strongly believe that that will never happen here, then l let's run a pilot. Let's see.

Matt Wilkinson (20:38)

Yeah. Agreed.

Resources and closing

Jasmine (20:41)

So the full post is on the Striven website and it carries the citations for every number we've used today, including the ones that argue against Matt. It's at striven.com forward slash thinking, and the title is two in five launched products fail commercially. Matt, where should people go if they'd like to read more about synthetic buyers?

Matt Wilkinson (21:09)

Well, in case you've been hiding under a rock, I recently wrote a book called The Buyer in the Loop. Yeah, it's about the force that pulls every commercial message and decision away from the buyer and what it takes to escape that force. It's out now as an ebook and a paperback, and if you'd like to read it then you can probably find it on Amazon or Lulu. But if you'd rather have a conversation feel free to get in contact through through the website or on LinkedIn.

Jasmine (21:42)

Great. So Coke ran a hundred and ninety thousand in person test and never asked the question that mattered. Scale is not the same thing as aim.

Matt Wilkinson (21:58)

Yeah, two in five launch products failing is is pretty shocking. You know, and and the number of stage gates that that that means that failed products pass through is outstanding. We really need to make sure that we're we're asking the right questions and that we're getting the right people in the rooms that to ask those qu ask and answer those questions.

Jasmine (22:20)

Well, thank you so much for opening our eyes to a totally different way of looking at keeping customers in the room and being customer centric. Thanks again for a great conversation, Matt.

Matt Wilkinson (22:33)

Thank you, Jasmine.

Jasmine (22:36)

Bye for now.

Matt Wilkinson (22:37)

Bye.

Q&A