S2: Ep22: the Tie is the Map

Matt Wilkinson and Jasmine Gruia-Gray work through a recent Strivenn blog post in which a LinkedIn poll on three book cover designs came back tied. What follows is the full conversation on why the tie was the useful result, how a synthetic customer panel predicted the shape of the disagreement, and how the real answer turned out to be a cover nobody had voted on.

The shrug that misreads a tied poll

Speaker: Jasmine [00:20]

Hey Matt, how's it going? So today we're going to discuss one of your more recent blog posts, where you ran a poll on LinkedIn and the results came back and three options land within a whisker of each other.

Speaker: Matt Wilkinson [00:21]

I'm good, how you doing?

Speaker: Jasmine [00:37]

No winner, just a number that looks like it says nothing. Most people shrug, they pick the one they liked anyway, and file the whole exercise under inconclusive. That shrug right there, in my opinion, is the mistake, and one that you really surface in your blog. I've been watching product and marketing teams do this for years. They run polls, surveys, different types of market research, and then they average the output into noise. The aggregate buries the answer every time. So in your blog, you put three book cover designs for your upcoming book in front of two audiences at once. The first was the humans on LinkedIn, and the second, a synthetic customer panel built from real buyer evidence. The headcount tied almost perfectly, and the three covers, the three audiences, and a number that looked useless. It was the most useful result you got all month. So, can you walk us through what you actually found?

Fifteen covers down to three

Speaker: Matt Wilkinson [01:50]

Yeah, so the setup was pretty simple. I started off with fifteen covers, and over a weekend I whittled those down to three. So there were three designs for the buyer in the loop, and the LinkedIn network voted publicly, my synthetic panel of four buyer personas voted as well. So we had quite a lot of interesting conversation and there was some really helpful feedback on all of them. But yeah, the public vote landed almost exactly tied. And of course with four personas across that synthetic customer panel, of course there was going to be a winner if they had to vote. But it was too close to call.

Speaker: Jasmine [02:25]

So some people's instinct is to call that a failed experiment. But you didn't.

Four markets hiding inside one count

Speaker: Matt Wilkinson [02:32]

Yeah, but I almost did. And then I went back and I looked at every commenter by their actual role, and that's when the data stopped being flat. The aggregate had hidden four completely different markets inside the one count.

Speaker: Jasmine [02:47]

So can we dig in a little bit more into what each segment actually did?

Speaker: Matt Wilkinson [02:51]

There were a number of different results. The most striking was that those that were leadership, founders, senior executives, people that I'd studied with on the MBA, they typically went for option two, which was in the middle. It had a very blue background, sort of a bit like a Gartner front cover, kind of authoritative. Their language always talked about credibility and positioning. It was the book that people were saying, this is the book I should have on my shelf. But I don't want it to be on someone's shelf. I want it to be read. And this isn't really a book for leaders and founders, it's more for marketers and sales. And so it's for everybody, but it's a marketing book ostensibly. And then when you look at the marketers, the marketers kind of broke across two groups. One for a simpler, plainer cover, the others for one that was very similar to number one, but a few extra design tweaks to it. The more experienced marketers went for one, the less experienced marketers preferred to have the human head on it. And what was interesting is that most people that worked in sales tended to like the image where there was more of a person on there as well. So there was really interesting kind of split across those groups. But it was what they said that was really important as well.

Buying a signal as much as the content

Speaker: Jasmine [04:03]

Say more.

Speaker: Matt Wilkinson [04:04]

Well, a lot of people were buying a signal as much as the content. Senior marketers, the people whose literal job is to choose between options, split evenly across all three, which was most interesting. The one that actually pictured the buyer was preferred by junior marketers and sales. Their reasons were practical, distinctiveness and visibility. And it was just really interesting that we had this group, but it really matched what the synthetic customers were saying as well in many ways. And so that showed the validity of the synthetic customer challenge, and also the fact that none of them could pick a specific output.

Speaker: Jasmine [04:40]

Same three covers, four audiences, four different answers, which means you're not really looking at one market.

Speaker: Matt Wilkinson [04:50]

Exactly. And a tie in aggregate is the market telling you it's more than one market. The only way to break that apart is to stop looking at the markets as a whole and start looking at who you really want to influence.

The panel that mapped the disagreement first

Speaker: Jasmine [05:00]

So here's the part I want to slow down on a bit, because this is where I think the blog goes somewhere genuinely interesting. Before any human voted, your synthetic panel had already mapped the terrain, and when the humans voted, the shape held.

Speaker: Matt Wilkinson [05:19]

Yeah. Leadership went solely to two. And when I say leadership, I mean people in more specific management roles. The two marketing voices in the panel split one for one, one for three. And the sales voice went to three. Then when the humans came in, they reproduced more or less the same structure. At least they found the fault lines. The individual votes moved around, but the shape of the disagreement really didn't.

Speaker: Jasmine [05:45]

So the panel never picked a winner. It predicted where the disagreement would live before a single person weighed in.

Speaker: Matt Wilkinson [05:55]

Absolutely. It gave me a really fast directional signal before I ever spent any time asking my network to jump in and help me decide.

Three objections a life sciences marketer would raise

Speaker: Jasmine [06:02]

So I want to push on this a bit more. I think there are three real objections that a life sciences marketer would bring to this methodology, and I'm curious about your answers.

Speaker: Matt Wilkinson [06:15]

Let's go.

Speaker: Jasmine [06:16]

Okay, so your LinkedIn audience is not really your buyer. You were polling peers, colleagues, people who you know, as you had just said. That's not a customer research sample. That's a professional social graph, in a sense. How do you defend what you learned from that?

Speaker: Matt Wilkinson [06:36]

Well, that's probably a weakness of my LinkedIn network, that it's not my buyers. But beyond that, I don't fully defend it as buyer research. I mean, this is a LinkedIn poll and it's a limit, but what I'd say is that the poll really wasn't designed to tell me what buyers will do. It was designed to surface the language different stakeholders would talk about and what people said and thought. And those words were really important. Some of the LinkedIn votes, the commercial leaders, the senior execs, they are buyers, and they sit right next to buyers. So the segmentation is useful, even if the sample is imperfect and probably too small to consider statistically significant. I can't extrapolate purchase intent just from that, but what I got was vocabulary and trade-offs, and that was really helpful.

Speaker: Jasmine [07:23]

Okay, fair enough. You're giving up purchase intent, but holding the segmentation value. I'll take that. My next objection is around your synthetic panel who predicted the structure of the disagreement. But your synthetic panel was built by you from buyer evidence, fair enough, that you curated using archetypes that you constructed. If it predicted the shape of human disagreement, how do you know it's because the panel is calibrated to reality and not because you already knew intuitively how these buyer types would split, and you built that into the personas without realising it? How do you rule that out, and that it isn't a sophisticated mirror reflecting your own priors back at you?

Speaker: Matt Wilkinson [08:16]

I can't rule that out. The panel is built on real evidence, not just invented archetypes. But the framing, the weighting, the way that the voices were represented, yeah, there's some of me in that, I'm sure. The panel's likely to reproduce it. What I would say, though, is that I didn't ever feed them a preferred cover. I didn't tell them any of my preferences. I knew which I wanted to win, but that's beside the point. The panel still found a structure that the humans confirmed. But does that prove the method? Probably not on its own from one data point, but we've got other data points to prove this sort of approach anyway.

Speaker: Jasmine [08:53]

So if the panel had diverged from the human result, you'd have said the panel gave you directional signal. It diverged because real people are more complex, which makes sense. If it converged, which it did, you said the panel predicted the structure. That's a hypothesis that can't lose. Both outcomes become evidence for the method.

Speaker: Matt Wilkinson [09:21]

I ran it as an experiment to see how well it would respond to imagery, honestly. I was surprised by just what it showed. And you know, the value is that it forced me to think in segments before I had any data at all. It did help me structure the question that I would ask of the data when it came in. Whether it predicted the answer is perhaps secondary to whether it sharpened how I'd looked for the answer and where I looked for the answer, but it's a fair critique.

Speaker: Jasmine [09:50]

So you're telling people to segment a poll that had what, a couple of dozen visible commenters, maybe? That's not a statistically meaningful sample. You can't build a segmentation strategy on maybe 20 votes sorted into four buckets.

Speaker: Matt Wilkinson [10:07]

Well, I think we had about forty, but you're absolutely right. To get to statistical significance you'd need a discrete population of at least four hundred votes to get that to a ninety five percent confidence level. So am I confident? No. But this isn't a quantitative study. This was qualitative signal gathering, and frankly also a marketing exercise. The question for qualitative work isn't, is this statistically significant. It's, is this useful? Do I get to a point where I get to data sufficiency? And I think the answer was yes, because after a while, we stopped hearing new comments. And so that's something where I think it was really important to hear. Was it directionally useful? Yes. Even with a small sample, the vocabulary clustering was really real. Leadership used different words than sales. The pattern was really worth looking into. Is it enough? I wouldn't bet a product launch on it necessarily, but I would bet a design conversation. And that's exactly what I did.

What to do in the first two hours

Speaker: Jasmine [11:03]

Okay, so let's make this useful for our audience. When you're sitting on a tied result, what's the first move? What do they do in the next couple of hours?

Speaker: Matt Wilkinson [11:16]

So I'd go back and I'd have a look at the comments and the responses and tag the role of the person, not just the vote they cast. You want to know which role is pulling which way. A small amount of time can turn a tied number into a bit of a segmentation.

The option nobody put on the table

Speaker: Jasmine [11:32]

So several commenters, I think I was one of them, independently described a design you never put on the table. How do you operationalise that?

Speaker: Matt Wilkinson [11:42]

There's an element of taste, right? And not everybody's going to like the same thing. You look for the option people describe that you never offered. Anybody who moves past your choices and starts designing in the comments section is giving you the real answer. They're not picking the nearest available compromise, they're telling you what they actually want. So those descriptions were really telling. They were the brief for the next iteration round, and they've already gone to Jim MacLeod, author of the Visual Marketer, and we've already come up with a final copy of what that cover's going to look like. And that means the vote really told us that the answer wasn't in those three covers at all. It was in something else. And I think we ended up with something much, much stronger. And I can't wait to show people what that final result is. But I think everybody will be a little bit surprised, and I hope a lot of people are going to buy it.

What this changes about preference testing

Speaker: Jasmine [12:32]

Absolutely. So if someone in life sciences marketing is running a preference test right now on product positioning or a campaign asset or a messaging hierarchy, what's the single thing this changes about how they run it?

Speaker: Matt Wilkinson [12:51]

Well, I would stop asking who wins or what wins. I would start asking who's choosing each option and why. That segment behaviour is your data. The count is really just the wrapper. Which buyer type is most decisive, which is most split, and what would they have chosen if you'd offered them something else? That's where your real positioning lives. And it was clear that the positioning for one group of people that were my target hadn't built the perfect option. I'm hoping I've got much closer now. But of course you're not going to capture the whole market with one option. There's always going to be variation within that. But it really helped us go from having no insight other than a synthetic customer panel, which was telling us something, to something that was much stronger. And what's really interesting, when I re-ran that test on the new panel, three of the four personas that are my key targets all voted for the new cover. We had a few options, but really I knew who I wanted to influence the most, and so that's where I'm headed now.

Where to read the full blog

Speaker: Jasmine [13:46]

If you want the full breakdown, including the language each buyer segment reached for, why the tied result was more useful than a winner would have been, and what Matt is taking to his book designer next, I'd encourage you to go read the full blog at strivenn.com. We'll also drop the direct link in the show notes.

Speaker: Matt Wilkinson [14:09]

Thank you. And if you're staring at a split result in your own research and you want to think through what's hiding in it, you can reach us either through the contact us button on strivenn.com or find either of us on LinkedIn. We try not to be hard to find.

Speaker: Jasmine [14:24]

So, in the sense of mythology, Tantalus reached for the fruit that kept pulling away. Your audience is doing the same thing with the option you haven't built yet. Stop counting the votes, read who's reaching instead.

Speaker: Matt Wilkinson [14:41]

The tie, as they say, is in the map.

Speaker: Jasmine [14:43]

Superb. Thank you again, Matt. And I am very much looking forward to reading the book once it comes out.

Speaker: Matt Wilkinson [14:51]

I'm looking forward to getting it in people's hands as well.

Speaker: Jasmine [14:54]

Thank you. Bye for now.

Q&A

I ran a poll and it tied. What is the very first thing I should do?

Go back to every response and tag the role of the person who cast it, not the option they picked. You are looking for which role pulls which way. Sort the votes into four or five buckets by job function and read the comments inside each bucket. An hour of tagging turns a flat number into a segmentation you can act on. The count was only ever the wrapper.

I do not have a synthetic customer panel. Can I still use this approach?

Yes. The panel gave Matt a fast directional read, but the core move needs nothing more than your existing responses. Take any preference test you already ran, one person, no budget, and re-read it by segment. If you want a panel later, start with three or four buyer archetypes drawn from real interview notes or CRM records. Build it from evidence, never from what you assume the answer will be.

My sample is tiny, maybe forty votes. Is segmenting it honest?

Forty votes will not reach statistical significance; you need roughly four hundred for that at a ninety five percent confidence level. Treat this as qualitative signal. The test is whether it is useful, and whether you reach data sufficiency, the point where new comments stop telling you anything new. Vocabulary that clusters by role is real signal even at small numbers. Use it to shape a design conversation, not to bet a launch.

How do I act on someone describing an option I never offered?

Treat it as the strongest answer in the room. Anyone who moves past your choices and starts designing in the comments is telling you what they actually want rather than picking the nearest compromise. Collect those descriptions and hand them to your designer or copywriter as the brief for the next round. Matt did exactly this, took it to Jim MacLeod, and landed a cover stronger than any of the original three.

How does this change a positioning or messaging test I am running now?

Stop scoring for the winner. Score for who chose each option and the words they used to justify it. Leadership and sales reach for different vocabulary, and that split is your positioning map. Ask which segment is most decisive, which is most divided, and what they would have chosen if offered something else. Run it on one live test next week. The answer to your positioning usually sits inside the disagreement.