The Wild West Rules of AI Discoverability

How AI Search is Rewriting Your Marketing Playbook

Right now, while you're reading this, an AI is deciding whether your brand exists. Not in a philosophical sense - in a practical, commercial one. A life science researcher is asking ChatGPT for recommendations on flow cytometry systems. Another is using Perplexity to compare qPCR platforms. Your product might be brilliant, your content might be comprehensive, but if AI can't find you, cite you, and recommend you - you're invisible.

For the first time since Google's early days, we're watching the rules of discoverability get rewritten in real time. And most life science marketers haven't noticed yet.

Here's what you need to know: the tactics that won SEO battles in 2015 won't save you in 2026. The buying journey has fundamentally changed, and with it, everything about how you need to show up.

It reminds me of the Wild West days of SEO (pre-2010), where gaming the system could have outsized effects where pages ranked in Google. When ChatGPT introduced its web search function, about 14% of the references were from Reddit. That has now dropped to around 2% as OpenAI quickly spotted marketers gaming the system.

So what is AI search anyway?

Before we go further, understand what AI search actually is: it's a branding channel, not a performance channel. As Mike King founder and CEO of iPullRank explained during a conversation on the Office Hours podcast with SEO guru and founder of SparkToro. Rand Fishkin:

"These platforms are more branding channels than they are performance channels... It tends to be more performant referral traffic... but your ROI is going to be reasonably low, at least right now."

AI search provides answers to questions and drives dramatically less traffic to websites than traditional search. But the traffic it does drive converts at extraordinary rates - up to 23 times better than organic search in some studies.

This isn't speculation. Ahrefs data from 2025 shows AI search traffic converts 23 times better than organic search but drives less than 1% of total traffic. Why? When AI answers the question completely, there's no reason to click through.

The question isn't whether to optimise for it; the question is whether you want your brand in the conversation when AI systems make recommendations.

The stakes? If your carefully crafted content doesn't exist in AI-mediated discovery, you are invisible. And your competitor who figured this out three months ago is getting the recommendations instead. This is particularly important as research by Forrester shows 90% of B2B buyers are using generative AI to research and shortlist vendors.

Now this is a long piece for me, but i wanted to download everything we understand about AI search right now, knowing full well, that it could all very well change tomorrow. In this blog I'll cover:

• Query fan out
• The AI crawler system
• The importance of text
• On-site fundamentals
• The importance of off-site signals
• Recency bias
• What this means for the buying journey
• The five steps you need to take now
  
  

The invisible mechanic: query fan out

Before we talk about crawlers, you need to understand the single biggest reason traditional SEO thinking fails in AI search:. This is a phenomenon called query fan out.

When someone asks ChatGPT "What's the best flow cytometry system for high-throughput screening?", the AI doesn't just search for that exact question. It generates five to ten related queries - sometimes more - and pulls information from all of them to synthesise the answer. That's query fan out.

As Mike King explains: "They're taking the query or the prompt that the user puts in and then extrapolating it to a series of synthetic queries." These might include "flow cytometry high-throughput comparison", "best cell analysis systems 2026", "flow cytometry system specifications", and half a dozen others you'd never predict.

This is why ranking number one for your target keyword no longer guarantees AI visibility. Studies show only a 25-39% overlap between traditional Google rankings and AI citations. Your competitor that only ranks fifth in Google might appear in ChatGPT's answer because the AI bots see they answer the synthetic queries the AI generated behind the scenes.

The implication? You're not optimising for one keyword anymore. You're optimising to own a semantic territory across a web of related queries. The more of those synthetic queries you can rank for, the more "raffle tickets" you have for appearing in AI-generated answers.

Understanding this invisible mechanic explains why early studies showed only 25% overlap between Google's top 10 and AI search results. Once researchers started accounting for query fan out, that number jumped to 39%. Without understanding what queries the AI is actually running behind the scenes, you're optimising blind.

Understanding the AI crawler ecosystem

Not all AI bots are created equal. Cloudflare's data from 2025 reveals three distinct types:

• Training crawlers account for 80% of AI bot activity. These are bots like GPTBot and ClaudeBot collecting data to improve language models. They're aggressive, often ignoring robots.txt files, and they take far more than they give back. Anthropic's ClaudeBot makes roughly 73,000 crawls for every single referral it sends back to websites.

• Search crawlers represent 15% of activity. These bots index content for AI-powered answer engines - the tools that are rapidly replacing traditional search for technical buyers doing product research.
• User-action crawlers grew 15-fold in 2025. These bots activate when someone asks an AI a question. ChatGPT-User, for example, doesn't crawl until a user asks ChatGPT to find information. Then it goes hunting.

The complexity? Googlebot and Bingbot serve dual purposes. They crawl for both traditional search and AI training. You can't separate the two. Block them for AI, and you lose search visibility entirely.

In July 2025, Cloudflare flipped a switch that gave people the option to "block the bots". I wrote about that here, and why you would be bonkers to block the bots and that if we want to be visible in AI search we have to do everything we can to make it easy for the LLMs to find and scrape our content.

Why text content dominates AI training

Here's something most marketers miss: not all content formats are equal in the eyes of AI systems. OpenAI has stated explicitly that it prefers text over audio and video because text is more straightforward to process. The algorithm doesn't need to interpret tone, facial expressions, or visual context.

The economics are stark. Processing text costs approximately 0.75 tokens per word. Audio jumps to 25 tokens per second. Video hits roughly 300 tokens per second. These numbers vary by model and improve over time, but the ratio tells you everything about priority.

When given a choice, language models will train on the most available and economical information. They default to text first. This means your carefully produced webinar or product demo video matters less for AI discoverability than the text transcript sitting on the same page. Your podcast episode matters less than the show notes and summary you publish alongside it.

This isn't an argument against video or audio content. It's an argument for making sure you're extracting maximum value from those formats by creating rich text versions that AI systems can efficiently index and cite.

I previously debunked the AI content-SEO penalty myth, but ensuring your content meets Google's  E-E-A-T (Expertise, Experience, Authoritativeness, Trustworthiness) guidelines is a very good idea.

On-site fundamentals: what works now

The good news is that foundational SEO principles still matter. The less good news is they're no longer sufficient.

Site structure, publishing frequency, and answering questions remain critical. AI systems prioritise content that provides clear, direct answers to specific queries. But here's where it gets interesting: you need to answer the questions your buyers are actually asking, not the questions you wish they were asking.

Use AI deep research tools to find those questions. Ask Claude or ChatGPT: "What questions are life science marketing managers asking on Reddit about content strategy?" Then answer them. On your site. In detail. With clear headers for Q&A format.

Semantic markup matters more than ever. AI needs help understanding what it's looking at. Tools like Schema.org markup tell the crawler "this is a product comparison" or "this is pricing information" or "this is a technical specification." It's the difference between being skimmed and being understood.

Your metadata is the advertisement to the LLM. In traditional Google SEO, meta descriptions get rewritten about 80% of the time. In AI search, they're critical decision points. As Mike King explains: "Your metadata is the advertisement to the LLM to determine whether or not they're going to use your content."

When an AI system gets your search result, it sees your URL, title, and description - the same as what appears in a SERP. Based on that metadata alone, it decides whether to request your page and use your content in the answer. Write meta descriptions that clearly signal what questions your content answers and what specific information it contains.

Semantic URLs deliver measurable lift. Research from Profound Strategy found that URLs with high semantic similarity to the query get 11.4% more citations in AI search. If your URL slug clearly relates to the topic - like /automated-liquid-handling-comparison rather than /product-page-47 - you're more likely to be selected and cited.

This isn't about gaming the system. It's about making it easy for AI to understand what value your page provides before it even loads the content.

Brand proximity to key topics. AI systems correlate words that appear close together. If you want to be found for "automated liquid handling," make sure your brand name appears within a few sentences of that phrase throughout your content. Repeatedly. Consistently. This isn't keyword stuffing - it's entity association. Chunking isn't a theory - it's measurable. One of the most contested topics in AI search optimisation is "chunking" - restructuring content into smaller, more focused sections. Critics call it unnecessary. The data says otherwise.

Mike King tested this by taking a single paragraph targeting both "machine learning" and "data privacy" and measuring its cosine similarity (how AI systems measure relevance). The original paragraph scored 0.648 for machine learning and 0.695 for data privacy.

He then split that one paragraph into two focused sections. The machine learning section jumped to 0.748 - a 15.4% improvement in relevance. The data privacy section hit 0.763 - a 9.8% improvement.

His conclusion: "Just me splitting this in half has made them more relevant and more [likely to] perform in those systems."

This doesn't mean writing robotic content. It means being deliberate about structure. One clear topic per section. Descriptive headers that signal what's coming. Data points that can be easily extracted. Information formatted for both human comprehension and machine parsing.

The best writers have always done this instinctively. Now we have quantifiable proof it matters for AI discovery.

Publishing frequency signals relevance. ChatGPT shows a strong recency bias, with 76.4% of its most-cited pages updated within the last 30 days. That's not a suggestion - that's a requirement. Your 2022 white paper isn't competing. Your quarterly blog post isn't competing. Fresh content wins in days, not months.

Page load speed matters differently now. Here's something most SEO guides won't tell you: if you're seeing 499 HTTP response codes in your logs, you're invisible to AI search.

The 499 code means the client gave up waiting for your server to respond. Because ChatGPT, Perplexity, and most AI platforms (except Google and Bing who have indices) fetch pages in real time, slow-loading pages simply don't get indexed. Mike King shared a case study where a client saw "a dramatic drop in their visibility in ChatGPT" directly caused by a spike in 499 responses.

Traditional SEO never taught you to monitor for this. Start now. Check your log files for 499 codes, especially from AI bot user agents. If you're seeing them, your performance problems are costing you AI visibility even if your Google rankings look fine.

Off-site signals: the new word-of-mouth economy

Here's where life science marketers need to wake up fast: off-site mentions now matter as much as backlinks. Perhaps more. I wrote about this here.

Press coverage and media mentions are essential. Wire distributions now help shape the narrative that Google's "no follow" links one overlooked.

Of course, real coverage in publications your buyers read is the best coverage you can get - but there are some sectors where those publications just don't exist. That's why I've purposefully been issuing press releases for Strivenn and even joined the Forbes Agency council to publish content - because every mention of your brand name - even without a link - is a signal to AI that you're worth citing.

Reddit and social media mentions have become ranking signals. Yes, really. Multiple studies in 2025 confirmed that AI platforms cite content 25.7% fresher than traditional search, and social platforms like Reddit are among the most-cited sources across ChatGPT, Perplexity, and Google AI Overviews.

In fact, as Mike King revealed: "Reddit is still the number one cited source across most [AI search] platforms." YouTube comes in second. These aren't fringe sources - they're the primary places AI systems pull information from when answering user questions.

This completely changes your content distribution strategy. That detailed technical discussion happening in a subreddit about microscopy? That's not just community engagement - that's AI training data. Those YouTube videos explaining your technology? If you are optimising your shownotes they will get cited far more than your polished pdf white papers.

This isn't just about volume. It's about context and sentiment. Natural language processing analyses the tone around your brand mentions. Complaints drag you down. Genuine enthusiasm elevates you. AI doesn't just count mentions - it reads them.

Wikipedia mentions signal authority. If your company or technology is significant enough for a Wikipedia entry, that's gold for AI visibility. Wikipedia consistently ranks among the top sources AI systems cite.

The Panda patent from 2012 referenced "implied links" - brand mentions without actual hyperlinks that could still signal authority. Google's John Mueller has since stated these aren't direct ranking factors for traditional SEO. But for AI search? The evidence suggests they absolutely matter. Brand awareness drives AI visibility in measurable ways.

The recency imperative

AI search has a recency bias that makes Google's freshness algorithm look patient.

Studies from late 2025 show that nearly 65% of AI bot hits target content published in just the past year. For ChatGPT specifically, content updated in 2025 receives roughly 50% of all citations from Perplexity. Google's AI Overviews follow similar patterns - the old strategy of "set it and forget it" content is actively harmful now.

This means your content strategy needs to shift from episodic publishing to continuous maintenance. That cornerstone guide you're proud of from 2023? Update it quarterly with new examples, fresh statistics, and current case studies. Change more than the publication date - add genuine new information.

The payoff is immediate. Unlike traditional SEO where ranking improvements take months, AI search can pick up and cite new content within days. Your updated comparison page can start appearing in AI-generated answers by next week.

What this means for the buying journey

The traditional B2B funnel is dead. What's replacing it is more chaotic than you thought.

Your buyers are going from ChatGPT to custom-generated comparison tables without ever hitting your website. They're asking AI to "compare the top three CRISPR screening platforms for high-throughput applications" and getting detailed feature matrices generated from your competitors' content - and yours, if you've done this right.

But that tiny percentage converts at extraordinary rates. These aren't tyre-kickers. They're qualified buyers who've already done the research and are ready to talk specifics.

The question isn't whether to optimise for AI discovery. It's whether you want to be in those automatically generated shortlists or watch your competitors dominate them.

Buyers enter at any point now. They loop back through earlier stages. They're guided by AI recommendations that pull from sources you don't control:

• Reddit threads

• Peer reviews

• Archived webinars from 2023.

Your job isn't to build a funnel. It's to be present in the web of information where buyers actually make decisions.

As Mike King explains: "AI search is effectively like a raffle... where you do have control is how many of those synthetic queries do you rank for. The more raffle tickets you have, the more likely you are to win."

Each semantic query where you rank well is another ticket. Fill the territory around your product category with comprehensive, well-structured content, and you dramatically increase your odds of being cited.

Most importantly: you're now writing for two audiences. Humans want story and context. AI wants clear structure and direct answers.

The best content serves both.

What to do next

This isn't a six-month strategy project. This is a "check your Cloudflare settings today" situation.

1. Audit your bot access. If you're using Cloudflare, verify whether you're blocking AI crawlers. If you are, decide which ones you want to allow. At minimum, don't block GPTBot, Google-Extended, Perplexity-User, or Claude-Web.

2. Check your log files for 499 HTTP response codes from AI bot user agents. If you're seeing them, your pages are timing out before AI systems can fetch them. This is invisible in traditional SEO tools but catastrophic for AI visibility. Fix your page load performance specifically for these bots - they're fetching in real time without the benefit of an index.

3. Identify your most important product pages and buying guides. Update them with semantic markup, using tools like Superschema. Add fresh statistics, current examples, and clear Q&A sections. Make sure your brand name appears in proximity to the key topics you want to own. Write meta descriptions that clearly signal what questions the page answers.

4. Look at your URL structure for new content. Use semantic slugs that clearly relate to the topic rather than generic identifiers. This isn't about going back and changing existing URLs (don't create redirect chains), but making better choices going forward.

5. Rethink your publishing and updating schedule. Fresh content matters, but strategic freshness matters more. Instead of just publishing more blog posts, identify your cornerstone content - the pages that answer critical buyer questions - and put them on a regular refresh cycle. Add new data, update examples, and extend sections where you can add unique value. This beats publishing generic content weekly.

The opportunity here is genuine. We're in the early days of AI search, which means the marketers who figure this out now will compound advantages for years - the data that gets searched is also likely to end up in the models - it's a win-win, discoverability now and in the future!

The ones who ignore it will wonder why they are invisible.

AI has changed the rules of discoverability. The question is whether you're ready to play by them.