Cloudflare Radar: Anthropic crawls 2,442 pages for every visitor it sends back; Google takes 5. Plus SEOmator's 48-site scan and 30+ AI SEO statistics.
Cloudflare Radar data shows Anthropic crawls 2,442 pages for every visitor it sends back; Google takes just 5. SEOmator's own scan of 48 top sites found 46% block at least one AI crawler. Meanwhile 2 billion people use Google AI Overviews, organic CTR falls 61% where they appear, and AI-referred traffic converts at 23x organic.
This data-driven guide presents 30+ verified AI SEO statistics for 2026 with original sources, organized by category to help you adapt your strategy — starting with a dataset nobody else has published, and a number we published last month that we are correcting today.
Almost every statistic in this article measures demand — how many people use AI search, and what they do afterwards. Nobody publishes the supply side: whether the pages those engines want to read are even open to them. So we measured it.
On 13 July 2026, SEOmator fetched and parsed the robots.txt of 48 high-traffic sites across news, ecommerce, SaaS, SEO, reference, travel, finance, and health. We recorded, per site, which AI user-agents carry an explicit Disallow rule.
62% of the sites name at least one AI crawler in robots.txt. 46% actually block one. 54% leave every AI bot free to crawl.
Here is the per-crawler breakdown:
| AI crawler | Operator | What it feeds | Sites blocking it | % of 48 |
|---|---|---|---|---|
| ClaudeBot | Anthropic | Claude training | 11 | 23% |
| GPTBot | OpenAI | ChatGPT training | 10 | 21% |
| PerplexityBot | Perplexity | Perplexity index | 10 | 21% |
| Google-Extended | Gemini / AI training | 10 | 21% | |
| CCBot | Common Crawl | Open dataset behind many LLMs | 10 | 21% |
| meta-externalagent | Meta | Llama training | 9 | 19% |
| Bytespider | ByteDance | Doubao / TikTok AI | 8 | 17% |
| OAI-SearchBot | OpenAI | ChatGPT Search index | 8 | 17% |
| ChatGPT-User | OpenAI | Live user-triggered fetches | 8 | 17% |
| Applebot-Extended | Apple | Apple Intelligence training | 7 | 15% |
| Claude-Web | Anthropic | Claude live fetches | 7 | 15% |
| Amazonbot | Amazon | Alexa / Rufus | 7 | 15% |
Source: SEOmator robots.txt scan of 48 high-traffic sites, 13 July 2026. The gap between ClaudeBot (11 sites) and GPTBot (10 sites) is a single site — see the correction below before reading an ordering into it.
Three findings stand out.
GPTBot and ClaudeBot are effectively tied — and no, ClaudeBot is not "the most-blocked bot." In our sample GPTBot is disallowed on 10 of 48 sites and ClaudeBot on 11. That is one site. A sample of 48 cannot resolve a one-site gap into a ranking, and we should not have written it as one. Read them as tied. For the web-scale ordering, use Cloudflare Radar's robots.txt data in the next section — it counts far more domains than we can, and it puts GPTBot clearly in front.
Blocking is bimodal, not gradual. Sites don't partially restrict. The 11 sites that block ClaudeBot are overwhelmingly the same sites that block GPTBot, CCBot, and Bytespider. News and health publishers (nytimes.com, bbc.com, washingtonpost.com, healthline.com, webmd.com) block broadly; almost every SaaS and ecommerce site in the sample blocks nothing at all.
Google-Extended is blocked at the same rate as GPTBot (21%), not more. Common Crawl's CCBot — the dataset behind many LLMs — is blocked by 21% of the sites we scanned. Blocking Google-Extended does not remove a site from Google Search or from AI Overviews; it only opts the site out of Gemini training. Some publishers appear to have blocked it without realizing that distinction cuts both ways.
Methodology, and its limit. A site counts as "blocking" a crawler only when that crawler has its own user-agent group with a non-empty Disallow rule. Two of the 48 sites returned no parseable robots.txt and are counted as blocking nothing. Most importantly: robots.txt is a declared preference, not an enforcement mechanism. It shows publisher intent. It does not prove a bot complied, and it doesn't tell you which bots actually reached your server. For that you need server-side logs — which is what SEOmator's agent analytics reads.
Our scan measures publisher intent on 48 sites. Cloudflare Radar measures what the crawlers actually do, across a slice of the internet orders of magnitude larger than anything we can fetch ourselves. It is a separate dataset from ours, with a different sample and a different method — and it is the reason for the correction above.
It also contains the single most important number in AI SEO right now, and almost nobody is quoting it.
The crawl-to-refer ratio counts how many pages an operator crawls for each one referral it sends back to the sites it crawled. It is a ratio, not a percentage — read it as "pages taken per visitor returned."
| Operator | Pages crawled per referral sent back |
|---|---|
| Anthropic | 2,441.6 : 1 |
| Mistral | 1,123.0 : 1 |
| OpenAI | 260.1 : 1 |
| Perplexity | 226.4 : 1 |
| Microsoft | 34.2 : 1 |
| 4.8 : 1 | |
| DuckDuckGo | 2.4 : 1 |
Source: Cloudflare Radar (radar.cloudflare.com), 28-day window ending 14 July 2026.
These ratios move, and they have been moving in one direction. When we tracked the same Cloudflare Radar metric month by month earlier this year, Anthropic's ratio fell from 45,458 : 1 in January 2026 to 11,736 : 1 by mid-March; the 2,441.6 : 1 above is the next step down that slope. Our AI crawler crawl-to-refer report breaks the full series out by operator and by industry — finance sites earn the best return of any vertical, shopping sites the worst.
That table is the economics of AI search in one line. Search engines crawled you because they intended to send you traffic; the crawl was the cost of the referral. AI assistants crawl you because they intend to answer instead of you. Google's 4.8 : 1 and DuckDuckGo's 2.4 : 1 are what a working exchange looks like. Anthropic's 2,441.6 : 1 is not an exchange at all.
This is why "just optimize for AI search" is an incomplete instruction. For most of these operators there is no traffic to optimize for. What you are optimizing for is a citation — a brand impression inside someone else's answer — and you should budget and measure it as brand media, not as an acquisition channel.
Radar's 28-day share of AI bot traffic ranks the crawlers by the volume they actually pull:
| AI bot | Operator | Share of AI bot traffic (28d) |
|---|---|---|
| Googlebot | 24.8% | |
| ClaudeBot | Anthropic | 19.3% |
| Meta-ExternalAgent | Meta | 10.6% |
| GPTBot | OpenAI | 9.6% |
| Bingbot | Microsoft | 8.5% |
| Bytespider | ByteDance | 7.1% |
| Applebot | Apple | 5.9% |
| Amazonbot | Amazon | 5.7% |
Source: Cloudflare Radar, 28-day window ending 14 July 2026.
Strip out the two conventional search engines and one fact jumps out: ClaudeBot is the biggest non-search AI crawler on the web, pulling double GPTBot's volume — 19.3% against 9.6%.
That connects Radar's dataset back to ours, and it explains something our 48-site scan could only hint at. ClaudeBot is a younger crawler than GPTBot, yet publishers block it at essentially the same rate. Now you can see why: it is the crawler they see most in their logs. Blocking decisions are not made from press coverage; they are made from bandwidth bills and server logs, and in the logs ClaudeBot is twice the presence GPTBot is. A newer bot earning an equal block rate is not a contradiction — it is what happens when a bot arrives at twice the volume.
Radar classifies each AI crawl by declared purpose:
| Crawl purpose | Share of AI crawls (28d) |
|---|---|
| Training | 46.8% |
| Mixed purpose | 39.2% |
| Search | 10.8% |
| User action | 2.5% |
Source: Cloudflare Radar, 28-day window ending 14 July 2026.
Only about 1 in 9 AI crawls is search indexing — the one purpose that could actually cite you and send a reader. Nearly half is pure model training, which returns nothing at all. The remaining 39.2% is mixed-purpose crawling that may or may not surface you in an answer.
The practical consequence is that the search-purpose crawlers are the scarce ones, and they are the ones worth making easy to serve. A curated llms.txt file points them at the pages you actually want quoted, rather than leaving them to reconstruct your site from whatever they happen to hit first.
Radar also scans robots.txt across a far larger set of domains than we do. This is where its data overrules ours on the ordering:
| AI crawler | Domains disallowing it | Of which, a full block |
|---|---|---|
| GPTBot | 602 | 458 |
| CCBot | 551 | 464 |
| ClaudeBot | 539 | 422 |
| Google-Extended | 502 | 383 |
| Bytespider | 496 | 443 |
| PerplexityBot | 308 | 188 |
| Googlebot | 306 | 27 |
Source: Cloudflare Radar robots.txt scan, 13 July 2026. These are raw domain counts, not percentages of the web — the denominator is not published, so do not read them as shares.
Two things to take from this table.
GPTBot leads the disallow list, and our sample was too small to see it. 602 domains disallow GPTBot against ClaudeBot's 539 — a gap Radar's sample can resolve and our 48 sites cannot. This is the correct web-scale ordering, and it is the one to quote.
Almost nobody blocks Google. Googlebot appears on 306 domains' robots.txt, but only 27 of them block it fully; the other 279 are partial rules carving out a checkout flow or a search-results directory. Compare that with GPTBot's 458 full blocks. Publishers will slam the door on an AI crawler and hold it wide open for the search engine — which tells you exactly how they price the two. The crawl-to-refer table above tells you they are right to.
One nuance worth keeping: GPTBot is simultaneously the most-disallowed and the most-allowed AI crawler. 602 domains disallow it, and 265 domains write an explicit Allow rule for it — more than for any other AI agent. GPTBot is not simply the most-hated bot on the web. It is the most deliberated-about bot on the web: it is the one publishers actually make a decision about, in both directions. Every other crawler mostly gets whatever the block-list template said.
Which raises the only question in this section you can act on today: which of these rules are in your robots.txt, and did anyone at your company choose them? Run it through a robots.txt tester and find out before you read another statistic.
Barely at all — and the gap is lopsided in a way that is worth exploiting. In a scan of 107,308 domains on 13 July 2026, Cloudflare Radar found that 80.5% already publish AI rules in robots.txt, but only 8.4% will serve an agent a clean, machine-readable version of a page, and only 6.8% publish content signals. The web has made up its mind about saying no to AI. It has done almost nothing about being understood by it.
| Signal | Share of the 107,308 domains scanned | What it means |
|---|---|---|
Has a robots.txt | 83.9% | Table stakes; 1 in 6 domains still has none |
Has AI rules in robots.txt | 80.5% | Near-universal — 96% of the domains that have a robots.txt at all |
| Has a sitemap | 69.2% | 30.8% of top domains have no sitemap |
| Markdown negotiation (serves agents a clean version) | 8.4% | Effectively empty |
| Content signals | 6.8% | Effectively empty |
| MCP server card | 0.2% | Not yet a thing |
Source: Cloudflare Radar agent_readiness scan (radar.cloudflare.com), 107,308 domains, 13 July 2026. Radar returns raw domain counts here, not percentages — the shares above are counts divided by the 107,308 domains successfully scanned. This dataset is Cloudflare's, not ours; the analysis of what it means for SEO is ours.
The asymmetry is the whole story. Deciding whether to block an AI crawler is a crowded room — four out of five domains have already taken a position. Deciding whether an AI crawler can actually read you is an empty one. If you want an edge in AI search, the block decision is not where it is hiding. Everyone has made that call. Almost nobody has made the second one.
One vertical breaks the pattern hard:
| Vertical | Markdown negotiation | Sitemap |
|---|---|---|
| Ecommerce | 33.1% | 82.3% |
| Shopping & Auctions | 29.5% | 80.7% |
| Sports | 9.8% | 70.1% |
| Business & Economy | 8.2% | 76.9% |
| Information Technology | 3.7% | 67.3% |
| Entertainment | 3.1% | 69.6% |
| Education | 2.7% | 58.4% |
| News & Media | 1.8% | 78.5% |
Source: Cloudflare Radar agent_readiness, by domain category, 13 July 2026. Ecommerce n=17,032; News & Media n=7,034.
Ecommerce serves agents machine-readable pages at 33.1% — roughly 9× the rate of Information Technology (3.7%) and 18× News & Media (1.8%). The Shopping & Auctions category corroborates it independently at 29.5%, so this is not a sampling artifact.
Here is the part worth sitting with: the tech industry writes about AI readiness and the shopping cart industry ships it. And ecommerce almost certainly did not ship it on purpose — the rate tracks hosted platforms (Shopify, BigCommerce and friends) turning the feature on by default for every store on them. Which is precisely why it is a leading indicator rather than a curiosity. This is what the default looks like once a platform decides for you. Every other vertical is still opting in by hand, and mostly not bothering.
The payoff for bothering is not small. Radar measures serving markdown instead of HTML to an agent as a 7.1× payload reduction (ai/markdown_for_agents, 28 days to 14 July 2026) — the same content, a seventh of the bytes, and none of the navigation, cookie banners and script tags a model has to discard before it can find your answer.
30.8% of the top 107,308 domains have no sitemap at all. In education it is 41.6%. This is not an AI problem or a 2026 problem — it is a 2005 problem, still unsolved on nearly a third of the most-visited sites on the internet, at exactly the moment a second class of crawler started reading the web.
The order of operations follows from the data, and it is not the order most AI-SEO advice gives you:
Most GEO advice starts at step 3. The data says a third of the web has not finished step 1.
AI is reshaping how users interact with search engines. The introduction of AI Overviews, ChatGPT Search, and Perplexity has moved users from clicking links to consuming AI-generated summaries.
AI now processes and interprets search queries with unprecedented sophistication. Rather than matching keywords, modern AI understands user intent, context, and the relationships between concepts.
One system retrieves documents; the other composes an answer — and the two reward different work. Our SearchGPT and traditional search comparison walks through what changes for keyword, backlink and structured-data strategy once the result is generated rather than ranked.
The traffic follows. AI search traffic grew 527% year over year, according to Semrush's 2026 AI SEO report — faster than any other referral channel, from a small base.
Google's AI Overviews have become the most significant change to search results since featured snippets. Here's what the data shows:
Not every keyword is exposed equally. Question-shaped queries draw an AI Overview roughly twice as often as the desktop average, which is why the format of your target keyword now matters as much as its volume. You can check which of your own terms trigger one with the Google AI Overview keywords checker.
Zero-click searches – where users get their answer without clicking any result – have become the dominant search behavior in 2026. But the headline number depends entirely on what you're measuring.
| Search context | Ends without a click | Source |
|---|---|---|
| Google searches with an AI Overview | 43% | ALM Corp, 2026 |
| Google AI Mode | 93% | ALM Corp, 2026 |
| All US Google searches (estimates vary) | 34%–68% | Multiple clickstream panels |
Source: ALM Corp, 2026; range for all-searches figure compiled from published 2024–2026 clickstream studies.
That third row is the honest one. Published estimates for the share of all Google searches ending without a click run from 34% to 68%, depending on whether clicks to Google's own properties count, which clickstream panel is used, and whether the sample is desktop or mixed. The direction isn't in dispute. The decimal is. Any post quoting a single all-searches zero-click figure to two decimal places is overstating what the data supports.
26% of users leave Google entirely after reading an AI Overview, up from 16% without one. (Source: Pew Research, 2025)
Looking forward: Gartner projects traditional search engine volume will drop 25% by the end of 2026, and by 2028 roughly 50% of all searches are expected to be generative. (Source: Gartner, 2024 prediction, reported via Incremys, 2026)
At a 93% zero-click rate, AI Mode is a visibility channel rather than a traffic channel, and it cites a different source mix than AI Overviews do. That makes it worth checking on its own terms with the Google AI Mode checker rather than inferring your presence from AI Overview data.
The AI search market has become increasingly competitive. Here's how the major players stack up in February 2026:
| AI Platform | Market Share (Feb 2026) | Change from Jan 2025 |
|---|---|---|
| ChatGPT | 64.5% | ↓ 22.2 percentage points |
| Google Gemini | 21.5% | ↑ 15.8 percentage points |
| Grok (xAI) | 2.3% | ↑ New entrant |
| Perplexity | 2.0% | ↓ 0.4 percentage points |
| Others | 9.7% | Various |
Source: First Page Sage, February 2026
ChatGPT remains dominant but has lost significant ground. Google Gemini has been the biggest winner, nearly quadrupling its market share in 12 months.
Published user-count estimates disagree, and they disagree in a specific way: they mix weekly and monthly actives. Figures for 2026 range from 700 million to 1 billion. Read the metric, not just the headline.
Perplexity has positioned itself as an AI-first search engine rather than a general chatbot:
Perplexity's 2.0% chatbot market share understates its SEO relevance: it cites sources on nearly every answer, and it is blocked by 21% of the sites we scanned. At web scale it is blocked less than the big training crawlers — Cloudflare Radar records PerplexityBot disallowed on 308 domains, roughly half GPTBot's 602 — so more of the web is open to it than to OpenAI or Anthropic. It also crawls 226.4 pages for every referral it returns, so treat presence in Perplexity as citation share, not as a traffic line.
Understanding how AI systems select sources is where optimization now starts:
The first bullet is contested, and it's worth saying so. Ahrefs puts 76.1% of AI Overview citations inside Google's top 10; other analyses have found a substantially larger share coming from below position 10. The studies used different query sets on different dates and can't be reconciled. What they agree on: a top-10 ranking helps your odds of citation but does not gate it. Pages that rank nowhere still get cited.
Despite lower volume, AI-referred traffic is dramatically more valuable:
| Traffic Source | Conversion Rate | Comparison |
|---|---|---|
| AI Search Traffic | 14.2% | Baseline |
| Google Organic | 2.8% | 5x lower |
| AI Referral (Signups) | 1.66% | 11x higher than organic (0.15%) |
Sources: Exposure Ninja, 2026; Ahrefs' 2025 report on its own AI-search traffic conversions
GEO is the practice of optimizing content for AI-powered search engines. The market is experiencing explosive growth:
That's not a hypothetical. Half the sites in our July 2026 sample that block ClaudeBot also publish content marketing aimed at exactly the audiences Claude answers for. Start with a GEO audit that checks crawler access before it checks anything else.
Research reveals surprising insights about what drives AI citations:
The branded-mention finding is the one that reorders most people's priorities, because it inverts the usual advice: the thing that best predicts whether an AI engine cites you is how often the web talks about you, not how many links point at you. That is a fundamentally different game from classic SEO, and we pull the two apart in our comparison of AI-powered and traditional SEO methods.
Both numbers point the same way, and neither says AI content ranks better. They say it ranks at all, in volume. The scarce input is no longer text.
Adoption is effectively settled. What it actually buys you is not. We put the two approaches side by side — cost, speed, conversion rate, and the places where manual work still wins — in AI SEO versus traditional methods.
Adoption and trust are not the same thing, and the gap between them is where the opportunity sits.
People are reading the answer and leaving. They are not, in any volume, following the citations. That means an AI citation is worth chasing for its brand impression, not for the click — and it changes what you should be measuring.
Treat single-vendor case studies as directional, not as benchmarks. A 48% lift in one vertical, reported by the vendor whose product produced it, tells you the mechanism works — not what it will do for you.
The statistics above converge on six moves. Each is expanded — with tooling, content structure and UX specifics — in our guide to optimizing for AI-driven search.
Before anything else, open your robots.txt. In our July 2026 scan, 46% of top sites blocked at least one AI crawler and 54% blocked none — and almost nobody in the second group had made that an active decision either. Both states should be deliberate.
Allowing a crawler is not the same as being served by it, though. The bots take far more than they give back: we break the exchange rate down in crawl-to-refer ratios for AI crawlers, and the split varies by market — see AI bot traffic by country. If you want to publish machine-readable guidance alongside robots.txt, llms.txt is the emerging convention, though adoption is still thin.
Focus on ranking in Google's top 10, as 76% of AI citations come from there. But don't ignore lower-ranking opportunities – a meaningful share of citations come from beyond page one.
Branded web mentions correlate 3x stronger with AI visibility than backlinks. Invest in PR, thought leadership, and earned media.
Use Q&A formats, clear headers, and structured data. AI systems prefer easily parseable content with direct answers.
The deeper shift is from matching keywords to matching meaning: topic clusters and named entities are what a model can actually resolve. See semantic search and contextual content for how to build both.
Monitor referrals from ChatGPT, Perplexity, and other AI platforms. Track citation frequency, not just rankings. Rankings still matter — 76.1% of AI Overview citations come from the top 10 — so keep a rank tracker on the keywords that trigger an AI Overview, and read position and citation together rather than as rival metrics.
AI can generate drafts and research, but human editing ensures brand voice and emotional resonance. The most successful content combines both. If you're putting a model to work on the research half, our library of SEO prompts for ChatGPT covers the tasks it handles reliably — and how to use ChatGPT for SEO is candid about the ones it doesn't.
One more behavioural shift belongs here, because it predates the current wave and is often mistaken for it: spoken queries are conversational, long, and question-shaped — the same shape assistants reward. The groundwork in voice search SEO transfers almost directly.
Our first-party data. The AI-crawler figures in this article come from a single run: on 13 July 2026, SEOmator fetched and parsed the robots.txt of 48 high-traffic sites spanning news, ecommerce, SaaS, SEO, reference, travel, finance, and health. A site counts as blocking a crawler only when that crawler has its own user-agent group carrying a non-empty Disallow. Two sites returned no parseable robots.txt. Sample size is 48 — large enough to show a pattern, too small to project onto the whole web, and we don't. Percentages are rounded to the nearest point. robots.txt states intent, not enforcement: it does not prove a crawler obeyed, and it cannot tell you which bots actually hit your server. Server logs answer that.
Cloudflare Radar data. The crawl-to-refer ratios, AI bot traffic shares, crawl-purpose split, web-scale robots.txt counts, the 107,308-domain agent-readiness scan and the markdown payload-reduction ratio are not ours. They come from Cloudflare Radar and are labelled as such everywhere they appear. What is ours is the analysis: which of these numbers change an SEO decision, in what order, and why. Source: Cloudflare Radar (radar.cloudflare.com) — bots/crawlers, ai/bots, ai/markdown_for_agents: 28-day window ending 14 July 2026; robots_txt scan dated 13 July 2026; agent_readiness scan dated 13 July 2026. Three normalization notes, because they are easy to get wrong: the crawl-to-refer figure is a ratio (pages crawled per one referral returned), not a percentage; the robots.txt figures are raw domain counts, not percentages of the web — Radar does not publish the denominator, so neither do we; and the agent-readiness figures are also raw counts, which we convert to shares only because Radar does publish that denominator (107,308 successfully scanned domains), and we state it every time.
Why we publish someone else's numbers. We could not run a 107,308-domain scan of the live web, and pretending otherwise would be the fastest way to lose the only thing a statistics page is for. Cloudflare can see traffic we cannot. Where they have the better instrument, we cite them and say so. Where we have the better instrument — our own audit and crawl runs — we say that too, and we mark the boundary in both directions.
When our data and Radar's disagree, Radar wins. That happened here. Our 48-site sample put ClaudeBot one site ahead of GPTBot; Radar's far larger robots.txt scan puts GPTBot clearly first. We retracted our ordering rather than defend a one-site gap, and left the correction visible at the top of the crawler section. A statistics article that quietly edits its own numbers is worth less than one that shows its work.
External statistics. Every third-party figure above carries a named source and a date. Where a widely-circulated statistic had no traceable original researcher, we removed it rather than repeat it. Where two credible sources disagree — zero-click rates, ChatGPT's market share, the share of AI citations coming from the top 10 — we've given the range and named the reason they disagree instead of picking the flattering number. Statistics sourced from competing SEO vendors are attributed to them by name; those links are nofollowed.
The data is clear: AI has fundamentally changed SEO in 2026. With 2 billion AI Overview users, 61% CTR drops, and AI traffic converting at 23x higher rates, ignoring AI optimization means leaving significant value on the table.
The single number to carry out of this article is Cloudflare Radar's crawl-to-refer ratio. Google takes about 5 pages for every visitor it sends you. Anthropic takes 2,442. The trade that built the open web — you may read my pages, you will send me readers — has not been renegotiated. It has been quietly voided, and the ratio is the receipt.
The loudest lesson from our own July 2026 scan is quieter still: nearly half of the web's biggest sites have already decided, often by accident, whether AI engines are allowed to read them. Most SEO teams have never checked which side of that line they're on. And when we checked our own headline number against a bigger dataset, it did not survive — GPTBot, not ClaudeBot, leads the web-scale disallow list. Publish the correction; it is cheaper than the alternative.
The winners in this new market will be those who:
Tools like SEOmator's AI SEO Assistant can help you adapt to these changes. The future of SEO is AI-powered – but success still depends on strategic, human-driven execution.
Related Resources:
At web scale, GPTBot is the most-disallowed AI crawler: Cloudflare Radar's 13 July 2026 robots.txt scan records 602 domains disallowing it (458 of them fully), ahead of CCBot (551), ClaudeBot (539), Google-Extended (502), Bytespider (496) and PerplexityBot (308). SEOmator's own smaller scan of 48 high-traffic sites found GPTBot and ClaudeBot effectively tied — 10 and 11 sites, a one-site gap we do not read an ordering into at that sample size — with PerplexityBot, Google-Extended and CCBot also at 21%. Overall, 46% of our 48 sites blocked at least one AI crawler and 54% blocked none.
Per Cloudflare Radar (28-day window ending 14 July 2026), Anthropic crawls 2,441.6 pages for every one referral it sends back, Mistral 1,123.0, OpenAI 260.1 and Perplexity 226.4. Traditional search sits in a different universe: Google is 4.8 : 1 and DuckDuckGo 2.4 : 1. This is a ratio, not a percentage. It means AI assistants take content at hundreds or thousands of times the rate at which they return readers — so plan for citations and brand impressions, not for sessions.
The most critical statistics are: 2 billion monthly AI Overview users, 61% organic CTR drop with AI Overviews, 93% zero-click rate in AI Mode, 23x higher conversion rates from AI traffic, and ChatGPT's 64.5% share of AI chatbot usage (down from 86.7%). Two more that get less attention: Anthropic crawls 2,441.6 pages per referral returned versus Google's 4.8 (Cloudflare Radar), and 46% of the top sites we scanned in July 2026 block at least one AI crawler in robots.txt (SEOmator).
AI Overviews reduce organic CTR by 61% (from 1.76% to 0.61%). Paid search sees an even larger 68% decline. However, being cited in an AI Overview increases your CTR by 35% compared to not being cited, and AI Overview CTR itself recovered from a 1.3% low in December 2025 to 2.4% in February 2026.
Because they measure different things. 93% applies to Google's AI Mode, 43% to Google searches that show an AI Overview, and estimates for all Google searches range from 34% to 68% depending on whether clicks to Google's own properties count and which clickstream panel is used. Any single decimal figure for "the" zero-click rate is overstating the evidence.
AI-referred traffic converts at 14.2% compared to Google organic's 2.8%. AI referral visitors convert to signups at 1.66% vs 0.15% for organic – an 11x advantage. Some B2B SaaS companies report 6x-27x higher conversion rates.
Generative Engine Optimization (GEO) is optimizing content for AI search engines like Google AI Overviews, ChatGPT, and Perplexity. The GEO market will grow from $886 million to $7.3 billion by 2031, with companies reporting 300-500% ROI within 6-12 months. It only works if the engines can crawl you — check robots.txt first.
Focus on ranking in Google's top 10 (76% of citations), build branded web mentions (3x stronger correlation than backlinks), use Q&A content formats, ensure fast page loading, and create deep, well-structured content that AI can easily parse. And confirm you aren't blocking the crawler that would need to read it.
Mistral crawls 3,389 pages for every referral it sends back — now the most extractive AI bot on the web, ahead of Anthropic (2,237:1) and OpenAI (217:1). I cross-checked Cloudflare Radar's global network against SEOmator's own 500+ site panel to rebuild the crawl-to-refer ratio for July 2026.
Nick Sawinyh
50+ ChatGPT prompts for keyword research, content optimization and competitor analysis — grouped by the job each one actually does well, with the outputs you need to verify before acting on them.
Charles Duncan
SEO pros: Master branded web search in the AI era! This ultimate guide covers branded links, anchors, & boosting search volume. Drive conversions now.
Nick Sawinyh
