AI search visibility for CMOs: eight questions to ask your team, and what good answers sound like
AI search visibility is how often AI answers mention and link to your brand. CMOs track share of answers, citation rate, AI referrals and pipeline.
Artificial intelligence (AI) search visibility is how often, and how accurately, your brand and pages appear inside AI-generated answers, and whether those answers link back to you. For a chief marketing officer (CMO), it sits next to share of search. Cloudflare reported in August 2025 that training drove nearly 80% of AI bot activity, and Google referrals to its news-related customers fell 15% from January to April 2025.
What AI search visibility means for a CMO, and the figures to know first
AI search visibility is how often, and how accurately, your brand, products and pages appear inside AI-generated answers, and whether those answers link back to you. For a chief marketing officer (CMO), it sits next to share of search as a measure of demand you can see before it reaches the funnel.
Googlebot held 39% of AI and search crawler traffic in July 2025, against 11.7% for GPTBot and 2.4% for Bytespider.
Share of AI and search crawler traffic, July 2025
| Label | Value |
|---|---|
| Googlebot | 39% |
| GPTBot | 11.7% |
| Meta crawler | 7.5% |
| Amazonbot | 5.9% |
| Bytespider | 2.4% |
The terms your team will use
| Term | Plain meaning |
|---|---|
| AI Overviews | Google's AI summaries in search results. Google says they help people get to the gist of a complicated topic and link out to sources. |
| AI Mode | Google's conversational search, particularly helpful for queries that need exploration, reasoning or comparison. |
| Generative engine optimization (GEO) | Our working term for the practice of making your content eligible, retrievable and citable in AI answers. |
One point to settle with your team early: Google states that to appear as a supporting link, a page must be indexed and eligible to be shown with a snippet, with no additional technical requirements.
Google for Developers, AI Features and Your Website, 10 December 2025
The figures to know first
Cloudflare's August 2025 analysis gave the scale of the shift:
- Training drove nearly 80% of AI bot activity, up from 72% a year earlier.
- In Cloudflare's dataset of news-related customers, Google referrals in March 2025 were down about 9% compared to January, and April was down 15% compared with January.
- Cloudflare tied the timing to Google's product changes: AI Overviews launched in the U.S. in May 2024, and AI Mode entered Labs in March 2025.
Cloudflare Blog, The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals, 29 August 2025
Those referral figures describe news sites, not your category. In our reading, the useful takeaway for a board is directional: more of your content is being read by machines, and fewer readers may follow a link back.
A first check, as a worked example
Take an example, not a benchmark: your team lists 50 priority pages and finds 8 that are not indexed or are blocked from showing a snippet. That leaves 42 pages eligible for AI features, and the 8 failures are a fix list with a clear owner. Run the same count each quarter and report it next to brand tracking results.
How Google's AI Overviews and AI Mode choose and link sources
Google's documentation says its generative AI features on Google Search are rooted in core Search ranking and quality systems. We think the grounding step covered earlier draws on that same index. The pages an AI answer links to come from rankings your search engine optimization (SEO) team already works on, so standard SEO practice still applies.
Google for Developers, Google's Guide to Optimizing for Generative AI Features on Google Search
Query fan-out, and what it does to your keyword list
The second mechanism is query fan-out, which Google defines as a set of concurrent, related queries generated by the model. In Google's example, the query "how to fix a lawn that's full of weeds" fans out to "best herbicides for lawns", "remove weeds without chemicals" and "how to prevent weeds in lawn".
Google for Developers, Google's Guide to Optimizing for Generative AI Features on Google Search
A page can therefore be linked for a subquery the user never typed. Here is an example, not a benchmark. Your team picks 8 priority buying questions and writes down 20 likely fan-out queries for each, so 8 questions with 20 queries each gives 160 queries to check by hand for which pages appear.
Use that list to find gaps in useful content. Do not use it as a page-production plan. Google says that creating separate content for fan-out variations primarily to manipulate rankings or generative AI responses in Google Search violates Google's scaled content abuse spam policy.
Google for Developers, Google's Guide to Optimizing for Generative AI Features on Google Search
Google also claims that clicks from results pages with AI Overviews are higher quality (meaning, users are more likely to spend more time on the site). That is Google's own finding. Check it against time on site in your analytics before you repeat it to the board.
Google for Developers, AI Features and Your Website, 10 December 2025
It also said that the number of people using its Preferred Sources feature has grown by more than 7x since the start of the year.
Google, More ways to connect with the web from Search, 15 October 2026
| Change Google announced | What to ask your team (our reading) |
|---|---|
| Inline links shown higher up in responses more often | Which of our pages earn inline links, and for which questions |
| Test of a dedicated top section for official pages, with context on why each link is useful | Whether our official task pages (pricing, support, sign-up) are clear enough to qualify |
| Related links at the end of some responses, now with image thumbnails | Whether key pages carry images that represent them well |
| Test of clickable creator and publisher names on video results in AI Mode | Whether our video publisher profile is set up |
In our reading, each of these changes rewards pages that Google can already rank and identify as the official source. That makes core SEO hygiene the first item on the CMO's checklist.
Which AI crawlers fetch for search and which collect training data
The split that matters for a CMO is simple. Some crawlers decide whether your pages can appear in an AI answer. Others collect content to train models. A third group fetches a page only when a user asks a question. Your team can block training without dropping out of search, but only if each token is set on purpose.
OpenAI states this directly: a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot. The cost of getting it wrong is visible. Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links.
OpenAI Developers, Overview of OpenAI Crawlers
Google's common crawlers always obey robots.txt rules when crawling automatically. User-triggered fetchers are different. For ChatGPT-User, robots.txt rules may not apply, and Perplexity-User generally ignores robots.txt rules.
Google for Developers, Google's common crawlers, 15 October 2026
OpenAI Developers, Overview of OpenAI Crawlers
Perplexity, Perplexity Crawlers
| Platform | User-agent token | What it does | Follows robots.txt | Time for changes to apply |
|---|---|---|---|---|
| Googlebot | Search inclusion, including all Google Search features | Yes, when crawling automatically | Not stated | |
| Googlebot-Image | Images, logos and favicons in Search features | Yes, when crawling automatically | Not stated | |
| Google-InspectionTool | Testing tools only, no effect on Search | Yes, when crawling automatically | Not stated | |
| OpenAI | OAI-SearchBot | Search inclusion in ChatGPT | Yes | ~24 hours |
| OpenAI | GPTBot | Model training | Yes | Not stated |
| OpenAI | ChatGPT-User | User-triggered fetch | May not apply | Not applicable |
| Perplexity | PerplexityBot | Search inclusion, not model training | Yes | Up to 24 hours |
| Perplexity | Perplexity-User | User-triggered fetch | Generally ignores | Not applicable |
A robots.txt template to hand your team
This is our starting point for a company that wants AI search presence but no training use. Adjust paths to your site.
User-agent: Googlebot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: GPTBot
Disallow: /
Two checks complete the job. First, confirm the firewall agrees with the file. Perplexity notes that with a Web Application Firewall (WAF) you may need to explicitly whitelist Perplexity’s bots to ensure they can access your content. We think a security rule that blocks a search crawler undoes the robots.txt work silently.
Perplexity, Perplexity Crawlers
Second, after any change, wait out the stated window before measuring. OpenAI says it can take ~24 hours from a site’s robots.txt update for our systems to adjust. A good answer from your team names each token, says which ones are allowed and why, and gives the date the file last changed.
OpenAI Developers, Overview of OpenAI Crawlers
Should we block AI crawlers like GPTBot
Treat crawler access as a governance decision with an owner, a written policy and a review date. Leaving it as a setting someone in IT changed once is how it goes wrong. Blocking bundles two separate choices: whether your content trains AI models, and whether it appears in AI answers. The previous section showed that OpenAI lets you make those choices independently and that changes take about a day to apply. Two further details matter for the policy.
First, allowing both OpenAI bots does not double the load. OpenAI says that when a site allows both, it "may use the results from just one crawl for both use cases". Second, if you run ads on ChatGPT, OpenAI states that OAI-AdsBot only visits pages submitted as ads and its data is not used to train foundation models.
OpenAI Developers, Overview of OpenAI Crawlers
Robots.txt is a request, not a lock
In response, it de-listed Perplexity as a verified bot. In the same test, Cloudflare found that ChatGPT-User fetched the robots file and stopped crawling when it was disallowed.
Cloudflare Blog, Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives, 4 August 2025
The practical point: a robots.txt line records your intent. Enforcement happens at the network or firewall layer, and that is a security budget conversation as much as a marketing one.
The trade-off to put in front of the CEO
Cloudflare reported in August 2025 that training drove nearly 80% of AI bot activity, up from 72% a year earlier. It also measured Perplexity at 194 crawls per visitor in July. In our reading, blocking training crawlers protects content but sends no visitors either way. Blocking search crawlers can remove you from answers your buyers read. In our reading, most companies selling to buyers who research in AI tools will allow search bots and decide training access on content value.
Cloudflare Blog, The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals, 29 August 2025
| Bot type | What it does | Default to discuss (our reading) |
|---|---|---|
| Search crawler (OAI-SearchBot, PerplexityBot) | Surfaces pages in AI search answers | Allow |
| Training crawler (GPTBot) | Collects content for model training | Decide by content type |
| User-triggered fetcher (ChatGPT-User, Perplexity-User) | Fetches a page when a user asks | Allow; robots.txt may not apply |
A one-page policy should name the owner (marketing, with security and legal consulted), list each bot and its status, state the reason in one sentence, and set a quarterly review. Bring it to the CEO and finance as part of the CMO, CEO and CFO alignment conversation, since the choice affects both content risk and pipeline.
Do AI Overviews reduce website traffic
The two sides measure different things, so a CMO should read them as two partial views rather than one verdict.
Cloudflare looked at publishers. In its dataset of news-related customers, Google referrals had been declining since February 2025. March 2025 referral traffic from Google was 9% lower than January, and April was down 15% compared with January. Cloudflare said the drop seems to coincide with some of Google's changes, which is a timing observation. It did not establish cause. The crawl-to-referral gap also stayed wide: Anthropic was at 38,000 crawls per visitor in July, down from January.
Cloudflare Blog, The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals, 29 August 2025
Google described visits, not volume. It said that with AI Overviews, people have been visiting a greater diversity of websites, and it described continued and substantial increases to inline links in its AI experiences. Google also noted that AI Overviews often don't trigger.
Google for Developers, AI Features and Your Website, 10 December 2025
Google, More ways to connect with the web from Search, 15 October 2026
| Claim | Who made it | Scope | What it means for you (our reading) |
|---|---|---|---|
| Google referrals to news sites fell | Cloudflare | News-related customers | Relevant if you publish timely content |
| People visit a wider range of sites | All AI Overviews users | Wider spread can still mean fewer clicks per site | |
| More inline links in AI responses | AI Mode and AI Overviews | More chances to be linked, not a guarantee of clicks |
In our reading, the two claims can both hold: more sites can get some traffic while any one site gets less.
How to read your own referral data
- Know what Search Console blends. Sites appearing in AI features are included in the overall search traffic in Search Console. You cannot separate AI Overview clicks there, so watch impressions and clicks together by query.
- Segment by query type. Have the team tag your top landing pages by whether an AI Overview shows for their main query.
- Size the change in money. Translate sessions into pipeline before you report it to the board (see marketing-sourced pipeline).
Google for Developers, AI Features and Your Website, 10 December 2025
A worked example, not a benchmark: say you get 50,000 Google sessions a month and they fall 8 percent, which is 4,000 fewer sessions. If 20 of your 50 top pages show an AI Overview, 40 percent of that set is exposed. Check whether the loss concentrates there. If it does, we think AI Overviews are a likely factor. If it spreads evenly, look at seasonality, rankings or demand first, and pair the analysis with share of search to see whether demand itself moved.
The eight questions to ask your team, with good and weak answers
Copy the table below into your next marketing review and ask each question as written. Pay attention to how specific each answer is and how quickly it comes. A team that has done the work names pages, dates and owners. A team that has not talks in generalities.
Three facts from Google sharpen the first two questions. A page must be indexed and eligible to show with a snippet to appear as a supporting link, and Google says there are no additional technical requirements. Its checklist extends crawl access past robots.txt to any CDN or hosting infrastructure. Its optimization guide adds that a site must be included in Search generative AI features in Search Console to be eligible.
Google for Developers, AI Features and Your Website, 10 December 2025
Google for Developers, Google's Guide to Optimizing for Generative AI Features on Google Search
| Question | Good answer (our reading) | Weak answer (our reading) |
|---|---|---|
| 1. Crawler policy: which AI crawlers do we allow or block, and who decided? | A written rule per crawler that separates search fetchers from training collectors, with a review date | We block all bots or IT handles that |
| 2. Indexing health: are priority pages indexed, snippet-eligible and reachable past our CDN and firewall? | Named pages checked in Search Console, CDN rules tested, eligibility setting confirmed | An answer about robots.txt alone |
| 3. Prompt set: which questions do we test, and where did they come from? | A fixed list drawn from sales calls and search data, grouped by buying stage, with related variants | Whatever prompts someone typed that week |
| 4. Sampling cadence: how often do we rerun the set, and how is it logged? | Same prompts on a schedule, each assistant, results stored with dates | Screenshots when someone remembers |
| 5. Citation sources: when we are absent, who is cited instead? | Named third-party domains, with a plan for each | We're not showing up, with no names |
| 6. Referral tracking: how do AI visits show up in analytics? | Referrer segments per assistant, plus engagement and conversions | Total organic traffic only |
| 7. Link to pipeline: what revenue do these visits touch? | AI-referred sessions tied to marketing-sourced pipeline in the customer relationship management (CRM) system | A visibility score with no revenue line |
| 8. Ownership: who owns this number? | One named owner drawing on SEO, content and web engineering | Everyone, or the agency |
For question 3, the prompt set should mirror how assistants search. Google describes query fan-out as a set of concurrent, related queries generated by the model to request more information, so test the follow-ups a buyer would ask. Do not answer it by publishing a page per variant: Google says doing that primarily to manipulate responses violates Google's scaled content abuse spam policy.
Google for Developers, Google's Guide to Optimizing for Generative AI Features on Google Search
For question 4, keep the scoring simple. As an example, not a benchmark: if your brand is cited in 20 of 50 prompts, your citation rate is 40%. Rerun the same set 8 times before reading a trend. Fixes take time to register, since Google says crawling can take anywhere from several days to several months.
Google for Developers, AI Features and Your Website, 10 December 2025
For question 6, Google reports that clicks from AI Overview pages are higher quality (meaning, users are more likely to spend more time on the site), so track engagement alongside volume. Put the answer to question 7 on your marketing scorecard, and settle question 8 in your org structure.
Google for Developers, AI Features and Your Website, 10 December 2025
How to measure AI search visibility and connect it to pipeline
Monitoring is now common. Proof is not. In our reading, per one AI search survey, 73% have invested in tools to monitor AI visibility, yet those tools estimate how a model presents a brand without linking that view to sales. A board needs both halves, so report four signals and say plainly which ones are proxies.
Digiday, CMOs are struggling to link AI visibility with sales, 13 August 2026
The four metrics to report monthly
| Metric | How to calculate it | What it tells the board (our reading) |
|---|---|---|
| Share of answers | Answers that mention you, divided by prompts in a fixed set | Whether buyers see you at the research stage |
| Citation rate | Answers that link to your pages, divided by prompts in the set | Whether your content is the source, not just the topic |
| AI referral sessions | Sessions from AI assistants, tracked as their own channel | Whether visibility produces visits |
| Branded search trend | Branded query volume over time, alongside share of search | Whether demand reaches you by other routes |
Freeze the prompt set for at least a quarter. If the team rewrites prompts each month, the trend line measures the rewrite.
A worked example, not a benchmark: take a fixed set of 50 buyer prompts. If you appear in 20 of the 50 answers, share of answers is 40%. If 8 of the 50 answers cite your pages, citation rate is 16%. The gap between those two figures is the work list for content.
The traffic line is growing fast enough to matter.
Connecting the signals to pipeline
No single report closes the loop.
A practical sequence:
- Feed AI referral sessions and branded search into your marketing mix model as inputs, not outcomes.
- Run a geographic or time-based incrementality test when you make a large content change, and compare branded search and marketing-sourced pipeline against a holdout.
- Report the result as a range, with the gap stated. Purchases that start in an AI answer and finish on a marketplace will not show up in your own data.
Put all four metrics on one slide of the board deck, with the model's range beside them.
A worked example and the decisions only the CMO can take
Here is our method applied to an example business-to-business company. The inputs below are round example figures, not benchmarks.
The worked numbers
The team builds a fixed set of 50 buyer prompts and runs it every month across the AI assistants its buyers use. In the first run, the company's pages are cited in 8 of 50 answers, a cited-answer share of 16%.
The team then rewrites the pages behind the weakest prompts. Each page gets a plain definition near the top, a comparison table and named, dated proof. Three months later the same set returns citations in 20 of 50 answers, or 40%.
| Run | Prompts in set | Answers citing us | Cited-answer share |
|---|---|---|---|
| Before the content fix | 50 | 8 | 16% |
| After the content fix | 50 | 20 | 40% |
That is 12 more cited answers (20 minus 8) on an unchanged set. The prompt set has to stay fixed. If the prompts change, the trend line tells you nothing.
On the revenue side, the site records 50 AI referral sessions in a month, and 8 of them become qualified leads, a 16% session-to-lead rate. We would report that rate next to marketing-sourced pipeline. We would treat it as directional until an incrementality test supports it.
The three decisions only the CMO can take
- Crawler access. Separate search access from training access. OpenAI says that each setting is independent, so a site can allow OAI-SearchBot and still disallow GPTBot. Sites opted out of OAI-SearchBot the documentation says "will not be shown in ChatGPT search answers." Ask the web team for the current robots.txt file. Then sign a one-page policy that lists each crawler, the decision and the owner. OpenAI notes changes can take ~24 hours to register for search, so schedule a check the next day.
- Content investment. Fund rewrites only for the prompts where your cited share is lowest and the deal value is highest. Set a stop rule before work starts. In our reading, if a quarter of rewrites does not move the share on the fixed set, the money belongs elsewhere.
- The measurement budget. Fund the monthly prompt run and referral tagging as a named line inside martech, not as a side project. Agree the reporting format with finance in advance, using the approach in our CMO, CEO and CFO alignment guide. The board then sees one cited-share trend and one pipeline figure each quarter.
OpenAI Developers, Overview of OpenAI Crawlers
What this page does not cover
This page does not compare monitoring tools or vendors, cover AI assistants beyond Google, OpenAI and Perplexity, or give traffic benchmarks for your category. It is not legal advice on crawler policy or content use, so involve your legal team when setting the policy.
Related on CMO Tribune: Marketing mix modeling, explained for the CMO who has to choose a measurement model; In-house vs agency cost, compared with your own numbers; Marketing team structure by revenue band: functional, pods, hub-and-spoke and business-unit models.
Questions
How do I know if my brand shows up in AI search?
Build a fixed set of buyer prompts, for example 50, drawn from sales calls and search data. Rerun the same set on a schedule in each assistant your buyers use, and store the results with dates. Count how many answers mention you and how many link to your pages. Also confirm that your priority pages are indexed and eligible to show with a snippet, since Google requires that for a page to appear as a supporting link.
Do AI Overviews reduce website traffic?
Among Cloudflare's news-related customers, Google referrals in March 2025 were 9% lower than in January, and April was 15% lower. Cloudflare said the drop seemed to coincide with Google's product changes, which is a timing observation, not proof of cause. Google said people visited a greater diversity of websites with AI Overviews, so more sites can get some traffic while any one site gets less. Check your own data by tagging top landing pages by whether an AI Overview shows for their main query, and see whether any loss concentrates there.
Should we block AI crawlers like GPTBot?
Treat it as a governance decision with an owner, a one-page written policy and a quarterly review. Blocking bundles two choices: whether your content trains models and whether it appears in AI answers. OpenAI lets a site allow OAI-SearchBot for search while disallowing GPTBot for training. A robots.txt line records intent, but enforcement happens at the network or firewall layer.
What is the difference between GPTBot and OAI-SearchBot?
GPTBot collects content for model training. OAI-SearchBot decides inclusion in ChatGPT search, and sites opted out of it will not be shown in ChatGPT search answers, though they can still appear as navigational links. OpenAI says each setting is independent and that robots.txt changes can take about 24 hours to apply. When a site allows both, OpenAI may use the results from one crawl for both uses.
What does Google say about optimizing for AI Overviews?
Google says its generative AI features in Search are rooted in core Search ranking and quality systems, so standard search engine optimization (SEO) practice still applies. A page must be indexed and eligible to show with a snippet, with no additional technical requirements. Google describes query fan-out, where the model runs related queries, and warns that creating separate content for those variations primarily to manipulate rankings or AI responses violates its scaled content abuse spam policy.
What should a CMO measure for AI search visibility?
Report four metrics monthly: share of answers, citation rate, AI referral sessions tracked as their own channel, and branded search trend. Freeze the prompt set for at least a quarter so the trend measures visibility, not prompt rewrites. Connect the signals to pipeline by feeding them into a media mix model and running incrementality tests against a holdout, then report the result as a range with the gap stated.
Sources
- AI Features and Your Website, Google for Developers, 10 December 2025
- Google's Guide to Optimizing for Generative AI Features on Google Search, Google for Developers
- The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals, Cloudflare Blog, 29 August 2025
- More ways to connect with the web from Search, Google, 15 October 2026
- Overview of OpenAI Crawlers, OpenAI Developers
- Google's common crawlers, Google for Developers, 15 October 2026
- Perplexity Crawlers, Perplexity
- Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives, Cloudflare Blog, 4 August 2025
- CMOs are struggling to link AI visibility with sales, Digiday, 13 August 2026
Written by the CMO Tribune Desk and edited by Lila Odinaev. Every factual sentence was checked against a passage on the source it cites. See how we work, or report an error on the corrections page.