WordPress SEO for AI: The Settings That Actually Matter

An AI answer that mentions your site passes through four stages. The model's built-in knowledge is frozen at a training cutoff, and nothing you change this quarter reaches it. Access decides whether a crawler may fetch and store your page. Retrieval decides whether your page enters the model's context for a given question. Generation decides which retrieved pages get used and cited.

On WordPress, almost everything you can configure sits in the access stage. That matters because access is where the evidence is strongest. A critical survey of 45 GEO studies (arXiv, July 2026) found that content already retrieved can change how it gets cited, but no technique showed a stable effect on getting discovered in the first place.

The practical upshot: before you install anything labelled "AI SEO", confirm that the bots can reach and read your pages. If you are weighing add-ons, our guide on what to check before installing AI SEO plugins covers the questions to ask.

WordPress serves a virtual robots.txt when no physical file exists in your site root. SEO plugins often edit that virtual file. A physical robots.txt uploaded to the root replaces it. Find out which one your site serves by opening yoursite.com/robots.txt in a browser.

Each AI company runs separate bots for separate jobs. OpenAI's crawler documentation states that each setting is independent of the others. Here is what disallowing each one does, as of October 2026:

There is no robots.txt token that blocks only AI Overviews or AI Mode. Googlebot is the control there.

A site that wants AI search visibility but no training use might serve:

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

One detail catches people out. A bot that matches a named group follows only that group and ignores the User-agent: * rules. Also note that user-triggered fetchers behave differently: OpenAI says robots.txt may not apply to ChatGPT-User, and Perplexity says Perplexity-User generally ignores it.

Google states that a page must be indexed and eligible to show with a snippet to appear as a supporting link in AI Overviews or AI Mode. It adds that there are no additional technical requirements. So the controls that remove snippets also remove you from those answers.

The on-page controls Google and Bing document are noindex, nosnippet, max-snippet:0 and the data-nosnippet attribute. Bing says content marked data-nosnippet is still indexed but excluded from snippets and AI summaries.

On WordPress, these settings hide in a few places:

• Settings > Reading. The "Discourage search engines from indexing this site" box adds a noindex directive. It is often left ticked after a staging site goes live.
• SEO plugin defaults. Most SEO plugins let you noindex post types, taxonomies or archives. Check that pricing, service and product pages are not caught by a blanket rule.
• Per-post toggles. A single page can carry its own noindex or snippet setting in the editor sidebar.
• Theme templates. A developer may have added data-nosnippet to hide a cookie banner and wrapped more than intended.

Google also runs a Search Console control for generative AI features. Inclusion is the default, but child properties can inherit an opt-out from a parent. Our free Search Console AI control check walks through finding it.

robots.txt is a request, not a gate. The real gate is your network layer. OpenAI's help documentation says eligibility for ChatGPT search means allowing OAI-SearchBot in robots.txt and confirming your host or CDN allows traffic from OpenAI's published searchbot IP addresses. Both conditions apply.

On a WordPress site, three layers can block a bot that robots.txt welcomes:

1. Security plugins with bot blocking, rate limiting or country rules. Some ship with lists of "bad bots" that include AI crawlers. See whether WordPress security plugins block AI crawlers for what to look for.
2. Managed hosts that run their own firewall in front of WordPress. You may not see these rules in your dashboard at all.
3. CDNs with bot management. As of September 15, 2026, Cloudflare blocks Training and Agent crawler categories by default on ad-displaying pages for new domains, new sites on existing accounts and existing free-tier customers, while Search stays allowed. A site can change access without anyone editing it.

The symptom is usually a 403 response or a JavaScript challenge served to the bot. Nothing in WordPress reports it. Our playbook on reviewing CDN bot settings for AI crawlers covers where these settings live.

To test it directly, the AI Crawler Checker requests a page as each AI crawler and shows which ones your firewall turns away.

Vercel's crawler study found that ChatGPT and Claude crawlers fetch JavaScript files but do not execute them, so they cannot read client-side rendered content. Googlebot does render JavaScript, so this gap mostly affects non-Google engines.

Classic WordPress themes build pages in PHP on the server, which is good news. The risk comes from what gets layered on top:

• Page builder widgets that load text through JavaScript after the page arrives.
• Tabs, accordions or "load more" sections that fetch content on click.
• Pricing tables or reviews injected by third-party scripts.
• Headless WordPress, where a JavaScript frontend renders the content. Read whether a headless CMS makes your site invisible to AI crawlers if this is your setup.

The test is simple. View the page source (not the inspector) and search for your price, your product names and your key claims. If they are missing from the raw HTML, a non-rendering crawler cannot see them. The fix is to output that content in the server-rendered template.

You control the access stage completely. robots.txt, index and snippet settings, firewall rules and rendering are all publisher-side and checkable.

You also control what your pages state. Putting a specific fact on a fetchable page (a price, a number, a one-line definition) gives an engine a sentence it can quote. That is the only on-page change with a positive experimental signal.

You do not control retrieval or citation. Engines rewrite your buyer's question into their own searches, decide per query whether to search at all, and pick sources by factors they do not publish. When engines cite other sites instead of yours, being named on those sites is associated with appearing in the answer, though no study promises an effect.

So the honest goal for a WordPress site is: no blocks, readable facts, and a clear view of each engine separately. Be wary of anyone who promises more.

Work from the most total failure to the least.

1. Check Settings > Reading. Make sure the discourage-indexing box is clear on the live site.
2. Open your robots.txt. Confirm Googlebot, bingbot, OAI-SearchBot, PerplexityBot, Claude-SearchBot and Meta-WebIndexer are allowed. Decide on training bots (GPTBot, ClaudeBot, Google-Extended) as a separate choice.
3. Test firewall reachability. Request key pages as each AI crawler. Look for 403s and challenges from your security plugin, host or CDN.
4. Audit noindex and snippet rules. Check SEO plugin defaults, per-page toggles and any data-nosnippet in theme files.
5. Check the raw HTML. Confirm prices, product names and key facts appear before JavaScript runs.
6. Fix contradictions. Make sure the same fact reads the same on your pricing, about and footer content.
7. Check each index. Use Search Console for Google and set up Bing Webmaster Tools for Bing, which backs Copilot.

If a fix needs a developer, the developer tickets playbook shows how to write it so it gets done. Once access is clean, the free checker on this site shows whether AI engines name you for up to three buyer questions.

• Installing a blanket "block AI" option. Many plugins and CDN toggles group training and search bots together. Blocking everything removes ChatGPT search eligibility along with training use.
• Blocking GPTBot and assuming you are hidden from ChatGPT. GPTBot governs training only. OAI-SearchBot controls search.
• Treating schema plugins as an AI lever. Google says structured data isn't required for generative AI search. Ahrefs' matched test of 1,885 pages found AI Overviews citations fell 4.6% after adding schema. Existing markup does no harm, but it is not a citation lever. FAQ rich results were deprecated in May 2026.
• Publishing an llms.txt and waiting. Google says Search ignores it, and no other engine documents reading it.
• Rewriting posts in an "AI-friendly" style. Google says you don't need to write in a specific way for generative AI search, and tests found citation-oriented rewrites can hurt.
• Trusting a WordPress setting you never re-tested. Plugin updates and CDN default changes can alter bot access silently. Re-run the access checks after any security or hosting change.