C-SEO Bench: Does Conversational SEO Work? The Study, Digested

C-SEO Bench: Does Conversational SEO Work? is a benchmark paper posted to arXiv as 2506.11097 and accepted to the NeurIPS 2025 Datasets and Benchmarks track. This digest uses version 3, dated 20 October 2025.

It tests conversational SEO (C-SEO): editing a document so a large language model is more likely to rank it highly or cite it when answering a question. The question is narrow on purpose. When a document is already among the sources a model has been handed, do C-SEO rewrites help it?

The paper follows the original GEO paper, which reported a 30 to 40% band across three methods, measured in an engine its authors built with sources supplied in context. C-SEO Bench is a stricter test of the same family of tactics.

The setup is a simulated answer pipeline, not a live search engine.

• A set of candidate documents goes into the model's context window.
• One target document is modified by a C-SEO method, such as a citation-oriented rewrite.
• The researchers measure how the target ranks in the model's output against its unmodified baseline.
• The tests ran across GPT-4o-mini, Claude-3.5-Haiku, o3 and o4-mini.

The comparison arm the paper calls traditional SEO did not involve earning a ranking. It was implemented by moving the target document to position 1 in the context. No index was queried. Think of it as a positional oracle: the researchers placed the document first by hand, which no publisher can do in a real engine.

The benchmark also varied how many documents in the pool used C-SEO at once. That models a market where your competitors run the same playbook.

The headline finding, in the authors' words: "Our experiments reveal that most current C-SEO methods are not only largely ineffective but also frequently have a negative impact on document ranking, which is opposite to what is expected."

On position, they wrote: "Instead, traditional SEO strategies, those aiming to improve the ranking of the source in the LLM context, are significantly more effective." Given the method, read this as evidence of position bias: being placed first in the context matters more than how the document is written.

On competition: "We also observe that as we increase the number of C-SEO adopters, the overall gains decrease, depicting a congested and zero-sum nature of the problem." Any edge a tactic gives shrinks once rivals copy it. The page on whether anyone wins when every competitor does GEO covers that dynamic.

These quotes are the paper's own summary statements. Per-method effect sizes are not reproduced here.

The study is strong on one point and silent on several others.

It is not evidence that classic SEO produces AI citations

The winning arm moved a document to slot 1 by hand. Nobody earned that slot through rankings, links or content. The result says position inside the context matters. It says nothing about how a real engine fills that context.

It is a within-context result

Every document started inside the model's context. The benchmark does not test whether anything helps a page enter the candidate set in the first place. That entry step is what most businesses care about.

It does not prove content quality is irrelevant

A null for rewriting tactics is not a null for having a relevant, accurate page.

The models are from 2025

As of October 2026, no replication exists on GPT-5.6, Claude Opus 5 or Gemini 3.

Corroboration points the same way

The 252,000-trial GEO factorial found that "formatting-only edits have little impact" and that topical relevance and list position drive being cited first. It also never called a search engine and used two candidates per trial, so it likely overstates position effects. The critical survey of 45 GEO studies warns that "citation-oriented rewrites can impair retrieval" and concludes that no reviewed technique shows a stable, longitudinal, cross-platform causal effect on organic discoverability.

Stop paying for citation-oriented rewrites. The best-designed test of these tactics found them ineffective or harmful, and any gain erodes as competitors adopt them. If an agency pitches rewrites with a promised uplift, ask which study measured that on pages outside the model's context.

Then spend the effort where the mechanism is documented:

1. Check rendering. Vercel found that ChatGPT and Claude crawlers fetch JavaScript files but "don't execute them. They can't read client-side rendered content." Put prices, product names and key claims in the raw HTML.
2. Check robots.txt per bot. OAI-SearchBot controls ChatGPT search. GPTBot controls training only.
3. Check the firewall. A CDN or WAF can return 403 to a crawler that robots.txt allows. The AI Crawler Checker requests a page as each AI crawler and shows which ones get turned away.
4. Check snippet controls. A stray nosnippet, max-snippet:0 or data-nosnippet keeps content out of AI answers.
5. State specific facts plainly. A price, a number or a one-line definition on a fetchable page gives an engine a sentence it can quote.

If earlier rewrites added appended quotation blocks, injected statistics or "according to authoritative sources" boilerplate, treat them as a risk, not an asset. The question can rewriting content for AI make my visibility worse goes deeper. None of these steps guarantees a citation. They remove the blocks you control.

Does C-SEO Bench prove that ranking well in Google gets you cited by AI? No. The traditional SEO arm moved the target document to position 1 in the model's context by hand. No search index was queried and no ranking was earned. The study shows that position inside the context matters a great deal. It does not show that organic rankings decide which documents reach that context in a live engine.

Is C-SEO the same thing as GEO? They overlap. Both describe changing a document's content so a generative engine ranks or cites it more. C-SEO Bench tested several such methods, including rewriting approaches that descend from the original GEO paper. Its verdict applies to that family of tactics when measured on documents already inside the model's context.

Why did gains shrink as more documents used C-SEO? The authors describe the problem as congested and zero-sum. A model has limited room to rank and cite sources. If one document gains by a tactic, another loses. When every document in the pool applies the same tactic, the relative advantage disappears and the overall gain falls.

Should I remove rewrites I already published? No study in the evidence measured the effect of reverting them. C-SEO Bench found these tactics often hurt ranking inside the context, so treat obvious artifacts as a risk worth reviewing. Prioritise the access checks first, because rendering, robots.txt and firewall problems can remove a page entirely, regardless of how it is written.