Why does ChatGPT give different answers about my brand each time?

Two runs of the same prompt are rarely the same request under the hood. Five variables move, and any one of them can change what ChatGPT says about you. This is what answer variability means in practice.

• Model routing. Your plan and the type of request decide which model answers. Secondary reports as of September 2026 say Free and Go users get gpt-5.6-luna, Plus gets gpt-5.6-sol, and higher tiers get a GPT-6 model. OpenAI has not confirmed that mapping itself.
• Training cutoff. Per OpenAI's model docs, the GPT-5.6 family has a February 16, 2026 knowledge cutoff. The Instant path (chat-latest) has an August 31, 2025 cutoff. A rebrand or price change between those dates exists in one and not the other.
• Search on or off. ChatGPT decides per question whether to search the web. A search-on answer can cite your current site. A search-off answer comes only from the training snapshot.
• Query rewriting. OpenAI's help page says ChatGPT search "typically rewrites your query into one or more targeted queries" before sending them to search providers. Different rewrites retrieve different pages.
• Memory and sampling. OpenAI says ChatGPT may use saved memories when rewriting a search query. On top of that, language models generate text with some randomness, so wording and which brands get named can shift even when everything else holds still.

The result: an answer that named you yesterday and skipped you today may not mean anything changed about your brand.

Retrieval state is the biggest hidden variable, and nobody outside OpenAI sees it directly. Published estimates of how often ChatGPT searches disagree widely. Profound reports about 18% of conversations trigger a web search. Semrush's US clickstream data put it at 34.5% of queries in February 2026. Cloro found 80.5% globally on a 100-prompt set. They measure different things on different populations, so treat the spread as the finding.

This matters because the two modes fail differently:

• Search off: the answer reflects training data. Nothing you publish this quarter reaches it. Errors here are often outdated facts, not inventions. Our guide on hallucination vs outdated answers shows how to tell them apart.
• Search on: the answer depends on which pages ChatGPT retrieved and cited. OpenAI notes the number of sources it consulted is often greater than the number it cites, so being retrieved and being named are different events.

The practical tell: visible citations or source links mean search ran. No citations means you probably saw a trained-knowledge answer.

Your goal is to hold every variable you can fixed, then measure the ones you can't across repeated runs. The full method is in our reproducible prompt testing protocol. The core steps:

1. Log out, or use a clean account with memory and custom instructions off. Logged-in history leaks into answers. See logged-in vs logged-out testing for what changes.
2. Start a new chat for every run. Earlier turns shape later ones.
3. Fix the wording. Write each buyer question once and paste it exactly.
4. Run each prompt several times, for example five, and count how often you're named. A rate is useful. One answer is not.
5. Log the retrieval state for every run: citations shown or not, and which URLs.
6. Record the date, plan and any model label shown. A new model release can shift answers overnight.
7. Note specific wrong facts, such as price, founding year or product names, not just whether you appeared.

For a quick snapshot before building your own log, the free checker on this site runs up to three buyer questions and scores the results. Treat it as one sample, not continuous tracking.

A few patterns produce misleading conclusions:

• API tests are not app tests. The API often runs with search off and on a single pinned model. The consumer app routes and searches at its own discretion. This is one reason AI visibility tools disagree with each other and with what you see.
• Your phrasing isn't the search query. Because ChatGPT rewrites prompts, tracking "the prompts customers type" measures a proxy for an input you can't observe.
• Cited is not the same as used. A source can shape an answer without being cited, and a citation can appear for a page that barely contributed.
• Averages hide modes. A blended score mixes search-on and search-off answers. Split them before drawing conclusions.

You can't control sampling, routing or the training snapshot. You can control whether search-on answers can reach your facts.

• Allow OAI-SearchBot in robots.txt. OpenAI's crawler docs say sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers. GPTBot is a separate setting that governs training only.
• Check your CDN or firewall. OpenAI says eligibility also requires your host to allow its published search bot IP addresses. A 403 here fails silently.
• Put key facts in the server HTML. Vercel's crawler study found ChatGPT's crawlers fetch JavaScript files but don't execute them, so client-rendered prices and product names may be invisible.
• State each fact once, the same way, everywhere. Contradictions on your own site give the engine conflicting sentences to quote.

If trained-knowledge answers are wrong, see fixing ChatGPT errors for what each mode allows.