Which version of ChatGPT should an AI visibility tool test?

The right ChatGPT to test is the one your buyers use, with the retrieval behavior they get. That means two settings, not one: the model and whether web search is on.

Most tools test ChatGPT through OpenAI's API. That is reasonable, because the consumer app has no measurement interface. But the API and the app are not the same product. The difference between API answers and consumer app answers comes down to three things:

• Plan. Free and paid plans are reported to serve different models.
• Route. ChatGPT moves questions between model paths with different knowledge cutoffs.
• Retrieval. The app decides per question whether to search. An API call does whatever the tool configured.

So the practical answer is this. Pick the model that matches your buyers' most likely plan. Test with search off to see what the model knows. Test with search on to see what it finds. Report the two separately. A tool that blends them into one number is telling you less than it seems.

"ChatGPT" is a product name covering several models. Secondary reports gathered in September 2026 say Free and Go users get gpt-5.6-luna, Plus users get gpt-5.6-sol, and Pro, Business and Enterprise users get GPT-6 Pro. OpenAI has not confirmed that mapping itself, so treat it as reported, not documented. If you want the detail, see whether ChatGPT answers differently on free and paid plans.

What OpenAI does document is the knowledge cutoff for each path:

That gap is about five and a half months. OpenAI describes chat-latest as the latest Instant model currently used in ChatGPT. So depending on how a question is routed, the same user can get an answer from a model that knows your 2026 launch or one that does not. Our page on model routing explains why this happens.

What it means for you: if your pricing, name or product changed between those two dates, a tool testing only one path can miss a real error. Ask which path the tool calls.

In the ChatGPT app, search is model-discretionary. The model decides, question by question, whether to look something up. When it does, OpenAI says ChatGPT typically rewrites the question into one or more targeted queries before sending them to its search providers. Nobody outside OpenAI sees those rewritten queries.

An API test works differently. The tool decides whether to enable web search. Many tools call the model with search off, which measures only parametric knowledge: what the model learned before its cutoff. That is a valid measurement. It is not what most app users see when the app chooses to search.

How often does the app search? Published estimates disagree sharply. Profound reports about 18% of conversations trigger a web search, Semrush's US clickstream put it at 34.5% of queries in February 2026, and cloro reports 80.5% globally on its own prompt set. They measure different things on different populations. None tells you how often ChatGPT searches for questions about your brand.

So the honest approach is to test with web search on and off and keep the results apart. You can try the difference yourself with the free search on vs off test.

Some things stay out of reach no matter which model a tool picks.

• Logged-in context. OpenAI says ChatGPT may use saved memories when rewriting a search query. A logged-out, memory-off test is reproducible. A real user's answer may not be.
• The real queries. Because ChatGPT rewrites prompts, any tracker's prompt list is a proxy for an input it cannot observe.
• Retrieval state of a past answer. If no citations appear, you cannot be sure whether the answer came from search or from memory.
• Model changes. OpenAI warns its model snapshots are regularly updated. A score from last quarter may have come from a different model.

The same problem exists outside ChatGPT. Google says AI Overviews and AI Mode may use different models and techniques, and the Gemini API generates its own search queries. Tools that track several models at once face this on every platform.

What it means: a ChatGPT score is a sample under stated conditions. It is useful when the conditions are stated. It misleads when they are hidden.

Use these questions in any demo or trial. A good tool answers each one in a sentence.

1. Which model do you call, by exact name? "ChatGPT" is not an answer. You want something like gpt-5.6-luna.
2. Which ChatGPT plan does that match? If your buyers are mostly on free plans, a test on a paid-tier model may not reflect them.
3. Is web search on or off? If on, does the tool record which URLs were retrieved as well as which were cited? OpenAI's API returns the two lists separately.
4. Do you report search-on and search-off results separately? A blended score hides the cutoff problem.
5. How do you flag model changes? Scores before and after a model swap are not directly comparable.
6. Do you test logged out with memory off? Otherwise results are not reproducible.

If a vendor cannot answer the first two, read whether tools test the same models people use in ChatGPT before signing. For a quick snapshot, the free checker on this site shows whether AI engines name you for up to three buyer questions. It is a starting point, not continuous tracking.