How to Build a Prompt Set for AI Visibility Tracking

A prompt set is the fixed list of questions you put to AI engines to see whether, and how, they mention you. It is useful. It is also a proxy, and you should build it knowing that.

The engines do not search for your prompt as written. OpenAI's help page on ChatGPT search says it typically rewrites the question into one or more targeted queries before sending them to search providers. Google says AI Overviews and AI Mode may use different models and techniques, and its Gemini API documentation says the model generates and runs its own search queries. So your prompt goes in, and a different, unobserved set of queries does the retrieving.

Two more things sit outside your control:

• Retrieval is discretionary. As of October 2026, ChatGPT decides per question whether to search, and Claude.ai search is a user toggle. Some answers come from the model's training data alone.
• Personal context changes the rewrite. OpenAI notes that saved memories can shape how ChatGPT rewrites a search query.

What it means: a prompt set measures a sample, not the whole. In Otterly's 2026 experiment, Microsoft's own AI Performance report counted 42,817 citations for one test site, while Otterly's prompt tracker logged 3,934 (the time windows may not match). The next move is to build the sample carefully and keep it stable, so changes over time mean something.

Work through these in order. Write everything down, because the documentation is what makes later runs comparable.

1. Name the decision. Write one sentence on what the set is for, such as "see whether AI engines recommend us when buyers compare payroll tools." Prompts that do not serve that sentence stay out.
2. Collect real questions. Pull wording from sales calls, support tickets, site search logs and the questions prospects send by email. Real phrasing beats brainstormed phrasing.
3. Group them into topics. One topic is one underlying need, for example "payroll for contractors." Several prompts can share a topic.
4. Tag each topic by funnel stage. Problem, category, comparison, and branded fact checks (see the next section).
5. Write wording variants. For each topic, write two or three phrasings: a short one, a long one with context, and one in the buyer's casual voice. Variants hedge against rewriting you cannot see.
6. Fix the run conditions. Record engine, interface (API or consumer app), country, language, and whether you are logged out with memory off. API answers and consumer app answers can differ, so pick one and stay with it.
7. Freeze and version it. Give the set a name and a date, such as v1, 2026-10-21, and store it where the whole team reads the same copy.

If you would rather start from a structure, the prompt set template lists categories and example prompts you can adapt.

No published study sets the right number of prompts, so size the set by coverage, not by a target count. The question is whether every topic and stage you care about appears often enough to notice a change. The how many prompts question goes deeper on the trade-offs.

A simple way to size it: topics times variants. Say you run a payroll software company (a hypothetical) and pick 10 topics with 3 variants each. That gives 30 prompts per engine per run. More topics widen coverage. More variants make each topic's result less dependent on one phrasing.

Spread topics across stages so one stage cannot dominate:

Keep branded prompts in their own group. An engine naming you when the prompt already contains your name tells you little about discovery. Those prompts are still valuable for a different job: checking whether engines state your price, plans and features correctly, since AI platforms do get specific facts about specific brands wrong.

If you sell in several markets, build one set per country and language rather than translating a single list. The multi-country prompt set playbook covers that.

A trend line only means something if the questions stay the same. Add five easy prompts and your visibility goes up without anything changing in the engines. Drop a hard comparison prompt and the same thing happens. The chart then reports your edits, not the market.

Rules that keep runs comparable:

• No silent edits. Every change creates a new version with a date.
• Overlap before switching. When you must change the set, run the old and new versions side by side for at least one cycle. That shows how much of the difference comes from the prompts themselves.
• Start a new series. Plot v2 as a new line. Do not splice it onto v1.
• Log outside events. Engines change underneath you. As of October 2026, the models behind consumer apps shift between releases and plan tiers, and Cloudflare changed its default AI crawler settings for many sites on 15 September 2026. Note such dates next to your results.

Freezing does not mean the set never improves. It means improvements are deliberate, dated and visible.

Run through this list before you collect a single answer.

• The set has a written purpose in one sentence.
• Every prompt comes from or resembles real buyer language.
• Every topic is tagged with a funnel stage.
• No single stage holds most of the prompts.
• Branded prompts sit in a separate group and are reported separately.
• Each topic has two or three wording variants.
• Run conditions are recorded: engine, interface, country, language, login and memory state.
• The set has a version name and a freeze date.
• You know which facts you will check in answers (price, plans, founding year, product names).
• You have a place to log engine changes and site changes between runs.
• You have decided how often to run it, and who runs it.

If you want a quick feel for how engines answer before building the full set, the free checker on this site runs up to three buyer questions as a one-off snapshot. A snapshot is a starting point, not a trend.

Good results are not "high scores." They are readings you can trust and act on.

Signs the set is working

• Results are reported per engine. In Otterly's experiment, Microsoft Copilot produced 2,718 of one site's 3,934 tracked citations while other engines barely cited it. A blended number would have hidden that. See one score versus per-platform scores.
• Movements line up with something you can name: a site fix, a model release, a crawler access change.
• Fact checks show which specific claims engines get wrong, so you know which page to fix.
• Non-branded and branded results tell different stories, and you read them separately.

Signs the set is misleading you

• Visibility jumped the same week someone edited the prompts.
• Most prompts contain your brand name.
• Every prompt is phrased the same way, so one rewrite pattern decides the whole result.
• Someone reports a "rank" or "position" in Google's AI answers. Google's own generative AI report exposes impressions only, so no position data exists to check that number against.
• Results are compared across runs with different login, memory or country settings.

The next move when results look bad is to check access before rewriting content. If a crawler cannot reach the page, or the page is not indexed, no prompt will surface it.

Pick a cadence you can sustain and keep it fixed. Monthly suits most teams. Weekly helps when you are actively fixing access or factual errors and want to see whether answers change. Irregular runs make trends hard to read, because you cannot tell timing from change.

Run an extra check after a known event: a site migration, a change to robots.txt or your CDN, a pricing change, or a major model release from an engine you track. Log the event so the next regular run has context.

Once a quarter, review the set itself. Ask whether the topics still match what buyers ask, and compare your prompts with any engine-reported data you can get. Bing Webmaster Tools exposes the grounding queries Copilot actually used, which is the closest view anyone has of the hidden input. Comparing grounding queries with your prompt set shows how to use that review to plan your next version.

Should my prompt set include prompts that name my brand? Yes, but keep them in a separate group. Branded prompts are the best way to check whether engines state your price, plans and product facts correctly. They are a poor measure of discovery, because the engine was told your name. If you mix them with category and comparison prompts, they lift the overall number and hide whether buyers find you when they do not already know you.

Can I add new prompts once tracking has started? You can, but treat it as a new version. Write the change down with a date, run the old and new sets side by side for at least one cycle, and plot the new version as its own series. Adding prompts to a running series makes the trend reflect your edits instead of engine behavior, and you lose the ability to say what actually changed.

Does the exact wording of each prompt matter? It matters less than people expect, and you cannot fully control it. ChatGPT rewrites questions into its own search queries, and Google may fan out into several related searches. Your wording shapes the starting point, not the final query. That is why each topic should carry two or three variants: if results agree across phrasings, the finding is more robust than any single prompt.

Can a prompt set tell me my ranking in AI answers? No. A prompt set can show whether you are mentioned or cited, and whether the facts are right, for the questions you asked. It cannot give a trustworthy rank. Google's generative AI report in Search Console shows impressions only, with no position data, and OpenAI says only that ChatGPT ranks results using multiple factors. Report presence and accuracy per engine instead.

Should I write prompts myself or let a tool generate them? Either can work if the result covers real buyer language and every funnel stage. Tool-generated prompts are fast but can drift toward generic phrasing. Prompts you write from sales calls and support tickets usually match how your buyers talk. A common approach is to generate a draft, then edit it against real questions before freezing. What matters most is that the final set stays fixed between runs.