How Often to Re-Run AI Visibility Checks

AI visibility tracking is several checks with different failure modes. Some fail suddenly and completely. Others drift. Each one deserves its own schedule.

Access runs most often because it is the gate with a documented consequence. OpenAI states that sites opted out of OAI-SearchBot "will not be shown in ChatGPT search answers." Its help center adds a second condition: the host or CDN must also allow traffic from OpenAI's published searchbot IP addresses. A deploy can break either half, and no dashboard will tell you.

Accuracy and prompt sets run less often because answers vary from run to run. A monthly run collects enough answers per platform to compare. A single flag on two answers looks like a collapse when it's really a partial run.

A change in your numbers is not proof that something on your site changed. Several things move on the provider's side.

• Models change. OpenAI's GPT-5.6 models carry a February 16, 2026 knowledge cutoff, while ChatGPT's Instant path (chat-latest) carries August 31, 2025. OpenAI also warns the snapshot "will be regularly updated." Routing between the two changes what ChatGPT knows without live search.
• App defaults change. Google's release notes added gemini-3.6-flash to the Gemini app's model menu on 2026-07-21, and its API changelog lists gemini-3.8-flash as generally available on 2026-09-02. Google AI Mode moved to a gemini-3.5-flash default on 2026-05-19. Google says AI Overviews and AI Mode "may use different models and techniques."
• Retrieval is discretionary. ChatGPT decides per query whether to search, and Claude.ai search is a user toggle. Published estimates of how often ChatGPT searches disagree widely: Profound puts it near 18% of conversations, Semrush's clickstream at 34.5% of US queries in February 2026, and cloro at 80.5% globally on a 100-prompt set. See how often ChatGPT searches the web.
• Infrastructure defaults change. From 2026-09-15, Cloudflare set Training and Agent crawlers to blocked by default on ad-displaying pages for new domains, new sites on existing accounts, and all existing free-tier customers. A site can lose AI access without its owner editing anything.
• APIs retire. Perplexity's Sonar Chat Completions retired on 2026-09-27. On the Agent API, web search is opt-in, so unmigrated tracking code can return ungrounded answers silently. Your tracker's numbers can change because the tool changed.

The next move is simple: keep a dated event log beside your results. When a number moves, you check the log before you touch the site. For more on this, read what changes between AI visibility checks.

1. List the facts you want AI to get right. Pricing, plan names, product names, founding year, location. Note the page where each lives, and fix any place your own site states one two different ways.
2. Freeze a prompt set. Write a fixed list of buyer questions and give it a version number. Changing one question starts a new series.
3. Fix the run settings. Run logged out with memory off, from the same location and language. OpenAI says ChatGPT may use saved memories when it rewrites a search query, so logged-in runs aren't reproducible. Record whether each answer showed citations, since that tells you if retrieval happened.
4. Put access checks in your deploy process. Request key pages as each AI crawler and diff the raw HTML against the rendered page. Our scripts for AI SEO checks cover the basics.
5. Book the calendar. Access weekly and after deploys. Index status and accuracy monthly. The prompt set on the same weekday every cycle.
6. Keep the event log. Your deploys, CDN or firewall changes, model launches, default changes, API retirements and any change to your tool's scoring formula.
7. Review each engine on its own. A blended score hides an engine that dropped to zero. Where an engine reports its own data, such as Bing Webmaster Tools' AI Performance report with its grounding queries, compare it against your prompt results. The tradeoffs are laid out in prompt-based tracking vs engine-reported data.

Run this after every release, and after any CDN, firewall, CMS or plugin change.

• robots.txt still allows the retrieval bots you want: OAI-SearchBot, PerplexityBot, Claude-SearchBot, Meta-WebIndexer, Googlebot and Bingbot.
• You haven't confused training and search bots. GPTBot governs training only, and OpenAI states each setting "is independent of the others."
• Each crawler user agent gets a 200, not a 403, a rate limit or a challenge page. The AI Crawler Checker requests a page as each crawler and shows which ones the firewall turns away.
• /robots.txt itself is reachable to those crawlers, not just your pages.
• Prices, plan names and key claims appear in the raw HTML before JavaScript runs. Vercel found ChatGPT and Claude crawlers fetch JavaScript files but don't execute them.
• No template change added noindex, nosnippet, max-snippet:0 or data-nosnippet to content you want shown.
• Canonical tags still point where they should.
• After any CDN plan or settings change, the AI crawler categories match what you intend.

Good

• Access checks return the same passing result every run.
• Accuracy flags on facts you control fall over time, or stay at zero.
• When a prompt-set number moves, the event log explains it.
• Engines move independently, and you can say why each one moved.

Bad

• One crawler gets a 403 while robots.txt allows it. That is a complete, silent failure for that engine.
• Key facts only exist after hydration.
• Every engine drops on the same day. That usually points to your side, so start with access.
• One engine drops while the others hold. That points to the provider side or to that engine's index. In Otterly's 2,000-post experiment, Google deindexed one site after about 14 days and the other after about 90 with no manual action, while Bing kept both and Copilot kept citing them.
• A report shows your "position" in a Google AI answer. No position data exists to check that against, as explained in why there's no AI rank to track.

Keep the schedule running indefinitely. A trend is the point, and one snapshot can't show one. If you have no baseline yet, start with the one-afternoon AI visibility check, or run the free checker on this site to see whether AI names you for three buyer questions. Then put the recurring checks on the calendar.

Start a new baseline, and stop comparing to the old one, when:

• You change, add or remove a question in the prompt set.
• Your tool changes its scoring formula. Scores from two formulas aren't comparable.
• A provider changes the default model, retrieval behavior or API you measure through.
• You move CDN or change bot-management settings.

Watch first-party reports for data problems too. Google Search Console's generative AI report shows impressions only, and a Google-side bug depressed impressions from around 2026-08-12. Exclude that window from any trend.

Once a quarter, review the prompt set itself. Retire questions buyers no longer ask and start a new versioned series. For the wider picture of how tracking works, see the AI visibility tracking guide.

Is daily tracking better than weekly? Not for prompt sets. AI answers vary between runs, so daily prompt tracking mostly adds noise and cost. Daily runs make sense for access checks, which are cheap and deterministic: a crawler either gets a 200 or it doesn't. Use a fixed weekly or monthly prompt cadence and rely on event-triggered access checks to catch sudden breaks.

Why did my results change when nothing on my site changed? The engines changed. Models get new versions and knowledge cutoffs, apps switch default models, and ChatGPT decides per query whether to search. CDNs change their bot defaults, as Cloudflare did on 2026-09-15. Tracking tools change too. Check your event log for a provider change on or before the date the number moved, and run an access check to rule out your side.

How soon after a fix should results change? Nobody can promise a timeline. Fixing access removes a block, but whether a page gets retrieved and cited is each engine's decision. Answers given without live search only change with a new model, bounded by its knowledge cutoff. Keep your schedule fixed and read the trend over several runs rather than expecting the next run to reflect the fix.

Should I re-run checks every time I publish? Run the access checklist on new pages, because a new template can carry a stray noindex or hide content behind JavaScript. Don't expect volume to move results by itself. In Otterly's experiment, two sites publishing about 1,000 AI-generated posts each were removed from Google's index, and Google's AI surfaces stopped citing them.

Can I compare results from two different tracking tools? Not directly. Tools ask different prompts, measure different engines, use different settings and score with different formulas. Some average answers given with and without live search. Pick one tool for your trend line, record its settings, and treat any other tool as a separate series.