Temperature in LLMs: definition and effect on AI answers

A language model writes one token (a word or word fragment) at a time. At each step it assigns a probability to every possible next token. Temperature decides how the model turns those probabilities into a choice.

• Low temperature sharpens the distribution. The most likely token wins almost every time, so output is repetitive and predictable.
• High temperature flattens it. Less likely tokens get picked more often, so output is more varied and sometimes less coherent.

Sampling temperature is a generation setting. It acts at the last stage, after the model has whatever it knows and whatever it retrieved in its context window. It does not change what the model knows. It changes which of its possible answers you get this time.

Say you run a hypothetical accounting software company and ask an AI assistant, "What's the best invoicing tool for freelancers?" The model might rank four tools as roughly equally plausible. At a non-zero temperature, one run names you first, the next names a competitor, and a third leaves you out.

None of those runs is the real answer. Each is a draw from the same distribution.

You can't set temperature in the consumer chat apps. The platform chooses it, and vendors don't publish the values they use. Even a setting of zero makes output more repeatable without guaranteeing identical answers.

Temperature is also not the only thing that moves. OpenAI says ChatGPT search rewrites your question into one or more targeted queries, and that saved memories can shape that rewrite. Google says AI Overviews and AI Mode may use different models and techniques. Each layer adds its own variation on top of sampling.

Temperature is the plainest reason a single check can't tell you how visible you are. If one answer is a sample, a week-to-week change in one answer can be noise.

What this means in practice:

1. Treat one run as a snapshot. It shows what happened once, for one question, on one platform.
2. Look for patterns across repeats. A brand named in most runs of a question is in a different position from one named once.
3. Compare like with like. Same questions, same platforms, measured over time, before you call a change a trend.
4. Ask how a tracker samples. Our page on whether prompt-tracking tools are accurate covers what a prompt sample can and can't show.

The free checker on this site is built on that logic: it runs up to three buyer questions and is labelled a snapshot, not continuous tracking.

What stays in your control is upstream of sampling. Whether crawlers can reach your pages, and whether your key facts are in the HTML before JavaScript runs, decides whether your content is a candidate at all. Temperature only reshuffles among candidates. The guide to what you can and can't influence in LLM optimization covers those levers.

• "Low temperature means accurate." It means consistent. A model can repeat the same wrong price every time at low temperature.
• "You can write content that beats temperature." Nothing on your page changes the setting. Research on predictable prose and AI citation found an association between model-predictable text and citation, but its causal test rewrote whole pages and never isolated style.
• "A score drop means something broke." It might. It might also be sampling. Check whether the change holds across repeat runs before acting.
• "Variation is random, so rankings are meaningless." Sampling is random within a distribution. The distribution itself reflects real factors such as position bias in the retrieved list and brand prominence.

• Context window: the text the model can see when it generates, including retrieved pages.
• Position bias: the tendency to favour documents placed earlier in the context.
• Brand prominence: how well-known a brand is, the main confounder in citation studies.
• Are prompt-tracking tools accurate?: what sampled answers can and can't measure.