Faithfulness (RAG metric): definition

Faithfulness scores one answer against one set of retrieved documents. The RAGAS documentation defines it as supported claims divided by total claims in the response.

Say a model answers a question about a hypothetical software company with five factual claims. Four of them appear in the pages it retrieved. One does not. Faithfulness is 4/5, or 0.8.

The metric belongs to retrieval-augmented generation, where a system fetches documents first and then writes an answer from them. Without retrieved context, there is nothing to score against. Some tools describe the same idea as groundedness, or as a consistency rate.

Most implementations follow three steps:

1. Split the answer into individual claims, in the spirit of the atomic facts used by FActScore.
2. Check each claim against the retrieved context, usually with a second model acting as judge.
3. Divide supported claims by total claims.

Two design details matter when you read a score.

• Unsupported is not the same as false. FActScore's not-supported class covers a source that "contradicts or lacks evidence for" a claim, and RAGAS likewise counts an unsupported claim against faithfulness. A true fact the model knew from training still lowers the score if the context omits it.
• Precision alone can be gamed. The SAFE paper added a recall term because precision-only scoring "does not penalize systems that abstain from answering too often or generate text with fewer facts." A short, cautious answer can score 1.0. See the SAFE and F1@K digest for how that fix works.

Vectara's hallucination leaderboard uses a binary version per summary: hallucination rate equals 100 minus the consistency rate.

Faithfulness and correctness answer different questions. Faithfulness asks: did the model stick to its sources? Factual accuracy asks: is the answer true? If an engine retrieves an outdated review site listing your old pricing, a perfectly faithful answer repeats the old pricing.

That shifts where the problem lives. When the engine reads the wrong page, the fix is upstream:

• Make sure AI crawlers can fetch your own pages, and that key facts sit in the HTML before JavaScript runs.
• State specific facts (a price, a number, a one-line definition) on fetchable pages, so the engine has a sentence it can quote.
• Remove contradictions between your own pricing page, about page and footer.
• Find third-party pages that describe you wrongly. Being named correctly on sites engines cite is associated with appearing in answers, with no guaranteed effect.

Two limits apply. First, citations are not proof of use: mechanistic work found that adjusting specific attention components repaired over 90% of missed citations without changing the sources (the attribution study). Second, manipulated pages exist. A model can be faithful to a page carrying injected instructions, which is a reputational risk, not a tactic.

• "Faithful means correct." No. It means consistent with the retrieved text.
• "High faithfulness means no hallucination." Only relative to the context. It cannot catch a wrong source.
• "A cited answer is a faithful answer." Citation emission is partly separable from source influence, so a citation does not show which text shaped the claim.
• "Every AI answer has a faithfulness score." Retrieval is discretionary on consumer apps. An answer written from model memory has no retrieved context to score against.
• "Publishers can raise an engine's faithfulness." It is a property of the model and its context. You control what your pages say and whether crawlers reach them.

• FActScore: atomic factual precision, scored against a knowledge source.
• Factual accuracy: whether an AI answer is true about your brand.
• Citation share: how often you are cited relative to other sources.
• Choosing AI visibility KPIs: where faithfulness and accuracy fit in reporting.