In short. Citation completeness measures whether the verifiable statements in an answer receive full support from their associated citations.

Definition: Citation completeness measures how much of an answer's verifiable content is fully supported by its associated sources. Citation recall expresses this as the proportion of statements that receive full support from their citations.

Key facts

Liu and colleagues' Evaluating Verifiability in Generative Search Engines defines citation recall over statements requiring verification, including claims that appear obvious or common sense.
Gao and colleagues' ALCE benchmark, short for Automatic LLMs' Citation Evaluation, checks a statement against its cited passages together, allowing combined evidence to supply full support.
Rashkin and colleagues' attribution framework evaluates whether an answer accurately represents its source; source trustworthiness receives a separate assessment.

How is completeness calculated?

Liu and colleagues' evaluation method segments an answer into sentences and excludes statements that require no verification, such as questions addressed to the user. Its definition translates into:

citation recall = fully supported verification-worthy statements
                  / all verification-worthy statements

An uncited factual sentence remains in the denominator and receives no credit. A sentence containing several independently verifiable claims needs support for all of them. The same paper defines citation precision as the proportion of citations supporting their associated statements. Completeness therefore requires examining the answer's uncited content alongside its cited content.

What counts as adequate source support?

The Attributable to Identified Sources framework, or AIS, checks whether information in a model response accurately represents the supplied evidence. Its annotation guidelines first establish whether the response is understandable in context. Pronouns and short answers may need the original question to make their meaning clear.

The guidelines then ask whether all the response's information is supported. Missing information or an inaccurate representation prevents a full-support judgment. A passage mentioning the same topic is insufficient when it leaves part of the claim unsupported.

AIS treats source quality and relevance to the question as separate evaluation tasks. A completeness assessment consequently needs a companion review of the evidence's reliability and the answer's usefulness.

How can an answer be audited?

Gao and colleagues' ALCE human evaluation protocol gives annotators the question, a sentence, and all the passages it cites. They judge whether that joint evidence supports every claim in the sentence, then average the sentence-level judgments across the answer.

For a website audit, this suggests a worksheet pairing every answer sentence with its associated passages and a full-support judgment, including rows for uncited factual sentences.

ALCE automates this check with natural language inference, which tests whether evidence entails a statement. Its recall metric requires an attached citation and full support.

FAQ

Can several sources jointly support a sentence?

Yes. ALCE's recall calculation combines the cited passages before checking support. You can assess their joint evidence even when an individual passage covers only part of the sentence.

Does full support establish that the source is reliable?

AIS's annotation guidelines separate accurate representation from source trustworthiness. You assess reliability independently of whether the answer faithfully represents the supplied evidence.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer