How do I check whether a citation supports an answer's claim?

Definition: Checking citation support means comparing an answer's statement with the evidence in its cited source. ALCE's citation evaluation tests whether the cited passages, taken together, fully support the statement using only the information they contain.

Key facts

• The ALCE paper separates citation recall, which checks support for answer statements, from citation precision, which checks for irrelevant citations.
• FActScore defines an atomic fact as a short sentence conveying one piece of information, allowing separate checks within a mixed sentence.
• Liu, Zhang and Liang's verifiability study asks reviewers to judge full, partial or absent support and to copy the minimal supporting set of source sentences.

How does ALCE test support?

In ALCE's evaluation method, answer statements generally follow sentence boundaries. The benchmark uses passages of 100 words as evidence. For each statement, it joins the cited passages and applies natural language inference: a test of whether the evidence entails, or supports the truth of, the statement.

The cited passages form the premise; the answer statement forms the hypothesis. A statement receives citation recall credit when it has a citation and the combined passages fully support it. The scores are averaged over statements.

Citation precision adds a separate check for each reference. ALCE treats a citation as irrelevant when it cannot support the statement alone and removing it leaves the remaining citations sufficient. Its method allows redundant references that each fully support a claim.

For a manual review, the corresponding question is whether the full statement follows from the cited evidence alone. A source about the same topic still needs a passage that supports the actual assertion.

How should a sentence with several facts be checked?

FActScore's method breaks an answer into atomic facts and labels each fact according to support from a specified knowledge source. Its motivation is that a sentence can contain a mixture of supported and unsupported information.

Applied to citation review, this suggests splitting a sentence into its individual assertions while retaining the connection to the original citation. Each assertion receives its own evidence check. This makes the unsupported part visible when a source supports only part of a sentence.

The source boundary matters: FActScore evaluates support against a chosen knowledge source. For the narrower question of whether an answer's citation supports its claim, the review uses the cited material. Evidence found elsewhere can be recorded separately as a further factual check.

What evidence should the review retain?

The verifiability study's annotation guidelines require interpreting the statement in the context of the question and the full response. This resolves references whose meaning depends on earlier text. Reviewers then check all the information in the statement against the cited document.

The study distinguishes full support from partial support and no support. It also records inaccessible articles separately. For multiple citations, reviewers check whether their combined content fully supports the statement and flag contradictions between sources.

A manual review can use this record, with the evidence field holding the minimal supporting sentences requested by the study:

{
  "answer_statement": "",
  "cited_url": "",
  "support_judgment": "",
  "evidence_sentences": [],
  "unsupported_assertions": []
}

The field names are a suggested audit format. The judgment and saved passage let another reviewer inspect the same claim-source comparison.

Related terms

• Citation correctness
• Citation completeness
• ALCE benchmark
• FActScore

FAQ

Can several citations jointly support a statement?

Yes. ALCE tests the concatenated cited passages for statement support. You can check each source's contribution and then assess their combined evidence.

What if I cannot open the cited page?

Record the access problem separately. The verifiability study's guidelines include an inaccessible-article judgment for cases such as a paywall or dead link.

Sources

1. Association for Computational Linguistics, "Enabling Large Language Models to Generate Text with Citations", December 2023
2. Association for Computational Linguistics, "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation", December 2023
3. Association for Computational Linguistics, "Evaluating Verifiability in Generative Search Engines", December 2023

---

Check how a page reads to answer engines. Open the GEO score analyzer