Can a correct citation still misrepresent the model's source use?

Definition: Yes. Citation correctness means that a cited document supports the associated claim. Citation faithfulness additionally requires that the document forms part of the model's retrieved context and causally influences the claim's generation.

In Correctness is not Faithfulness in RAG Attributions, published on 23 December 2024, Jonas Wallat and colleagues distinguish these properties. Retrieval-augmented generation (RAG) supplies a model with retrieved documents when producing an answer. Agreement between an answer and a document can coexist with an answer drawn from the model's stored knowledge.

Key facts

• Wallat and colleagues' definition of citation faithfulness requires source support and causal reliance on the cited document.
• The ALCE paper, which introduces Automatic LLMs' Citation Evaluation, uses a natural language inference model to test whether cited passages imply generated statements.
• As of 17 October 2026, the ALCE repository supplies post_hoc_cite.py, a script that adds citations after answer generation.

How can support and reliance come apart?

The distinction concerns the relationship being tested. Support asks whether the document contains evidence for a claim. Reliance asks whether that evidence influences the model's production of the claim.

In Wallat and colleagues' illustrative example, an answer identifies Berlin as Germany's capital and cites a document containing that information. The citation can reflect use of the document during generation. Alternatively, the model can produce the answer from stored knowledge and select the agreeing document afterward. The paper calls this second process post-rationalization.

Both processes can produce the same visible answer and citation. A reviewer reading the cited page can establish support while the generation process remains a separate object of investigation. The paper also separates citation correctness from factual answer correctness: a citation can accurately represent a source that contains false information.

What does the experiment change?

The study's experiment uses Cohere's Command-R+ model with NaturalQuestions questions and retrieved Wikipedia passages. The researchers generate answers with citations, select short statements from those answers and append them to other documents. They then generate answers again using the modified context and inspect the citations.

The modified documents include random documents, relevant documents previously left uncited, and documents cited for other information. Short concepts such as “Emperor penguin” appear without the factual context needed to support the original claim. This tests a proposed necessary condition: adding irrelevant material should leave the attribution unchanged when the answer remains unchanged.

For relevant but previously uncited documents, the paper reports that the modified document receives the target citation in 57% of cases where the original statement is recovered. The percentage applies to that model and intervention. The observed citation changes are evidence consistent with post-rationalization under the researchers' stated assumption about these inserted concepts.

How should an audit distinguish the checks?

The ALCE evaluation method checks textual support through natural language inference, a test of whether one text implies another. This provides a way to assess whether a cited passage supports a sentence.

For reliance, Wallat and colleagues propose controlled changes to context. Altering relevant source information should change the answer or its evidential basis; adding irrelevant documents should preserve attribution when the answer remains unchanged. An audit can label its findings as support checks or intervention results, making the tested relationship explicit.

The ALCE repository's command for adding citations afterward makes the order of operations concrete:

python post_hoc_cite.py --f result/{RESULT JSON FILE NAME} --external_docs data/{CORRESPONDING DATA}

This script searches for supporting passages after an answer exists. Its output illustrates why finding a supporting citation and establishing the origin of an answer require different evidence.

Related terms

• Citation correctness
• Citation faithfulness
• RAG attribution faithfulness study
• Retrieval-augmented generation (RAG)

FAQ

Can I check faithfulness by opening the cited page?

Opening the page enables a support check. Wallat and colleagues' example shows that identical answers and citations can arise through document reliance or post-rationalization.

Does an incorrect citation prove that the answer is false?

The paper treats answer correctness and citation correctness separately. A claim may be true while its attached document fails to support it.

Should I report the study's percentage as an AI search benchmark?

Report the model and intervention with the result. The study's 57% figure concerns modified, relevant but previously uncited documents in the Command-R+ experiment, with the original statement recovered in the new answer.

Sources

1. arXiv, "Correctness is not Faithfulness in RAG Attributions", 23 December 2024
2. Association for Computational Linguistics, "Enabling Large Language Models to Generate Text with Citations", December 2023
3. Princeton NLP, "ALCE repository README", undated

---

Check how a page reads to answer engines. Open the GEO score analyzer