In short. BrowseComp accuracy measures finding reference answers, while ALCE separately evaluates citation support and relevance.
Definition: Citation quality measures how fully an answer's statements are supported by cited evidence and how relevant its citations are. BrowseComp scores answer-finding accuracy; ALCE evaluates citation support separately from answer correctness.
Key facts
- The BrowseComp paper, published in April 2025, describes 1,266 questions with short reference answers and grades semantic agreement with those answers.
- The ALCE paper, published in December 2023, separates answer correctness from citation recall and citation precision, which measure support coverage and citation relevance.
- The ALCE evaluation instructions provide a dedicated
--citationsoption alongside other evaluation options.
What does a BrowseComp result establish?
BrowseComp measures success at finding hard-to-find information on the web. In the paper's grading procedure, an AI model compares the predicted answer with the reference answer for semantic equivalence. The grading target is the final answer's agreement with the reference.
The same paper identifies generating long answers and resolving ambiguous queries as challenges outside its focus. Its accuracy therefore answers a specific question: how often does the evaluated system find the target answer on this question set? That score alone supplies no measurement of whether the citations in a longer response support each statement.
How does ALCE check citation support?
ALCE, short for Automatic LLMs' Citation Evaluation, requires a system to retrieve passages and generate an answer with citations. The paper's evaluation framework distinguishes the following checks:
| Metric | What the evaluation checks |
|---|---|
| Answer correctness | Whether the response covers the expected answer content, using a metric tailored to the dataset. |
| Citation recall | Whether each statement has citations whose combined passages fully support it, averaged across statements. |
| Citation precision | Whether individual citations are relevant, averaged across citations. |
ALCE uses natural language inference, a test of whether evidence logically supports a statement, to automate support checks. Its recall calculation requires both a citation and full support from the cited passages. Its precision calculation treats a citation as irrelevant when it fails to support the statement alone and removing it leaves the remaining citations sufficient. Precision also requires the statement to pass the recall check.
These distinctions explain why finding a correct answer and supplying supporting citations require separate evaluation. A relevant topic match between a page and a statement is weaker than the full support required by ALCE's recall test.
How can an agency inspect the evaluation behind a score?
ALCE provides a concrete example. As of October 2026, its repository instructions give this command for evaluating ASQA, one of its question-answering datasets:
python eval.py --f {path/to/result/file} --citations --qa --mauve
The braces mark the result-file placeholder. The same instructions store model outputs with reference answers and run configurations in JSON files under result/; evaluation writes a file with the suffix .score.
For a report about a website's citations, the practical inference is to request the answer text and cited evidence alongside the score. The BrowseComp grading target and ALCE's support tests provide separate checks: whether the answer matches the expected fact, and whether its attached evidence supports its statements. Keeping those results separate makes the reported measurement interpretable.
Related terms
- BrowseComp benchmark
- ALCE benchmark
- Citation correctness
- Citation completeness
- How do I check whether a citation supports an answer's claim?
FAQ
Can I use BrowseComp accuracy as a citation-quality percentage?
No. BrowseComp's grader compares answers with reference answers. A citation-support percentage requires a separate evaluation of the answer statements against their cited evidence, such as ALCE's citation recall.
Does adding more citations improve ALCE precision?
ALCE's precision test checks relevance. A citation that neither supports the statement independently nor contributes necessary evidence when combined with the other citations counts as irrelevant, provided the statement otherwise has full citation support.
Sources
- OpenAI, "BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents", 16 April 2025
- Association for Computational Linguistics, "Enabling Large Language Models to Generate Text with Citations", December 2023
- Princeton NLP, "ALCE repository README", accessed 17 October 2026
Check how a page reads to answer engines. Open the GEO score analyzer