In short. Split the answer into atomic factual claims, check each against a declared evidence set, and report unsupported claims separately from missing citations.
Definition: Measuring unsupported claims means splitting an AI answer into individual factual assertions and checking each against a declared evidence set. The result is the share of assessed claims for which that evidence provides no support; citation checking separately measures support from the sources attached to the answer.
Key facts
- The FActScore paper defines an atomic fact as a short statement conveying one piece of information and measures the percentage supported by a reliable knowledge source.
- The ALCE paper defines citation recall through full support from cited passages and citation precision through the relevance of individual citations.
- As of 17 October 2026, the official FActScore implementation exposes an unpenalized score, a responding ratio and an average count of atomic facts per response.
How should an answer be split into claims?
The FActScore method uses atomic facts because a sentence can combine supported and unsupported information. Its annotation process lets human reviewers revise a model's proposed decomposition before another set of reviewers checks support.
A practical audit adapts that process: retain the original answer, list its factual assertions, and split statements that combine independently checkable details. Preserve qualifications such as dates, quantities and conditions in the resulting claims. Record excluded material and the exclusion rule so that greetings or subjective advice do not silently change the denominator.
The assessment unit matters. ALCE generally splits answers at sentence boundaries, with list entities used for its list-answer dataset. An atomic-claim audit should identify its unit explicitly when compared with ALCE results.
Which sources count as support?
FActScore checks claims against a chosen knowledge source. For a website audit, define that evidence set before assigning labels: the answer's linked pages for a citation audit, or a separately selected reference collection for factual checking. Record the relevant passage and the decision for each claim. Support remains relative to that declared collection.
The ALCE evaluation checks whether the cited passages, considered together, fully support the associated statement. It averages these statement-level decisions to obtain citation recall. A link beside a sentence therefore requires a passage-level check, including any qualifications within the sentence.
Keep missing citations separate from failed evidence checks. An uncited claim can have support in the independent reference collection; a cited claim can lack support in its linked sources. ALCE's citation precision adds a further check for irrelevant references, allowing a report to distinguish coverage from citation relevance.
What should the report contain?
For a manual audit using a binary supported or unsupported decision for every assessed atomic claim, the FActScore proportion yields these calculations:
supported_fraction = supported_atomic_claims / assessed_atomic_claims
unsupported_fraction = unsupported_atomic_claims / assessed_atomic_claims
Report both counts alongside the fractions and retain the claim-to-passage worksheet. Label citation recall separately and state whether its denominator consists of sentences or atomic claims.
The FActScore implementation's output fields provide additional context when using the package. After scoring, its documented output can be inspected with:
print(out["init_score"])
print(out["respond_ratio"])
print(out["num_facts_per_response"])
Here, init_score excludes the length penalty. The accompanying response and fact counts help interpret comparisons involving abstentions or shorter answers.
Related terms
FAQ
Can I count missing links as unsupported claims?
You can report missing links as a separate count. Under ALCE's citation-recall definition, a statement needs cited evidence that fully supports it; FActScore checks support against the selected knowledge source.
Can I automate the claim decomposition?
Yes. FActScore's human evaluation starts from model-generated atomic facts that reviewers can revise. You can use that approach while retaining a review step for the claim boundaries.
Should I compare the package's score directly with my manual fraction?
Check the score field first. The official implementation distinguishes score from init_score, which excludes its length penalty; use the latter when comparing with an unpenalized supported-claim fraction under matching assessment rules.
Sources
- Association for Computational Linguistics, "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation", December 2023
- Association for Computational Linguistics, "Enabling Large Language Models to Generate Text with Citations", December 2023
- FActScore authors, "FActScore README", accessed 17 October 2026
Check how a page reads to answer engines. Open the GEO score analyzer