In short. FActScore measures the share of atomic facts supported by a chosen knowledge source, with factual coverage assessed separately.

Definition: FActScore evaluates factual precision by measuring the share of atomic facts in generated text that a chosen knowledge source supports. An atomic fact is a short statement conveying one piece of information.

Key facts

Min and colleagues' FActScore paper gives each atomic fact equal weight and averages response scores over prompts that the model answers.
The ACL Anthology publication record places the paper at EMNLP in December 2023; its human evaluation concerns generated biographies.
The authors' implementation supports a custom reference collection containing documents with title and text fields.

How does atomic-fact evaluation work?

The paper's definition treats a sentence as potentially containing several independently checkable statements. Splitting it into atomic facts allows supported and unsupported details within that sentence to receive separate labels.

The response-level calculation and the model-level aggregation can be expressed as this pseudocode matching the paper's equations:

response_score = supported_atomic_facts / total_atomic_facts
FActScore = mean(response_score over responses that answer the prompt)

Each fact has equal weight, irrespective of its importance. The reference collection is part of the evaluation's definition: a support decision depends on the evidence in that collection.

In the human annotation procedure, annotators review and revise facts initially produced by InstructGPT. Another set of annotators marks irrelevant statements and labels relevant facts Supported or Not-supported using English Wikipedia. This separates fact extraction from evidence checking.

What does the original study evaluate?

The study's data section describes a sample of 183 people from Wikidata with corresponding Wikipedia pages. The human evaluation covers biographies generated by InstructGPT, ChatGPT and PerplexityAI. The authors choose biographies because they contain specific, verifiable statements and allow comparison across professions, nationalities and levels of rarity.

The paper also introduces an automated estimator, a model that approximates the human evaluation. One variant retrieves passages from the knowledge source and asks a language model to judge an atomic fact using those passages. The paper distinguishes error in an aggregate score from accuracy on individual fact labels: errors on separate facts can offset one another in the aggregate.

These experiments assess factual precision in the tested biography task. The limitations section identifies nuanced or debatable claims and frequently conflicting reference material as unsuitable conditions for the method's assumptions.

How should an evaluation report its scope?

Abstention means declining to answer a prompt. The paper's limitations section recommends reporting FActScore together with the abstention rate and average number of atomic facts. Factual recall means coverage of information; the score's precision calculation can favor responses that contain fewer facts or answer fewer prompts.

As of 17 October 2026, the implementation README documents a default Wikipedia snapshot dated 1 April 2023. It also documents a length penalty controlled by --gamma, with a default of 10 that penalizes responses containing fewer than 10 facts; --gamma 0 disables it. The implementation exposes both the penalized score and the unpenalized init_score.

For comparisons, this means recording the reference collection and its date, the estimator and the penalty setting alongside the score. Those choices determine which evidence is available and how short responses are treated.

FAQ

Does a high FActScore mean your answer is complete?

FActScore measures support for the facts present in a response. The paper explicitly separates this from factual recall, which concerns coverage, and discusses answers that omit significant information despite containing supported facts.

Can you use a source other than Wikipedia?

Yes. The authors' README documents register_knowledge_source for a custom collection. You supply documents in JSONL format, with one document per line and title and text fields, then select that collection when computing the score.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer