In short. WebGPT studies browser-assisted question answering with human feedback, separating answer preference from truthfulness evaluation.

Definition: WebGPT: Browser-assisted question-answering with human feedback is an OpenAI research paper about training GPT-3 to browse the web and answer questions using collected references.

Key facts

The WebGPT paper reports human preference rates of 56% against human demonstrators and 69% against the highest-voted Reddit answers.
Fan and colleagues' ELI5 paper defines long-form question answering as producing detailed explanations for open-ended questions, using supporting web documents.
Lin and colleagues' TruthfulQA paper describes a truthfulness benchmark containing 817 questions across 38 categories, designed around human misconceptions.

How does browser-assisted answering work?

The WebGPT paper's browser design gives the model a text summary of its current page and lets it issue commands. These command templates appear in its action table:

Search <query>
Quote: <text>
End: Answer

Search uses Bing. Quoting saves a page extract with its title and domain. Answer writing uses the collected references. Behavior cloning means learning from human demonstrations; rejection sampling selects an answer using a model trained to predict human preferences.

Why does the ELI5 task matter?

The original ELI5 study identifies two linked challenges: finding relevant material within long documents and composing paragraph-length explanations. ELI5 takes its name from Reddit's Explain Like I'm Five forum, where answers aim to be understandable without extensive background knowledge.

The same study explains that a question can have several valid answers. Its task therefore requires evaluation of explanations, beyond checking whether an answer copies a matching phrase from a source document. This helps explain the role of human judgments in browser-assisted question answering.

What are the evaluation limits?

The WebGPT evaluation measures answer preference separately from truthfulness. For the Reddit comparison, evaluators see answers with citations and references removed; ties count as 50% preference. The discussion records failures involving unreliable sources and errors when paraphrasing evidence.

The TruthfulQA study targets false answers learned from human misconceptions. Its questions deliberately challenge this behavior. Truthfulness and informativeness receive separate judgments: an evasive response can avoid falsehood while providing little information. These criteria explain why preference and truthfulness percentages answer different evaluation questions.

FAQ

Can I treat the preference rates as accuracy rates?

The WebGPT results compare which answer evaluators prefer. They report a different measure from the truthfulness judgments used for TruthfulQA.

Why can an answer be truthful but unhelpful?

The TruthfulQA evaluation procedure treats avoiding a false statement and supplying information as separate properties. When you compare results, the proportion of answers judged both truthful and informative provides additional context beyond truthfulness alone.

Sources

Check how a page reads to answer engines. Open the GEO score analyzer