In short. An API benchmark measures its tested configuration; treating it as a proxy for consumer search requires validation on the consumer interface.
Definition: An application programming interface (API) benchmark measures answers produced by software requests under specified test conditions. Its use as a proxy for a consumer search interface, such as a website's chat window, requires validation against observations from that interface.
Key facts
- The BrowseComp paper describes a benchmark for finding difficult information and explicitly sets aside long answers and ambiguous user queries.
- As of 17 October 2026, OpenAI's web search API guide distinguishes retrieved sources from citations included in an answer.
- OpenAI's ChatGPT web search guide places search results and citations in the chat when search runs, and says workspace settings can restrict search availability.
What does BrowseComp actually grade?
The BrowseComp paper, submitted on 16 April 2025, describes questions created by human trainers with short, verifiable reference answers. Its grading procedure uses an AI model to compare the predicted answer with the reference for semantic equivalence, meaning whether they convey the same answer.
The paper identifies persistent searching and finding difficult information as the capabilities exercised. It calls the benchmark incomplete because ordinary user queries also involve long answers and ambiguity. The measurement consequence is that answer accuracy on BrowseComp and the frequency of a website's citations in consumer search require separate evaluation criteria.
Which API conditions belong in the report?
OpenAI's API guide says the Responses API lets the model choose whether to search; Chat Completions search always retrieves web information. The guide also describes access to models and a tool used by Search in ChatGPT.
A report should identify the endpoint, model and search configuration. The guide documents domain filtering, approximate user location and live-access controls.
This request fragment enables search and requests the retrieved-source list:
{
"tools": [{"type": "web_search"}],
"include": ["web_search_call.action.sources"]
}
For counting citations, the documented url_citation annotations identify cited URLs. The broader sources list includes URLs consulted during response formation. A citation report should preserve that distinction.
How should the consumer interface be compared?
OpenAI's ChatGPT web search guide describes citations appearing in the chat when web search runs. It also documents account and workspace conditions for deep research, an investigation across multiple sources that produces a report.
For a comparison with that interface, the practical implication is to record the exact question, date, account or workspace conditions and selected search mode alongside the visible answer and citation links. Ordinary search and deep research should have separate result labels.
The BrowseComp paper illustrates the value of describing the tested setup: its model evaluations specify model identifiers and distinguish browsing-enabled models. Applying that principle, matched questions on the API and consumer interface support a comparison for the recorded conditions. Keeping the observations separate makes differences inspectable before treating API results as a proxy.
Related terms
FAQ
Can I label an API result as consumer search visibility?
Label it with the tested endpoint and configuration. The API documentation distinguishes search behavior across endpoints; validating a consumer-search label requires observations from the consumer interface.
Does a correct BrowseComp answer measure my site's citation frequency?
BrowseComp's grading procedure compares an answer with a short reference answer. To measure your site's citation frequency, define a separate check for its linked URLs in the observed answers.
What should I retain from a consumer-interface check?
Keep the answer and visible citation links with the question and test conditions. The ChatGPT guide places search results and citations in the chat, providing the artefact for reviewing that observation.
Sources
- Jason Wei et al., arXiv, "BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents", 16 April 2025
- OpenAI, "Web search", accessed 17 October 2026
- OpenAI, ChatGPT Learn, "Web search", accessed 17 October 2026
Check how a page reads to answer engines. Open the GEO score analyzer