# OpenAI released BrowseComp for hard-to-find web information tasks OpenAI released BrowseComp on April 10, 2025, with 1,266 short-answer tasks designed to test persistent web browsing and verification. Canonical: https://rangoon.ai/insights/archive-openai-browsecomp-browsing-agent-benchmark-april-2025/ Author: Rangoon Editorial (https://rangoon.ai/insights/editorial/) Published: 2026-10-24 Updated: 2026-10-24 Event date: 2025-04-10 Tags: [OpenAI](https://rangoon.ai/insights/tags/openai/), [AI evaluation](https://rangoon.ai/insights/tags/ai-evaluation/), [agent benchmarks](https://rangoon.ai/insights/tags/agent-benchmarks/), [browser agents](https://rangoon.ai/insights/tags/browser-agents/) Source note: Historical reporting: OpenAI released BrowseComp on April 10, 2025 and said the open benchmark contained 1,266 hard-to-find, short-answer browsing problems. The release describes model screening, search-result checks, human verification and explicit limits around open-ended user queries. Its linked simple-evals repository is the public implementation context. OpenAI’s model results and test-time-compute experiments are supplier records on BrowseComp; they do not prove general research quality, citation reliability, safe browsing or performance on a changing live web. Image: Hard-to-find clue routed to checked short answer. Original Rangoon explanatory diagram: Source boundary: dated primary release reporting; original editorial analysis. No vendor interface, photograph, measurement result, or endorsement implied. ## Key takeaways - BrowseComp separated hard-to-find fact location from broad, open-ended question answering. - Short answers made checking practical while narrowing what the benchmark represents. - Model results depend on browsing policy, search state, tool access, samples and the protected task set. ## OpenAI released 1,266 browsing tasks on April 10, 2025 [source 1](https://openai.com/index/browsecomp/) OpenAI dated BrowseComp to April 10, 2025 and described an open-source benchmark of 1,266 challenging problems for agents locating hard-to-find information online. The release’s premise is deliberate: a useful browsing agent may need to connect clues across many sites, not merely retrieve a familiar fact. BrowseComp restricts answers to short, single-answer targets so they can be checked. That makes the event a release of a defined browsing evaluation, not a release of a general web-research certification. A task can be difficult to locate and easy to verify while still differing sharply from a reader’s ambiguous, multi-part or time-sensitive request. ## The construction process filtered for questions that were hard to find but bounded to grade [source 1](https://openai.com/index/browsecomp/) OpenAI says trainers created questions from evidence-backed facts, then used checks involving contemporary models, simple searches and human attempts to assess difficulty. The release calls this an asymmetry of verification: finding an answer can require broad exploration, while checking a proposed short answer can be direct. That design gives BrowseComp a clear grading target, but it also sets a boundary on the conclusion. It does not evaluate whether an agent writes a useful explanation, handles a disputed source, asks a clarifying question, maintains citations or decides that a query is unsafe. The exact task-writing and filtering process is part of what a BrowseComp score represents. ## Tool use and inference-time effort were explicit conditions in OpenAI’s reported runs [source 1](https://openai.com/index/browsecomp/) [source 2](https://github.com/openai/simple-evals) The launch compares models with and without browsing and describes experiments that vary browsing effort and aggregate repeated attempts. It reports that additional test-time work affected outcomes, while also noting that the Deep Research model was trained on data aimed at BrowseComp-style tasks. Those facts make it misleading to treat the numbers as attributes of a base model alone. The agent policy, model version, browser capabilities, search availability, maximum effort, number of samples and aggregation rule all help produce a run. A result obtained on the 2025 collection may also become less comparable as the live web changes, sources disappear or benchmark examples leak into training material. ## BrowseComp measures hard answer retrieval while deliberately separating it from open-ended browsing help [source 1](https://openai.com/index/browsecomp/) BrowseComp was designed around a particular asymmetry: locating a fact may require searching many sources, while verifying a proposed short answer can remain direct. OpenAI therefore used short, in-principle single-answer questions instead of long responses or disputed conclusions, and it says the benchmark does not aim to measure a common user-query distribution. This makes its 1,266 tasks suited to probing persistent clue-driven retrieval, not to scoring every part of a browsing assistant. The release also compares browsing policies, test-time effort and aggregation of repeated attempts, so a result belongs to an agent configuration as well as a model. Reading the benchmark this way preserves its method: it is evidence about controlled retrieval under its task construction, not a generic score for explanation quality or source judgment. ## Sources 1. [OpenAI: BrowseComp launch](https://openai.com/index/browsecomp/) 2. [OpenAI: simple-evals BrowseComp implementation](https://github.com/openai/simple-evals) ## Related reading - [OpenAI released PaperBench to evaluate agents replicating AI research](https://rangoon.ai/insights/archive-openai-paperbench-ai-research-replication-april-2025/) - [Terminal-Bench released version 2.0 with Harbor and renewed task verification](https://rangoon.ai/insights/archive-terminal-bench-two-harbor-verified-terminal-evaluation-november-2025/) - [OpenAI introduced SWE-Lancer to test coding agents against freelance software work](https://rangoon.ai/insights/archive-openai-swe-lancer-freelance-software-benchmark-february-2025/) ## More information - [Documentation index](https://rangoon.ai/llms.txt) - [AI access and policies](https://rangoon.ai/ai/) - [Source repository](https://github.com/hypler-dev/rangoon)