# agent benchmarks insights

Rangoon Insights reporting and guides tagged agent benchmarks.

Canonical: https://rangoon.ai/insights/tags/agent-benchmarks/

## Topics
- [Agent systems](https://rangoon.ai/insights/topics/agents/): Agent runtimes, handoffs, tools, and the boundaries that make them reviewable.
- [Governance](https://rangoon.ai/insights/topics/governance/): Policy, identity, authorization, approvals, and evidence for consequential work.
- [Infrastructure](https://rangoon.ai/insights/topics/infrastructure/): Model serving, sandboxes, tool services, and deployment boundaries.
- [Models](https://rangoon.ai/insights/topics/models/): Inference choices, local serving, compatibility, and operational tradeoffs.

## Stories
- [Amazon released SOP-Bench for agent evaluation on business procedures](https://rangoon.ai/insights/archive-amazon-sop-bench-business-procedure-agent-evaluation-august-2026/)
  Amazon released SOP-Bench on August 21, 2026, combining expert-authored procedures, tools and ground-truth task outcomes across business domains.
- [Terminal-Bench released version 2.0 with Harbor and renewed task verification](https://rangoon.ai/insights/archive-terminal-bench-two-harbor-verified-terminal-evaluation-november-2025/)
  Terminal-Bench released version 2.0 and Harbor on November 7, 2025, rebuilding terminal-agent tasks around stronger task verification.
- [Meta introduced ARE and Gaia2 for asynchronous agent-environment evaluation](https://rangoon.ai/insights/archive-meta-are-gaia2-asynchronous-agent-evaluation-september-2025/)
  Meta introduced ARE and the Gaia2 benchmark on September 22, 2025, focusing agent evaluation on dynamic, asynchronous environments.
- [OSWorld-Verified updated computer-use tasks, infrastructure and comparison rules](https://rangoon.ai/insights/archive-osworld-verified-computer-use-evaluation-july-2025/)
  XLANG Lab’s July 28, 2025 OSWorld-Verified release reported fixes to 300-plus issues, cloud parallelization and a refreshed basis for comparing computer-use agents.
- [Terminal-Bench launched an 80-task terminal-agent benchmark with containerized verification](https://rangoon.ai/insights/archive-terminal-bench-first-terminal-agent-release-may-2025/)
  Terminal-Bench’s May 19, 2025 launch introduced an 80-task terminal-agent set, dedicated Docker environments and tests that checked the state left by a configured agent run.
- [SWE-smith published a pipeline for turning Python repositories into software-agent training tasks](https://rangoon.ai/insights/archive-swe-smith-synthetic-software-agent-data-april-2025/)
  The April 30, 2025 SWE-smith paper described a pipeline that builds executable Python environments, injects test-breaking changes and produces large software-agent training sets.
- [OpenAI released BrowseComp for hard-to-find web information tasks](https://rangoon.ai/insights/archive-openai-browsecomp-browsing-agent-benchmark-april-2025/)
  OpenAI released BrowseComp on April 10, 2025, with 1,266 short-answer tasks designed to test persistent web browsing and verification.
- [OpenAI released PaperBench to evaluate agents replicating AI research](https://rangoon.ai/insights/archive-openai-paperbench-ai-research-replication-april-2025/)
  OpenAI released PaperBench on April 2, 2025, using ICML papers, hierarchical rubrics and an open implementation to evaluate research replication.
- [METR proposed task-completion time horizons for measuring long software work](https://rangoon.ai/insights/archive-metr-task-completion-time-horizons-march-2025/)
  METR published its time-horizon study on March 19, 2025, relating agent success to expert human task duration across multi-step tasks.
- [Epoch AI introduced FrontierMath with unpublished expert mathematics problems](https://rangoon.ai/insights/archive-epoch-frontiermath-expert-mathematics-benchmark-november-2024/)
  Epoch AI introduced FrontierMath on November 8, 2024, pairing original expert mathematics problems with automated answer verification.
- [Meta published PARTNR to test planning and coordination in simulated human-robot household tasks](https://rangoon.ai/insights/archive-meta-partnr-embodied-agent-benchmark-october-2024/)
  Meta published the PARTNR embodied-agent benchmark on October 31, 2024, with 100,000 simulated household tasks designed to expose planning and coordination limits.


## More information

- [Documentation index](https://rangoon.ai/llms.txt)
- [AI access and policies](https://rangoon.ai/ai/)
- [Source repository](https://github.com/hypler-dev/rangoon)