# Agent evaluation insights

Rangoon Insights reporting and guides tagged Agent evaluation.

Canonical: https://rangoon.ai/insights/tags/agent-evaluation/

## Topics
- [Agent systems](https://rangoon.ai/insights/topics/agents/): Agent runtimes, handoffs, tools, and the boundaries that make them reviewable.
- [Governance](https://rangoon.ai/insights/topics/governance/): Policy, identity, authorization, approvals, and evidence for consequential work.
- [Infrastructure](https://rangoon.ai/insights/topics/infrastructure/): Model serving, sandboxes, tool services, and deployment boundaries.
- [Models](https://rangoon.ai/insights/topics/models/): Inference choices, local serving, compatibility, and operational tradeoffs.

## Stories
- [Amazon released SOP-Bench for agent evaluation on business procedures](https://rangoon.ai/insights/archive-amazon-sop-bench-business-procedure-agent-evaluation-august-2026/)
  Amazon released SOP-Bench on August 21, 2026, combining expert-authored procedures, tools and ground-truth task outcomes across business domains.
- [Microsoft released STATE-Bench to test agent memory in stateful tasks](https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/)
  Microsoft released STATE-Bench on May 19, 2026, evaluating memory systems through stateful enterprise tasks, repeated runs and user interaction.
- [IBM Research launched an open agent leaderboard with Exgentic evaluation tooling](https://rangoon.ai/insights/archive-ibm-open-agent-leaderboard-exgentic-may-2026/)
  IBM Research launched the Open Agent Leaderboard on May 18, 2026, reporting quality and cost for full agent systems across shared benchmarks.
- [Anthropic released Petri for broad alignment audits, while its generated transcripts still needed human interpretation](https://rangoon.ai/insights/archive-anthropic-petri-open-source-auditing-october-2025/)
  Anthropic released Petri on October 6, 2025 as an open-source auditing tool that runs simulated multi-turn interactions and scores the resulting model behavior for review.
- [Meta introduced ARE and Gaia2 for asynchronous agent-environment evaluation](https://rangoon.ai/insights/archive-meta-are-gaia2-asynchronous-agent-evaluation-september-2025/)
  Meta introduced ARE and the Gaia2 benchmark on September 22, 2025, focusing agent evaluation on dynamic, asynchronous environments.
- [SWE-rebench proposed continuously collected software-agent tasks to limit benchmark contamination](https://rangoon.ai/insights/archive-swe-rebench-fresh-software-agent-evaluation-may-2025/)
  The May 26, 2025 SWE-rebench paper introduced an automated pipeline for fresh executable Python tasks and a benchmark designed to separate generalization from known public issues.
- [Sakana reported an AI-generated workshop manuscript passed review while documenting its preliminary limits](https://rangoon.ai/insights/archive-sakana-ai-scientist-v2-workshop-review-march-2025/)
  Sakana AI reported on March 12, 2025 that an AI Scientist-v2 manuscript passed an ICLR workshop review process, then published code and a report.
- [OpenAI’s low-resource coding research: validate behavior before migration](https://rangoon.ai/insights/openai-low-resource-coding-migration/)
  OpenAI reports progress on low-resource coding and COBOL migration. Analysis explains behavior checks, protected fixtures and release evidence for modernization teams.


## More information

- [Documentation index](https://rangoon.ai/llms.txt)
- [AI access and policies](https://rangoon.ai/ai/)
- [Source repository](https://github.com/hypler-dev/rangoon)