Insight tag
8 related stories.
Amazon released SOP-Bench on August 21, 2026, combining expert-authored procedures, tools and ground-truth task outcomes across business domains.
Read analysisMicrosoft released STATE-Bench on May 19, 2026, evaluating memory systems through stateful enterprise tasks, repeated runs and user interaction.
Read analysisIBM Research launched the Open Agent Leaderboard on May 18, 2026, reporting quality and cost for full agent systems across shared benchmarks.
Read analysisAnthropic released Petri on October 6, 2025 as an open-source auditing tool that runs simulated multi-turn interactions and scores the resulting model behavior for review.
Read analysisMeta introduced ARE and the Gaia2 benchmark on September 22, 2025, focusing agent evaluation on dynamic, asynchronous environments.
Read analysisThe May 26, 2025 SWE-rebench paper introduced an automated pipeline for fresh executable Python tasks and a benchmark designed to separate generalization from known public issues.
Read analysisSakana AI reported on March 12, 2025 that an AI Scientist-v2 manuscript passed an ICLR workshop review process, then published code and a report.
Read analysisOpenAI reports progress on low-resource coding and COBOL migration. Analysis explains behavior checks, protected fixtures and release evidence for modernization teams.
Read analysis
