Insight tag
5 related stories.
The May 26, 2025 SWE-rebench paper introduced an automated pipeline for fresh executable Python tasks and a benchmark designed to separate generalization from known public issues.
Read analysisThe April 30, 2025 SWE-smith paper described a pipeline that builds executable Python environments, injects test-breaking changes and produces large software-agent training sets.
Read analysisAmazon introduced SWE-PolyBench on April 23, 2025, with 2,110 repository issues across Java, JavaScript, TypeScript and Python.
Read analysisOpenAI introduced SWE-Lancer on February 18, 2025, using freelance software tasks, end-to-end tests and manager-choice evaluations.
Read analysisOpenAI introduced Codex CLI on April 16, 2025 as an open-source terminal coding agent, linking local repository access with multimodal reasoning.
Read analysis
