Insight tag
13 related stories.
Anthropic released Petri on October 6, 2025 as an open-source auditing tool that runs simulated multi-turn interactions and scores the resulting model behavior for review.
Read analysisMicrosoft announced its AI Red Teaming Agent public preview on April 4, 2025, bringing PyRIT-based adversarial testing to Foundry evaluation, reporting and logging surfaces.
Read analysisOpenAI introduced GPT-5.2 on December 11, 2025, describing Instant, Thinking, and Pro variants for different kinds of work.
Read analysisScale AI introduced SWE-Bench Pro on September 19, 2025, a 1,865-problem coding-agent benchmark spanning public, held-out, and commercial repositories.
Read analysisMistral introduced Medium 3 on May 7, 2025. Its benchmark, price, and deployment claims require a workload-specific quality and cost basis.
Read analysisARC Prize launched ARC-AGI-2 on March 24, 2025, with grid-based reasoning tasks, 120-task evaluation sets, pass@2 scoring, and cost reporting.
Read analysisScale AI and the Center for AI Safety released Humanity’s Last Exam on January 23, 2025: 3,000 multimodal, closed-ended academic questions.
Read analysisDeepSeek submitted its DeepSeek-R1 paper to arXiv on January 22, 2025. Its account of reinforcement learning and distillation called for claim-by-claim model evaluation.
Read analysisGoogle DeepMind’s January 6, 2025 FACTS Grounding paper described a 1,719-example benchmark for long-form answers tied to supplied documents.
Read analysisDeepSeek documented a DeepSeek-V3 upgrade for deepseek-chat on December 26, 2024. A stable endpoint name does not preserve checkpoint identity, prompt behavior, or evaluation.
Read analysisMicrosoft introduced Phi-4 on December 12, 2024 as a 14-billion-parameter small language model. Its training account made data and curriculum review part of evaluation.
Read analysisAmazon introduced the Nova family on December 3, 2024. Its text, multimodal, image, and video models made an input-output map more useful than one family label.
Read analysisAnthropic’s engineer-training commitment raises a practical question for development firms: what evidence shows that an AI workflow is ready for a client handoff?
Read analysis
