# Sakana reported an AI-generated workshop manuscript passed review while documenting its preliminary limits Sakana AI reported on March 12, 2025 that an AI Scientist-v2 manuscript passed an ICLR workshop review process, then published code and a report. Canonical: https://rangoon.ai/insights/archive-sakana-ai-scientist-v2-workshop-review-march-2025/ Author: Rangoon Editorial (https://rangoon.ai/insights/editorial/) Published: 2026-10-24 Updated: 2026-10-24 Event date: 2025-03-12 Tags: [Sakana AI](https://rangoon.ai/insights/tags/sakana-ai/), [AI Scientist](https://rangoon.ai/insights/tags/ai-scientist/), [scientific publishing](https://rangoon.ai/insights/tags/scientific-publishing/), [Agent evaluation](https://rangoon.ai/insights/tags/agent-evaluation/) Source note: Historical reporting: Sakana AI’s March 12, 2025 post reported that a manuscript generated by AI Scientist-v2 passed an ICLR workshop peer-review process conducted with workshop and ICLR cooperation. The post says the team later released a technical report and code, and also says none of the three submissions met its internal standard for the ICLR conference track; the described paper was withdrawn before meta-review. The technical-report abstract describes template-free hypothesis generation, experiment execution, analysis and manuscript generation under progressive agentic tree search. The repository warns that the code executes LLM-written code and may use dangerous packages, uncontrolled web access and unintended processes. Workshop review and a code release do not independently validate research claims or authorize unsandboxed execution. Image: Constrained automated-research loop. Original Rangoon explanatory diagram: Source boundary: dated primary release reporting; original editorial analysis. No vendor interface, photograph, measurement result, or endorsement implied. ## Key takeaways - The reported review outcome concerned a workshop experiment, not an ICLR conference-track acceptance or a settled scientific result. - Sakana’s own account records citation errors and an internal quality bar the generated manuscripts did not meet. - A research agent that writes and runs code needs a constrained execution environment and human review of methods and results. ## Sakana dated its workshop-review report to March 12 and described the experimental setting [source 1](https://sakana.ai/ai-scientist-first-publication/) On March 12, 2025, Sakana AI reported that a paper produced by its AI Scientist-v2 had passed the peer-review process at an ICLR workshop. The post says the workshop organizers and ICLR leadership cooperated in an experiment with three AI-generated submissions, while reviewers were told that AI-generated manuscripts could be among the papers. That context narrows what the event means. A workshop review experiment can test how a manuscript fares under a defined process; it is not a finding that the system has established a durable scientific discovery. Sakana says the generated paper investigated a negative result about compositional regularization. The source supports reporting the stated review outcome and process, not treating the manuscript as an independently confirmed research conclusion. ## The publisher disclosed that its own review found preliminary work and citation mistakes [source 1](https://sakana.ai/ai-scientist-first-publication/) Sakana’s report contains an important qualification often lost in a headline about peer review. It says the team assessed all three generated papers against an ICLR conference-track standard and concluded that none met its internal bar in their current form. It also describes an erroneous citation of an LSTM-based neural network and says the team encouraged repeat experiments to improve scientific accuracy, reproducibility and statistical rigor. The post further says the reported manuscript was withdrawn after workshop review, so it did not receive a further meta-review. These disclosures make the event more informative, not less: the system produced a paper that cleared one configured workshop process while its authors still identified material limitations. A review score is evidence about that process, not a replacement for verifying methods, citations, data and claims. ## The technical report described an experiment manager and tree search, not a universal research method [source 2](https://pub.sakana.ai/ai-scientist-v2/paper/paper.pdf) Sakana’s technical report presents AI Scientist-v2 as an end-to-end system that formulates hypotheses, designs and executes experiments, analyzes and visualizes data, and writes manuscripts. It says the v2 design removes reliance on human-authored code templates and uses a progressive agentic tree search with an experiment-manager agent. Those are design and evaluation claims for the reported system. They do not show that every proposed experiment is scientifically suitable, that a generated comparison uses representative data, or that an output generalizes beyond the machine-learning domains tested by the authors. Scientific work still needs a question with operational definitions, data provenance, controls, uncertainty analysis and a route for independent reproduction. Automation can expand the search space while also expanding the need to inspect the paths it explored. ## The open repository warns that the system can execute LLM-written code and needs containment [source 1](https://sakana.ai/ai-scientist-first-publication/) [source 3](https://github.com/SakanaAI/AI-Scientist-v2) The AI Scientist-v2 repository explicitly warns that the code executes LLM-written code and identifies risks including dangerous packages, uncontrolled web access and unintended processes. It recommends a controlled sandbox such as Docker. That warning is directly relevant to the research workflow: an agent that proposes an experiment and then runs generated code has a different security boundary from a text-only assistant. A safe evaluation should restrict filesystem and network access, bound compute and subprocess creation, record dependencies and commands, and require review before any result reaches a shared environment or external system. The March event was about a research-and-publishing experiment and an associated release. It did not demonstrate safe general deployment, autonomous authority over resources, or correctness of every generated experimental result. ## Sources 1. [Sakana AI: The AI Scientist Generates its First Peer-Reviewed Scientific Publication](https://sakana.ai/ai-scientist-first-publication/) 2. [Sakana AI: The AI Scientist-v2 technical report](https://pub.sakana.ai/ai-scientist-v2/paper/paper.pdf) 3. [SakanaAI GitHub: AI-Scientist-v2 repository](https://github.com/SakanaAI/AI-Scientist-v2) ## Related reading - [SWE-rebench proposed continuously collected software-agent tasks to limit benchmark contamination](https://rangoon.ai/insights/archive-swe-rebench-fresh-software-agent-evaluation-may-2025/) - [Microsoft released STATE-Bench to test agent memory in stateful tasks](https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/) - [Meta introduced ARE and Gaia2 for asynchronous agent-environment evaluation](https://rangoon.ai/insights/archive-meta-are-gaia2-asynchronous-agent-evaluation-september-2025/) ## More information - [Documentation index](https://rangoon.ai/llms.txt) - [AI access and policies](https://rangoon.ai/ai/) - [Source repository](https://github.com/hypler-dev/rangoon)