# OSWorld-Verified updated computer-use tasks, infrastructure and comparison rules XLANG Lab’s July 28, 2025 OSWorld-Verified release reported fixes to 300-plus issues, cloud parallelization and a refreshed basis for comparing computer-use agents. Canonical: https://rangoon.ai/insights/archive-osworld-verified-computer-use-evaluation-july-2025/ Author: Rangoon Editorial (https://rangoon.ai/insights/editorial/) Published: 2026-10-24 Updated: 2026-10-24 Event date: 2025-07-28 Tags: [OSWorld-Verified](https://rangoon.ai/insights/tags/osworld-verified/), [computer use](https://rangoon.ai/insights/tags/computer-use/), [agent benchmarks](https://rangoon.ai/insights/tags/agent-benchmarks/) Source note: Historical reporting: Source 0 is XLANG Lab’s dated July 28, 2025 OSWorld-Verified announcement. Source 1 is XLANG’s official project repository, reviewed as current technical documentation on October 24, 2026. It provides environment and execution detail but is not used to backdate current setup choices into the event. Image: Verified tasks run inside a versioned desktop environment. Original Rangoon explanatory diagram: Source boundary: dated primary release reporting; original editorial analysis. No vendor interface, photograph, measurement result, or endorsement implied. ## Key takeaways - OSWorld-Verified reported fixes for more than 300 community-raised issues and asked readers to compare latest-version results with the refreshed benchmark. - The release changed both task quality and operating infrastructure, including AWS-based parallelization, so version identity matters for score comparison. - The repository’s VM, Docker and KVM instructions show that host virtualization, system images and setup behavior can shape a computer-use result. ## The update treated benchmark maintenance as research work [source 1](https://xlang.ai/blog/osworld-verified) On July 28, 2025, XLANG Lab announced OSWorld-Verified as a major refinement of its computer-use benchmark. The dated post says a roughly 10-person team spent about two months collecting, verifying, validating and fixing more than 300 pieces of community feedback. It names web changes, ambiguous instructions and evaluation robustness among the issues addressed. The post also asks readers to compare results against the new verified results when using the latest version. This is a substantive release event because it changes the task and measurement baseline, not merely a website label. A score from the original OSWorld and a score from OSWorld-Verified may differ because the benchmark, its environments or its checks changed, even if the evaluated agent remained unchanged. ## AWS parallelization changed the operating envelope [source 1](https://xlang.ai/blog/osworld-verified) The release post presents OSWorld-Verified as both a task-quality and infrastructure update. It says the project moved from VMware and Docker toward AWS support, reporting a change from more than 10 hours to minutes through 50-fold parallelization. It also announced a public evaluation platform intended to support verified like-for-like comparisons. Faster execution does not by itself mean harder or easier tasks, but it can alter practical sampling, retry discipline, failure observation and evaluation cost. Those variables matter when teams compare reported success rates. The event supports the publisher’s infrastructure claims and its stated measurement goal. It does not demonstrate that every cloud run is identical, that each platform yields the same graphics, timing or application behavior, or that a public platform independently audits every submitted system. ## A real desktop benchmark still has a host contract [source 1](https://xlang.ai/blog/osworld-verified) [source 2](https://github.com/xlang-ai/OSWorld) XLANG’s official repository describes several execution paths rather than one universal host: VMware or VirtualBox for desktop hardware, Docker for KVM-capable servers, and cloud providers for parallel runs. Its current instructions require Python 3.10 or newer, identify Ubuntu or Windows as Docker experiment choices, and warn that interrupted Docker experiments can leave residual containers. The repository also says some tasks require Google-account access and OAuth configuration, while other tasks can need proxy settings depending on site defenses and network location. These details draw the boundary of an execution environment. A computer-use score relies on more than screenshots and clicks: virtualization provider, operating-system image, application state, account setup, network configuration and cleanup can all affect whether an agent reaches a task or a verifier sees a result. ## Verified examples did not make computer actions generally reliable [source 1](https://xlang.ai/blog/osworld-verified) [source 2](https://github.com/xlang-ai/OSWorld) XLANG Lab called OSWorld-Verified a more stable and scalable foundation for computer-use research, while also saying the benchmark remained challenging and had room to improve toward human-level performance. That is a measured claim about a maintained evaluation artifact, not evidence that an agent can be trusted with unbounded desktop access. The announcement acknowledges the need to repair dynamic websites, unclear instructions and fragile evaluation signals; these are precisely the kinds of conditions that can reappear in production systems. Results must therefore retain the verified release revision, task selection, operating platform, run policy and evaluator. The July event provides dated evidence of a benchmark maintenance release and its stated scope. It does not provide authorization for an agent to access an account, guarantee correct interpretation of a visual task, or establish reliability in a different application fleet. ## Sources 1. [XLANG Lab: Introducing OSWorld-Verified](https://xlang.ai/blog/osworld-verified) 2. [XLANG: OSWorld official project repository](https://github.com/xlang-ai/OSWorld) ## Related reading - [Terminal-Bench launched an 80-task terminal-agent benchmark with containerized verification](https://rangoon.ai/insights/archive-terminal-bench-first-terminal-agent-release-may-2025/) - [Terminal-Bench released version 2.0 with Harbor and renewed task verification](https://rangoon.ai/insights/archive-terminal-bench-two-harbor-verified-terminal-evaluation-november-2025/) - [SWE-smith published a pipeline for turning Python repositories into software-agent training tasks](https://rangoon.ai/insights/archive-swe-smith-synthetic-software-agent-data-april-2025/) ## More information - [Documentation index](https://rangoon.ai/llms.txt) - [AI access and policies](https://rangoon.ai/ai/) - [Source repository](https://github.com/hypler-dev/rangoon)