Meta published PARTNR to test planning and coordination in simulated human-robot household tasks

The October research record defined PARTNR as a benchmark for collaboration^{[source 1][source 2]}

On October 31, 2024, Meta published PARTNR, short for Planning And Reasoning Tasks in humaN-Robot collaboration. The research page describes a benchmark for household activities in which tasks have spatial, temporal and heterogeneous-agent capability constraints. It reports 100,000 natural-language tasks across 60 houses and 5,819 objects. Those figures describe the scale and composition of a simulated research environment. They do not mean a robot has encountered that many real homes or demonstrated reliable behaviour with the full range of everyday objects.

The release framed the work around coordination between a human and a robot, not around an autonomous machine completing every task alone. That distinction matters because a collaborative system must decide who can do which part, maintain task state and respond when an earlier step changes the plan. Meta’s source records support analysis of those research questions. They do not establish how a commercial robot would handle privacy, physical safety, accessibility, liability, hardware faults or a person who gives an ambiguous instruction in an unscripted home.

Simulation-in-the-loop generation was intended to ground a large task set^{[source 1][source 3]}

Meta says PARTNR used a semi-automated pipeline in which language models generated tasks and simulation was kept in the loop for grounding and verification. The project page describes the same design as a way to reduce generation errors while creating tasks at scale. This is a method for building an evaluation set, not evidence that the language models performing the generation solve the benchmark tasks. A grounded simulation can catch some impossible or inconsistent tasks while still differing from sensing, movement and human communication in a real household.

The benchmark was designed to expose failures as well as measure progress^{[source 1][source 2]}

Meta’s research record says its analysis of state-of-the-art language models found poor coordination and failures in task tracking and recovery from errors. The project page reports that people solved 93 percent of tasks while the evaluated language-model planners solved 30 percent. These measurements are useful precisely because they describe shortcomings under a defined protocol. They should not be treated as a general score for every model, a prediction of one robot’s success rate or a claim that a human collaborator eliminates failure in a physical setting.

The paper also reports comparisons where language models paired with real humans required more steps than two humans and more steps than a single human, and describes a smaller fine-tuned planner matching a much larger model in its tested setting. Such results can motivate further research on planning data and task division. They remain conditional on the environment, planner, human-in-the-loop method, model version and metric. A speed or step comparison does not resolve whether a physical robot has enough perception, force control or safe recovery behaviour for a particular job.

A benchmark release did not certify a household robot or a human-robot workflow^{[source 1][source 2][source 3]}

This account does not use later coverage, later hardware demonstrations or updated model results to enlarge the October 2024 event. It records a specific research milestone: Meta published a benchmark intended to reveal limitations in human-robot coordination and planning. The appropriate conclusion is that PARTNR provided a structured way to evaluate those limitations, not that it certified any model or robot for household work, established an autonomous product, or showed that benchmark performance transfers without further testing and safety controls.