{
  "schema_version": "rangoon.article.v1",
  "id": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/",
  "url": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/",
  "markdown_url": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/index.md",
  "json_url": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/index.json",
  "title": "Microsoft released STATE-Bench to test agent memory in stateful tasks",
  "summary": "Microsoft released STATE-Bench on May 19, 2026, evaluating memory systems through stateful enterprise tasks, repeated runs and user interaction.",
  "language": "en",
  "author": {
    "name": "Rangoon Editorial",
    "url": "https://rangoon.ai/insights/editorial/"
  },
  "publisher": {
    "name": "Rangoon",
    "maintainer": "Hypler",
    "url": "https://rangoon.ai/"
  },
  "format": "News analysis",
  "topic": {
    "slug": "agents",
    "label": "Agent systems",
    "url": "https://rangoon.ai/insights/topics/agents/"
  },
  "tags": [
    {
      "slug": "microsoft",
      "label": "Microsoft",
      "url": "https://rangoon.ai/insights/tags/microsoft/"
    },
    {
      "slug": "ai-evaluation",
      "label": "AI evaluation",
      "url": "https://rangoon.ai/insights/tags/ai-evaluation/"
    },
    {
      "slug": "agent-evaluation",
      "label": "Agent evaluation",
      "url": "https://rangoon.ai/insights/tags/agent-evaluation/"
    },
    {
      "slug": "memory",
      "label": "memory",
      "url": "https://rangoon.ai/insights/tags/memory/"
    }
  ],
  "dates": {
    "published": "2026-10-24",
    "updated": "2026-10-24",
    "event": "2026-05-19"
  },
  "event_source_id": 1,
  "retrospective": true,
  "reading_minutes": 3,
  "takeaways": [
    "STATE-Bench assessed whether memory changes task outcomes, not merely whether an agent can retrieve an old fact.",
    "Repeated runs made consistency a distinct measure alongside average completion.",
    "Stateful task success still depends on domain procedures, simulated users, scorers and tool permissions."
  ],
  "source_note": "Historical reporting: Microsoft’s Open Source Blog announced STATE-Bench on May 19, 2026 as an open, memory-agnostic benchmark. The release describes 450 tasks across customer support, travel and shopping, pre-populated stateful environments, deterministic assertions for state-changing tasks, repeated-run reliability and user-experience grading. The linked public repository is implementation context. Microsoft’s baseline outcomes and simulator-variance estimate are publisher results under its task and agent setup; they do not prove that any memory architecture is reliable in another enterprise, policy or production database.",
  "sources": [
    {
      "id": 1,
      "title": "Microsoft Open Source Blog: STATE-Bench launch",
      "url": "https://opensource.microsoft.com/blog/2026/05/19/introducing-state-bench-a-benchmark-for-ai-agent-memory/",
      "published": "2026-05-19"
    },
    {
      "id": 2,
      "title": "Microsoft: STATE-Bench public repository",
      "url": "https://github.com/microsoft/STATE-Bench",
      "published": null
    }
  ],
  "image": {
    "url": "https://rangoon.ai/assets/editorial/archive-microsoft-state-bench-agent-memory-evaluation-may-2026-diagram-v1.webp",
    "thumbnail_url": "https://rangoon.ai/assets/editorial/archive-microsoft-state-bench-agent-memory-evaluation-may-2026-diagram-v1-thumb.webp",
    "alt": "Original evaluation explanatory diagram, Memory-equipped agent evaluated through repeated stateful runs. Labels: History, Memory, Agent, Tools, State, Score. Source boundary: dated primary release reporting; original editorial analysis.",
    "type": "original-diagram",
    "caption": "Memory-equipped agent evaluated through repeated stateful runs. Original Rangoon explanatory diagram: Source boundary: dated primary release reporting; original editorial analysis. No vendor interface, photograph, measurement result, or endorsement implied.",
    "credit": "Rangoon / Hypler",
    "source_url": "https://rangoon.ai/brand/",
    "rights_url": "https://rangoon.ai/terms/#media-use",
    "rights_note": "Hypler artwork permissions do not grant rights to third-party marks or imply endorsement."
  },
  "sections": [
    {
      "id": "may-launch",
      "url": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/#may-launch",
      "heading": "Microsoft made STATE-Bench publicly available on May 19, 2026",
      "paragraphs": [
        "Microsoft’s Open Source Blog dated the STATE-Bench announcement May 19, 2026 and called the release open-source and memory-agnostic. Microsoft framed the target question narrowly: whether an agent improves with experience on realistic enterprise tasks, rather than whether a memory layer retrieves a fact from a long conversation. The release lists 450 tasks across customer support, travel and shopping. This establishes a public benchmark and stated scope. It does not establish that memory will improve every agent, that a particular retrieval method is superior or that a live enterprise should permit an agent to alter records without its own controls and review."
      ],
      "bullets": [],
      "source_ids": [
        1
      ]
    },
    {
      "id": "stateful-loop",
      "url": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/#stateful-loop",
      "heading": "The task loop combined multi-turn interaction with stateful tools and bounded simulation",
      "paragraphs": [
        "Microsoft describes each task as a self-contained scenario with pre-populated records, a customer problem and deterministic state assertions. The orchestrator runs a multi-turn loop: the agent receives conversation history, calls tools and interacts with a user simulator that reveals information as needed. This is more than a text-retrieval setup because actions can affect a booking, order, cart or account state. The release says the simulator follows task-specific rules and estimates about one percent simulator-induced variance in testing. That estimate belongs to Microsoft’s implementation, not to all conversational-agent evaluation. A task can be reproducible in its supplied environment while differing from a real organization’s identity checks, exceptions and escalation paths."
      ],
      "bullets": [],
      "source_ids": [
        1,
        2
      ]
    },
    {
      "id": "four-metrics",
      "url": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/#four-metrics",
      "heading": "The release separated completion, repeated-run reliability, efficiency and user experience",
      "paragraphs": [
        "STATE-Bench reports four dimensions: task completion, pass-to-the-fifth-power reliability across five runs, efficiency and a user-experience score. For state-mutating tasks, Microsoft says a deterministic scorer compares final state with ground truth; other task types can use an LLM judge for expected process. This separation is a meaningful design choice. A one-off completion rate can conceal fragile behavior, while an efficient run can still provide a poor customer experience. The metrics also have different sources of uncertainty: repeated samples, simulator behavior, state assertions, judge instructions and cost accounting. A single composite headline would hide those choices, so a reader should retain the component results and conditions."
      ],
      "bullets": [],
      "source_ids": [
        1,
        2
      ]
    },
    {
      "id": "memory-design",
      "url": "https://rangoon.ai/insights/archive-microsoft-state-bench-agent-memory-evaluation-may-2026/#memory-design",
      "heading": "STATE-Bench tests memory through repeated stateful procedures rather than a one-off recall question",
      "paragraphs": [
        "STATE-Bench’s method centers on whether memory changes outcomes in pre-populated, stateful tasks. Microsoft runs each task five times, reports average completion and pass-to-the-fifth-power reliability, and uses deterministic final-state comparison for state-mutating tasks; procedural and informational tasks can use an LLM judge. This makes a memory comparison more demanding than retrieving a past chat fact. The same memory layer can change tool choices, the number of turns, consistency and the user interaction that precedes an action. Microsoft’s no-memory baseline is therefore a starting configuration for the supplied customer-support, travel and shopping environments, not a claim about every memory architecture. The meaningful comparison holds task setup, agent loop and scoring method visible while varying the memory component."
      ],
      "bullets": [],
      "source_ids": [
        1,
        2
      ]
    }
  ],
  "project_context": null,
  "related_links": [],
  "permissions_url": "https://rangoon.ai/permissions.txt",
  "content_hash": "sha256:8598a9e0e8148c768a4fea73822481e4a2c941c090cfe80f9892f3bf03459951"
}