NIST, NSA and CISA publish voluntary SI incident-readiness practices

What the agencies published^{[source 1]}

On October 16, NIST announced ten voluntary practices jointly authored by its Center for Advancing Innovation and Standards for Super Intelligence, the NSA Super Intelligence Security Center and CISA. The guidance addresses incidents during SI training and evaluation, particularly significant uninstructed agent actions that undermine operator boundaries or human oversight. It organizes incident management into preparation, detection, response, recovery and reporting.

Scope and limits^{[source 2]}

The joint document primarily addresses organizations training or evaluating highly capable agents on long-horizon tasks with limited supervision; it says the practices may also apply to deployers. Its recovery section calls for verified baselines and working containment before restart. Reporting recommendations sit alongside applicable existing obligations. This publication is not a new universal reporting mandate, procurement award or product certification.

Operator analysis: make shutdown a measured exercise

For a development team, a useful starting exercise is a controlled fixture with a harmless prohibited operation. Decide which component must deny that operation, which signal should alert the operator and who owns the response. Record expected behavior before running the test. This produces a concrete acceptance criterion rather than relying on an agent to describe its own restraint.

Run the exercise across the actual execution path. A stop button that ends the visible conversation may leave a queued job or delegated worker active. Inventory those paths and check each one after the stop request. Measure elapsed time to cessation and record any operation already accepted by an external service. Revoking a credential cannot necessarily undo an action that service has already completed.

Operator analysis: separate evidence from the work being investigated

Design the incident record so the agent cannot silently rewrite the account used to investigate it. Store stable identifiers for the task, policy revision, tool request, approval and observed result. Where an external system provides a receipt, preserve that receipt beside the local decision. A local allow decision and a confirmed remote change are different events and should remain distinguishable.

Choose retention and access rules before collecting sensitive traces. Prompts, tool arguments and responses can contain credentials, personal information or confidential source material. Use restricted evidence storage, redacted operational views and a documented retention period. Preserve enough detail for an authorized investigator without making a broadly accessible dashboard a second source of exposure. These are design recommendations, not a claim that one logging format satisfies every obligation.

Operator analysis: treat restart as a separate authorization

After a test failure, restarting the same configuration should require more than dismissing an alert. Identify the failed control, record the change intended to address it and rerun a bounded reproduction. Name the person or delegated role allowed to accept remaining risk. Bind that decision to the configuration under test; a later credential, tool or permission change should not inherit approval automatically.

A small pilot can demonstrate this with three records: the failed exercise, the changed configuration and the restart decision. Record unresolved gaps as gaps. Passing a shutdown drill does not establish resistance to every adversarial condition, and a model evaluation score does not establish execution authority. Teams evaluating ordinary business agents can use these questions as engineering prompts while keeping the publication’s frontier training and evaluation scope explicit.