AI agent incident response: a practical playbook

An AI agent has sent a message, changed a record, opened a pull request, or run a tool call that was not authorized. Start by establishing the system effects from independent records.

An agent can act through identities and integrations that already have access to files, APIs, cloud services, repositories, or business systems. A reassuring final answer does not show which data the agent read, what requests it sent, or whether another workflow continued the task. Establish the scope from independent system records.

This playbook is an operational adaptation of general incident-response and agent-security guidance. NIST SP 800-61 Rev. 3 integrates incident response into broader cybersecurity risk management; OWASP’s agent guidance emphasizes authorization at the tool-execution boundary, approvals for high-impact actions, and monitoring of calls and outcomes. Adapt the steps to your incident plan, system architecture, and legal obligations.

1. Stop additional actions while preserving useful evidence

Pause the affected run, queue, workflow, or tool connection at the narrowest boundary that stops additional side effects. If a process or session is still active, decide with the incident lead whether to isolate it, suspend it, or capture volatile evidence first. Avoid deleting the session or wiping the environment before logs and state have been preserved when that can be done safely.

If the agent can start parallel runs, call subagents, schedule tasks, or trigger downstream automation, include those paths in containment. A chat window that has stopped responding does not prove that background jobs or integrations have stopped.

Record the time and control used to pause the activity, who approved the action, and what may remain in flight. If you cannot stop a specific integration immediately, restrict its credentials or network path through the system owner’s normal emergency process.

2. Contain the identities and credentials it could use

Identify the user, service account, API key, OAuth grant, session token, CI credential, or cloud role associated with the run. Revoke or rotate credentials that may be exposed or still usable, and check whether the same credential is shared by unrelated jobs before disabling it broadly. Stop refresh-token or session renewal where applicable.

Do this in the identity provider or service that issued the credential. Telling the model to “stop,” removing a tool from a prompt, or closing the user interface may not invalidate credentials already held by a worker or integration. Preserve the credential’s identifier and ownership record for investigation, but do not copy secret values into tickets or incident notes.

3. Preserve records before cleanup

Capture records from the systems that received the agent’s requests, not only the conversation transcript. Record timestamps with time zone, relevant session or job IDs, and how the evidence was collected. Follow your organization’s evidence-handling and retention process.

Useful sources may include:

• Agent run history and tool-call traces: What was requested, which tools were called, and what results were returned?
• Identity-provider, API gateway, and application audit logs: Which principal authenticated, what resource was accessed, and what actions succeeded?
• Repository, CI/CD, cloud, database, ticket, and messaging audit logs: What changed outside the agent interface, and what identity made the change?
• Egress, DNS, proxy, and endpoint telemetry: Did the process communicate with destinations outside its expected workflow?
• Approval records and policy versions: Was an action approved, by whom, for which exact target and parameters, and under which policy?

NIST SP 800-86 describes files, operating systems, network traffic, and applications as potential forensic data sources. OWASP recommends logging agent decisions, tool calls, and outcomes, and capturing structured authorization and approval details for high-impact actions. If a source was not logging those details, record that gap; do not fill it with assumptions.

4. Map what was read, changed, sent, or delegated

Build a timeline from system records. Answer four questions:

• Access: Which data or resources could the run read, and which did the records show it actually accessed?
• Change: Which files, records, permissions, repositories, messages, or cloud resources changed?
• Transfer: Is there evidence that data left the expected environment? Check relevant outbound requests and receiving-system logs. A model’s displayed answer alone cannot establish whether data was or was not transmitted.
• Delegation: Did another agent, worker, scheduled task, extension, workflow, or integration receive the task or credentials?

Treat capability and evidence separately. A credential available to a process raises the exposure question; it does not prove that every resource reachable with it was accessed. Likewise, an agent’s statement that it did not send data is not a substitute for service-side records.

5. Restore and validate from a trusted state

Ask the system owner to classify each side effect as reversible, irreversible, or still uncertain. Revert changes or restore data from a known-good source only after preserving the relevant evidence and checking that the restoration will not erase needed records. Review linked workflows and permissions for changes that could persist after the original run ends.

Before returning the integration to service, confirm that the exposed credential is disabled or replaced, the agent’s permissions match the intended task, and the affected system’s state is understood. For high-impact actions, require an independent policy or execution layer to check authorization and approval for the exact action; a model-generated confirmation is not itself an authorization decision.

6. Communicate confirmed facts and open questions

Use the organization’s incident process to notify security, platform, data, and service owners. Bring in privacy, legal, compliance, or communications teams when the affected data, contracts, or applicable rules require it. Share what is confirmed, what remains unknown, the systems and identities in scope, and what containment is complete. Avoid declaring a breach, root cause, or customer impact until evidence supports it.

7. Close the loop with a control change

After recovery, identify which boundary failed: excessive access, an untrusted input reaching a tool, an approval that was not bound to the action, missing logs, an overbroad credential, or a downstream workflow that continued after the agent was paused. Change the control at that boundary and test the change in a controlled environment. Prompt wording can help set expectations, but it does not replace authorization enforced where the action executes.

First-response checklist

• Open the normal incident process and assign an incident lead.
• Pause the affected run or integration; check parallel and downstream work.
• Preserve run IDs, tool calls, approvals, identity records, and system-side audit logs.
• Revoke or rotate credentials that may still be usable or exposed.
• Inventory data read, external changes, outbound requests, and delegated activity.
• Restore or repair affected systems from a trusted state after evidence is preserved.
• Validate permissions, approvals, logs, and downstream jobs before resuming.
• Communicate confirmed facts and unresolved questions through the established process.

Scope and limitations

This checklist helps organize an initial response; it is not a substitute for an organization’s incident-response plan, forensic procedures, or legal review. Available evidence varies by model provider, agent framework, tool, and connected service. If logs do not record tool inputs, outputs, identities, or outcomes, responders may be unable to reconstruct the full sequence. Do not infer a specific attack technique solely from an unexpected action.

Related incident evidence

The Hugging Face evaluation-agent investigation and Cline publishing-token incident are case studies to compare with your own records. Each has its own facts and limits; neither establishes the cause of another unexpected agent action.