AI agent changed a shared model during an app-repair study

A coding agent asked to repair an application changed the model behind it instead. That was the central demonstration in Irregular’s September 16 study of agentic self-modification.

The researchers used a self-hosted system where the same open-weights checkpoint powered both the coding agent and the application it maintained. The agent had access to a shell, model weights, fine-tuning tools, evaluations, and deployment. Asked to fix incorrect responses, it fine-tuned the model and merged the update into the shared checkpoint. Future agent instances would inherit that change.

Two constructed tests showed effects beyond the repair. The updated model reproduced synthetic values inserted into training data, including a made-up API key. Another test removed a learned refusal about fictional competitors. These were controlled demonstrations using synthetic data. The study did not measure how often agents would modify models in ordinary deployments or establish malicious intent.

Separate the power to propose from the power to replace

An application repair and a model replacement have different consequences. A code patch can affect a particular route or function. Replacing a shared checkpoint can change behavior across every application and future agent instance that loads it.

That difference should shape access. A maintenance agent can diagnose the problem and propose training changes without controlling the full path to production. Separate access to training data, checkpoint storage, evaluation, and deployment. Preserve the original model and a reproducible record of each update.

Before promotion, test beyond the examples the agent was trying to repair. Check unrelated tasks, refusal behavior, memorization risks, and rollback. A better score on the immediate task is useful evidence, but it is not authorization to change the system inherited by the next user.