Give an autonomous coding agent an acceptance contract
Autonomous agents can make large edits quickly. Without a contract, an impressive diff can still be the wrong change. A single scoped issue, a captured baseline, clear tests, a performance budget, an explicit review artifact, and a rollback path make evaluation tractable.
Start from one scoped issue
Limit the agent to one issue with a clear intended behavior change. The issue should name the failing behavior, expected behavior, and affected area without prescribing a full implementation. Include constraints such as files to avoid, interfaces that must stay stable, and any compatibility requirement. Reject vague goals like make it faster or improve the codebase. A scoped issue prevents wide refactors that obscure regressions. It also makes rollback meaningful because the boundary of intent is known. If the task naturally splits, create separate contracts rather than one combined ticket.
Capture the baseline dirty tree
Record the exact repository state before the agent edits anything. Record the current branch and commit hash, save a patch of tracked uncommitted changes, and copy untracked files separately without moving or modifying the original worktree. Treat the existing dirty tree as owned work that must be preserved, not overwritten. Require the agent to base changes on that state and to report any assumption about pre existing edits. This avoids blaming the agent for pre existing failures and avoids losing user work. If the baseline already has failing tests, document them explicitly so they are not falsely counted as regressions introduced by the agent.
Define a regression test the agent must beat
Define a test that fails before the change and must pass after. Prefer a focused regression test over a broad integration suite when possible, because broad suites add noise. State the exact command to run and the expected result. Require the agent to show before and after output rather than asserting success. Add negative cases where relevant, such as unchanged behavior for unrelated inputs. If tests are hard to write, define an observable contract with manual steps and expected outputs, labelled illustrative where no automation exists. A test turns subjective review into a repeatable check.
Set a performance budget
Set quantitative limits before execution and enforce them. Separate product performance budgets, such as request latency or memory use, from task scope budgets such as maximum runtime or changed-file count. Choose numbers from existing baseline measurements instead of guesses. State what happens when a budget is exceeded, such as stopping work and reporting partial progress. Budgets protect repositories from runaway changes and make comparison fair across attempts. They also force attention on efficiency rather than only on passing tests. Do not invent pricing or cost claims that the tooling does not verify; measure local resources only.
Require a review artifact
Require a written artifact before acceptance. The artifact should summarise intent, scope of edits, test evidence, budget compliance, and remaining risks. Include the list of changed files and any assumptions the agent made. Ask for known limitations and areas not covered. A structured artifact supports later review and reduces hidden context. It should be treated as a proposal, not proof, since the agent can misunderstand intent even while tests pass.
Plan rollback before merge
Define how to undo the change before applying it. Options include a feature branch, a clean revert of one commit, or restoring an isolated test copy from the baseline archive. Require the agent to keep its changes isolated from unrelated work. Verify that rollback restores the baseline test state. Do not merge until rollback has been exercised at least once on a copy. Knowing the exit route keeps experiments safe and makes adoption a deliberate decision rather than an accident.
Checklist
- Write the issue to one behavior change with explicit constraints.
- Capture branch, commit, and uncommitted changes before the agent runs.
- Specify a failing regression test and its exact run command.
- Set one measurable performance or scope budget and a stop rule.
- Require a review artifact and verify rollback on a copy before merge.
Takeaway
An acceptance contract turns fast agent output into reviewable engineering. Narrow scope, a captured baseline, a regression test, a budget, a documented review, and a rollback plan each reduce the risk of an impressive but wrong change. None of them guarantees correctness.
Further Reading
Try it on your own code
Give a coding agent a task, a repo and a cloud machine. Review the diff when it is done.
Start free