Claude Code team has Claude test the app before committing

Anthropic’s Claude Code team stamps every commit in its own repos with how it was verified, and “none” is an allowed answer so unverified commits can be found later, the team wrote in a November 16 post on Anthropic’s blog. The steps to start the app or simulator go into a skill named verify, kept in a .claude/skills/ folder in the repo, and Claude knows to run it before committing changes.
Claude writes and reviews about 90% of the team’s code. Without checks an agent can run on its own work, the post says, you can only run one or two Claudes at a time, because you have to check everything yourself; with them, dozens or hundreds can run in parallel. A second opinion also pays: in internal testing around structured file generation, having a separate grader check Claude’s work against a rubric improved task success by up to 10 points over a standard prompting loop, with the largest gains on the hardest problems. Anthropic’s guide to verification loops lists a grader of that kind among its options: Rubrics in Claude Managed Agents, in beta, which uses a separate grader agent.
Making verify worth running
The skill is meant to use the app the way a human or QA engineer would. The team has Claude use the Claude Chrome extension for web changes, an iOS or Android MCP for mobile, computer use for desktop, or run the service locally for backend work. Each skill should sit as close to the relevant code as possible, and a repo can hold as many as it needs. The advice is to have Claude iterate on the skill and stress test it until it is fast and reliable, since it serves as the smoke test harness and should be “rock solid.”
Cheaper checks come before it. On the team, Claude runs the tests, build and linter before opening a pull request, because an agent that pushes untested code spends shared review and CI time on a bug it could have caught itself. The post names two ways to enforce that: a hook that runs the tests when Claude finishes, or a /goal of a passing test suite.
Repeated failures turn into rules. If an agent keeps failing a test that checks logs for user data, for example, the post suggests a hook or a more explicit rule in CLAUDE.md, which “(mostly)” stops the issue from coming back.
Which tests to write
If you pick one kind, the team says, make it end-to-end tests, which walk through a real user flow such as adding an item to a cart, checking out and landing on a confirmation page. Either the whole feature works or it doesn’t, so they are hard to fool. Unit tests, which check small pieces in isolation, are faster and less flaky but can pass while the feature is broken: where one might report “Error: Timeout after 30 seconds,” an end-to-end test can return a screenshot with a description of what’s broken.
Two more types make the list. Property tests start from a statement that should never fail, like “account balance can never become negative,” and use a library (fast-check for JS/TS, Hypothesis for Python, proptest for Rust) to throw thousands of generated inputs at it, edge cases that don’t share the bias of the agent that wrote the code. Differential tests suit migrations: old and new systems get the same inputs, randomly generated or sampled from production traffic, and any difference that isn’t on a list of approved changes is a bug or a breaking change.
Whatever the type, flaky tests, the ones that pass or fail at random, teach the agent to rerun them instead of fixing the code. Slow ones mean fewer test runs, which makes problems harder to pinpoint.
When a human gets pulled in
Claude runs the checks and tries to fix whatever fails. If a check is still failing after three rounds, Claude stops and brings in a human, and the check should say why it pulled in a reviewer, so nobody has to work it out from the diff. The people who own the code write its checks and refine them until they flag what the owner would flag.
Past outages live in a reliability checklist of 39 items, each with a trigger that can be read from a diff, a check, and the incidents or PRs behind it. One is “--resume loads old transcripts”: when session storage or message types change, it asks whether a new field loads safely, since older transcripts won’t have it. Claude reads the diff and checks only the items whose trigger matches, usually 3 to 10.
Changes can’t edit the rules that judge them. Code Review reads REVIEW.md, the file that tells the reviewer what to watch in this code, from the base or default branch, never from the pull request. The team’s commit gate, which can block a commit if the session did not include /code-review or /verify, has off, observe and enforce modes; observe only logs what would have been blocked, so false positives can be reviewed before anyone is stopped. Its escape hatch is refreshingly blunt: once two commits have been blocked in a session, the next one goes through so Claude doesn’t get stuck.
Reviews are also reserved for commits, since a push during a review makes its result stale. In the team’s words, “A passing review on an out-of-date commit means nothing.”
Up next
Claude Code 2.1.326 runs Code Mode with agent teams on
Developers running a team of Claude agents in the terminal can switch on Code Mode too, where Claude writes JavaScript that calls its own tools, in the latest Claude Code. Would you let Claude script its own tool calls, or approve each one yourself?
Related stories
- Claude Code team hands most code reading to Claude
- Claude Code team defines a loop by its stop condition
- Claude Code writes the eval, grader included
- Claude went from bug hunt to abuse report on New Year’s Eve
- Claude Code sessions keep running after you close the laptop
- Claude Code reads AGENTS.md when CLAUDE.md is missing
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

