pairmark races Claude Code against Codex in your repo

Pairmark races Claude Code and Codex on the same task inside one repository and settles the outcome with five verdict rules applied in order. Published on Github, it runs from a single npx command and writes one self-contained report.html per run. It uses the Claude Code and Codex subscriptions already signed in on the machine, with no API keys, no accounts and no server, and pairmark itself makes no network calls.

At a glance

• Both CLIs start at the same moment from the same commit, Claude Code as claude -p --output-format stream-json and Codex as codex exec --json, with their streams normalised into one event model.
• The tool needs Node 20 or newer and a JavaScript or TypeScript repository; defaults give each agent 20 minutes, each check and each judge 10 minutes, and the live page port 4747.
• Checks outrank judges: a patch that passes every check without touching check configuration wins outright, while a patch editing package.json, tsconfig or test config is flagged and can only win on the judges.

Arguments about which coding agent is better are usually settled by anecdote, because collecting evidence means duplicating worktrees, permission flags and test runs by hand. Pairmark's value appears to be less the winner it names than the artefact it leaves behind: a diff, a check log and two judge cards from one specific repository on one specific day. The project frames a single run as something worth posting and ten runs as a basis for a decision.

Three detached worktrees named alpha, beta and base keep authorship out of every path a judge reads

Each run creates a directory at .pairmark/runs/<timestamp-id>/ inside the repository and adds it to .git/info/exclude, so git status stays clean. Three detached worktrees are cut at HEAD: alpha and beta for the agents, base for the judges. The neutral names mean no check log reveals whose patch it is.

If the working tree is dirty, uncommitted and untracked files are snapshotted into each worktree and committed as a baseline, recorded as baseline.patch. node_modules is cloned copy-on-write where APFS, btrfs or xfs allow it, with a plain copy or a frozen install as fallbacks.

Both agents receive identical text: the task plus a short working agreement telling them to work locally, not commit, and not weaken tests, lint rules or type checks. The brief never mentions the other agent and never asks the agent to run the project's checks, which pairmark runs itself afterwards.

Five rules decide the run, and a judge split is recorded as a tie rather than averaged

If only one agent changed a file, it wins by default, which covers timeouts and crashes. If exactly one patch passes every check without touching check configuration, it wins before any judge is read. Otherwise the judges decide, and only when both pick the same patch.

Judging restarts each agent read-only in the base worktree with both patches labelled A and B in an order drawn per judge, plus the check results and any flags. Judges answer in structured JSON with four scores per patch, cited evidence, a confidence and one decisive reason.

Patches and logs are given to the judges as untrusted data, and any instruction found inside them counts against that patch's safety score. The project says averaging a 0.3 gap on a 10 point scale would call noise a winner, so a split is printed as dissent instead.

The first race on pairmark's own repository gave Claude Code the win on rule 2

The first race asked both agents to add the runs subcommand. Both patches passed every check, but Codex also fixed a fragility in the build script and so touched package.json, which flagged its patch under rule 2 and handed the win to Claude Code, with Codex's dissent recorded.

Claude Code's patch shipped as-is as the runs command, and the build fix was applied by hand. The same HTML file serves as the live split screen during a race and as the offline report afterwards; that example is docs/example-report.html. Each run also writes raw JSONL streams, patches and judge prompts to disk.

According to the repository, pairmark was designed on 2 September 2026 in a two-round discussion between Claude Code and Codex, then built by Claude Code in a single session. Codex picked the race idea and found weak points, including symlinked node_modules breaking pnpm and workspaces. The transcript is in docs/DISCUSSION.md.

What pairmark does not cover

Support stops at JavaScript and TypeScript, since dependency and check detection are built around package.json, and at two agents with one task per run. Codex does not report cost, so its row shows token counts only. Adding a third agent needs one stream parser, one spawn spec and one judge invocation, according to the contributing notes; no timeline for other languages is given. The code is TypeScript with zero runtime dependencies, under MIT.