A Claude Code mod that reverts its own failed ideas

A Claude Code mod that bn-l published as claude-autoresearch-mod on GitHub on October 1 brings pi-autoresearch’s optimization loop to Claude Code: the agent tries an idea, benchmarks it, keeps whatever improves the result, reverts whatever makes it worse and repeats. Its sessions use pi-autoresearch’s .auto/log.jsonl format, so a log written by either tool works in the other, but the mod runs only on Claude Code 2.1.285 or newer with CLAUDE_CODE_ENABLE_FUNCTION_HOOKS=1 set in ~/.claude/settings.json.
The README calls it an almost 1:1 port of pi-autoresearch that also fixes some small bugs, and credits karpathy/autoresearch as the inspiration. The new mods API is what lets it change Claude Code’s UI to match the experience in pi. Claude Mods are Anthropic’s function hooks, packaged inside plugins.
Installing takes two commands: claude plugin marketplace add bn-l/claude-autoresearch-mod, then claude plugin install autoresearch@bn-l. The browser dashboard needs Node 18 or newer, or Bun, and the README calls it “very optional”.
In pi-autoresearch, the tool this mod copies, three tools run the loop. init_experiment sets up the session once: its name, the metric, the unit, and whether lower or higher is better. run_experiment runs any command, times it and captures the output. log_experiment records the result, commits it and updates the widget and dashboard. Each line of .auto/log.jsonl holds a commit hash, a metric value and whether the change was kept or discarded, Ry Walker’s profile of the project says. The same profile says .auto/prompt.md keeps enough context for a fresh agent to resume the loop, and lists ports that already existed: ozeron/autoresearch as a Claude Code plugin and cursor-autoresearch for Cursor.
The loop is only as trustworthy as its measurement, Hysen Labs warns in its review of pi-autoresearch. The tool expects a command that prints METRIC name=number lines, and the reviewer says the user should be able to review the commits the loop writes. Ry Walker’s profile also describes optional backpressure checks (tests, types or lint that block a change from being kept when they fail) and green, yellow and red confidence scores after three or more experiments, meant to separate real gains from benchmark jitter.
To try it on a toy project first, copy examples/sort-bench out of the repository, make that folder a git repository and run /autoresearch make sort.js faster in it.
Up next
One email check hides Claude’s Android debug menu
Claude’s Android app hides a full internal debug menu behind one check, whether the account email ends in @anthropic.com. Flip that check in a repackaged APK and it opens, with feature flag overrides and a staging backend switch inside.
Related stories
- Claude Code ships mods, the same tool behind its /diff
- Claude Code shipped its own TypeScript in npm source maps
- Show HN: Generate a Claude Code or Codex agent fleet from one HTML file
- pairmark races Claude Code against Codex in your repo
- Oh My Subagents keeps Claude Code teams running
- Show HN: Agent-hop – a Rust TUI that hops a live Claude Code chat into Codex
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

