Five Claude models audited the LangChain monorepo

The CTRL NODE team ran a four-phase audit (Discovery, Audit, Strategy, Task Plan) across five models: Opus 4.8, Fable 5, Sonnet 5, Sonnet 4.6, and Haiku 4.5. Each model analyzed the Python LangChain monorepo with strict requirements for file:line citations and no hallucinated references.
Fable 5 earned an A- grade, matching Opus. Its strength lies in strategy and planning: defining themes, non-goals, and M0 through M3 milestones complete with effort badges, risk assessments, and acceptance criteria. Fable was the only model to identify a vendored 704-line Mustache engine and a disabled C90 complexity linter.
However, Fable missed specific threats: Opus successfully caught TOCTOU and DNS rebinding vulnerabilities on SSRF paths. The authors conclude that no single model replaces a multi-model pipeline: use Haiku for discovery, Sonnet for auditing, Opus for threat detection, and Fable for backlog generation.
Related stories
- Anthropic releases Claude Fable 5 and Mythos 5 models
- Opus 5.5 aced a test suite at 3.4x GPT-6 Sol's cost
- Opus 5.5 finds new bugs for CodeRabbit and misses old ones
- Fable 5.1 refuses the knife but heats a gas can anyway
- Andon Labs puts GPT-6 Astra ahead of Claude Fable 5.1
- Claude Opus 5 tops Sierra's agent-building test at 23.9%
Comments
No comments yet. Be the first.
Join the conversation
Sign in with Google to leave a comment. Your name and avatar come from your Google profile, and the comment appears after moderation.
