claude-api-guard fails CI on broken Claude API calls

claude-api-guard, a GitHub Action published on Github, fails a pull request when code calls a Claude or OpenAI SDK shape that a dated provider release note already broke, and it ships with 10 hand-extracted Anthropic rules plus 4 for OpenAI. Scanning runs entirely on rules baked into the repository: no API key, no model call at runtime, and only HIGH-severity findings fail the job by default.

At a glance

• The scanner walks the Python syntax tree node by node instead of grepping file text, so comments, docstrings and other vendors' identically named methods cannot register as findings.
• A weekly job reads Anthropic's release notes and extracts new breaking changes with an LLM into a human-reviewed PR; one run produced 14 rules, including 8 hand-extraction had missed.
• Detection covers Python for both SDKs and, as a first pass, JS/TS for Anthropic only; the Business Source License converts automatically to the MIT License on 2030-09-01, with commercial self-hosting free from day one.

Provider SDK churn is a maintenance cost that usually surfaces as a production error rather than a changelog entry: a sampling parameter disappears, an HTTP client is swapped, a response field is renamed. What appears to distinguish this project is that its own rule set is treated as code that can be wrong, since nearly every provider and language added to it produced a false-positive class that only showed up once the scanner was aimed at real downstream repositories.

The regex prototype produced 1,576 findings across six public repositories, nearly all false positives

The first version matched patterns over whole files and was tested against anthropic-cookbook, anthropic-sdk-python, llm, aider, OpenHands and litellm. Most of what it returned came from comments, docstrings, string literals and, in multi-provider codebases, other vendors' calls that happened to share a method name with Anthropic's.

The AST rewrite matched rules against individual syntax nodes, and its first generic run still returned 1,720 findings in litellm alone, 1,604 of them from one httpx rule. Adding structural preconditions, that the file actually imports anthropic and that the call sits inside an async def, brought litellm to 2 and aider from 11 to 0; a later dedupe pass removed 427 duplicate findings in anthropic-sdk-python.

Auto-fix covers 5 of about 19 rules and refuses any patch that stops the file parsing

Patches are generated only where the fix is a pure deletion or a one-to-one string swap, with no judgment call attached: no model migration, no rephrased system prompt, no effort level chosen on the developer's behalf. Every patched file is re-parsed before anything is written, and a patch that fails that gate is refused and logged.

Run with --write against a copy of anthropic-cookbook, the fixer made 20 edits across 9 files, every one of which still parsed; a rescan left only the 4 assistant-prefill findings the tool was never meant to touch. One temperature argument assembled in a **kwargs dict was correctly left alone, since the fixer edits only keyword arguments written at the call site.

The OpenAI rule set went from 195 findings in litellm to 59 after three fixes

The four OpenAI rules were hand-extracted from the SDK's httpx2 migration guide, its changelog's breaking-change markers and the 2023 v1.0.0 migration guide. Triage against litellm found that the import precondition matched files borrowing openai.types as a cross-provider vocabulary, that bare httpx imports were not actionable, and that one hit was a string literal in a logging payload. All 59 remaining findings were reviewed, and OpenAI's cookbook, 224 Python files, returned nothing.

The JS/TS scanner, built on Babel rather than a tree-sitter grammar, covers the Anthropic SDK only. Four fixes, including pulling the assistant-prefill rule out of the generic engine after 1,619 matches, cut vercel/ai from 1,734 findings to 67, all of which check out on inspection.

The weekly sync needed four attempts on GitHub Actions: one crash on invalid JSON escapes in a generated regex field, one failure on a repository setting that blocks Actions from opening pull requests, then a 59-second run that opened the first rule-update PR.

What the Monday schedule still has to prove

The OpenAI half of the rule sync has only run in dry mode against past history; its first real test waits for openai-python to ship a version carrying a breaking-changes section. JS/TS support does not read the OpenAI rules at all, the string-literal false-positive class has not been audited across the other 21 rules, and the git-URL install used by the weekly auto-fix workflow stays unproven until that schedule fires once.