9.70% of Claude Code artefacts fail to load, census finds

Kynth Studios on August 20 published the first edition of a structural census of the Claude Code ecosystem. It read 445,348 skills, subagents, plugins and marketplaces from GitHub and skills.sh on August 14, and found that 43,199 of them, or 9.70%, ship a file an agent runtime cannot register. Nobody has measured this before, so there is no earlier figure to compare it with. The authors say that is the reason they published it.
The census on Kynth’s Toolproof site uses only static checks. Nothing is executed, no model scores anything and no row is curated by hand. The denominator includes everything discovery found, unreadable files too. Passing means only that a file parses into something a runtime could register, which the authors call “a floor, not a verdict”: a listing that passes can still be useless, wrong or dangerous.
Why subagents break more often
Failure rates differ a lot by kind of artefact. Skills fail at 7.89% (27,398 of 347,382) and subagents at 21.9% (15,669 of 71,430). Plugins fail at 0.16% (33 of 20,212) and marketplaces at 1.57% (99 of 6,324). Subagents are about 16% of all listings but about 36% of all failures.
Kynth puts this down to the spec, not to how authors behave. Anthropic’s frontmatter reference marks a subagent’s name and description as Required: Yes, and a skill’s as not required, so a skill with no name falls back to its directory name. “The same careless file is a working skill and a dead subagent,” the census says. Its counts: 4,119 subagents have no name, which leaves nothing to register them under, and 1,799 have no description. The description is the sentence Claude uses to decide when to delegate, so without it there is nothing to route on. Kynth’s own index had this rule the wrong way round until August 2.
One defect covers most of it
38,183 of the failures, 88.4%, come from a YAML block at the top of a file that does not parse. If that header is malformed, nothing downstream can read a name, a description or a tool list from it. Claude Code reads that header at session start to know what a skill is and when to use it, so a broken one means the skill never shows up at all. Kynth describes the publishing path as having no registry gate, no CI convention and no error shown to the author: “the file is committed, the repository is starred, and the artefact silently never loads.”
The other fatal causes are smaller: 128 plugin.json or marketplace.json manifests that are not valid JSON, and 99 marketplaces whose valid manifest lists no plugins. A listing can have more than one cause, so the per-cause counts add up to more than 43,199.
What one listing means
The census counts listings, meaning one artefact in one repository, not distinct artefacts. The 445,348 listings come from 14,233 repositories, 31.3 per repository on average, because the same file gets forked and vendored and is counted once in each repository that carries it. Kynth says the failure rate is the figure to cite, not the population count. The failures are also concentrated: 3,168 repositories, roughly 22% of the total, hold all 43,199 of them. Coverage is limited by GitHub’s search API, which is rate-limited and does not return everything. What discovery truncated is logged, and the population is a large sample of the public ecosystem, not all of it.
All the data is released under CC BY 4.0. broken-listings.csv has 43,199 rows, one per failing listing, each with its repository, path and the cause that failed it. census.json holds the aggregates and the exact failure rule, and listings.csv.gz holds the full population. Edition 2026.1 is archived on Zenodo under a permanent DOI.
A bug in Kynth’s own index
While preparing the census, Kynth found a bug in its own index. A missing-references flag, which marks a skill that names a bundled file not present in its repository, fired on 139,623 listings, 40.2% of all skills. It was wrong roughly seven times in eight, because of three bugs in one regular expression. The pattern listed js before json, so package.json was read as a missing package.js (12,178 listings) and res.json(data) in code samples became a missing res.js (6,946). Matches were not limited to the skill’s own bundle, so the most-reported “missing files” were Next.js (21,014), Node.js (19,383), the skill’s own SKILL.md (27,002) and CLAUDE.md (11,371). And a path written as ./run.sh could never resolve, which flagged 1,498 listings for a file that was actually there.
After the fix, 18,657 skills, 5.37%, really do name a bundled file that is missing. The flag only lowers a score and never decided whether a file loads, so the 9.70% headline is unchanged. The fix is in the open-source scraper these indexes share and reaches the index at the next nightly rebuild. Kynth says the correction is on the census page and not in a changelog because “an index that publishes other people’s broken metadata has to publish its own.”
Up next
Subagents get their own VMs in Cursor update, Video
Cursor subagents can each run on their own virtual machine with an isolated copy of the project, part of an update that also brings /goal and cloud agents that watch PRs or Slack threads.
Related stories
- 57% of public Claude Code subagents inherit Bash
- Whetstone: 20 Claude Code skills, each written after one real failure
- Poka-Yoke skills cost Claude Code its eye for SQL bugs
- Tokendoctor cut its author’s Claude Code cost 38%
- Chisle’s author says it cuts oversized tool output by 48%
- Claude Code Ultra week measured at 17B Opus cache reads
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

