anthropic

Mythos found more bugs than Rubrik's engineers could review

Promtime

anthropic

Security and AI company Rubrik spent one month with Mythos Preview, Anthropic's model available only to vetted partners inside Project Glasswing, and concluded that its own engineers could not keep pace with the vulnerabilities the model surfaced. CTO and co-founder Arvind Nithrakashyap described the experience to The New Stack.

At a glance

  • Rubrik wrapped the model in a software harness that manages tool calls and checkpoints and adds business context, security context and trust boundaries before findings reach engineers for review.
  • Automated remediation is restricted to a deliberate, tightly-scoped subset of vulnerability classes where machine-driven fixes are reliable and well defined; everything else is routed to engineers for human judgment.
  • Rubrik joined Project Glasswing in June, when Anthropic widened the invitation-only program to roughly 150 organizations across 15 countries, and assembled a joint engineering and infosec team for the work.

Why it matters The bottleneck Rubrik describes appears to be structural rather than specific to one company: if a model finds issues faster than people can triage them, the constraint moves from detection to the software surrounding the model. That likely makes harness design, context injection and routing logic the real engineering work of AI-assisted security, and it puts the question of which fixes may be applied without human sign-off at the center of the process.

Rubrik's first response was to consider hiring more reviewers

When the first readout of potential issues came back from Mythos, which Anthropic calls its most capable model, Rubrik treated it as a capacity problem and weighed adding human reviewers to absorb the larger volume of findings, Nithrakashyap told The New Stack. The plan was abandoned quickly.

We quickly abandoned the plan…as we realized there was no way human-driven remediation could keep pace with AI-speed discovery.

Mythos was surfacing complex vulnerability chains that Rubrik's usual security tools and methodology missed, including relationships between components across large codebases that a conventional scan or a single engineer's review could not catch. The surge in findings created a prioritization bottleneck the existing engineering team was not set up to absorb.

Rubrik runs a whole-repository scan first, then narrows with targeted passes

Rubrik assembled a multi-functional engineering and infosec team for the work, leaning on automation wherever possible rather than scaling its base of human reviewers. The team first runs Mythos across a whole repository, then uses those initial findings to shape progressively more targeted passes.

Those later passes weed out noise so that only high-quality findings reach the appropriate teams and are prioritized accordingly. The stated goal of the harness was to cut the number of findings that ultimately land with engineers for review and remediation. According to Nithrakashyap, only after the targeted passes were introduced could Rubrik build a workflow that relays high-priority, actionable findings from Mythos.

Automated fixes are confined to a predefined set of vulnerability classes

Nithrakashyap said the question of what not to automate came up more often than expected, and he described a conflict between trustworthy automation and maximum automation. To preserve trust, Rubrik limited automated remediation to a deliberate, tightly-scoped subset of vulnerability classes.

Those classes cover cases where machine-driven fixes are highly reliable and well defined, and only vulnerabilities inside them move along the automated path. Everything else Mythos surfaces is routed to the relevant engineering teams, where human judgment owns the final fix. Nithrakashyap says the model has raised, rather than lowered, the engineering bar for the systems around it:

Mythos has shown us that AI actually increases the demand for engineering rigor in the systems that surround it.

What's next Rubrik's account covers a single month of use, and the next stage of the work is integrating structural context directly into the harness and building systems that categorize and filter Mythos findings into actionable items for remediation. Mythos Preview remains invitation-only through Project Glasswing, and no wider availability date has been given.

Comments

No comments yet. Be the first.

Join the conversation

Sign in with Google to leave a comment. Your name and avatar come from your Google profile, and the comment appears after moderation.

We only use your name and avatar from Google. We never store your email address.