AWS Emulator mocks 438 AWS APIs, written by agents

AWS on November 11 released AWS Emulator, a local mock of more than 200 of its services that a team of agents built and will keep maintaining, under the public-domain CC0 license, The New Stack reports. Code that coding agents write against AWS can now be exercised offline, with no AWS account and no credentials handed to the agent, but only as far as API behaviour goes: there are no real backends behind the responses.

The emulator is written in Rust, runs on Linux, macOS and Windows without a network connection, and covers 438 public AWS API models. “There’s no AI inside of AWS Emulator,” Tom ‘Spot’ Callaway, a principal open source software strategist at AWS, told The New Stack. “It does not do an LLM callout at all.” Nor does it do what the services do. “I run this SQL query, and I don’t get the right results,” he said. “Well, guess what? There’s no database.” The tool “really was built for the purposes of testing coverage.”

Callaway’s case for it rests on the emulators that came before, AWS’s own included: “The challenge is that they get out of date very quickly. They don’t always reflect reality on the ground, and because of that, they’re often not trusted. And we didn’t even trust them internally.” The team also wanted somewhere agents could test AWS code without holding credentials. The alternative is costly: testing against the real cloud usually meant a dedicated account, “often” at $10,000 or more, The Letter Two reports. The New Stack sees the launch raising questions about the future of LocalStack, the existing third-party emulator.

How the agents divide the work

Matt Goldberg, a senior principal engineer on the AWS Developer Tools team, frames the project in a blog post after Anthropic’s agent-built C compiler: given a spec and a reference implementation, agents can build to it, and “the AWS API Models are our spec and the actual AWS cloud is our reference implementation.” Those models are the Smithy files from which AWS already generates all its SDKs. His first prompt to Kiro, AWS’s agentic coding tool, produced a passable SQS emulator in 15 minutes; Kiro built out the rest over a weekend, and his team was testing its own software against that gappy version the following Monday.

The production system splits into three teams of agents. The first writes test scenarios from the Smithy models, often spanning several services, and runs them against real AWS to pin down correct behaviour. “If the docs don’t tell the truth, then the cloud becomes the source of truth,” Callaway said. A recorded scenario looks like creating a queue, sending a message and deleting the queue, per The Letter Two. The second team replays the scenarios against the emulator, groups failures that seem to share a cause and files issues. The third fixes them, and a change merges only after build and test checks plus agent reviews for security, coding standards and behaviour regressions. SiliconANGLE puts the system at dozens of agents running on Amazon Bedrock AgentCore.

Engineers work on the system that directs the agents and leave the emulator’s code alone. “We don’t edit it. We don’t modify it,” Callaway said. A bad patch gets traced back to the agent that wrote it, and the agent gets fixed. One earlier version of the agents decided to write its own Python interpreter so the emulator could support Lambda functions; they now have instructions to reuse existing libraries. Early sandboxing “did not have as aggressive a sandbox as they needed to,” Callaway said, and there was concern about agents reading the source of AWS’s internal services to work out how to implement something like Aurora.

New services should land quickly. Callaway expects the gap after a launch to be “days, not weeks, not months,” while noting the team has not yet seen how that plays out in public. AWS plans to say more at re:Invent, which starts November 30.

What outsiders can send in

The CC0 license follows from the authorship: AWS was not sure it could copyright the agents’ work and did not want to assert a copyright that may not exist. It is also publishing the benchmark it tests the emulator with, “so that you can not trust us,” Callaway said, and others can use it on their own projects. Given his account of why earlier emulators failed, that benchmark is the piece a team weighing the switch can actually lean on.

Code contributions are off the table. “We’re not going to take pull requests on the project at all,” Callaway said. “The agents will be doing that. We will take issues from customers.” He expects those issues to be mostly scenarios the emulator does not yet handle, and if the agents keep getting a particular API wrong, “that’s a bug in the agent.”