Two sample agents run code to drive browser and desktop

On September 7, OpenAI published a reference version of the loop it uses for computer-use agents. In this loop the model doesn’t return one click per turn. It writes code against a live runtime, and that code can run several steps, check them and report back. The repository on GitHub holds two agents built this way. One is in JavaScript and drives a browser through Playwright. The other is in Python and drives a desktop through PyAutoGUI. Both run on the Responses API and ship under the MIT License.

OpenAI’s earlier agent worked differently. Operator, announced on January 23, 2025, ran on the Computer-Using Agent (CUA) model. That model combined GPT-4o’s vision with reinforcement learning, read screenshots of the page and acted with mouse and keyboard input. It did not go through site APIs. The new repository keeps the same basic cycle: inspect an interface, choose an action, execute it, check the result. The difference is that the action is now a program.

“Code lets the model combine actions, process observations, and choose when to look again,” the repository says. A persistent runtime keeps useful state and helper functions available between calls. So the model can write a loop that fills several form fields, checks that each change took effect, and returns only the text or screenshots it actually needs. OpenAI says this can cut model round trips and repeated input context, while also giving the model feedback it can use to fix its own mistakes.

The two agents split the work differently. The JavaScript agent uses Playwright locators, screenshots and browser controls in a persistent session, with a TypeScript server and agent loop. In the Python agent, the server, the agent loop and a persistent PyAutoGUI worker all run in Python. It controls a visible browser on your desktop through screenshots, mouse input and keystrokes. Both use the same web console and the same labs. OpenAI frames them as a demonstration of the pattern built with standard libraries, and says its own products use their own runtimes and additional controls.

Those products have changed shape several times. Operator was folded into ChatGPT as ChatGPT agent on July 17, 2025, and the standalone app shut down on August 31, 2025. ChatGPT agent was later removed, and ChatGPT Atlas was sunset on August 9, 2026. The capability now lives in a cloud browser on OpenAI’s servers, a ChatGPT Chrome extension and the ChatGPT Work desktop app. The sample agents work through the user interface. For the other route, in August OpenAI turned on WebMCP in the ChatGPT desktop app’s built-in browser, a standard that lets sites expose structured tools that agents call directly.

The code is laid out so the loop is easy to find. In each app the core sits in one file, responses-loop.ts or responses_loop.py, which the layout describes as “model, code, feedback”. Next to it is a persistent execution worker: javascript-worker.ts on one side, desktop/worker.py on the other. On the Python side, runtime.py handles worker lifecycle and interruption. A shared contracts/ directory defines the request, event and replay types, and scripts/launch.mjs starts one app together with the shared console.

The labs are three self-contained practice apps: a kanban board with cards, columns and tasks, a drawing canvas with shapes, layers and tools, and a hotel search with mock reservations. Scenario defaults and task prompts live in labs/catalog.json. Every run gets a fresh copy of a lab template. Workspaces, screenshots and replays stay in the selected app’s data/ directory, which Git ignores. Replay JSON lets you inspect the recorded trace, and a screenshot timeline shows earlier states of the environment.

The console’s “Run finished” status means only that the agent loop and cleanup ended normally. It is not a verdict on the task. To decide whether the job got done, you inspect the screenshots, the recorded trace and the model response. The repository says it directly: “A final answer does not prove the task succeeded.” For scale, OpenAI’s launch figures for CUA were 38.1% on OSWorld, a benchmark of full computer use, plus 58.1% on WebArena and 87% on WebVoyager, both browser tasks. OpenAI noted at the time that most WebVoyager tasks are relatively simple.

Setup takes Node.js 22.20.0 and an OpenAI API key with access to the configured model. Corepack supplies the pinned pnpm 10.26.0. After cloning, you run corepack enable and pnpm install --frozen-lockfile, copy .env.example to .env and export OPENAI_API_KEY. Both apps read the root .env, and shell variables take precedence. To start the JavaScript agent, run pnpm playwright:install and pnpm dev:js. The Python agent needs uv and Python 3.10 or newer (the repo pins 3.12), then pnpm python:install, pnpm python:playwright:install, a preflight check with uv run --project python-app python -m app.desktop.worker --check, and pnpm dev:python. From the console you pick a scenario, edit its prompt and select Start Run. Runs make real API calls. Stop interrupts a run, and before switching apps you press Ctrl+C and wait for shutdown.

OpenAI is clear about the risks. Generated code runs with your user permissions, and the samples have no operating-system sandbox and none of OpenAI’s production action-review controls. The advice is to stick to the labs or other environments you control. The Python agent moves your real mouse and keyboard, and its screenshots may capture other windows. So it should run in a dedicated desktop session, with the lab window in front on your primary monitor. API keys and run artifacts should stay private. Environment files and generated run data are excluded from Git, and services should stay on their default loopback addresses.

A crash can leave desktop input held down. The repository asks you to read the Python app’s interruption and recovery notes before running it.