OpenAI’s Pando cut monorepo push latency nearly 5x

OpenAI has moved all development in its monorepo onto Pando, a Git backend it built with Codex in 10 weeks, and says the switch cut median push latency by nearly 5x, the company wrote in a November 2 OpenAI engineering post. Pando itself stays in-house, with OpenAI saying only that it is “exploring ways to share the technology that makes Pando possible” with its users, so what other teams can run today is OpenAI’s Git client work: several fixes already in upstream Git, with more in OpenAI’s public Git build.

Engineers and agents kept the standard Git protocol and client throughout the move, and Pando now delivers nearly a petabyte of Git data a day.

What other teams can use now

Pando hands out history through packfile-uri, a Git protocol feature that lets a server answer a fetch with links to packs the client downloads on its own. A Google engineer first proposed it in 2019, The New Stack notes. Relying on it for every fetch exposed client edge cases in concurrent downloads, cleanup after interrupted transfers and resuming downloads that had already completed. OpenAI fixed those paths and added validation when the client accepts an existing pack for a fetch. It also added tracing for packfile-uri downloads and byte counts for pack generation, so a slow fetch shows whether its time went into generating packs or transferring them. That work has gone upstream.

Two other changes sit in OpenAI’s own Git build for now, The New Stack reports. One lets the client download and index independent packs in parallel. The other, “explicit haves”, lets a server advertise a branch’s current tip separately from the history a pushing client may leave out of its pack. On Pando, this lets a client use --force-with-lease to check that a feature branch hasn’t moved while building its pack against trunk alone. OpenAI ships this build to developer machines and inside the ChatGPT desktop app, and tracks the upstreaming work in a public patch tracker.

Branches that don’t wait on each other

OpenAI modeled the two usual ways to scale a Git service before building its own. A three-phase-commit design keeps full copies of the repository on several replicas and needs a majority of them to agree on every branch update, over several round trips. A write-ahead-log design orders every push through one shared manifest in blob storage, updated with compare-and-swap. In both, pushes to unrelated branches still compete for the same coordination step.

Pando coordinates each branch separately. On a push, it validates the incoming packfile (Git’s format for compressing many objects together) while streaming it to blob storage, without unpacking it into a local repository. A single transaction then records a “ref version” in a metadata store, holding the new tip, the pack that supplied it and the earlier versions it depends on, and moves the branch pointer. That pointer is the only contended key, so clients conflict only when they push to the same branch at the same moment. Adding Pando servers, metadata store capacity or blob stores raises write throughput.

Reads reuse what writers uploaded. A conventional Git server computes and compresses a fresh pack for each fetch, which costs CPU and disk I/O and resists caching. Pando maps the client’s haves and wants to stored ref versions and returns links to existing packs. A cache service called Express serves them from regional caches with hit rates above 99%, and on standard cloud VMs clones and fetches saturate 10 Gbps links. OpenAI says Pando has served more than 57 terabytes of packs in a single hour, with more than 99% of those bytes coming from local disk cache.

A client that is a day behind on trunk would otherwise need tens of thousands of per-push packs. Background workers merge them into larger covers, the way log-structured merge-trees compact data, reaching level L4 in the monorepo today. The covers also act as a skip list over ref versions, so Pando picks a client’s packs in logarithmic time. No pack is discarded or garbage-collected, and packs at every level keep serving fetches.

What Pando gives up

The first cost is a repository-wide snapshot. A client reading several branches can see them at different points in time: if branches foo and bar both move during a full clone, one client may get foo at version 1 and bar at version 2, another the reverse. OpenAI’s position is that users and agents depend on the order of updates to a single branch and are agnostic to the global order.

The second comes from what the server shows a pusher. A conventional server lists every branch, and with millions of branches that advertisement alone can exceed 100 MiB. When a client pushes a new feature branch, Pando advertises only a designated trunk by default, so anyone holding that trunk history can fetch the branch with the one pack the writer uploaded. The price lands on engineers who push the same branch many times or stack one feature branch on another: each push resends everything outside trunk, packs grow and client bandwidth goes up. In OpenAI’s monorepo the median pack produced this way was under 20 KiB, and the 99th percentile under 3 MiB.

The comparisons with other designs come from OpenAI’s own simulation: 1,000 clients, each pushing unique commits to a distinct branch, on hardware from commodity to premium VMs. In it, the three-phase-commit model slowed on writes as replica disks saturated, and the write-ahead-log model slowed on reads as servers fell behind the journal. For the three-phase-commit run, OpenAI also relaxed repository-wide consistency.

Why the old setup broke

Codex let each engineer run more work in parallel: one person now has several changes ready for CI at once, and each push can set several jobs fetching the same commit. From around the end of 2025, the systems behind the monorepo produced slow fetches and pushes and failed merges, with reliability at times below 90%. The New Stack reports that OpenAI expects quarterly commits to the monorepo to be up by over 2,400% on the first quarter of 2024 by the end of this year.

Pando went into production in April. Its core was designed by a single engineer, implemented in Rust by teams of agents and deployed by a handful of engineers. Agents running GPT-6 Astra wrote Lean and TLA+ specifications of a “theory of Pando”, which the team used to find and fix bugs in the implementation. The first deployment ran as a read- and write-through proxy in front of the old system, so traffic moved over without downtime, and Codex skills let developers ask the agent to migrate their own environments.

The same load is hitting the big forges. GitHub said at its Universe conference on October 28 that it now sees 7.4 billion commits a month, and it blames its recurring outages on growth driven mostly by “agentic demand.” Inside OpenAI, Pando’s next candidates are workloads where large numbers of agents share one Git repository.

Other answers split on Git. Entire, the startup from former GitHub CEO Thomas Dohmke, and Cursor’s Origin build on Git, The New Stack writes, while Anthropic started from scratch with a new client and architecture in Scion. Anthropic has said it has no plans to release Scion either.