OpenAI’s Pando cut monorepo push latency nearly 5x

OpenAI now runs all development in its monorepo on Pando, a Git backend it built with Codex in 10 weeks. According to a November 2 post on OpenAI’s blog, Pando cut median push latency by nearly 5x. Pando itself stays in-house, and OpenAI says only that it is “exploring ways to share the technology”. What other teams can actually run is client code. The packfile-uri download fixes OpenAI sent upstream ship in Git 2.56.0, but parallel pack downloads and “explicit haves” exist only in OpenAI’s own Git build.
Codex is the reason OpenAI gives. One engineer now has several changes ready for CI at once, and each push can set several CI jobs fetching the same commit. From around the end of 2025 the old system answered with slow fetches and pushes, failed merges and reliability “at times falling below 90%.” The New Stack reported that OpenAI expects quarterly commits to its monorepo to be up more than 2,400% by the end of this year compared with the first quarter of 2024. The authors, Aaron Friel, Taylor Blau and Ted Nyman, concluded that the limit was the way conventional servers serve Git, not the Git client or protocol. That is why engineers and agents kept their usual Git commands.
Two things Pando gives up
The first is a consistent snapshot of every branch at once. Pando orders updates one branch at a time. Each push records a “ref version” in a metadata store: the new tip, the pack that delivered it and the earlier versions it depends on. The only contended key is that branch’s pointer, guarded by a compare-and-swap-like operation, so pushes to unrelated branches don’t wait for each other. OpenAI also modeled the two usual ways to scale a Git service: three-phase commit across full replicas, and one shared write-ahead log in blob storage. In both, every push still goes through a single shared point.
The second is a fresh packfile for every client. A packfile is Git’s format for bundling compressed objects. A standard server works out which objects each fetching client lacks and compresses a pack just for that client. Pando validates each incoming pack in a single streaming pass and stores it in blob storage without ever unpacking it. On fetch it returns links to the stored packs the client needs, using packfile-uri. That protocol feature lets a server send clients to download packs separately, and Google’s Jonathan Tan first proposed it in 2019. A regional cache service called Express serves those links with hit rates above 99%. On standard cloud VMs, OpenAI says, clones and fetches saturate 10 Gbps network links.
Stored packs are only reusable if every reader can be shown to have what they depend on. So when a client pushes a feature branch, Pando advertises only the trunk as a base rather than every ref. With millions of branches, a full ref advertisement alone can exceed 100 MiB. OpenAI’s Git build adds “explicit haves”: the server reports a branch’s current tip separately, so a client can use --force-with-lease to confirm the branch hasn’t moved and still build its pack against trunk alone.
Every push leaves a small pack behind, and a client catching up on trunk would otherwise download tens of thousands of them a day. Background workers merge them into larger “covers”, much like compaction in log-structured merge-trees, and the monorepo now uses levels up to L4. Pando keeps packs at every level instead of garbage-collecting them, and the levels let it pick the packs a client needs in logarithmic time.
Numbers since April
Pando started serving production traffic in April. One engineer designed the core, teams of agents wrote it in Rust, and a handful of engineers deployed it. Agents running GPT-6 Astra wrote Lean and TLA+ specifications that the team used to find bugs and prove properties of the system. For the cutover, Pando first ran as a read- and write-through proxy in front of the old system, and OpenAI now ships its own Git build to developer machines and inside the ChatGPT desktop app.
OpenAI gives these production figures: median push latency down nearly 80%, nearly a petabyte of Git data a day served “with just a handful of CPU cores”, and a peak of more than 57 terabytes of packs in a single hour. Over 99% of the bytes in that hour came from local disk cache. By our calculation that hour ran at about 126.7 Gbit/s, and at that rate for a full day Pando would serve 1.37 PB. The comparison with other designs comes from OpenAI’s own simulation, with 1,000 clients each pushing to a separate branch on hardware ranging from commodity to premium VMs. In it, the modeled three-phase commit system slowed on writes as its disks saturated, and the write-ahead log system slowed on reads.
What the design costs
OpenAI states the consistency tradeoff itself: “A client reading several branches cannot assume those reads describe a single point in time across the repository.” If two branches change during a clone, one client may see foo at version 1 and bar at version 2 while another sees the reverse. OpenAI argues that users and agents depend only on the order of updates to a single branch.
Advertising only trunk costs client bandwidth. When an engineer pushes one branch many times or stacks one feature branch on another, the client resends everything that isn’t on trunk. In the monorepo the median pack produced this way was under 20 KiB, and the 99th percentile under 3 MiB.
Pando also isn’t being open-sourced, The New Stack noted, and Anthropic has said it has no plans to release Scion, its own internal system. Entire, from former GitHub CEO Thomas Dohmke, and Cursor’s Origin build on Git, while Anthropic started from scratch with a new client. GitHub, where many teams push, said in October that it sees 7.4 billion commits a month and blamed its recurring outages on growth driven mostly by “agentic demand.”
Our analysis
What we checked
On November 3 we matched OpenAI’s client patches against Git’s release tags and its master, next and seen branches. We also read OpenAI’s public Git build (branch codex) and its patch tracker, and searched the Git mailing-list archive.
OpenAI’s claim that several fixes are already upstream holds. They are in a Git release from late September, so teams on that release or newer have them without OpenAI’s build. The two features that matter most to Pando’s push and fetch paths are further off. Parallel downloads depend on a protocol change that was withdrawn from upstream review, and explicit haves have not been sent to the Git list at all.
| Change | Where it is | Detail |
|---|---|---|
| Safe concurrent packfile-uri downloads, cleanup after interrupted transfers, resuming finished downloads, download and pack-byte tracing | Git v2.56.0, tagged September 27 | Seven of eight commits by Ted Nyman; the byte counter by Aaron Friel |
| Parallel pack download and indexing | OpenAI’s build only | fetch.packfileUriJobs defaults to 1; higher values apply only when the server advertises the no-ref-delta promise, whose upstream series Git’s August “What’s cooking” report listed as “Retracted for now.” |
| Explicit haves | OpenAI’s build only | No results on the Git mailing-list archive; not among the tracker’s patches for upstream |
By our inference, parallel download cannot land upstream until the capability it depends on is proposed again. OpenAI’s public build carries 92 commits on top of upstream, the same count its tracker shows. Taylor Blau wrote 69 of them and Friel 12. We matched patches by commit subject, which misses any that were reworded upstream. The build’s base is a Git snapshot from September 7, before 2.56.0, roughly two months behind Git’s master as of November 1.
Up next
Claude threads shrink long requests from 800 KB to 1.8 KB, Video
The median request in long conversations shrinks from about 800 KB to 1.8 KB with Claude’s message threads. Token usage stays the same, since Claude still reads the full conversation. Does your agent resend the full history every turn or trim it first?
Related stories
- OpenAI to buy Pierre, whose diff tools already run in Codex
- GitHub hit 7.4B monthly commits, up from 2.9B in August
- Open-source Scion clones only the slice you ask for
- Claude’s merge rate outgrew git, so Anthropic built Scion
- GitLab and Anthropic rebuild Git for AI agents
- Codex Desktop hid 323 GB of crash dumps on one Mac
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

