developer-tools

A maxed-out data center behind GitHub's August 17 outage

Promtime

developer-tools

GitHub now takes in 2.9 billion commits a month, along with 130 million merged pull requests and 24 million new repositories. The figures come from the company's postmortem of its August 17 outage, reported by Thenewstack, and mark a jump from the 1.4 billion monthly commits GitHub was already straining to serve in April.

At a glance

  • The outage began when traffic reached a new peak and a critical infrastructure component in GitHub's Central US data center failed to scale with it, spreading capacity pressure into authentication failures across multiple services.
  • Azure now carries 58% of platform load, up from 12% in May, and half of all Git operations, while GitHub added 3 million CPU cores and 120 petabytes of high-speed storage this year.
  • GitHub's own data centers are at their power limit, and Fedorov says operational practices did not keep up with the pace of change, and that the company is accelerating its migration to Azure.

GitHub sits under most of the open source ecosystem, so a capacity ceiling there is a shared constraint rather than a vendor problem. Coding agents produce commits and pull requests at machine pace, and the doubling since April suggests the load curve is steeper than the hardware curve GitHub can build against. That gap appears to be what makes rival platforms commercially interesting while GitHub's availability record stays uneven.

Capacity, not a code change, caused the eight-hour outage

The outage ran for almost eight hours and was GitHub's second major one in August. Fedorov states that no code change was involved: the infrastructure ran out of capacity, and the failure spread from there. The same postmortem carries the new usage figures, including the 130 million monthly merged pull requests.

Our investigation found that the outage began when traffic reached a new peak, and a critical infrastructure component in our Central US data center failed to scale with it. The resulting capacity pressure spread through our systems, causing authentication failures and disrupting multiple GitHub services.

Fedorov describes the modernization effort as real but insufficient in pace. GitHub has, in his words, "made progress, but these incidents make clear that we must accelerate this work," a reference to the platform's long-running shift toward Azure and away from its own data centers.

Azure now serves 58% of GitHub's platform load, up from 12% in May

Fedorov disclosed that Azure now serves 58% of GitHub's platform load, against 12% in May. Half of all Git operations run on Azure as well, the clone, fetch and push traffic behind those 2.9 billion monthly commits recorded in the postmortem.

The buildout on GitHub's own side is also large: 3 million CPU cores and 120 petabytes of high-speed storage added this year. Its own facilities are now at their limit, with GitHub having "installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure," Fedorov writes. The constraint he names is available power.

Cursor and ex-CEO Thomas Dohmke's Entire are building alternatives as GitHub isolates critical systems

Fedorov writes that operational practices did not keep up as the pace and complexity of change increased. GitHub has redirected teams and resources toward availability and invested in stronger testing, safer rollouts, better observability and more effective alerting, work he says is not complete.

A parallel effort isolates critical systems and removes shared dependencies between them, which Fedorov says is designed to reduce the likelihood of an outage and limit its impact when one occurs. The August 17 failure moved through that kind of shared path, reaching authentication before other services.

GitHub's position under the open source ecosystem makes those failures visible well beyond its own customers, and rivals have moved. Entire, founded by former GitHub CEO Thomas Dohmke, is betting on a distributed system for agent-driven development workloads, and Cursor has entered the same space with Origin.

No date for the Azure cutover

For now GitHub is applying consistent retry limits and budgets and tuning timeouts across service-to-service interactions, which Fedorov says is meant to prevent retry storms and cascading load. The postmortem gives no completion date for the Azure migration, no target for how much load will eventually leave GitHub's own data centers, and no schedule for the isolation work.

Comments

No comments yet. Be the first.

Join the conversation

Sign in with Google to leave a comment. Your name and avatar come from your Google profile, and the comment appears after moderation.

We only use your name and avatar from Google. We never store your email address.