Fewer than 100 people built OpenAI’s Jalapeño chip

OpenAI’s hardware team took Jalapeño, its first AI accelerator, from the first RTL code describing its logic to tapeout in nine months. Tapeout is the point where the finished design goes to manufacturing, and OpenAI’s own language models helped speed up the work. Hardware VP Richard Ho and Chris Leary, a member of technical staff, described the workflow in an IEEE Spectrum report published September 15.

OpenAI fully unveiled the chip on August 25. It delivers up to 13.4 petaflops of 4-bit compute and connects to 232 GB of memory at 15.4 TB/s. In benchmarks cited by OpenAI, Jalapeño cut end-to-end latency (the time from prompt to last token) by up to 3.6x compared with Nvidia’s GB300, the chip OpenAI currently relies on, and drew less power doing it. Those figures still have to hold up once the chip is in wide service across OpenAI’s inference fleet. Jalapeño is designed to be deployed in pods of 2,048 chips.

The team behind it averaged fewer than 100 people over the course of the project. It is still about 100 people today as it works on second- and third-generation designs. That count covers the whole hardware group, from system design to software and supply chain. It does not include Broadcom, which partnered with OpenAI on the chip. Andrew Kahng, a distinguished professor at UC San Diego, calls the pace “likely best in class today.” In 2016 Kahng co-organized an IEEE Design Automation Futures workshop where Ho, then an engineer at Google, gave a keynote. Ho argued there that the time a chip design takes depends on how many iterations a team can complete in a day.

From o3 to Astra

The project spanned several generations of OpenAI’s models:

• October 2024: work on Jalapeño begins.
• April 2025: o3, one of the models the team started with, is released to the public. The Jalapeño team had access to it earlier.
• October 13, 2025: OpenAI and Broadcom announce 10 gigawatts of custom AI accelerators, designed by OpenAI and developed and deployed with Broadcom. Rack deployments are targeted to start in the second half of 2026 and finish by the end of 2029.
• May 2026: the first chips come back from the foundry.
• August 25, 2026: OpenAI publishes Jalapeño’s first results.
• Hot Chips 2026: OpenAI presents the architecture, including silicon it deliberately left idle, and Ho and Leary give figures for AI-guided physical design.
• September 3, 2026: GPT-6 Astra is publicly released. By the end of the project the team was already using its precursors.

Ho and Leary had guessed the broad strokes of the AI-assisted workflow up front. Even so, improvements in OpenAI’s models brought a few surprises along the way. Leary says the newer model can work directly in Verilog, the hardware description language chips are written in, without a translation step. He adds that it is close to running proprietary design tools on its own. The team also used internal LLMs fine-tuned for chip design that have not been released. Ho would not say which models were used, but said the hardware team worked with OpenAI’s research team. The plan is to carry what the project taught into the commercial models. “It’s safe to say that Astra and following models will be very good at chip design,” Ho says.

Why the front end ran through XLS

“Automation itself has existed in chip design for many decades. It’s not a new problem,” says Ankur Srivastava, director of semiconductor initiatives and innovation at the University of Maryland. He says LLMs differ from earlier automation tools because they understand language and code. That suits them to tasks that “are still in the linguistic domain of the problem.”

OpenAI built its front-end workflow to make use of that. It runs on Accelerated Hardware Synthesis (XLS), an open-source high-level synthesis toolchain originally developed at Google. High-level synthesis lets engineers describe hardware in something close to an ordinary programming environment. In XLS, designers write in C++ or DSLX, a domain-specific language inspired by Rust, and XLS turns that into Verilog.

“We were thinking about how to leverage AI to make the project faster, and the AI was much better at software-looking things,” Leary says. “XLS in some ways looks like software, so it got that benefit.” Leary also knew exactly how XLS should behave, because he started it while he was at Google.

Kahng says aiming the models at high-level synthesis is “more natural for the LLM to work with” and allows fast iteration. “I see this as a generally useful workflow, and it’s one that ‘has legs’ going into the future.”

40 hours on one kernel

The team applied the same reasoning after first silicon. When the chips came back from the foundry in May, it pointed its internal models at writing the software that runs benchmarks such as SemiAnalysis’ InferenceX. On DeepSeek’s multi-head latent attention kernel benchmark, performance rose from 0.31% to 88.94% of the theoretical ceiling in roughly 40 hours. That ceiling is set by the chip’s compute and memory bandwidth.

Ho says the result is repeatable, so the gap between the first chips arriving from the foundry and production ramping up can be shortened. “All our schedule assumptions are going to be based on the fact we have this capability now.”

The back end stayed with Broadcom

The two companies mostly split the work between design and implementation. OpenAI handled end-to-end system design: the inference accelerator, the memory hierarchy and networking. Broadcom took “physical design from the gates onward,” Ho says. That covers most of the back end: routing interconnects, completing and verifying the clock and power specifications, and sending the design data the foundry needs. Broadcom then carried the chip through production on its own internal workflow. Its engineers had access to OpenAI’s public commercial models but not to the internal ones.

OpenAI still has physical design engineers on the Jalapeño team, and they give their Broadcom counterparts guidance on floorplan and routing, among other things. At Hot Chips, Ho and Leary said AI-guided physical design optimization shrank the matrix multiplication units by 10% in area compared with an optimized human baseline. OpenAI says this means more circuitry fits into the same silicon.

The Broadcom partnership tempered some of the praise for OpenAI’s speed. David Chin, co-founder of agentic chip design startup Verkor.io, says “the schedule they gave us is quite credible,” but believes Broadcom’s help was essential: “If you have somebody else start from scratch, it won’t be possible.” His co-founder Ravi Krishna called the pace “a relatively impressive result.” He expects better LLMs could make a project started today faster still.

Krishna also thinks OpenAI’s approach to the back end was a bit conservative, and puts that down to when the work began. “The models from the last four to five months have improved. From April [2026] onwards… is when they really started to be able to handle those tasks better,” he says. Another Verkor.io co-founder, Suresh Krishna, agrees: “there’s no reason you couldn’t have an agentic loop that largely accelerates the backend of the process as well.”

What the second generation changes

Ho and Leary suggested Jalapeño’s workflow may look old-fashioned next to their next projects. “As you can imagine with [Jalapeño], we were trying to go as fast as we could. So there’s a trade-off between ‘do we want to take time to do some innovation, or do we want to do things that we know work historically?’” Leary says. “With the second generation, we have a kind of reset opportunity to ask about all the things we want to get set up for.”

Ho says the second-generation workflow has “a lot of places that we are introducing [AI],” and names verification and physical design. Leary says the team now has tools for automatic waveform manipulation and viewing. They automatically pick out the clock signals linked to a failure, which could speed up debugging while the hardware is still being designed.

Neither of them expects chip design to become fully automated. “We’re not saying that anyone can come and just build state-of-the-art, frontier AI/ML accelerator chips using just [OpenAI’s coding platform] Codex,” Ho says. “We are saying some very specific things about how to be better at Codex, and how we are focusing on a small team and fast timelines to reach quality results.”