Jalapeño finally has numbers attached to it, and OpenAI’s first set looks strong. Measured on SemiAnalysis’ InferenceX benchmark, the chip delivered both more tokens per user and more throughput per kilowatt than the best inference processors currently on the market.
The complication is timing. That comparison was run against an Nvidia Blackwell system, and Jalapeño isn’t shipping in any meaningful quantity until 2027.
What OpenAI showed at Hot Chips
Tuesday’s presentation at the Hot Chips conference was where the company walked through the specifics — the fullest public view of the system since it was announced back in October.
“The bottom line is that the results show a very, very significant performance advance over state of the art,” said Richard Ho, OpenAI’s head of hardware, in a press call. “Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It’s very efficient to serve a lot of customers, but it can also be very low latency.”
The claim, then, is throughput and latency together. Those two normally pull against one another, which is exactly why the assertion deserves scrutiny once the hardware is sitting in real racks.
The 2027 problem
According to Ho, Jalapeño arrives at the end of 2026 “in very small volumes,” with the larger rollout following in 2027. Winning a paper comparison against Blackwell in 2026 is a different thing entirely from outperforming whatever Nvidia has on the market once Jalapeño is handling production traffic at volume.
No custom silicon effort escapes this. The benchmark is always against today's hardware, and then come two years of watching that target drift forward.
Built with Broadcom, and by OpenAI’s own models
Broadcom worked closely with OpenAI on Jalapeño’s development, and the company says its own models contributed to the process as well. The intent is for the chip to serve as a multigenerational platform, with AI products, models, chips and memory all developed in concert.
That full-stack story is the real argument being made. It’s the reason OpenAI could aim at particular phases of inference that tend to create friction, instead of optimizing for one headline throughput figure.
Where the bottlenecks actually are
Prefill and the communication phase are the design’s targets — stages OpenAI describes as frequent bottlenecks in inference processing. If you’ve ever watched a long-context request stall out before the first token shows up, you already know prefill is where the waiting happens.
“We designed Jalapeño to minimize data movement and communication delays,” the company said in a blog post presenting the results. “This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase.”
Explicit placement and local retention of the KV cache is the least glamorous piece of the engineering — and the piece that decides whether that tokens-per-kilowatt figure holds up under genuine serving loads.
The late-2026 small-volume deployments are the milestone to track. That’s the moment the InferenceX figures stop living on a slide and become something OpenAI has to run its own traffic through.