OpenAI has released the first benchmark results for Jalapeño, its custom inference chip, arguing that the processor can serve more model output with less power than today’s leading alternatives. The figures offer a first technical test of a project previously known mainly through its design goals.
The results were presented at the Hot Chips conference and measured through SemiAnalysis’ InferenceX benchmark. OpenAI says Jalapeño produced more tokens per user and higher throughput per kilowatt than current state-of-the-art inference processors, including Nvidia’s Blackwell platform.
Benchmarks put efficiency at the centre
Inference turns a trained model into answers, making latency and operating cost recurring constraints for every user interaction. OpenAI hardware leader Richard Ho described Jalapeño as a significant advance in both efficiency and response time. The company is positioning the chip as a way to increase useful output without increasing electricity consumption at the same rate.
The comparison is a vendor presentation of benchmark results, not evidence from broad independent deployment. SemiAnalysis created the benchmark, but the source does not provide enough configuration detail to establish how the tested systems would compare across every model, workload or data-centre environment. Results may also change as competing hardware and software mature.
Metrics can also favor different product objectives. Tokens per user emphasize interactive responsiveness, while total throughput and performance per kilowatt reflect capacity and energy efficiency. A complete evaluation needs all three rather than one headline number.
Even with those limits, the emphasis on throughput per kilowatt is revealing. OpenAI is not judging the chip only by peak arithmetic capability. It is measuring how much service the complete platform can deliver to users under energy constraints, which is closer to the operational problem faced by a large AI provider.
Co-design targets wasted data movement
Jalapeño was developed with Broadcom and forms part of what OpenAI calls a multigenerational, full-stack hardware platform. The company says its chips, models, memory and products are being designed together, with OpenAI models also assisting the development process.
The architecture is intended to reduce delays around two distinct phases of inference. Prefill processes a user’s input, while later generation repeatedly consults the model’s key-value cache to produce output. OpenAI says Jalapeño keeps that cache local and activates compute, memory and networking according to the needs of each phase. The objective is to avoid unnecessary communication and data movement that can leave expensive components waiting.
That approach trades some generality for closer alignment with OpenAI’s own workloads. A purpose-built platform can remove inefficiencies that a broadly usable accelerator must tolerate. It also ties the value of the hardware to the accuracy of OpenAI’s forecasts about future models and product demand.
Deployment arrives after the competition moves
OpenAI expects only very small Jalapeño volumes near the end of 2026, followed by significant deployment in 2027. That schedule is the largest caveat attached to the benchmark. Nvidia and other chip developers have time to ship new products, improve software and alter the comparison before OpenAI operates Jalapeño at meaningful scale.
Manufacturing volume introduces questions that a conference benchmark cannot answer. Yield, reliability, integration, networking and sustained performance under diverse production traffic will determine whether the efficiency survives outside a controlled evaluation. The source does not quantify expected chip counts, capital costs or the share of OpenAI inference the platform might eventually handle.
The delayed ramp also means today’s comparison is a snapshot of systems at different stages. Jalapeño’s measured design must become a manufacturable platform, while incumbent suppliers can iterate on products already serving customers.
Today’s disclosure nevertheless advances the story beyond an announced chip. OpenAI has now attached performance claims and a deployment horizon to Jalapeño, making the project more testable. Its strategic value will depend less on winning one benchmark than on converting the co-designed architecture into reliable capacity before rival systems erase the measured advantage.
