OpenAI’s Jalapeño Chip Stakes a Claim Against Nvidia Blackwell in Real-World AI Inference

OpenAI Jalapeno Chip Just Beat Nvidia's Best at Its Own Game

Estimated reading time: 10 minutes

Key Takeaways

• The race to build the world's most powerful AI chip just got a serious new contender.
• Independent engineers from SemiAnalysis physically visited OpenAI's labs, supervised the tests, and published the numbers on a public leaderboard for anyone to check.
• The chip that runs AI answers in the background — faster responses, lower costs, less power — is about to become one of the most contested pieces of technology on the planet.
• Here is exactly what happened, what the numbers mean, and why this shift will shape the AI tools you use every day.

The race to build the world's most powerful AI chip just got a serious new contender. OpenAI's OpenAI Jalapeno chip — an in-house inference accelerator built alongside Broadcom — has posted benchmark results that put Nvidia's GB300 Blackwell on notice. This is not a marketing claim. Independent engineers from SemiAnalysis physically visited OpenAI's labs, supervised the tests, and published the numbers on a public leaderboard for anyone to check.

For businesses, developers, and AI watchers, this matters. The chip that runs AI answers in the background — faster responses, lower costs, less power — is about to become one of the most contested pieces of technology on the planet.

Here is exactly what happened, what the numbers mean, and why this shift will shape the AI tools you use every day.

What Is the OpenAI Jalapeno Chip and Why Was It Built

OpenAI has spent years relying on Nvidia GPUs to run its AI models. Every time someone sends a message to ChatGPT, a Nvidia GPU somewhere in a data center processes that request. That arrangement is expensive. Nvidia's chips are powerful, but they were designed to do many things — training AI, running games, handling scientific simulations. General-purpose hardware costs more per task than a chip built for one specific job.

That is exactly why OpenAI built Jalapeno. It is what engineers call an Application-Specific Integrated Circuit, or ASIC. Instead of being a Swiss army knife, it is a scalpel — designed from the ground up to do one thing extremely well: run AI inference at massive scale.

Inference is the process of taking a finished AI model and using it to answer questions or complete tasks. It is different from training, which is the process of teaching a model in the first place. Training requires brute computing force. Inference requires speed, efficiency, and the ability to handle thousands of users at once without falling over.

OpenAI co-developed Jalapeno with Broadcom, one of the world's leading semiconductor design companies. The chip is manufactured by TSMC on its N3P process node — one of the most advanced chip-making processes available today. It uses HBM4 memory, which delivers roughly 15.4 terabytes per second of memory bandwidth. It supports MXFP4 numerical format, a precision format designed to maximize throughput without sacrificing model quality. Its theoretical compute sits at around 13.4 petaflops.

The package draws up to 700 watts of power — but in actual tests, consumption stayed below 550 watts. That detail will become very important in a moment.

Sources: openai.com/index/jalapeno-first-results | shattered.io/openai-jalapeno-beats-nvidia-blackwell-2026

How the InferenceX AI Benchmark Works and Why It Matters

Numbers only mean something if the measuring stick is trustworthy. That is where the InferenceX AI benchmark comes in.

InferenceX is an open-source agentic inference benchmarking platform run by SemiAnalysis, a highly respected semiconductor research firm. It measures AI hardware — chips from Nvidia, AMD, Google, and now OpenAI — using standardized tests across real large language models. The metrics it tracks include:

• Throughput per kilowatt — how much AI work a chip completes per unit of power consumed
• End-to-end latency — how long a user waits from sending a message to getting a full response
• Interactive workload performance — how the chip handles chatbot-style, back-and-forth conversations

What makes InferenceX especially credible is that SemiAnalysis does not simply take a company's word for it. Engineers from the firm traveled to OpenAI's labs in person, observed the benchmark runs directly, and validated the methodology. The results were then published on a public platform where Nvidia GB300, AMD MI355X, and other leading chips are listed side by side.

The models used in the tests were:

• GPT-OSS 120B — a large open-weight model at 120 billion parameters
• DeepSeek R1 670B — an extremely large reasoning model at 670 billion parameters
• Kimi K2.5 1T — a one-trillion-parameter model from Moonshot AI, one of the largest models tested publicly

These are not toy benchmarks. These are some of the biggest, most demanding AI models running today. Testing on them gives a realistic picture of what the hardware actually does under real-world pressure.

The 2026 Hot Chips conference in Silicon Valley also featured Jalapeno's architectural details, placing it squarely among the most closely watched chip announcements of the year.

Sources: inferencex.semianalysis.com | newsletter.semianalysis.com/p/openai-jalapeno-better-than-nvidia | techcrunch.com/2026/08/25/openais-jalapeno-chip-is-built-for-fast-inference-at-scale-benchmarks-show

The Raw Numbers — What Jalapeno Actually Scored Against Nvidia GB300 Blackwell

This is where things get striking. The Nvidia GB300 Blackwell was the top-ranked system on InferenceX before Jalapeno entered the picture. The GB300 sits inside rack systems that consume roughly 1,200 to 1,400 watts per unit. Jalapeno, at under 550 watts of observed consumption, uses less than half that power. Now look at what it produced.

GPT-OSS 120B Results

| Metric | Jalapeno | Nvidia GB200 |
|---|---|---|
| Throughput per kW | ~85,448 tokens/s/kW | ~44,960 tokens/s/kW |
| End-to-end latency | ~1.03 seconds | ~1.80 seconds |

That is roughly a 1.9x efficiency advantage and a 1.7x latency improvement for a 120-billion-parameter model. A user waiting 1.8 seconds for an AI response versus 1.03 seconds does not sound enormous, but across millions of daily conversations, that gap translates directly into user satisfaction and infrastructure cost.

DeepSeek R1 670B Results

| Metric | Jalapeno | Nvidia GB300 |
|---|---|---|
| Throughput per kW | ~19,641 tokens/s/kW | ~11,781 tokens/s/kW |
| End-to-end latency | ~1.65 seconds | ~5.99 seconds |

Here the latency gap jumps to 3.6x. A response that took six seconds on Nvidia Blackwell comes back in under two seconds on Jalapeno. For a 670-billion-parameter reasoning model, that is a stunning result.

Kimi K2.5 1T Results

| Metric | Jalapeno | Nvidia GB300 |
|---|---|---|
| Throughput per kW | ~18,195 tokens/s/kW | ~11,862 tokens/s/kW |
| End-to-end latency | ~1.56 seconds | ~5.31 seconds |

Even at one trillion parameters, Jalapeno returns answers in under 1.6 seconds versus over 5 seconds for Nvidia's best current chip.

The Interactive Workload Finding

One data point in the research deserves special attention. In a specific low-latency, interactive regime — the kind of setting that mimics real chatbot and agentic AI tools — Jalapeno was measured at approximately 12,258 mixed tokens per second per kilowatt. Nvidia GB300 at that same interactive setting measured around 118 tokens per second per kilowatt.

That is a more than 100x throughput-per-watt advantage in chat-style workloads. SemiAnalysis' analysis confirmed that Jalapeno can sustain up to 104x the concurrent task throughput of GB300 for DeepSeek R1 at matched single-user generation speeds.

To be clear about what this means in plain terms: if you are running a chatbot or AI assistant that thousands of people use at the same time, Jalapeno can potentially handle more than 100 times as many simultaneous conversations per watt of electricity compared to Nvidia's current best chip. That is the kind of number that makes CFOs and infrastructure teams pay very close attention.

Sources: tomshardware.com/tech-industry/semiconductors/openai-says-its-jalapeno-chip-beats-nvidias-gb300-in-first-published-benchmarks | newsletter.semianalysis.com/p/openai-jalapeno-better-than-nvidia | aitoolsrecap.com/Blog/openai-jalapeno-benchmarks-nvidia-gb300-2026

Stop Losing Hours to Work Your Business Could Handle Automatically

Running a business means wearing too many hats. The emails, the follow-ups, the data entry, the scheduling — it piles up fast. At Flowgrammer, we build custom business automation systems that remove the manual grind from your operations so you can focus on the work that actually grows your company.

👉 See What Automation Can Do For Your Business (flowgrammer.ca)

What This Means for the AI Industry and Nvidia's Position

The benchmark results are impressive. But understanding what they actually change — and what they do not — requires some context.

Nvidia Is Not Going Anywhere

Nvidia just posted a $96 billion quarter. Demand for Blackwell chips remains enormous across every major cloud provider and hyperscaler. Its CUDA software ecosystem — the programming environment that developers have used for over a decade to build AI workloads — is deeply embedded in the industry. Training massive AI models still runs almost exclusively on Nvidia hardware.

Jalapeno does not challenge any of that. It is an inference-only ASIC. It was not designed to train models, and its benchmarks do not claim otherwise. OpenAI still needs Nvidia — or similar hardware — for training new models.

What Jalapeno does is reduce OpenAI's dependence on Nvidia for the most expensive, highest-volume part of their business: serving millions of users every single day with fast AI responses.

The Strategic Value for OpenAI

Every token that Jalapeno delivers more cheaply and quickly is a direct cost saving for OpenAI and a quality-of-service improvement for users. At scale — billions of tokens per day — even modest efficiency gains translate into hundreds of millions of dollars in annual savings.

There is also a negotiating dimension. OpenAI no longer has to accept whatever price Nvidia sets for inference hardware, because it now has a credible in-house alternative. That changes the power dynamic in procurement conversations, even if Jalapeno never fully replaces Nvidia chips.

Richard Ho, OpenAI's chip chief, framed this clearly: the Jalapeno processor gives OpenAI customers a choice between models optimized for lower cost or faster answers. That kind of flexibility has direct value for enterprise customers who need to manage AI spending carefully.

The Broader Industry Shift

OpenAI is not the first to do this. Google built TPUs for exactly the same reason — to reduce dependence on third-party hardware and improve the economics of serving AI at scale. Amazon has Trainium and Inferentia. Microsoft has the Maia chip. Meta has MTIA.

What makes Jalapeno notable is the scale of its reported performance advantage and the transparent, third-party verification process used to publish the results. By putting the numbers on InferenceX — a public benchmark that anyone can inspect — OpenAI invited scrutiny rather than avoiding it.

Analysts at SemiAnalysis, Tom's Hardware, and TechCrunch have each noted the same pattern: this represents a meaningful acceleration in the shift toward custom AI inference accelerators by major model providers. It mirrors the same vertical integration strategy that allowed Google's AI products to scale with relatively predictable economics compared to pure GPU-dependent competitors.

The antitrust and regulatory angle is also worth noting. Policy researchers and legal analysts watching AI hardware concentration have begun flagging custom accelerator development as a key front in future competition policy debates. When the largest AI model providers build their own chips, it changes who controls the economics of AI access — a question that goes well beyond semiconductor specs.

Sources: techcrunch.com/2026/08/25/openais-jalapeno-chip-is-built-for-fast-inference-at-scale-benchmarks-show | shattered.io/openai-jalapeno-beats-nvidia-blackwell-2026 | forklog.com/en/openai-says-jalapeno-outperforms-nvidia-blackwell-in-tests

What This Means for Businesses Using AI Tools

If you are a business that uses AI products — ChatGPT, API-based AI integrations, automation workflows powered by large language models — these developments have direct implications for you.

Faster responses are coming. As OpenAI deploys Jalapeno at scale, the latency improvements shown in benchmarks should translate into meaningfully faster AI responses for end users. For agentic workflows where AI takes multiple steps to complete a task, cutting latency from 5.99 seconds to 1.65 seconds per step compounds quickly into dramatically shorter overall completion times.

Costs should decline over time. The efficiency gains Jalapeno delivers — nearly double the work per watt compared to Nvidia Blackwell — reduce the cost of running AI inference. OpenAI has historically passed some of those savings on to API customers through price reductions, and that trend is likely to continue as custom silicon scales up in deployment.

Model access will expand. Richard Ho's framing of Jalapeno as giving customers a choice between "lower cost or faster answers" suggests OpenAI is building tiered infrastructure: some requests routed to the cheapest available hardware, others to the fastest. Businesses that use OpenAI's API will likely see new pricing and latency tiers that map to this underlying hardware flexibility.

The competitive landscape will keep shifting. InferenceX currently compares Jalapeno, Nvidia GB300, and AMD MI355X. Google TPUv7 and v8 are expected to appear on the platform soon, along with Nvidia's next-generation Rubin architecture. The pace of improvement in AI inference hardware is accelerating, not slowing down.

For any business building products or workflows on top of AI infrastructure, the practical message is simple: the underlying hardware is getting faster and cheaper at a pace that makes today's cost and latency assumptions outdated within months, not years.

Sources: openai.com/index/jalapeno-first-results | hyper.ai/en/stories/1c720390ec1624022de3ccb7c09a7e62 | x.com/rohanpaul_ai/status/2092259674741711160

Frequently Asked Questions

1. What is the OpenAI Jalapeno chip?
The OpenAI Jalapeno chip is a custom-built inference ASIC (Application-Specific Integrated Circuit) developed by OpenAI in partnership with Broadcom. It is designed specifically to run AI inference — delivering AI model responses to users — at high speed and with significantly lower power consumption than general-purpose GPU hardware like Nvidia's Blackwell chips. It uses TSMC's N3P manufacturing process, HBM4 memory, and MXFP4 precision format.

2. What does "inference" mean and why does it matter?
Inference is the process of using a trained AI model to answer questions or complete tasks. Every time you ask ChatGPT a question, that is an inference request. Training — teaching the model in the first place — is a separate, more compute-intensive process. Inference is where AI companies spend the most money at scale because it happens billions of times per day. Faster, cheaper inference hardware directly reduces operating costs and improves user experience.

3. How was Jalapeno tested and who verified the results?
Jalapeno's benchmarks were run on the InferenceX platform, an open-source inference benchmarking system operated by SemiAnalysis. Engineers from SemiAnalysis physically visited OpenAI's labs, supervised the benchmark runs in person, and published the results on a public leaderboard. This is a significantly higher standard of verification than a company simply publishing its own internal numbers.

4. What models were used in the benchmark tests?
Three large open-weight AI models were tested: GPT-OSS 120B (120 billion parameters), DeepSeek R1 670B (670 billion parameters), and Kimi K2.5 1T from Moonshot AI (one trillion parameters). These represent a range of model sizes, from large to extremely large, giving a realistic picture of how Jalapeno performs across different types of AI workloads.

5. How much better is Jalapeno than Nvidia's GB300 Blackwell?
Across the tested models, Jalapeno delivers between 1.5x and 1.9x more throughput per kilowatt of power than Nvidia GB200/GB300, and between 1.7x and 3.6x lower end-to-end latency. In specific interactive (chatbot-style) workloads, the throughput-per-watt advantage can exceed 100x, because Jalapeno is specifically optimized for fast, low-latency responses while consuming under half the power of Nvidia's reference systems.

6. Does this mean Nvidia is being replaced?
No. Nvidia remains the dominant force in AI hardware, particularly for training large models. The company posted a $96 billion quarter recently, reflecting enormous and continuing demand for Blackwell chips. Jalapeno is an inference-only chip and does not challenge Nvidia's position in AI training. Its primary impact is reducing OpenAI's reliance on Nvidia for serving users, which affects costs and gives OpenAI more negotiating leverage.

7. What is the power consumption difference between Jalapeno and Nvidia Blackwell?
Jalapeno's package TDP (Thermal Design Power) is rated at 700 watts, and actual measured consumption in tests ran below 550 watts. Nvidia's GB200/GB300 Blackwell reference rack systems consume roughly 1,200 to 1,400 watts. This means Jalapeno delivers its performance advantages while using less than half the electricity of the hardware it is outperforming on inference tasks.

8. When will Jalapeno's improvements affect regular ChatGPT users?
OpenAI has not published a specific deployment timeline, but the benchmarks and the Hot Chips 2026 presentation suggest the chip is moving toward production use. As Jalapeno is deployed at scale, users should expect faster response times for AI-powered products and, over time, lower pricing for API customers as the cost of inference decreases.

9. What is the InferenceX benchmark platform?
InferenceX is an open-source agentic inference benchmarking platform built and operated by SemiAnalysis. It provides standardized, publicly visible performance data for AI inference accelerators including Nvidia GB300, AMD MI355X, OpenAI Jalapeno, and — coming soon — Google TPUv7/v8 and Nvidia Rubin. It measures throughput per kilowatt, end-to-end latency, and interactive workload performance under consistent, controlled conditions. You can view it at inferencex.semianalysis.com.

10. What does this mean for the AI hardware industry more broadly?
Jalapeno is part of a broader shift in which major AI companies — Google with TPUs, Amazon with Trainium and Inferentia, Microsoft with Maia, Meta with MTIA, and now OpenAI with Jalapeno — are building custom silicon to reduce their dependence on third-party GPU manufacturers and improve the economics of AI at scale. This trend toward vertical integration is drawing attention from competition regulators and policy researchers, because it concentrates both AI model development and the hardware needed to serve those models within the same organizations. As more of these custom chips come online, the economics and competitive landscape of AI services will continue to shift rapidly.