OpenAI and Broadcom unveil Jalapeño custom chip for AI inference

OpenAI and Broadcom have unveiled Jalapeño, a custom chip designed specifically for running large language models (LLMs). According to OpenAI, the new processor is focused on AI inference, the stage where trained models generate responses to user prompts. The announcement marks OpenAI's first custom AI accelerator and reflects the company's growing efforts to build more of the infrastructure required to operate AI services at scale.

Jalapeño is a custom inference chip developed jointly by OpenAI and Broadcom. OpenAI said the processor was designed around the requirements of large language model inference and is intended to support the efficient deployment of AI models across its services.

Unlike training chips, which are used to develop and refine AI models, inference processors are responsible for serving those models to users. As AI applications continue to attract millions of users and process large volumes of requests, inference has become a critical part of the AI computing stack.

OpenAI described Jalapeño as the first accelerator in a broader multi-generation hardware roadmap being developed with Broadcom.

The launch comes as demand for AI computing resources continues to grow rapidly. Companies developing advanced AI models are increasingly looking beyond off-the-shelf hardware and investing in custom silicon tailored to their workloads.

Building dedicated chips allows AI developers to optimise performance, improve efficiency, and potentially reduce the cost of delivering AI services. It also gives companies greater control over their infrastructure at a time when access to AI computing resources remains a strategic priority.

For OpenAI, custom hardware could help address the growing computational demands associated with serving large language models to users worldwide.

The announcement highlights OpenAI's ambitions to play a larger role in the infrastructure layer of AI. Historically, the company has relied on external hardware platforms to train and deploy its models. With Jalapeño, OpenAI is moving towards a more vertically integrated approach that combines software, models, networking, and purpose-built compute infrastructure.

Broadcom's role in the project also underscores the increasing importance of custom AI accelerators in the semiconductor industry. The company has emerged as a key partner for organisations seeking specialised chips for AI workloads.

The introduction of Jalapeño could have implications beyond OpenAI's own operations. As inference becomes one of the largest cost centres in AI, purpose-built processors may become increasingly important in determining the economics of deploying advanced models.

If successful, custom inference chips could help improve efficiency, reduce operating costs, and support the wider availability of AI services. For OpenAI, Jalapeño represents the first step in what the company describes as a longer-term effort to build dedicated hardware for the next generation of AI systems.