OpenAI President Greg Brockman on new chip: This is a real performance improvement

Overview

• CNBC exclusive interview with OpenAI President and co-founder Greg Brockman and Broadcom CEO Hock Tan following the announcement of their jointly developed AI chip.
• Early testing shows a substantial improvement in performance per watt, with gigawatt-scale deployment targeted to begin later this year.

Performance claims

• Brockman says the chip "absolutely does" decrease the cost of each token at a fixed level of intelligence.
• He describes it as a real performance improvement in performance per watt and performance per dollar.
• It also means OpenAI can serve much more intelligent models much more broadly at scale.
• Specific numbers are not yet public, but Brockman says a detailed technical report will come over the next couple of months and initial numbers from the team are looking "very, very strong."

Design and differentiation

• The chip was designed from the ground up for LLM inference, not adapted from existing hardware.
• It went from design to tape out in nine months, which Brockman calls blazing fast for the industry.
• It was designed with OpenAI's model, targeting knowledge of where LLM inference workloads are and where they are going, across the whole industry, not just OpenAI's models.
• Responding to a competitor's (Cerebras) skepticism, Brockman says the market is very big and the chip carves out its own unique niche.

Broadcom's perspective

• Hock Tan says the nine-month design-to-tape-out schedule has always been Broadcom's aspiration.
• He says the project validates Broadcom's business model: every frontier model developer will eventually design and build its own silicon because they can do it much better.
• The accelerator, named the Jalapeno, is built on the fundamentals of OpenAI's LLM and ChatGPT, spanning the full stack from the model through hardware and software down to the basic transistor.