# OpenAI President Greg Brockman on new chip: This is a real performance improvement

*CNBC Television* · https://www.youtube.com/watch?v=-ORtSHo6wP4

## Summary

In a CNBC exclusive, OpenAI President and co-founder Greg Brockman and Broadcom CEO Hock Tan discuss their jointly developed AI chip, which early testing shows delivers a substantial improvement in performance per watt, with gigawatt-scale deployment targeted to begin later this year. Brockman says the chip reduces the cost per token at a fixed level of intelligence and that a detailed technical report will follow in the coming months. Tan says the project validates Broadcom's business model that frontier model developers will build their own custom silicon.

## Key points

- Greg Brockman says the new chip delivers a real performance improvement in performance per watt and performance per dollar.
- OpenAI plans to publish a detailed technical report with benchmark numbers over the next couple of months.
- The chip was designed from scratch for LLM inference and went from design to tape out in nine months.
- Brockman argues the chip carves out its own niche in a very big market, designed around knowledge of LLM inference workloads across the industry.
- Hock Tan says the project validates Broadcom's business model that every frontier model developer will eventually build its own custom silicon.
- The accelerator, called the Jalapeno, was built on the fundamentals of OpenAI's LLM and ChatGPT, spanning the full stack from model down to the transistor.
- Early testing shows substantial improvement in performance per watt, with gigawatt-scale deployment targeted to begin later this year.

## Notes

### Overview
- CNBC exclusive interview with OpenAI President and co-founder Greg Brockman and Broadcom CEO Hock Tan following the announcement of their jointly developed AI chip.
- Early testing shows a substantial improvement in performance per watt, with gigawatt-scale deployment targeted to begin later this year.

### Performance claims
- Brockman says the chip "absolutely does" decrease the cost of each token at a fixed level of intelligence.
- He describes it as a real performance improvement in performance per watt and performance per dollar.
- It also means OpenAI can serve much more intelligent models much more broadly at scale.
- Specific numbers are not yet public, but Brockman says a detailed technical report will come over the next couple of months and initial numbers from the team are looking "very, very strong."

### Design and differentiation
- The chip was designed from the ground up for LLM inference, not adapted from existing hardware.
- It went from design to tape out in nine months, which Brockman calls blazing fast for the industry.
- It was designed with OpenAI's model, targeting knowledge of where LLM inference workloads are and where they are going, across the whole industry, not just OpenAI's models.
- Responding to a competitor's (Cerebras) skepticism, Brockman says the market is very big and the chip carves out its own unique niche.

### Broadcom's perspective
- Hock Tan says the nine-month design-to-tape-out schedule has always been Broadcom's aspiration.
- He says the project validates Broadcom's business model: every frontier model developer will eventually design and build its own silicon because they can do it much better.
- The accelerator, named the Jalapeno, is built on the fundamentals of OpenAI's LLM and ChatGPT, spanning the full stack from the model through hardware and software down to the basic transistor.

## Transcript

[00:01] THE COMPANY SAYING EARLY TESTING SHOWS A SUBSTANTIAL IMPROVEMENT IN PERFORMANCE PER WATT, WITH GIGAWATT SCALE DEPLOYMENT TARGETED TO BEGIN LATER THIS YEAR. JOINING US NOW IN A CNBC EXCLUSIVE, OUR OPENING EYES PRESIDENT AND CO-FOUNDER GREG BROCKMAN AND BROADCOM CEO HOCK E. TAN. GENTLEMEN, THANKS TO YOU BOTH FOR BEING HERE. GREG, LET ME JUST START WITH YOU. WE SPOKE TO YOU LAST OCTOBER WHEN YOU AND BROADCOM UNVEILED THIS EFFORT. AND I'M I'M SORT OF CURIOUS. YOU KNOW AT THE TIME YOU SAID THAT THE GOAL WAS TO DECREASE THE COST OF EACH TOKEN AT A FIXED LEVEL OF INTELLIGENCE. DOES THIS DO THAT. AND IF IT DOES IT IS IT SUBSTANTIALLY BETTER OR IS IT MORE INCREMENTAL. >> FIRST OF ALL THANK YOU FOR HAVING ME. IT ABSOLUTELY DOES

[00:45] THAT. THIS IS A REAL PERFORMANCE IMPROVEMENT ON PERFORMANCE PER WATT IN TERMS OF PERFORMANCE PER DOLLAR. AND SO FOR A FIXED AMOUNT OF INTELLIGENCE FOR A FIXED MODEL, IT IS SO MUCH MORE PERFORMANT. BUT ALSO IT MEANS WE'RE GOING TO BE ABLE TO SERVE JUST MUCH MORE INTELLIGENT MODELS MUCH MORE BROADLY AT SCALE. >> AND WHEN YOU SAY SUBSTANTIALLY BETTER, WHEN YOU TALK ABOUT PERFORMANCE PER WATT, YOU AREN'T GIVING US NUMBERS YET. AND I'M CURIOUS, YOU KNOW, ARE WE GOING TO GET THEM IN TERMS OF TECHNICAL BENCHMARK REPORTS AND THE LIKE? >> ABSOLUTELY. YEAH. WE'LL HAVE A DETAILED TECHNICAL REPORT OVER THE NEXT COUPLE OF MONTHS. I'VE SEEN THE INITIAL NUMBERS FROM OUR TEAM. THEY'RE LOOKING VERY, VERY STRONG. SO WE'RE

[01:23] GOING TO BE VERY EXCITED TO SHOW ALL THE DETAILS TO THE WORLD. >> ALL RIGHT. I ASSUME YOU HEARD OUR LAST GUEST SAYING, WELL, THEY'RE NOT GOING TO BE GOOD ENOUGH TO MEET WHAT WE'RE DOING AT CEREBRAS. DO YOU DISAGREE? >> LOOK, I THINK THAT THE MARKET IS VERY BIG, THAT THIS IS A CHIP THAT IS DESIGNED FOR LM INFERENCE FROM THE GROUND UP. WE REALLY WENT FROM SCRATCH AND HAD AN AMAZING TEAM WORKING ON THIS, ACTUALLY DESIGNED WITH OUR MODEL. SO WE WERE ABLE TO DESIGN THE WHOLE THING FROM END TO END TO TAPE OUT IN NINE MONTHS, WHICH IS REAL, JUST LIKE BLAZING FAST FOR THIS INDUSTRY. AND IT'S REALLY TARGETED AROUND OUR KNOWLEDGE OF LM INFERENCE, WHERE THE WORKLOADS ARE, WHERE THEY'RE GOING, NOT JUST OUR MODELS, BUT

[02:02] THE WHOLE INDUSTRY. AND SO IT'S SOMETHING THAT'S UNIQUE AND DIFFERENT AND REALLY CARVES OUT ITS OWN NICHE THAT DOESN'T EXIST RIGHT NOW. >> YEAH. I WANTED TO COME TO YOU TO THAT ON THE NINE MONTH, YOU KNOW, DESIGN TO TAPE OUT. IT WAS AN AGGRESSIVE SCHEDULE. HAS IT CHANGED THE WAY YOU DO THINGS AT ALL AT BROADCOM? DOES IT BECOME THE NEW NORMAL FOR SORT OF CUSTOM AI SILICON. >> IT IS FIRST OF ALL THANKS AGAIN FOR HAVING ME. APPRECIATE SEEING YOU AGAIN DAVID. BUT ALWAYS. >> LIKE TO SEE YOU AS WELL. THANK YOU. >> WELL IT'S A GREAT OPPORTUNITY BECAUSE THIS IS A KEY MILESTONE FOR US. BUT ANSWERING YOUR QUESTION, IT HAS ALWAYS BEEN OUR ASPIRATION TO DO THAT. WHAT WE HAVE HERE AND IN A BROADER CONTEXT, IT'S IT

[02:53] VALIDATES VERY WELL THE BUSINESS MODEL WE PUT IN PLACE, WHICH IS SIMPLY THAT EVERY MODEL MAKER, MLM, FRONTIER MODEL, DEVELOPER, WORLD EVENTUALLY GO TOWARDS BUILDING, DESIGNING, BUILDING THEIR OWN SILICON COMPUTE CAPACITY, COMPUTE DEVELOPMENT, SIMPLY BECAUSE THEY CAN DO IT MUCH BETTER. AND HERE WHAT WE HAVE IS WORLD CLASS, LEADING EDGE FRONTIER MODEL DEVELOPMENT TO COMBINE WITH A TEAM, ONE OF THE BEST TEAM IN THE WORLD, IN THE WORLD OUT THERE TO CREATE AI CHIP DESIGN, WORKING WITH US AND WITH ALL THIS IP SILICON TECHNOLOGY. AND WHAT YOU GET IS A IS AN AI ACCELERATOR, THE JALAPENO, WHICH IS BUILT GUIDANCE ON THE FUNDAMENTALS OF OPEN AI. LLM IT'S BUILT ON CHATGPT AND IT'S THE FULL STACK FROM THE MODEL ALL THE WAY DOWN

[04:03] HARDWARE AND SOFTWARE TO THE BASIC TRANSISTOR. WHEN YOU HAVE THAT KIND