Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology

Background

• Naveen Rao learned programming as a child on a computer his family got in 1978, became an electrical engineer, and later earned a PhD in neuroscience to figure out how to make computers intelligent.
• He founded Nirvana Systems in 2014, the first AI chip company, sold it to Intel (admitting he sold too early), and started and ran Intel's AI group.
• After 2020 he built a GPU infrastructure company that took off after ChatGPT in 2022, joined forces with Databricks in 2023, and now represents a quarter of Databricks' total revenue.

The Energy Wall

• Google crosses 3.2 quadrillion tokens per month; at 10 joules per token that is about 12 gigawatts, against under 100 gigawatts of global data center energy.
• Rao estimates AI runs out of energy in about 3 years given exponential market growth against linear energy supply.
• About 50% of the cost of serving a token is energy; data center planning now starts with the energy contract, not floor space or GPUs.

Biology as Proof

• The human brain runs on about 20 watts; a monkey's brain on 1 watt (like a cell phone); a squirrel's brain on 8 milliwatts while jumping between branches with perfect accuracy.
• The human cortex moves only about 16 billion bits per second, versus nearly 30 trillion bits per second in and out of memory for a high-end GPU — moving information is what drives energy demand.
• Computers since ENIAC (1945) have been built for speed, not energy efficiency, and Moore's law scaling of efficiency has largely ended.

The Approach: 4D Computing

• Each abstraction layer (digital, floating point, neural networks) is lossy; Unconventional AI connects an abstraction of semiconductor physics directly to the neural network.
• Inspired by dynamical systems theory — emergent behavior in bird flocks and ant colonies — and demonstrated by synchronizing metronomes on a rolling plank.
• The open-source Uno model generates images using coupled oscillators; sparsity lets them throw away n² connections and get better, more trainable behavior.
• The first physical dynamical computer was taped out June 1st and generates images at roughly 500 nanojoules per image, versus millijoules on a GPU.
• 4D computing uses the time dimension plus physical 3D die stacking, unifying compute and memory instead of von Neumann separation.

Outlook

• Goal: 1,000x power efficiency in three and a half years, hitting the limits of 2D lithography; today's computing is about 10 billion times from the thermodynamic limit, while mammalian brains are within one or two orders of magnitude of it.
• Rao expects a shift from gigawatt data centers to many small, local ones, enabling billions of robots, and a Jevons paradox effect making this the largest market humanity has ever seen.
• Product path: a full rack data center product within about 2 years; models port at the model layer, not the operations layer, with Python libraries (not CUDA) for expressing time-varying stochastic elements.