NVIDIA's CUDA moat is disappearing in inference
The guest argued that NVIDIA's proprietary CUDA software stack does not act as a moat in the rapidly growing AI inference market, making it easy for customers to switch to alternative hardware.
The argument
Switching from GPUs to Cerebras chips requires only 10 keystrokes to redirect an API, and CUDA has already lost significant market share in frontier model training, with two out of three major frontier models not built on CUDA.
The thesis, stress-tested
✓ What validates it
- ✓Major frontier labs publicly migrating training workloads away from CUDA-dependent frameworks
- ✓Cerebras or AMD gaining significant market share in inference API traffic
▸ Risks discussed
- ▸NVIDIA introducing new software lock-ins specifically optimized for inference
- ▸Customers remaining loyal to NVIDIA due to hardware familiarity and ecosystem inertia
Hear it yourself
"Why is that? Because nobody wants to wait. Right? So if you're engaged with the AI, speed is of the essence. But if the AI is doing agentic work and your competitor gets three times, five times, 10 times as much work done in twenty minutes than you do, you're gonna get smoked. And so this notion somehow that Ben proposed that speed isn't very important in agentic flows is dead wrong. That speed is important in in all aspects of productive work and that your ability to get more done in less time is a fundamental advantage that that accrues over time."
AFFILIATE LINK · ZORTIX MAY EARN A COMMISSION · NEVER A RECOMMENDATION TO TRADE