AI inference shifts to the edge
The guest argued that AI infrastructure and inference will increasingly shift to the edge and on-device environments to avoid high public cloud costs.
The argument
Michael Dell stated that the lowest-cost token is generated directly where data is created, such as on phones, PCs, and embedded industrial or medical equipment. He noted that while companies initially embrace the public cloud, high bills are driving a rebalancing toward localized, on-device computing.
The thesis, stress-tested
✓ What validates it
- ✓Increased shipment volumes of AI-capable PCs and embedded edge devices
- ✓Public cloud providers reporting a deceleration in basic inference workloads as edge deployments grow
▸ Risks discussed
- ▸Public cloud providers could aggressively lower token costs to retain workloads
- ▸Edge devices may face hardware limitations for complex model architectures
Hear it yourself
"It's happening in 4,000 enterprises where we're building these these Dell AI factories. It's happening in Sovereign AI, you know, like with Palantir and, you know, people wanna protect their data, but also use AI on it. They wanna bring the AI to where their data is."
00:00 / 00:20
AFFILIATE LINK · ZORTIX MAY EARN A COMMISSION · NEVER A RECOMMENDATION TO TRADE