What is TPU?

Anthropic committed over $100B to compute deals, 5 GW of TPUs (Google/Broadcom, starting 2027) and 5 GW of Trainium (Amazon).

“It was an over hundred billion dollar commitment.”

GPUs can move data between vector and matrix units through 16 wiring lines versus only two in TPUs, giving GPUs substantially higher internal data movement bandwidth.

“The amount of data you can move between a vector unit and a matrix unit is actually much higher in a GPU than in a TPU. Instead of having to move all the data through just two lines, you're moving it through 16 lines of wiring in a GPU.”

CPUs with deterministic latency are technically feasible but have been abandoned because the market does not reward that design trade-off.

“You can actually design a CPU that has deterministic latency as well. In fact, the processors inside a lot of AI chips also have deterministic latency. Groq has advertised this. TPUs have that in the core as well. The challenge is getting deterministic latency and high speed at the same time. Non-deterministic latency comes from specific design choices in a CPU. It's actually possible to remove those design choices and make a CPU with deterministic latency, but those are not very attractive in the market, so people don't make those CPUs anymore.”

Every GPU, TPU, and AI chip currently serving modern AI models was designed before ChatGPT and is fundamentally retrofitted, not purpose-built, for today's workloads.

“Every GPU, every TPU, every AI chip that was serving these models were just like fundamentally built before this and are retrofit to serve these modern models.”

Google has reportedly placed an order for 3 million TPUs to be manufactured in 2028, a notable signal of long-horizon AI hardware commitment.

“Google has reportedly placed an order for 3 million TPUs to be manufactured in 2028.”

Google is selling TPU hardware to external customers outside its own data centers, a notable shift in its hardware commercialization strategy.

“You are already seeing them step outside of their own data centers for this exact reason.”

Google TPU: Jensen Huang predicts that once TPU becomes a large-scale business, Google will shift to customer-owned tooling rather than ASIC contractors.

“Where will TPUs go when they when it becomes a large business? Customer own tooling. There's no question about it.”

Google has already sold approximately 20% of its TPU capacity to Anthropic.

“Google already made a deal to sell like 20% of their TPUs to anthropic.”