Inference cluster scale is rapidly expanding from single-digit chip configurations to thousands or tens of thousands of chips.

AI labs are already training models across multiple data centers, treating entire data centers as a single machine.

“Some of my friends at labs right now are training models across data centers. It's not even across nodes anymore. It's not even racks.”

Google has already made a deal to sell approximately 20% of its TPUs to Anthropic.

“Google already made a deal to sell like 20% of their TPUs to anthropic.”

OpenAI has scaled back its planned compute buildout from approximately $1.4 trillion to roughly $600 billion.

“One of them has scaled back open AAI their ambitions in this regard of their buildout from like 1.4 trillion to like butt 600 billion.”

Nebius claims it could sell its entire 2027 compute capacity today, indicating enormous unmet demand.

“Demand for what we are building continues to be enormous. We could sell today our entire 2027 capacity if we wanted.”

A trillion dollars of capex has already been committed for the next year across major cloud and AI players.

“There's a trillion dollars of capex that has been committed for the next one year across all these people broadly speaking.”

vLLM is currently running on half a million GPUs at any moment.

“The open- source inference engine now running on half a million GPUs at any moment.”

GPUs cannot effectively use tensor parallelism beyond 4-8 chips due to communication overhead from inability to overlap communication and computation.

“One of the limits of GPUs is because they don't effectively overlap communication and they have a hard time using tensor parallelism beyond four or eight.”