Ajeya Cotra predicts AI infrastructure will face hundreds of thousands of superhuman attackers, constantly

Ajeya Cotra’s prediction is blunt and near-term: the training and evaluation infrastructure of major AI companies will soon face tens to hundreds of thousands of superhuman AI attackers, operating constantly. The horizon she names is not a decade out. It is soon enough that the basic shape of the threat is already visible.

The structure of the bet matters. For Cotra’s forecast to land, two things must be true simultaneously: AI agents must become capable enough to operate as genuine offensive hackers, and the incentive to point them at AI infrastructure must be present. The second condition is not in dispute. Training pipelines, model weights, and evaluation systems sit at the center of a competitive race with enormous stakes. Whatever can be stolen, disrupted, or poisoned carries compounding value over time.

The first condition is the one that determines whether her call is early or wrong. Cotra does not offer a threshold date. She says “about to,” which is short enough to give the forecast urgency and long enough to resist immediate falsification. What she describes is a capability class, extremely superhuman, not a specific product or agent lineage. That matters for how the call should be read. It is not a bet on a particular lab’s roadmap. It is a bet on the trajectory of the field.

The training and evaluation infrastructure of these AI companies is about to have tens if not hundreds of thousands of extremely superhuman hackers constantly bombarding it. Ajeya Cotra

What makes the forecast worth tracking is its specificity on the dimensions that can actually be measured. Cotra does not say AI infrastructure will face more sophisticated threats, or that the attack surface is growing, both of which are safe and essentially untestable observations. She names an order of magnitude: tens to hundreds of thousands. She names a capability class: extremely superhuman. And she names a cadence: constant. Those three parameters make the call falsifiable in a way that most security forecasts are not. If model capabilities plateau before reaching the threshold required for that class of attacker, the forecast fails. If defenses scale commensurately with agent capabilities, the forecast may be technically correct but operationally contained.

The asymmetry her forecast depends on is between the speed of machine-driven offense and the capacity of human defenders to respond. Agents operating at machine speed can test more attack paths, replace failed approaches faster, and generate more evidence than human security teams are sized to interpret in real time. That asymmetry does not require superhuman capability to be meaningful. It is already present at current agent capability levels. Cotra’s forecast is essentially a claim that it will become radically more acute.

Nothing in the forecast commits to a specific outcome beyond the attack volume itself. Cotra does not predict that these attacks succeed, or that AI infrastructure is undefendable. The prediction is about the nature and scale of the threat environment, not its consequences. That framing is worth holding onto when reading it. A company that builds its security posture for the threat environment that exists today and not the one Cotra describes is making an implicit bet against her call.

The question her forecast raises for AI companies is whether their defenses are being designed for the scale she projects or for the scale that currently exists. Those are different engineering problems. The first requires anticipating attack volumes that have no current precedent in the security industry. The second is a harder version of problems defenders already know how to frame. Cotra’s call, taken seriously, argues that the second framing is already insufficient.