Who is Anima Anandkumar?

Physics-informed neural operators can generalize to higher resolution than training data by enforcing PDE and conservation-law constraints at a finer resolution than the available data, providing guidance where labels are absent.

“You could give it partial differential equation constraints, conservation laws and you can now enforce them at a finer resolution than the data you have. Then there's more guidance in a way. So that way it can now come up with the right answers even at higher resolution because you're you know giving it that's how we can ensure that these physics informed neural operators can work at higher fidelity and higher resolution than even the training data that was available.”

Neural operators treat physical fields as continuous functions, so at inference time they can accept inputs and produce outputs at any resolution, unlike standard neural networks fixed to their training resolution.

“At inference time you can give it now inputs and ask for outputs at any resolution. So you're not just limited to the resolution of training that we see in standard neural networks and that's what neural operators enable.”

A high-resolution global weather neural operator model was trained on only about 50,000 samples, orders of magnitude smaller than language datasets.

“Our weather model like had about like 50,000 samples, right? 50,000 samples of fairly high resolution like global weather maps but it's nothing like what we see with language and in other domains it's even less.”

ForecastNet predicted Hurricane Lee's landfall several days earlier than traditional weather forecasting models.

“Our forecast net was able to correctly predict that the hurricane making the landfall several days earlier compared to the standard weather forecasting models.”

Industrial-scale physical simulations require hundreds of billions to a trillion tokens of context length, far beyond any feasible transformer.

“If each of the dimension is even a few hundred grid points which is where you know industrial scale starts at like a thousand grid points in each dimension we're talking like hundreds of billions to even a trillion context length right so forget ever having a transformer for anything of this scale all of the world's compute will not be enough.”

Purely data-driven AI cannot achieve scientific discovery in any domain because, by definition, no training data can exist about a finding not yet made.

“We will never have data about a new discovery.”

ForecastNet, Anima Anandkumar's neural operator weather model, demonstrated earlier and more accurate hurricane landfall prediction than traditional forecasting models.

“Our forecast net was able to correctly predict that the hurricane making the landfall several days earlier compared to the standard weather forecasting models.”

AI trained on a few thousand fusion plasma samples predicts plasma disruptions a million times faster than traditional simulations.

“We barely have a few thousand samples but we are able to accurately predict events like disruption very well and we are able to do that a million times faster than what traditional simulations were able to do.”

Extreme weather events like hurricanes may actually be easier for AI to learn than assumed, because their strong physical structure means fewer training samples are needed, not more.

“I think this is where more broadly the lesson is the physical world may be more forgiving because you know where there are extreme events like hurricanes that have very specific physical signature Right? So it's like extreme but in a very specific way. So maybe you don't need as many samples because the physical world has a lot of structure.”

Anima Anandkumar on why the physical world is more forgiving for AI than commonly assumed.

“I think this is where more broadly the lesson is the physical world may be more forgiving because you know where there are extreme events like hurricanes that have very specific physical signature Right? So it's like extreme but in a very specific way. So maybe you don't need as many samples because the physical world has a lot of structure.”