What is Liquid AI?

Liquid AI published the closed-form solution to liquid neural network neuron-interaction equations in Nature Machine Intelligence in November 2022.

“We solved the liquid neural network kind of interaction of neurons with each other in closed form for the first time. this became a nature machine intelligence paper published in November of 2022.”

LFM2 architecture composition: 80% double-gated 1D convolutions and 20% group query attention.

“The architecture is basically 80% doublegated convolutions you know 1D convolutions plus some group query attention basically 20% of query.”

Liquid Foundation Models process over 1 billion requests per month on the Shopify Shop app alone.

“We are passing through like some amazing statistics exponential statistics as well in terms of use. It seems like clients are really enjoying those features and now we are exponentially at over a billion request per month you know so that's like the amount of requests that goes through like the liquid foundation models like across like that shop app kind of just that kind of metric.”

Liquid AI's AFMD automated architecture search produced the double gated convolution as the optimal efficient architecture, eliminating hand-tuned gating from models like Mamba and gated delta nets.

“Turns out all of this has to go away if you want to get to the most efficient form of format of architecture and it became the double gated convolution that actually came out of this massive search space like AFMD.”

Liquid AI's architecture search strategy: rather than picking a single alternative to transformers, they built a meta AI system that searches the operator space given a specific deployment environment, producing hardware-aware hybrid architectures automatically.

“Let's build an unbiased way let's build a meta AI system that searches through the operation space so we build a meta system that searches through operators of interest given the deployment environment.”

Liquid AI open source models have surpassed 40 million total downloads and approximately 1.5 million downloads per week.

“They have an overall over 40 millions of downloads and we have like 1.5 about 1.5 million downloads per week.”

Hand-tuned gating mechanisms used in popular efficient architectures like Mamba are architecturally unnecessary; automated search eliminates them entirely in favor of a simpler double gated convolution.

“Turns out all of this has to go away if you want to get to the most efficient form of format of architecture and it became the double gated convolution that actually came out of this massive search space like AFMD.”

Liquid AI's Apollo App runs a 1B parameter hybrid attention plus gated convolution model on-device on an iPhone for private local data search and classification.

“Even a 1 billion parameter model, which combines a small number of attention layers with a very simple gated learned convolution, while admittedly far from the frontier, can run fast enough on an iPhone to be a real option for basic use cases such as privately searching through and classifying one's own local.”

Liquid AI's Mercedes-Benz deployment model is 600 MB, designed for over-the-air updates to all North America Gen 3 cars running on a roughly $100 chip.

“This model itself is about 600 megabyte so imagine like if you have like a 600 megabyte intelligence that goes inside every car you can do an overthe-air update or OTAA you can do overthe-air update of every car on their on the planet.”

Ramin Hasani on the architectural lesson from their massive search: all hand-tuned gating mechanisms are unnecessary.

“Turns out all of this has to go away if you want to get to the most efficient form of format of architecture and it became the double gated convolution that actually came out of this massive search space like AFMD.”