Due to quadratic scaling, halving bit precision should yield more than double FLOP count, contradicting historical assumptions.

Nvidia's product specs for B300 and beyond show FP4 as only 3x faster than FP8, whereas the theoretical expectation from quadratic scaling implies it should be 4x faster.

“Nvidia's product specs have started acknowledging that in B300 and beyond, where the FP4 is three times faster than the FP8. Though it should be 4x.”