FP8/FP4 tensors in Metal 4.1

[Yoused]

The downside I see is that you can't have individual zeros within a block, since the 00 bit pattern would be interpreted as 1.0. And its not quite clear how signed stuff would work — does that leave you with a single mantissa bit, for effective ±1.0 and ±1.5? Using two's complement would give you -2, -1, 0, and 1 before scaling, which IMO might be more useful?

The format of FP32 is

S

exp

mantissa

In the 2-bit signed format, the S bit is part of the 2-bit value and the mantissa has one extra bit.

Imagine your block has 1KB arranged to represent a 64x64 array. If the exponent is 0x81, the signed values in the block can be 2, 3, -2 or -3. Obviously there would be a gap between 2 and -2, and as the exponent increases, the gap gets larger.

Imagine the array represents a texture definition. If the exponent were 0x84 and the values were interpreted as unsigned, each entry in the array could have a topographical value of 16, 20, 24 or 28. As the exponent gets larger, obviously, the topographical offset increases. This is also true for a signed array, although there would only be 2 possible offsets on each side.

I could also imagine using a 64KB array to describe the density or refractive index of a 64x64x64 heterogenous cube, where the exponent defines a baseline around which the mantissas specify localized variations.

These, of course, are models we can visualize. I suspect this would mostly get used for abstract correlations (like node weightings).