FP8, FP4, MXFP4 and NVFP4 by hand

Table of Contents

• Why by hand?
• The cast of characters
• Everything is a grid and a scale
• The E2M1 grid, enumerated by hand
• Our running example tensor
• Format 1: FP8 (E4M3) by hand
• Aside: FP8 is two formats — and FP4 is only one
• Format 2: FP4 by hand
• Format 3: MXFP4 by hand
• Format 4: NVFP4 by hand
• The scoreboard
• Appendix A: the E4M3 grid by hand
• Appendix B: why scale factors round up, not to nearest
• Appendix C: the full 32-value table
• Appendix D: Why E2M1 stops at 6, E4M3 stops at 448

Why by hand?

I was interested understand how NVFP4 works, why it is different from MXFP4, and why I should care about NVFP4. The diagram in Introducing NVFP4 for Efficient and Accurate Low-Precision Inference is not the easiest to follow either, hence this guide, which will work out NFVP4, MXFP4, and FP8 by hand.

FP4, MXFP4 and NVFP4 look almost identical in that each stores value in 4 bits, i.e. 1 sign bit, 2 exponent bits, and 1 mantissa bit (E2M1), and the only difference is a footnote about scaling factors. This footnote makes all the difference, and is the subject of this article.

By the end you should be able to look at any block-scaled format and immediately answer: what is the grid spacing here, and which values fall through the cracks?

This article was written with the assistance of AI.

The cast of characters

† FP8's 4 exponent bits per element already reach ±448, so this article casts to FP8 directly with no scale factor at all. (Appendix A works through the E4M3 grid in full, addressing questions such as where does the figure 448 come from.) Production FP8 inference often adds one FP32 scale per tensor to use that range well — still just one number for the whole tensor.

How the bits work: sign, exponent, mantissa

Four bits give $2^4 = 16$ distinct bit patterns, and E2M1 reserves none of them for NaN or infinity, so all 16 decode to finite numbers, which are laid out as S EE M — sign, 2 exponent bits, 1 mantissa bit.

The exponent field picks a bracket: A bracket is the stretch of number line between one power of two and the next, $[2^n, 2^{n+1})$ (formally called a binade). Every number inside the same bracket shares the same exponent. A bracket's base is its lower endpoint, i.e. $2^n$ , where the mantissa bit is zero.

Why exactly four? Because the exponent field is 2 bits wide, and 2 bits have $2^2 = 4$ possible settings — 00, 01, 10, 11. Each setting names one bracket, so the count of brackets is fixed by the width of the exponent field:

Consequence of the exponent width: E2M1 with 2 exponent bits and 1 mantissa bit stops at 6, E4M3 with 4 exponent bits and 3 manitssa bits reaches reaches 448. See (Appendix D for an explanation.

What "width" means, and why it doubles: A bracket's width is its length on the number line: $[1,2)$ is 1 wide, $[2,4)$ is 2 wide, $[4,8)$ is 4 wide. Each is twice the previous one because moving up a bracket doubles both endpoints, and doubling both ends of an interval doubles the distance between them. Concretely:

The mantissa field splits the bracket: One mantissa bit gives two points per bracket, at 1.0× and 1.5× the base. Four brackets × two points each = 8 magnitudes:

Every bracket holds the same number of points, but each is twice as wide as the one below — so the gap between points doubles too. The grid is non-uniform because exponents step in powers of two.

Applying the sign bit to those 8 magnitudes gives the 16 codes. Since 0000 and 1000 both mean zero, that is fifteen distinct values:

Putting the three fields together, each of those 16 codes answers three questions in order — which sign, which bracket, which of the two points inside it:

In the above table, first two rows have the same sign and bracket, and differ only on the mantissa bits, which is what "the mantissa splits the bracket" means. Rows three and four differ only in the exponent field, and the value quadruples, because two brackets up is two doublings. The sign bit flips the result and touches nothing else, which is why the grid is perfectly symmetric about zero and why 0000 and 1000 collide on zero.

The same three fields, on real numbers

Below are some more examples of how nubmers convert to E2M1 and E4M3:

Row four is where 4-bit precision bites. 5.05 sits in bracket $[4, 8)$ with base 4, so its significand is $5.05 / 4 = 1.2625$ — but one mantissa bit offers only 1.0 or 1.5, and nothing in between. It rounds to 1.5 and comes back as 6.0. Rows five and six show what three mantissa bits buy: eight significand settings per bracket instead of two, so −3.54 lands on −3.50 rather than being flung to −4.

What is the scale for?

Notice what every one of those rows has in common: whatever bits you feed in, the result is a member of that same fixed set of fifteen. E2M1 has no other numbers available. It cannot represent 0.0003, and it cannot represent 4700.

However, real tensors contain numbers like 0.0003 and 4700. So the scale applies a separate multiplier s that stretches or shrinks the fixed grid onto wherever the data actually lives.

One mental image is to think of a ruler with fifteen tick marks. The 4 stored bits say which tick; the scale says how far apart the ticks are. The same ruler measures a molecule or a bridge depending only on the units you print on it. I will explain how to apply s with specific examples later.

What does it mean to "share" a scale?

Every element's exponent field is a private per-element scale. For example, FP8 E4M3 is a 3-bit mantissa grid, and a private 4-bit power-of-two scale for every element. When we say "share" a scale in the context of MXFP4 or NVFP4, we are referring to scales shared across elements:

Note that block scaling does not replace the per-element exponent. Instead, it replaces the exponent's range.** For MXFP4 and NVFP4, the shared scale exists to slide that narrow 4-bracket window onto wherever the data lives. Each element still picks its own bracket inside the window. The bargain made with block scaling is to let a group of values share one scale, and amortise its cost across the group. One 8-bit scale per 32 values costs 0.25 bits per value; per 16 values the 8-bit scale costs 0.5 bits each.

How many scale factors are there, really?

Here is a count of the number of bits for a 32-value tensor:

NVFP4 stores three numbers of metadata for thirty-two values, and MXFP4 stores exactly one. At 32 values the FP32 per-tensor scale is enormous relative to the data, so NVFP4 works out to 5.5 bits per value here rather than its headline 4.5. Real tensors are where the amortisation happens — a 4096 × 4096 weight matrix holds 16.7 million values, so that same 32-bit scale is spread across all of them:

Because every number in the group is stretched by the same amount, so the ruler has to be long enough for the group's largest member. A single outlier forces a long ruler with widely spaced ticks, and every one of its groupmates is measured on that coarse ruler whether it needed to be or not.

This is why group size matters:

• Bigger groups → less metadata, but one outlier contaminates more neighbours.
• Smaller groups → more metadata, but damage stays local.

Per-tensor scaling (plain FP4) is the extreme case where the group is everything, so a single outlier anywhere sets the ruler for the whole tensor. MXFP4 shrinks the group to 32, NVFP4 to 16.

Why is scale encoding important

There are two independent precisions in play:

1. Element precision — how finely you can pick among the ticks. This is the 4 bits, E2M1. It is identical across FP4, MXFP4 and NVFP4. Lack of element precision results in element rounding error, which is per-value and roughly random. Some values round up, some down, and across a dot product the errors partially cancel.

2. Scale precision — how finely you can set the tick spacing itself. This is E8M0 vs E4M3 vs FP32, and it is where the three formats genuinely differ. Lack of scale precision leads to scale error which is systematic. Get s wrong and all 16 or 32 values in the block are stretched wrongly in the same direction, at once. It is a bias, not noise, and biases do not cancel — they accumulate through the matmul.

So a bit spent on the scale buys more accuracy than a bit spent on elements. That is the central design insight of MXFP4 and NVFP4.

Everything is a grid and a scale

Every format here does the same three things. Strip away the names and each one is:

where s is a scaling factor shared by some group of values, and q is the nearest point on a small fixed grid. $\text{Q}$ is shorthand for snap this number to the nearest value the format can actually represent. The subscript names which set you are snapping to — for all three 4-bit formats here, the E2M1 grid ${0, \pm 0.5, \pm 1, \pm 1.5, \pm 2, \pm 3, \pm 4, \pm 6}$ . Written out, it is picking the closest grid point:

Below are some examples:

The first row lands on 1.5 — a value ordinary rounding cannot even produce — because 1.452 sits 0.048 from 1.5 but 0.452 from 1.0. The third row is the MXFP4 outlier we will meet shortly: 5.05 snaps all the way up to 6 because the grid holds nothing between 4 and 6. (Exact ties go to the even grid point where the mantissa bit is 0.)

The natural choice of scale for a group is the one that stretches the grid to exactly cover the largest magnitude in that group, i.e. absolute maximum (or amax):

because 6 is the largest magnitude E2M1 can represent. Any smaller s clips the outlier; any larger s wastes grid points on a range the data never visits.

The E2M1 grid, enumerated by hand

Before quantizing anything, let us write out the entire FP4 grid. It has 16 codes and we can list all of them.

E2M1 lays each code out as S EE M — 1 sign bit, a 2-bit exponent field, 1 mantissa bit — and decodes with the usual floating-point formula:

Three rules turn the bits into numbers, and a fourth explains why the grid reaches as high as it does:

1. The mantissa bit gives 1.0 or 1.5. Normal floats carry an implicit leading 1 that is never stored, so with one stored bit the significand is $1 + M/2$ — binary 1.0 or 1.1. This is why every magnitude below is a power of two or 1.5× one.
2. The bias is 1. For a 2-bit exponent field the bias is $2^{E-1} - 1 = 2^{2-1} - 1 = 1$ , so a stored field $e$ means an actual exponent of $e - 1$ . (Where that formula comes from is worth its own section — see just below.)
3. Field 00 means subnormal. The implicit leading 1 is dropped (significand becomes $M/2$ ) and the exponent is pinned at $1 - \text{bias} = 0$ . This is where 0.5 comes from — it is not a normal value.
4. Field 11 is not reserved. IEEE formats spend their top exponent field on infinity and NaN. E2M1 has too few codes to spare, so 11 encodes ordinary numbers. That is why 4.0 and 6.0 exist, and why all 16 bit patterns decode to finite values.

Running all four exponent fields against both mantissa values:

Four exponent fields × two mantissa values = 8 magnitudes. Applying the sign bit gives 16 codes, and since 0000 and 1000 both mean zero, 15 distinct values:

Here is a rewritten, drop-in replacement for your text. It keeps your original formatting (including the math tags and table) so you can copy and paste it directly, but it translates the dense technical jargon into easy-to-understand concepts and analogies.

Where the bias comes from

Earlier, we used a formula for the bias ( $2^{E-1} - 1$ ) that might have looked like it was pulled out of thin air. Here is what it actually means.

Why do we need a bias at all?
In a floating-point number, the "exponent" part is stored as a simple, positive-only counting number (starting from 0). There is no plus or minus sign attached to it. However, we need negative exponents to represent tiny fractions below 1.
To solve this without wasting space on a minus sign, we "shift the goalposts." By automatically subtracting a fixed number (the bias) from whatever positive number is stored, we instantly slide the lower half of our numbers down into negative territory.

Why use that specific math formula?
That specific formula perfectly centers the window. By subtracting exactly $2^{E-1} - 1$ , our available exponents are split right down the middle—roughly half fall below zero, and half sit above zero.
More memorably, it puts 1.0 right in the middle. To get an actual exponent of 0 (which is required to make the number 1.0), the computer just stores the bias number itself. This same elegant rule works across all sizes of floats:

Why use a bias instead of the computer's normal negative numbers?
The answer is sorting. Using a bias keeps all the binary 1s and 0s in perfect numerical order. If you take positive floating-point numbers and tell the computer to read their raw bits as if they were just basic, simple integers, they will naturally sort correctly from smallest to largest. As Appendix B shows, because everything is perfectly ordered, rounding a number up to the next available slot doesn't require complex math or a lookup table—the computer literally just adds 1 to the raw integer value. This perfect sorting is what makes NVFP4's "scale selection" so incredibly cheap to run.

You can see this smoothly working in the E4M3 format: as the actual numbers get larger, their raw underlying bit patterns just step up like a normal counter—0, 1, 2, 3... climbing perfectly in step with the values themselves. The creators of FP8 deliberately kept this feature because it allows hardware and software to compare and sort these complex decimal numbers using basic, lightning-fast integer operations.

Conversion by hand with a sample tensor

We will use one 32-element tensor, which is exactly one MXFP4 block and exactly two NVFP4 blocks. The values are split into two halves with deliberately different characters, to represent activation tensors in LLMs which routinely mix quiet channels with channels carrying huge outliers.

Block A — a "quiet" group (elements 1–16), amax = 0.47:

 0.31  -0.47   0.12   0.28  -0.19   0.41  -0.35   0.07
 0.22  -0.44   0.38  -0.11   0.16   0.29  -0.25   0.44

Block B — a "loud" group with an outlier (elements 17–32), amax = 10.10:

-3.54  -3.14   1.33  10.10  -0.07   2.71  -1.62   0.88
 3.01  -2.35   0.15   1.97  -0.53   2.24  -1.11   0.66

The tensor-wide amax is 10.10, driven entirely by one element in block B. Block A's largest magnitude is more than 21 times smaller.

Format 1: FP8 (E4M3) by hand

FP8 E4M3 gives us 1 sign, 4 exponent and 3 mantissa bits, with a maximum finite magnitude of 448. Our values all sit comfortably inside that, so no scaling is needed.

To quantize by hand, find the binade (the power-of-two bracket) each value falls in, then round to the nearest of the 8 mantissa steps within it.

Element 20, x = 10.10. It lies in $[8, 16)$ , so the exponent is $2^3 = 8$ and the mantissa step is $8/8 = 1.0$ . Available points: 8, 9, 10, 11, 12.

Element 17, x = -3.54. It lies in $[2, 4)$ , exponent $2^1 = 2$ , mantissa step $2/8 = 0.25$ . Available points: 3.00, 3.25, 3.50, 3.75.

Element 8, x = 0.07. It lies in $[0.0625, 0.125)$ , exponent $2^{-4}$ , mantissa step $2^{-4}/8 = 0.0078125$ .

The tiny value got a tiny absolute error, because FP8 carries its own exponent per element. It does not care that another element in the tensor is 144 times larger. Every value gets roughly 6% relative precision regardless of magnitude.

FP8 result: MSE = 0.00127, max absolute error = 0.110. No value is driven to zero.

Aside: FP8 is two formats

The FP8 standard defines two encodings, and which one you use depends on what the tensor is for. The convention, from NVIDIA's FP8 Formats for Deep Learning and baked into Transformer Engine's Format.HYBRID:

E4M3 for the forward pass (weights and activations). E5M2 for the backward pass (gradients).

Why that split

Both are 8 bits. They differ only in how those bits are divided between exponent and mantissa — which is the same range-versus-resolution trade we saw between E8M0 and E4M3 scales, now applied to the elements themselves:

Deriving E5M2's maximum the same way we did E4M3's: 5 exponent bits give 32 fields with a bias of 15, and the top field is reserved for infinity and NaN. So the largest usable field is $e = 30$ , exponent $30 - 15 = 15$ , base $2^{15} = 32{,}768$ . Two mantissa bits reach a significand of $1 + \tfrac{3}{4} = 1.75$ :

That is 128× E4M3's reach, bought by moving exactly one bit from the mantissa to the exponent — one bit doubles the bracket count while halving the points inside each, so range grows exponentially while resolution drops by half.

Why E5M2 for backward pass: Weights and activations are well-behaved in magnitude but need precision; gradients are the opposite. Gradient magnitudes swing across many orders of magnitude during training, and the small ones matter — a gradient that underflows to zero contributes nothing to the update. Gradients also tolerate coarse values well, since they are averaged over a batch and noisy to begin with.

Our tensor demonstrates the forward-pass half directly. It looks like activations — moderate magnitudes, no extreme range — and E4M3 is the clear winner:

Flip to gradient-like values and the verdict reverses, because E4M3's floor is $2^{-9}$ :

A gradient of $10^{-5}$ simply ceases to exist in E4M3. In E5M2 it survives, imprecisely, but still preferable to underflow.

Why this matters for the rest of the article

Because FP8 uses one scaling factor for the whole tensor, E5M2 is needed for backward pass. With a single per-tensor scale, every value must fit inside one format's dynamic range at once, so a tensor spanning many orders of magnitude forces us to spend bits on range — meaning E5M2 and its coarser mantissa.

Block scaling dissolves that constraint. NVIDIA's Transformer Engine documentation notes that MXFP8, by giving each block of 32 values its own scale, reduces the dynamic range any single block has to cover — so MXFP8 uses E4M3 everywhere, forward and backward. The recipe is Format.E4M3 for MXFP8 versus Format.HYBRID for plain FP8.

(The split is a convention, not a law. DeepSeek-V3 trained with E4M3 in both directions. Recent work on FP8 reinforcement learning found the hybrid recipe tracked a BF16 baseline while pure-E4M3 destabilised around 500 steps, gradient norms spiking as the range ran out — so the trade is real, and it depends on how wide your gradients actually spread.)

And for FP4? There is only one

The natural follow-up is whether FP4 has an equivalent pair. It does not. The OCP microscaling specification defines two element encodings for MXFP8 (E4M3 and E5M2) and two for MXFP6 (E2M3 and E3M2) — but for MXFP4 it lists exactly one: E2M1. NVFP4 uses the same scales for both forward and backward pass too.

Two reasons for not splitting the precisions between forward and backward pass:

First, four bits leaves nothing to trade. After the sign bit, three bits must be divided between exponent and mantissa, which gives only four possible layouts — and the ones on either side of E2M1 are degenerate:

Quantizing our tensor with each, holding the scaling identical (NVFP4-style 16-value blocks, ideal fractional scale) so that only the element layout varies:

E2M1 has the lowest overall MSE. On the quiet block, the uniform layouts are best — E1M2 beats E2M1 by more than 2×, because when values are evenly spread, evenly spaced grid points fit them better. E2M1 wins on the strength of block B, which is a non-uniform grid with points bunched near zero and stretched at the top.

(Note E1M2 and INT4 score identically. That is not a coincidence — E1M2's grid is 0, 0.5, 1.0 … 3.5, which is INT4's 0…7 scaled by one half, and a free per-block scale absorbs the half. E1M2 is INT4. This is also why INT4 remains genuinely competitive for weight-only quantization, where distributions are well behaved, and loses to FP4 on activations, where outliers are the norm.)

Second, block scaling removes the reason a second format would exist. With one scale per tensor, gradients spanning many orders of magnitude need a format that carries its own range. Give every 16 or 32 values their own scale and that pressure vanishes; the scale supplies the range, so element bits are better spent on mantissa.

AMD's engineers put the same point about FP6, observing that E2M3 usually outperforms E3M2 in practice because the scaling factor already captures most of the dynamic range, which makes the extra mantissa bit the more valuable one. The identical logic pushes FP4 toward mantissa — but with only three bits to allocate, pushing one step past E2M1 lands on E1M2, which surrenders the non-uniform grid that outliers require. E2M1 is not one option among several; it is the only balance point that fits.

So the progression across the MX family tells a consistent story: 8 bits leave enough room for two encodings, 6 bits still leave two, and by 4 bits the trade space has collapsed to a single point. The fewer element bits you have, the more the work shifts from the format to the scale.

Format 2: FP4 by hand

Plain FP4 is E2M1 with a single FP32 scaling factor applied in software across the whole tensor.

Step 1: find the scale.

Step 2: read off the effective grid. Multiply the E2M1 grid by s:

0    0.84   1.68   2.53   3.37       5.05       6.73            10.10
|      |      |      |      |          |          |                |

The dead zone is everything below $0.25 \times 1.68333 = 0.42$ .

Step 3: quantize a few elements.

Element 20, $x = 10.10$ :

The outlier is perfect, by construction. Now block A.

Element 1, $x = 0.31$ :

Element 8, $x = 0.07$ :

Element 2, $x = -0.47$ :

That last one is worth staring at. The value -0.47 was nearly in the dead zone, and rounding it out to the first grid point nearly doubled its magnitude. When a value sits near the dead-zone boundary, FP4 has only two options and both are badly wrong.

Thirteen of block A's sixteen values collapse to zero. A single scale chosen for a tensor whose amax is 10.10 simply cannot see values around 0.3.

FP4 result: MSE = 0.06735, max absolute error = 0.410. Block A alone has MSE 0.0849; block B has 0.0498.

Format 3: MXFP4 by hand

MXFP4 introduces the first innovation: hardware-supported block scaling. Instead of one scale for the tensor, each 32-value block gets its own scale — and the Tensor Core handles the grouping and rescaling natively.

The catch is how the scale is stored. MXFP4 uses E8M0: 8 exponent bits, no sign bit, and no mantissa bits at all. An E8M0 value is exactly a power of two, $2^n$ . It is compact and rescaling becomes an exponent addition, but it means the scale can never be 1.68 — only 1, or 2, or 0.5.

Step 1: compute the shared exponent. The OCP microscaling spec sets

where $\text{emax}{\text{elem}} = 2$ for E2M1, because $2^2 = 4$ is the largest power of two on the E2M1 grid. Note that $\text{emax}{\text{elem}}$ refers to the exponent of the largest normal number. In computing, a normal number is a non-zero floating-point number that falls within the standard, balanced range supported by a hardware format without losing precision. $\lfloor \cdot \rfloor$ denotes floor operation, which is used because the scaling is E8M0.

Our tensor is exactly one 32-value block, amax = 10.10:

Step 2: read off the effective grid.

MXFP4 (s = 2):    0     1     2     3     4        6        8              12
FP4   (s = 1.68): 0    0.84  1.68  2.53  3.37     5.05     6.73            10.10

Look at where the top of the grid sits: 12.0, when our data only reaches 10.10. The power-of-two constraint forced us to overshoot.

Step 3: quantize the outlier.

The neighbouring grid points are 4 and 6. Distances: $|5.05 - 4| = 1.05$ and $|5.05 - 6| = 0.95$ . So it rounds up:

The largest value in the tensor is reconstructed 19% too large. This is the failure mode NVIDIA's Figure 3 illustrates, and it is a direct consequence of the coarse scale: with a fractional scale we could have placed a grid point exactly on 10.10.

Step 4: quantize block A. With a dead zone of 0.50 and block A's largest magnitude being 0.47:

All sixteen values of block A quantize to zero. The entire quiet half of the tensor is deleted.

MXFP4 result: MSE = 0.19549, max absolute error = 1.900. Block A MSE 0.0941; block B MSE 0.2969.

MXFP4 scored worse than plain FP4 on this tensor. With only 32 values, our tensor is a single MXFP4 block, so MXFP4 gets none of the block-scaling benefit it was designed for while paying the full cost of the coarse E8M0 scale. On a large real tensor with many blocks, MXFP4 comfortably beats a single per-tensor scale. What this example isolates cleanly is the cost of the power-of-two constraint, and we will pay that debt off properly in the ablation section.

(Some round the shared exponent up rather than using the floor rule, which trades the overshoot on amax for a wider dead zone.)

Format 4: NVFP4 by hand

NVFP4 changes both knobs at once.

Knob one: the block shrinks from 32 values to 16. Our tensor now has two blocks instead of one, and block A finally gets a scale chosen for block A.

Knob two: the scale is stored in E4M3 instead of E8M0. E4M3 has 3 mantissa bits, so it can express fractional scales — 1.75, 22, 3.5 — not just powers of two. This is what NVIDIA means by "high-precision scale encoding."

But E4M3 tops out at 448 and bottoms out around 0.00195, which is a much narrower range than E8M0's $2^{\pm 127}$ . A tensor whose blocks need wildly different scales could run off the end. This is what the second-level FP32 per-tensor scale is for: it pre-normalizes the whole tensor so that every block's scale lands inside E4M3's range.

Step 1: the per-tensor FP32 scale

The standard recipe picks the global scale so that the block containing the tensor's amax maps to E4M3's maximum, 448:

Conceptually we are working with $x' = x \times s_{\text{global}}$ , and each block's scale is computed on the stretched values. The 6 accounts for the E2M1 grid maximum; the 448 for the E4M3 maximum. Together they guarantee every block scale fits.

Step 2: block A's scale

Note that the microscaling rule with E4M3 is to divide amax by 6, which is what is used in the formula above.

Then round 20.848 into E4M3. It lies in $[16, 32)$ , so the mantissa step is $16/8 = 2$ . The candidates are 20 and 22.

Nearest would be 20 — but scale factors are rounded up, not to nearest, because a scale that is too small clips the block's outlier. (Appendix B works through why.) So:

Decode side, the effective scale is:

Block A's grid, obtained by multiplying E2M1 by 0.082664:

0   0.0413  0.0827  0.1240  0.1653  0.2480  0.3307       0.4960
|      |       |       |       |       |       |            |

The dead zone is now $0.25 \times 0.082664 = 0.0207$ . Compare that to MXFP4's 0.50 — it is 24 times narrower. Block A is no longer invisible.

Step 3: quantize block A by hand

Element 1, $x = 0.31$ :

Element 2, $x = -0.47$ :

Element 8, $x = 0.07$ :

Element 3, $x = 0.12$ :

Every one of these was exactly 0.00 under MXFP4. Zero of block A's sixteen values are lost.

Step 4: block B's scale

Exactly 448 — no rounding at all. This is because we defined $s_{\text{global}}$ so the block holding the tensor amax would land precisely on E4M3's ceiling. The effective scale is

which is the ideal scale from our formula, recovered exactly.

The outlier that MXFP4 reconstructed as 12.00 is now exact.

Other elements in block B still take real damage, because a block whose amax is 10.10 has a coarse grid no matter how precisely you encode the scale:

Based on the calculations for block B, we can see that NVFP4 does not rescue a block that genuinely contains a 144:1 dynamic range. Block B's MSE is 0.0498 under both plain FP4 and NVFP4. What NVFP4 buys is that the outlier's damage is contained within its own 16 values instead of spreading across 32.

NVFP4 result: MSE = 0.02555, max absolute error = 0.357, SQNR = 23.0 dB. Block A MSE 0.0013; block B MSE 0.0498.

The scoreboard

NVFP4 delivers 7.6× lower MSE than MXFP4 and 2.6× lower than plain FP4, at 4 bits per element in all three cases.

The gap between NVFP4 and FP8 is still significant — NVFP4 has x20 more MSE than FP8. But most of NVFP4's errors are in block B, and specifically in the values that share a block with a 21× outlier. On a tensor without that pathology, NVFP4 closes a lot of the remaining distance.

Appendix A: the E4M3 grid by hand

Understanding NVFP4's scale rounding requires knowing what E4M3 can actually represent. It has 1 sign bit, 4 exponent bits (bias 7), and 3 mantissa bits.

Within any binade (bracket) $[2^e, 2^{e+1})$ there are exactly 8 representable values, spaced $2^e/8$ apart — three mantissa bits split each bracket eight ways instead of E2M1's two. So the absolute spacing doubles with each binade while the relative spacing stays at about 12.5%.

Here is the complete list — all 16 exponent fields, not a sample:

Why the brackets do not start at 1. The list runs down to $[0, 0.015625)$ . Fractional values get brackets exactly like large ones, because the bias of 7 makes stored exponent fields decode to negative exponents: field 1 means $1 - 7 = -6$ , giving the bracket $[2^{-6}, 2^{-5})$ . Bias exists precisely so that a non-negative bit pattern can express a negative exponent; it is $2^{E-1} - 1 = 2^3 - 1 = 7$ for E4M3's four exponent bits, and §4 derives that formula. The smallest positive value E4M3 can hold is a subnormal at $2^{-9} \approx 0.00195$ , not 1.

Where the 0000 row's numbers come from. Both follow from the subnormal rule. When the exponent field is 0, two things change: the implicit leading 1 is dropped, so the significand becomes $m/8$ instead of $1 + m/8$ , and the exponent is pinned at $1 - \text{bias} = -6$ rather than $0 - \text{bias}$ . Enumerating the eight mantissa settings:

giving 0, 0.00195, 0.00391, … 0.01367. The spacing is the step between consecutive $m$ , which is one eighth of $2^{-6}$ :

The upper limit is then forced: the largest subnormal is $\tfrac{7}{8} \times 2^{-6} = 0.01367$ , and one step further lands on $\tfrac{8}{8} \times 2^{-6} = 2^{-6} = 0.015625$ — which is exactly the smallest normal value, the 0001 row's base. So the subnormal region ends precisely where normals begin, which is why the row is written $[0,\ 0.015625)$ .

Why the exponent is pinned at $1-\text{bias}$ , not $0-\text{bias}$ . This is the part that looks arbitrary and is not. Using $-6$ makes the subnormal spacing $2^{-9}$ identical to the spacing inside the first normal bracket $[2^{-6}, 2^{-5})$ , which is also $2^{-6}/8 = 2^{-9}$ . The two regions therefore join seamlessly: every value from 0 up through $2^{-5}$ is a multiple of $2^{-9}$ , a single uniform ladder of 17 rungs with no discontinuity at the boundary.

Had the exponent been $0 - \text{bias} = -7$ instead, the largest subnormal would be $\tfrac{7}{8} \times 2^{-7} = 0.00684$ , leaving a gap of 0.0088 before the smallest normal at 0.015625 — a hole nine times wider than the local step size. Dropping the implicit leading 1 is what lets the significand reach 0 at all (so zero is representable, and values shrink gradually toward it instead of falling off a cliff), and pinning the exponent one notch up is what stops that fix from tearing a hole in the grid.

E2M1 uses the identical rule with its own numbers: exponent $1 - 1 = 0$ , significand $m/2$ , giving 0 and 0.5 — and 0.5 is exactly the spacing inside $[1, 2)$ as well. That is why the spacing column in the bracket table stalls at 0.5 for the bottom two rows instead of continuing to halve.

Why the top bracket is $e = 15$ , not $e = 16$ . Four bits count from zero: the sixteen patterns are 0 through 15, and 1111 is 15. There is no bit pattern for 16 — that would need a fifth bit ( $16 = \texttt{10000}$ ). So the field spans $e = 0 \ldots 15$ , which after subtracting the bias gives exponents $-6$ through $+8$ . Sixteen fields, one spent on subnormals, fifteen normal brackets.

Why the range ends at 448 rather than infinity. Because E4M3 has no encoding that means infinity at all. IEEE formats spend their entire top exponent field on Inf and NaN; E4M3 spends one code on NaN and none on Inf, so there is simply no bit pattern left over to represent "unbounded." Values that exceed 448 saturate to 448 instead of becoming Inf. Note also that every float format ends at some finite maximum — in FP32 it is about $3.4 \times 10^{38}$ , and infinity is a separate special encoding beyond it, not the endpoint of the numeric range. E4M3 just omits that special encoding. The final row is written $[256, \mathbf{448}]$ with a closed bracket because it is a truncated bracket: it would have run to 512, but the NaN reservation removes its top code.

Where 448 comes from. The top bracket is $e = 15$ , so the exponent is $15 - 7 = 8$ and the base is $2^8 = 256$ .

Why the significand reaches 1.875. Three mantissa bits hold an integer $m$ from 0 to 7, and the significand is $1 + m/8$ — the denominator being $2^M$ , one for each mantissa bit. Read as a binary fraction it is simply 1. followed by the three stored bits:

So 111 gives $1 + \tfrac{7}{8} = 1.875$ , the largest significand any bracket can hold. Note it never reaches 2.0 — it stops exactly one step short, which is what keeps a value inside its own bracket instead of spilling into the next.

Why the top bracket stops at 1.75. The single code reserved for NaN is exponent 1111 with mantissa 111 — which is precisely the code that would otherwise have meant $1.875 \times 256 = 480$ . With 111 unavailable at $e = 15$ , the largest usable significand there is 110, i.e. 1.75:

NVFP4 needs the second-level FP32 scale because E4M3 spans about $2^{18}$ in magnitude, as compared to E8M0 which spans $2^{254}$ . Without pre-normalizing the tensor, a block whose amax is very small relative to the tensor could need a scale below E4M3's floor and would be unrepresentable.

Appendix B: why scale factors round up, not to nearest

When we computed block A's scale, the ideal was 20.848 and the E4M3 candidates were 20 and 22. Nearest-rounding says 20, but we chose 22.

The block's job is to represent amax = 0.47. The grid's largest point is $6 \times s$ , so we need:

With $s' = 20$ , the effective scale is $20 / 266.1386 = 0.075147$ , and the grid maximum is:

The block's largest value clips. It reconstructs as 0.4509, an error of 0.019 — and clipping is worse than it looks, because it is a systematic bias rather than symmetric rounding noise. Every value near the top of the block gets pulled inward in the same direction.

With $s' = 22$ , the effective scale is 0.082664 and the grid maximum is:

No clipping. The cost is that the grid is now about 5% wider than necessary, so every value carries slightly more rounding error.

The tradeoff is asymmetric. A slightly coarse grid spreads a little extra error evenly across all 16 values; a clipped grid inflicts concentrated, one-directional error on the largest values — which, in a matrix multiply, are the ones contributing most to the output. Rounding scales up is the safe direction.

Note that block B avoided this question entirely: its ideal scale was exactly 448. That is a designed property of the global-scale recipe, not a coincidence, and it means the block containing the tensor's amax always gets a mathematically perfect scale.

Why rounding up is cheap. Choosing the next representable value above a target is arithmetic because of monotonic bit ordering. Read a positive float's encoding as a plain unsigned integer and you get its position in the sorted list of representable values — E4M3's 127 non-negative values encode as exactly 0, 1, 2, … 126, with no gaps:

Think of them as house numbers on a street. The houses sit at wildly uneven distances — 0.002 apart near zero, 32 apart near 448 — but the numbers still run 1, 2, 3, 4, so "the next house up" is always just add one, regardless of the physical gap.

Hence, vonverting 20.848 lands on house 90, the value 20. It is too small, so move to house 91: the value 22. Conversion hardware offers a round-toward- $+\infty$ mode that does it directly. Rounding scales up is the numerically safe choice and the cheap one, which is a large part of why the convention exists.

Appendix C: the full 32-value table

Block A (quiet, amax = 0.47)

Block B (loud, amax = 10.10)

Scale factors used

Appendix D: Why E2M1 stops at 6, E4M3 stops at 448

Think of floating-point math like a machine with two dials:

1. The Multiplier Dial (The Exponent): This takes huge power-of-two jumps (1, 2, 4, 8, 16, 32...).
2. The Fine-Tuning Dial (The Mantissa): This is a decimal slider that always sits between 1.0 and just under 2.0.

To get your final number, the simply multiplies the two dials together:
Final Number = Multiplier × Fine-Tuner

To find the maximum possible number in any format, we turn both dials all the way up to their maximum settings.

1. Why E2M1 stops at 6

E2M1 only has 4 bits.

• 1 bit is for the plus/minus sign.
• 2 bits are for the Multiplier dial.
• 1 bit is for the Fine-Tuning dial.

The Multiplier Dial (2 bits):
With 2 bits, you only have 4 possible settings (00, 01, 10, 11). The computer assigns these to the multipliers: 0.5, 1, 2, and 4.

• Maximum Multiplier = **4**

The Fine-Tuning Dial (1 bit):
With 1 bit, the dial only has two clicks: 0 or 1. The computer assigns these to the decimals 1.0 and 1.5.

• Maximum Fine-Tuner = **1.5**

The Math:
To get the biggest number, multiply the maximums together.

4 x 1.5 = 6.0

With only 4 bits, the machine simply doesn't have enough notches on its dials to reach anything higher than 6.

2. Why E4M3 stops at 448

E4M3 (used for FP8 and NVFP4 metadata) has 8 bits. It gives us much bigger dials.

• 1 bit is for the plus/minus sign.
• 4 bits are for the Multiplier dial.
• 3 bits are for the Fine-Tuning dial.

Let's turn these dials to the max:

The Multiplier Dial (4 bits):
With 4 bits, we have 16 settings. The computer scales these powers of two much higher: 1, 2, 4, 8, 16... all the way up to 256.

• Maximum Multiplier = **256**

The Fine-Tuning Dial (3 bits):
With 3 bits, we have 8 clicks. The computer slices the space between 1.0 and 2.0 into eighths: 1.0, 1.125, 1.250, etc. The very highest click on this dial is 1.875.

The Math (and the Twist):
If we multiply the maximums together, we get:
256 x 1.875 = 480

Why is the limit 448 and not 480?
Because of a safety rule: computers need a special code to represent an "Error" or "Not a Number" (NaN) — for example, if you try to divide by zero.

Instead of adding an extra bit for errors, the engineers designed E4M3 to sacrifice its absolute highest setting. The computer is not allowed to turn both dials to their absolute max at the same time. That specific combination (256 + 1.875) is locked and reads as "Error".

So, to get the highest real number, we keep the Multiplier at 256, but we have to click the Fine-Tuning dial down by exactly one notch.
The notch just below 1.875 is 1.75.

So the final math is:
256 x 1.75 = 448

Summary

• E2M1 maxes out at 6 because its biggest multiplier is 4 and its biggest fine-tuner is 1.5. (4 x 1.5 = 6).
• E4M3 maxes out at 448 because its biggest multiplier is 256, and its biggest legal fine-tuner before triggering an error code is 1.75. (256 x 1.75 = 448).

References

• Introducing NVFP4 for Efficient and Accurate Low-Precision Inference — NVIDIA Technical Blog
• OCP Microscaling Formats (MX) Specification — for the E8M0 shared-exponent rule
• FP8 Formats for Deep Learning — Micikevicius et al., the paper defining E4M3 and E5M2
• Using FP8 and FP4 with Transformer Engine — the hybrid recipe, and why MXFP8 needs only E4M3
• NVIDIA TensorRT Model Optimizer and LLM Compressor — for quantizing your own models to NVFP4