Undefined type Float8_e4m3fn on Apple Silicon: BF16 and GGUF Workarounds for FP8 Models

Two FP8 image models crashed with Undefined type Float8_e4m3fn on an Apple Silicon Mac, and each needed a different fix — a BF16 swap for one, a GGUF build (plus a loader patch) for the other — with measured generation times for both.

When you load an FP8-quantized image generation model on an Apple Silicon Mac (MPS backend), it can stop with this error:

Undefined type Float8_e4m3fn

I hit this on a 32GB unified-memory Apple Silicon machine (M5) with a ComfyUI-based setup, on two separate models — FLUX.2 and Krea 2 Turbo — and worked around each in a different way, taking measurements afterwards. This article is that record.

• Case 1 (FLUX.2): solved by swapping only the main model for the BF16 build
• Case 2 (Krea 2 Turbo): no realistic BF16 option, so I switched to a GGUF-quantized build (plus a patch for the loader's architecture tag)

What is actually happening

FP8 (float8_e4m3fn) is usually distributed as a quantization format aimed at NVIDIA GPUs, and the MPS backend does not support arithmetic on that dtype. In my environment, loading the official FLUX.2 FP8 build (flux-2-klein-4b.safetensors, about 4.07GB) stopped with Undefined type Float8_e4m3fn the moment it entered the first linear layer.

The key point is that this is not "out of memory" — it is an unsupported dtype. Lowering the resolution or cutting the step count will not fix it. FP8 builds look attractive because the files are small, but on a Mac you have to plan your setup on the assumption that they simply will not run.

Case 1: Swap only the main model for BF16 (FLUX.2)

FLUX.2 ships an official BF16 build, so I replaced only the diffusion model itself with the BF16 version. That was a single file; everything else was reused as-is.

What is interesting is that the fp8_scaled text encoder ran on MPS without any change. Not every file with "FP8" in its name is doomed; what breaks is the place where a float8_e4m3fn tensor actually enters an operation. Reading which layer the error came from, and keeping the swap as small as possible, also turned out to be the better choice for disk space (deleting the failed FP8 model reclaimed about 4GB).

Measurements after the BF16 swap (32GB machine, MPS)

BF16 files are roughly twice the size of FP8, but with 32GB of unified memory even an 800x1408 generation with reference images stayed at a peak of about 27GiB. On a 16GB machine this setup will be rough — in that case the GGUF route below is the main option.

Case 2: Switch to a GGUF-quantized build (Krea 2 Turbo)

The FP8 build of Krea 2 Turbo failed with the same Float8_e4m3fn error. Here, loading BF16 outright was too heavy, so I moved to a GGUF-quantized build (Q6_K, 10.6GB), which ran stably on MPS.

GGUF is a quantization format that came out of the LLM world, but it can be used for image generation models through ComfyUI-GGUF. For a Mac that cannot use FP8, it is the realistic way to "keep the file small and still run".

The trap: mismatched GGUF architecture tags

With community-distributed GGUF files (for example krea2_raw_bf16-Q5_1.gguf), the architecture tag inside the GGUF is a custom value (krea2), and the ComfyUI-GGUF loader rejects it as an unknown architecture.

I solved this with a local patch that adds a one-line alias to the architecture check in ComfyUI-GGUF/loader.py (treating krea2 as a compatible architecture name). The idea looks like this:

# near the architecture check in loader.py (conceptual code)
if arch == "krea2":
    arch = "qwen_image"  # alias-resolve to a compatible architecture

Two caveats:

1. A git pull wipes the patch. If a GGUF that worked yesterday suddenly fails to load after a custom node update, suspect this first.
2. Spoofing the tag is only valid when the contents really are a compatible architecture. If the model is genuinely something else, it will break, as you would expect.

Decision guide

1. Undefined type Float8_e4m3fn is an unsupported dtype. Resolution and step tweaks will not fix it
2. If an official BF16 build exists, swapping only the component that errors is the shortest path (the surrounding files often keep working unchanged)
3. If BF16 is too heavy for your memory, a GGUF-quantized build is the practical answer. At the Q6_K level I could confirm the quality is usable
4. If the loader rejects your GGUF, check the architecture tag. Manage the patch on the assumption that updates will erase it

I also published a model-agnostic version of this triage procedure (a four-way classification of the errors, what to record, and a decision table) on ACS Developer: Apple Silicon FP8 model errors and BF16 alternatives, measured

Originally published in Japanese on Zenn: https://zenn.dev/acs_developer/articles/apple-silicon-mps-float8-e4m3fn-workaround