Technical
The FP8 File That Silently Runs at FP16: A VRAM Gotcha Worth Knowing
ยท RenderBob team
A checkpoint saved in FP8 does not guarantee it runs at FP8. Some loaders and backends upcast the weights, so the file is small and the memory it consumes is not.
A checkpoint file saved in FP8 does not guarantee it actually runs at FP8. The classic trap is to grab an FP8 variant specifically to fit a VRAM-constrained card, confirm the file is smaller on disk, and assume the VRAM saving will follow. Then you OOM anyway, not because FP8 cannot work, but because the loader, the hardware backend, or an unset quantisation control upcast the weights before they reached the GPU.
ComfyUI's load log makes this easy to misread. Most loaders print a line such as model weight dtype torch.float8_e4m3fn, manual cast: torch.bfloat16. That manual-cast field is often normal even when FP8 matrix multiplies are running, because accumulation still happens in higher precision. It is not, by itself, proof that the VRAM saving is gone. On some backends it is the real failure. ComfyUI issue #11519 documents AMD ROCm setups, including gfx1201, where FP8 models are force-loaded and computed at FP16 or BF16, wasting the memory the filename promised. HunyuanVideo-Foley's own loader docs warn about a simpler version of the same bug: leave quantisation unset on an FP8 safetensors file and it is upcast to FP16 in memory.
How to check what you actually loaded
Read the load log, then cross-check against resident VRAM during a run, not the file size on disk. A model that is genuinely running at FP8 should show meaningfully lower memory than its FP16 equivalent. If it does not, something in the load path is upcasting. Where a node exposes a quantisation mode, set it to the format you intend rather than leaving auto. Launch-level flags such as --fp8_e4m3fn-unet, covered later this week, are a more forceful alternative when a node-level setting is not producing the reduction it should.
A technique existing in the ecosystem is not the same as it working on your card, your backend, and your loader. Verify FP8 on real production shots and real target hardware before you trust the filename, or an unexplained OOM will eat the deadline on hardware that should have had headroom.
More from the blog
- Block Swap, Explained: The Technique Behind Almost Every Low-VRAM Video Workflow
Scroll through almost any low-VRAM video workflow in ComfyUI and you will find block swap doing the heavy lifting under a different node name each time.
- W4A4 and W4A8: The New Quantization Toolkit for Squeezing More Out of a Checkpoint
ComfyUI's Quantization Toolkit, formerly the INT8 Toolkit, adds native W4A4, W4A8 and W8A8 quantisation, including LoRA support. Here is what the notation means and how to pick a mode.