esatapedico's laboratory

Notes on AI, open-source, DevOps, hardware, and technical experiments.
devto
NVFP4 tiers, triaged: which file for your card, what it costs, what breaks first

NVFP4 tiers, triaged: which file for your card, what it costs, what breaks first

Objective: you leave knowing which file to download for your card. Our lab publishes tiered NVFP4 versions of Qwen3.8-27B, from VERY-LOW to HIGHEST, plus two tiny tiers for small cards. Developers keep filing the same tickets. Here is the fix that held up for each, the measurement behind it, and what it costs. Symptom: fastest Qwen3.8 at home on a Blackwell card. True fix: the LOW tier. What does it cost? A few points of fidelity for a large multiple of speed. → A community member measured one hundred forty eight tokens per second on average with peaks near two hundred. The llm-bench aggregator recorded one run of our flagship file at one hundred thirty four tokens per second with quality near eighty seven. Perplexity testers measured the same files below plain Q4 at equal size. One wrote that every tested NVFP4 file was worse than his daily Q4 driver, and his follow-up landed at seven point four nine, still behind at equal speed. That is the whole deal.