Z.ai’s GLM-5.3-Flash (320B total / 18B active, natively multimodal MoE with hybrid KDA linear attention + DeepSeek sparse attention, 1M context) shipped in August 2026 as a ~643 GB BF16 repo, with a 328 GB “main” repo that is actually a mixed FP8 export (BF16 attention, FP8 experts). NVFP4 — 4-bit E2M1 weights with FP8-E4M3 per-16 block scales and an FP32 global scale, a native Blackwell tensor-core format — is the obvious fit for it on GB10/GB300/RTX-class hardware, and within weeks five distinct quantizations had accumulated 100K, 87K, 51K, 26K and 7K downloads respectively. This note is a deep dive into what actually differs between them: which tensors are quantized and how, on-disk size, claimed accuracy, real measured speed on Spark-class hardware, and the serving gotchas each one carries.

Popularity here means Hugging Face download counts of repos tagged base_model:quantized:zai-org/GLM-5.3-Flash (or its BF16 sibling), as of 2026-10-09. The most-downloaded GLM-5.3-Flash quant overall is unsloth’s GGUF (~1M downloads) — excluded, it is not NVFP4, and llama.cpp has no glm5_next support anyway. ¹

The field

(The literal #5 by downloads is dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4 at 7,198 — covered separately at the bottom, because it is not a distinct quantization.)

Reference points: the BF16 source is 642.7 GB ⁶, the FP8-mixed main repo 328.3 GB ⁷. Every NVFP4 lands within a 10 GB band (195-204 GB, 30-32% of BF16), because in all five the routed experts (~90-97% of parameters) go to 4 bits and everything else stays high precision. The differences are entirely in which everything else, which precision the activations use, and how the weights were produced.

TL;DR

• Want the safest default: nvidia/GLM-5.3-Flash-NVFP4 — the only one with a controlled same-harness BF16-vs-NVFP4 accuracy table (and the deltas are noise-level), broadest engine support (vLLM and SGLang), plus the community-validated Spark recipes.
• Want SGLang with NEXTN speculative decoding: RadixArk — the SGLang-native card, validated on 4× GB300, with audit files (tensor-audit-b.json, precision-contract-b.json) nobody else ships.
• Want vLLM with llm-compressor/compressed-tensors tooling: RedHatAI — cleanest W4A4 expert-only scope, MTP layer kept at FP8 for speculative decoding.
• Want minimal GPU memory / 2× GB10-class boxes: LibertAI — the only weight-only (NVFP4-A16) build, smallest of the five, and by far the best-documented pitfalls (including a wild input-scale saga, below).
• Want a checkpoint whose quality was trained against quantization: local-inference-lab — the only quantization-aware-distilled build (expert weights retrained under a BF16 teacher); no published benchmarks of its own, which is the main caveat.

The precision map (verified from tensor counts, not just READMEs)

All five are “MoE expert” quants — attention (both KDA linear layers and the 11 DSA sparse layers + indexer), routers, embeddings, lm_head, norms, and the vision tower stay BF16 everywhere. The real differences:

The scale-tensor arithmetic in each repo’s model.safetensors.index.json pins these down harder than the READMEs do, and it resolves three card-vs-card contradictions:

• RadixArk’s README says W4A4 with “abs-max scaling, no calibration needed”, but ships 36,423 input_scale tensors anyway — exactly 36,288 (routed) + 126 (shared experts, 42×3) + 9 (dense MLPs 0-2). That is a static, calibrated W4A4 checkpoint (their card does list the cnn_dailymail calibration), and the count also independently confirms the shared-expert and dense-MLP scope their README claims. ²
• NVIDIA’s card sentence “only the linear operators within sparse MoE shared experts and dense MLP are quantized” is misleadingly worded: the ignore list and the scale counts (36,297 input scales = 36,288 routed + 9 dense-MLP; block+global scales 2×36,297) show the shared experts are not quantized — matching their own ignore list and the GLM-5.2 card convention. ³
• LibertAI’s “what’s quantized” table says the MTP head stays BF16, but the config ignore list contains no layer-45 expert entries and the index ships 37,152 = 43×288×3 weight scales — the MTP routed experts are NVFP4 like every other layer (only MTP’s attention/eh_proj stay BF16). Their 37,152 input_scale tensors are the August-30 fix: 36,288 real calibrated values taken from RedHat’s calibration of the same base, 864 MTP entries filled with the median. ¹
• RedHatAI’s index shows exactly 36,288 NVFP4 weight scales + 864 FP8-era globals for layer 45, matching their claim of FP8 MTP experts; it is the only checkpoint where the draft layer’s experts are neither NVFP4 nor BF16. ⁴

Size and memory footprint

Two nuances on “which source is better”. Quantizing experts from an FP8 export means the 4-bit values derive from already-8-bit-rounded weights — dabsLabs built their (much-less-downloaded) quant specifically from the BF16 repo to avoid this “double quantization”. ⁸ In practice the measured penalty looks negligible: LibertAI (FP8 source) reports per-expert relative reconstruction error ≈0.0925 vs NVFP4’s intrinsic floor ≈0.093, and tattrongvu’s FP8-sourced quant measures 0.0918 — both at the format’s limit. ⁹ The other nuance runs the other way: on a memory-per-token basis what matters is that all five fit a 2× DGX Spark cluster at TP=2 (≈90-100 GB/rank) where the BF16 original cannot, which is exactly why NVFP4 became the Spark-community default.

Accuracy: what’s claimed, and what it can be compared against

Only NVIDIA publishes a same-harness BF16-vs-NVFP4 A/B — the single most useful table, since it isolates the quantization effect (GB200, temp 1.0, top_p 0.95): ³

That is noise-level degradation — consistent with NVFP4’s track record on other MoE families (NVIDIA’s GLM-5.2 and 5.1 NVFP4 cards show the same pattern). ¹⁰

The others publish absolute numbers under different harnesses/hardware, so they cannot be ranked against each other — treat the following as “is it broken?” checks, not rankings:

• RadixArk (4× GB300, SGLang, NEXTN spec): GSM8K 97.14% (4 seeds), AIME 2026 92.45% (1,920 generations), Terminal-Bench 2.1 83.1% pass@1. Note their TB 2.1 lands within 0.1 pp of NVIDIA’s 83.15% on a different stack — a decent cross-check. ²
• RedHatAI (vLLM, lm-eval-harness/lighteval, 3 seeds): GSM8K Platinum 97.74%, MATH-500 94.87%, AIME 2025 86.67%, GPQA Diamond 90.57% — no baseline row, so the −1.5 pp GPQA gap vs NVIDIA’s table is harness + hardware + seed variance, not evidence of a worse quant. ⁴
• LibertAI publishes nothing, deliberately: “we would rather publish nothing than publish a number we did not measure.” ¹
• local-inference-lab publishes no benchmarks either — the claim is structural (distillation compensates quantization error) rather than scoreboarded. ⁵

One qualitative counter-signal from the field: a DGX Spark forum user reported severe vision hallucinations on non-UI imagery while running the RedHatAI checkpoint (a second user suggested trying an EXL3 encode instead and reported the same images working there). ¹¹ One anecdote, wrong-tool-or-quant unknown — but it is the only “NVFP4 broke something for me” report I found for any of the five.

Measured speed on Spark-class hardware

All forum numbers below are for NVFP4 checkpoints (mostly NVIDIA’s), on GB10 DGX Sparks (121 GB unified memory each, ~270 GB/s bandwidth) unless noted. ^{12 13 14 15}

• 2× Spark, vLLM TP=2 (docker-compose patch chain): 262K ctx ≈ 14.3 tok/s bf16 KV → ≈21.8 tok/s with fp8 KV + MTP-4; later standing config 512K ctx, 24-30 tok/s c1, 180-364 tok/s chunked prefill, 1.26M-token fp8 KV pool, 89/100 tool-eval hardmode, byte-exact 440K needle retrieval.
• 3× Spark, TP=3 (LibertAI checkpoint, MTP-4): 35.2 tok/s c1 decode, ~1,800 tok/s prefill, 1.5M KV pool, 512K context. Author’s own conclusion: TP=3 is a memory upgrade, not a throughput upgrade — prefill and decode identical within noise vs TP=2; only weights/rank (90→64 GiB) and KV pool moved.
• 3× Spark, kindling stack TP=3 (NVIDIA checkpoint + DFlash2 drafter): ~3,100-3,500 tok/s cold prefill (64K prompt ready in 20 s), 156/179 tok/s aggregate prose at 8/12 streams (code ~480), 3.05M-token fp8 KV pool, 64-way serving; GSM8K 96.4%, HumanEval 94.5%. A second user reproduced: prefill ~3,200 tok/s at 260K prompts, decode 56.5 c1 prose / 98.7 JSON, TTFT 151-189 ms.
• Concurrency scaling is the weak spot: NVFP4 + DFlash2 on 2× Sparks went 33 → 43 tok/s aggregate from c1 to c4, versus 31 → 98 for DeepSeek-V4-Flash DSpark on identical hardware — the GLM stack “does not scale with concurrency” in that user’s testing; a native-MTP k=3 run on a W4A16 Intel quant scaled 58 → 73 from a higher floor.
• 4× RTX PRO 6000 Blackwell (96 GB each): the local-inference-lab checkpoint runs over PCIe-only TP=4 — no performance numbers posted. ¹⁶

For single-stream decode the NVFP4 Spark numbers cluster around 20-35 tok/s depending on stack, spec decoding, and context — nobody claims more, and the DFlash2-drafter variants trade concurrency headroom for that single-stream speed.

Serving gotchas that differ per checkpoint

• The LibertAI input-scale saga (a W4A16-in-vLLM hazard). vLLM’s ModelOptNvFp4FusedMoE requires an activation scale; a weight-only checkpoint without one gets an uninitialized scale — every expert multiplied by zero, model emits one token repeatedly, no error. LibertAI’s first fix (input_scale = 1.0) was itself wrong: the underflow bound is per-16-block, so 1.0 flushed fp8 block scales to zero for low-amax blocks and degraded output intermittently, worse with context (reproduced on GB300). Final fix: real per-projection calibrated scales (median 1.58e-03 — the 1.0 placeholder was 632× too high). The cleaner path — the compressed-tensors branch with no activation quant at all — still failed to load in their test. ¹ If you run weight-only NVFP4 through vLLM’s ModelOpt MoE path, check for exactly this.
• Reasoning/tool parsers are engine-specific and fail silently. SGLang wants --reasoning-parser glm45; vLLM’s glm45 is a different parser that silently discards the whole reply against this model (use deepseek_r1). Tool calls: --tool-call-parser glm (the GLM-4.5 parser) fails silently — empty content, tool_calls: null — use glm47. ¹
• GB10-specific: the 34 KDA linear-attention layers need a per-request recurrent state cache, so on tight memory concurrency, not KV, is the first thing that runs out (fix: lower --max-running-requests before context); the DSA TileLang backend is bf16-KV-only and needs a shared-memory tile patch on GB10 (169,984 B requested vs 101,376 B available — block_I=32, num_stages=1, threads=128). ¹ RadixArk’s card adds that fp8 KV ≈ 1.8× token capacity vs bf16 KV with the TRT-LLM DSA backends. ²
• Vendor image pinned per model: all five assume the glm53-flash per-model vLLM/SGLang images (glm5_next is not in upstream vLLM main); LibertAI additionally documents that vLLM launches must point at a local directory, not the repo id (a processor_config.json path-resolution bug). ¹

What about dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4 (the literal #5)?

It is an abliterated (“uncensored”) derivative of the base model, not a new quantization technique: 194.7 GB in 121 shards, 37,152 weight scales + globals and zero input scales — i.e. the same weight-only ModelOpt recipe as LibertAI’s original August-26 build (same shard count, same size), with no evidence of LibertAI’s August-30 input-scale fix. Its card’s real content is an unusually detailed serving-warning block (reasoning_effort levels resolving to max, empty replies under small max_tokens, media parameters silently ignored). If you download it for vLLM use, assume the uninitialized-input-scale failure mode applies and use the marlin backend or a scale plugin; for SGLang it should behave like any weight-only checkpoint. ¹⁷

Which one, practically

For this cluster’s world (GB10 Sparks, vLLM/SGLang, agentic serving): nvidia’s for the validated-accuracy default and the best-covered Spark recipes, RadixArk’s if you are SGLang-only and want NEXTN spec, RedHatAI’s if your tooling is compressed-tensors-based, LibertAI’s when memory is the binding constraint (and you read its warnings), and local-inference-lab’s if you want to bet that distillation beats post-training quantization — a bet its own repo, unusually, does not ask you to take on faith alone but also does not yet evidence with numbers.

Sources

Appendix, same day: the top 20 of every precision, accuracy only

Ranked strictly by HF downloads (2026-10-09 snapshot) among quants of the base model, excluding zai-org itself; any format counts. The NVFP4 five above are #2, #3, #4, #8, #20; the other fifteen slots go to GGUF (8 repos), FP8 (2), MXFP4 (2, AMD and NVIDIA-flavored), MLX (1), W4A16 int4 (2), and two structurally different builds (expert-pruned, expert-disk-streamed). This appendix is deliberately accuracy-only: precision/quality claims and measurements, no speed. ¹⁸

(For reference the original publisher’s repos are 6.35M / 67.7K downloads.) ¹⁸

What the accuracy evidence actually shows

The twenty cards split into three evidence cultures, and the differences between them matter more than any single number:

Culture 1 — benchmark scoreboards against an external baseline (the vendor-style quants): NVIDIA, RadixArk, RedHatAI, AMD, canada-quant, autotrust E224, dealignai-FP8, OneNexus. Absolute scores on GSM8K/AIME/GPQA/MMMU-class suites. Canada-quant’s companion page is the most transparent of the whole set (every grid, protocol, and a logged retraction of its own earlier NVFP4 comparisons — including one where NVIDIA’s checkpoint beat theirs — after discovering a drafter bug had corrupted the read). ²¹ OneNexus is the only repo that runs a paired statistical test against a competitor checkpoint on identical fixed samples, and reports its own +3.5 pp GPQA win over AMD as not significant (p = 0.265). ²³

Culture 2 — token-level fidelity against a reference run (the llama.cpp/MLX ecosystem): aj9o9, pfeifferj, orcarouter (×2), peasantsmith, patrickbdevaney, bartowski. Same idea everywhere: run the reference (BF16, FP8, or Q8_0) and the quant on identical text and measure perplexity ratio, KL divergence, and top-1 token agreement. The consistent finding across all six:

• 4-bit is nearly free, 2-bit is where models break. orcarouter-MLX: 4-bit PPL +2.96% (KLD 0.013), 3-bit +9.96%, 2-bit +56.9%, “2bit-lite” +141%. orcarouter-GGUF: Q4_K_M KLD 0.087, Q3_K_M 0.176, Q2_K 0.486 (“pick it for fit, not quality”). ^{24 25}
• 3-bit mixed-precision layouts can land within ~2 pp of an 8-bit reference on MMLU-Pro (pfeifferj: 60.55/60.00 vs 61.95 at 3.5/3.0 bpw) — non-uniform allocation buys a lot at small budgets. ²⁶
• Careful 2-bit + pruning can retain most capability on narrow domains while collapsing on breadth: autotrust’s 79 GiB 2-bit build passes 35/40 hard reasoning cases, and peasantsmith’s Maya-L holds 99.2% of FP8 zero-shot accuracy while being ~2-bit in its experts — but REAP50’s own per-domain table shows the cost is pushed onto generic text (top-1 agreement 0.92 on math, 0.58 on “ballast” prose). ^{27 28 29}
• The measurement harness itself is a hazard on this architecture: aj9o9 documents that llama-perplexity’s default multi-sequence batch path silently returns NaN or diverges on glm5_next, and that their own earlier published numbers used it (“any perplexity figure for this architecture published without -b 512 should be treated with suspicion”). ³⁰

Culture 3 — no measurement at all: unsloth (the #1 repo by 10× — its card here is a pointer to docs and its own Dynamic-3.0 marketing claim, with the base model’s vendor benchmark table reproduced) ³¹, huihui-ai (re-quant of unsloth’s files with layers 15-35 ablated, zero quality data) ³², Justvugg (honest about it: “no independent benchmark of this container has been published yet”, offers a GLM-5.2 analogy) ³³, LibertAI (deliberately, as covered above), wtdcode (a card listing only what’s quantized — no numbers) ³⁴, and the two prune/stream builds’ GGUF wrappers beyond what’s cited. The most-downloaded GLM-5.3-Flash quant in the world publishes no accuracy evidence for this model.

Cross-format accuracy reading, in one paragraph

Where numbers overlap, they tell one consistent story. Full-precision-class formats are all noise-level from their sources: NVIDIA’s NVFP4 A/B, dealignai’s +0.59 pp MMLU after weight editing, bartowski’s layout variants within ±8% KL of each other. Everything at ~4 bits (NVFP4, MXFP4, MLX 4-bit, GGUF Q4_K_M, int4 W4A16) lands in the same band — PPL penalties of roughly +3-9% and KLD of 0.01-0.09, or benchmark deltas within a couple of points — regardless of vendor or format, which is the practical meaning of “4-bit is free”: scheme choice matters far less than bit-width at this range. Quality differentiate at 3 bits and below, where the two learned-allocation attempts (GSQ-RCO’s tensor-type assignment, Maya’s expert-tiering) measurably beat naive uniform layouts of the same width, and where 2-bit files should be chosen for whether they fit, not for what they preserve — every measurement culture agrees on that cliff. The two caveats that survive scrutiny: pruning moves damage onto the domains its calibration didn’t protect (REAP’s own tables show it), and any perplexity number on this architecture measured with default batching is suspect (aj9o9’s NaN finding). ^{30 29 26 28}

Sources (appendix)

Download/like counts: Hugging Face Hub API, 2026-10-09. Popularity ranking method and all precision-scheme verification (scale-tensor counts, config ignore lists, quantization_config groups) are reproducible from each repo’s public model.safetensors.index.json and config.json.

Appendix B, day two: canada-quant’s int4 W4A16 vs NVIDIA’s NVFP4

The two most heavily audited cards in the GLM-5.3-Flash quant ecosystem ^{35 36} — and the two that mostly don’t overlap on benchmarks — side by side. Short version: they are different answers to different constraints, the schemes overlap less than the “4-bit” label suggests, and no clean accuracy head-to-head exists yet — canada-quant published one against NVIDIA’s checkpoint, then retracted it for cause and has not re-run it as of 2026-10-10.

Scheme, from the configs

Two asymmetries are worth dwelling on. First, weights-only vs weights+activations: W4A16 keeps activations in BF16, so its accuracy exposure is confined to weight rounding (GPTQ’s error feedback makes that small); NVFP4 additionally rounds every expert activation to 4-bit float at inference — a second error source NVIDIA’s A/B says costs nothing measurable on their six benchmarks, but which is structurally the riskier half of the scheme. Second, the MTP head: canada-quant keeps the speculative-draft head in BF16 inside the checkpoint and validated num_speculative_tokens: 2-3 per GPU; NVIDIA’s card ships the head but publishes no speculation recipe. If you serve with speculation — which on this model is worth +34-70% single-stream — canada-quant’s is the checkpoint with that path documented and measured.

Accuracy: what each card actually publishes

NVIDIA publishes the ecosystem’s only same-harness BF16-vs-quant A/B (GB200, temp 1.0, top_p 0.95): GPQA-D 0.9217→0.9211, SciCode 0.5621→0.5769, MMMU-Pro 0.7688→0.7630, AA-LCR 0.7100→0.7106, IFBench 0.6130→0.6054, Terminal-Bench 2.1 0.8258→0.8315 — noise-level on every row. canada-quant publishes absolute numbers with protocol detail (n=120 AIME cells split 4×30, 131K-token budgets, per-hardware breakdowns, miss autopsies — 16 of their 18 AIME 2026 misses were budget-exhaustion truncations, not wrong answers): AIME 2025 0.8833 (H100), AIME 2026 85.0% (2× DGX Spark), GSM8K 0.970-0.975, GPQA-Diamond 0.8586, MMMU 0.747, OCRBench 888.

Can these be compared? Only partially and only in one direction:

• vs the base model: there is no official zai text table (the vendor card’s scoreboard is an image), so canada-quant’s absolute scores have no same-protocol BF16 anchor. The closest cross-check is internal: their AIME 2026 85.0% beats the EXL3 quant’s 80.0% under a matched protocol on identical hardware — a real, paired quant-vs-quant result.
• vs each other: the one overlapping benchmark is AIME 2025, where canada-quant reports 0.8833 and RedHatAI’s NVFP4 86.67% — but different harnesses, hardware, and seeds, so that is a “nobody looks broken” data point, not a ranking. NVIDIA’s card does not report AIME or GPQA in canada-quant’s harness; canada-quant’s GPQA-D 0.8586 and NVIDIA’s 0.9211 differ in protocol (n=198 @131K vs GB200 temp-1.0) as much as in value.
• The retracted comparison is the honest centerpiece. canada-quant’s BENCHMARKS.md carries a dated retraction: earlier tables compared against two different NVFP4 checkpoints (LibertAI’s and NVIDIA’s) under one “NVFP4” label and ran NVIDIA’s checkpoint without its MTP head, which they had misread as missing — and they removed every NVFP4 comparison, “wins and losses alike”, including the 2× DGX Spark head-to-head where NVIDIA’s stack led ³⁷. A same-pair comparison with each side at its best configuration was running on 2× DGX Spark as of 2026-10-04 and one is registered for H100; neither result is published as of 2026-10-10. That is the correct scientific behavior — the opposite of card-inflation — and it means the fair answer to “which is more accurate?” is genuinely pending the authors’ own re-run.

Evidence-quality verdicts

• NVIDIA: narrow but deep. Six benchmarks, one direction (vs BF16), same harness, control experiment included; nothing on AIME/GSM8K-class suites, nothing on non-Blackwell hardware, no perplexity/KLD fidelity numbers.
• canada-quant: broad, transparent, and self-auditing — build gates (exactly 36,288 packed tensors, vision keys 348/348 identical to source, zero dtype drift, no collapsed expert scales), per-hardware grids, miss autopsies, a retraction log, and two upstream vLLM bugs filed with evidence. Absolute scores without a BF16 anchor, so its quality case rests on “sane protocol + matches expectations + beats EXL3 in a paired test”, which is honest but not an isolation of the quantization effect.
• Neither publishes token-level fidelity metrics (PPL ratio / KLD / top-1 agreement) — that measurement culture lives in the llama.cpp/MLX ecosystem (Appendix A), where 4-bit weight-only builds show PPL +3-9% / KLD 0.01-0.09. W4A16 int4 sits in that family; NVFP4 W4A4 has no equivalent published measurement anywhere for this model.

Which one, practically (accuracy lens)

For maximum preserved accuracy on Blackwell: NVIDIA’s NVFP4 — the only quant of this model with a controlled A/B, and the deltas are noise. For accuracy per GB and accuracy with speculation validated: canada-quant’s W4A16 — 177.7 GiB (13% smaller than NVFP4), GPTQ int4 with BF16 activations is the lower-variance scheme on paper, its AIME 2026 paired win over EXL3 is the cleanest quant-vs-quant result anyone has published, and it is the only card whose speculative path is benchmarked end-to-end. For Hopper/RTX-class hardware canada-quant is simply the portable choice (NVFP4’s W4A4 needs Blackwell tensor cores or the marlin fallback). The honest caveat either way: until canada-quant’s re-registered head-to-head publishes, no one has measured these two checkpoints against each other under a fair protocol — and the one attempt that existed was retracted by its own authors for good reason.

Sources (appendix B)