Searcharxiv⌕ Search

arXiv · 2611.15671

The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models

Abstract

A quantized model is usually judged by comparing it with its BF16 original, but that comparison changes more than the weights. It often changes the serving engine, it always runs on a different server instance, it may change how requests are batched, and on some stacks the library selects a different kernel depending only on which packages import. We measure each of these sources on the same 2,400 public decisions for two of Cloudflare's decision models, Clef-flash (9B) and Clef (27B), on an NVIDIA GB10, and state every quantization effect as the excess over them. Sources that are not quantization move between 0 and 0.92% of decisions. llama.cpp restarts are bit-identical on every record for both models. vLLM restarts flip 0, 5 and 20 decisions at BF16, FP8 and NVFP4 on Clef-flash and 4, 0 and 22 on Clef; a third instance of Clef's quantized builds (exploratory) leaves FP8 at 0 and puts NVFP4 at 4 to 22, depending on which two instances are compared. An engine change at BF16 flips 6 to 16, a batch-size change 8 to 15, and the Gated DeltaNet kernel transformers selects unasked 10 to 14. Quantization inside one engine moves more: for Clef-flash and Clef, 0.88% and 1.04% at Q8_0, 2.12% and 1.62% at FP8, 5.79% and 3.50% at Q4_K_M, and 5.79% and 5.67% at NVFP4. On both models, every 4-bit interval excludes every non-quantization source. Two of our five pre-registered predictions failed, and the same two fail when the tests are rerun on the 27B: FP8 is not inside restart noise, and changing engine moves fewer decisions than 8-bit quantization does. Accuracy hides most of this. Over the full 31,275 public questions, Clef's Q4_K_M build moves accuracy by a net six questions (-0.02 points [-0.20, +0.16]) while 436 of its decisions get worse (exploratory), and NVFP4 costs Clef-flash 0.95 points and Clef 0.65. We release the code, quantization recipes, per-record outputs and the pre-registration.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Aleks Blagojevich, Trevor Sunderland. 2026-11-11. The Noise Floor of a Quantization Comparison: Engine, restart, batch and kernel drift, measured against 8-bit and 4-bit effects in two decision models. https://searcharxiv.com/abs/2611.15671

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Algorithmic Insurance

When artificial intelligence (AI) systems make errors in high-stakes domains such as medical diagnosis or autonomous driving, a single algorithmic flaw across varying contexts can generate heterogeneous losses that challenge traditional insurance assumptions. Algorithmic insurance, financial coverage for AI-induced damages, is an emerging market, yet insurers struggle to price these risks and AI developers lack frameworks linking system design to liability exposure. We connect a key operational choice in binary classification, how practitioners set decision thresholds, to tail risk exposure. Using conditional value-at-risk (CVaR) to capture extreme losses, we prove that established approaches such as accuracy maximization can significantly increase worst-case losses relative to tail risk optimization, with penalties growing at least quadratically as thresholds deviate from optimal. We propose a liability insurance contract that mandates risk-aware thresholds and characterize when it creates value for AI providers, extending the analysis to degrading model performance and human oversight. In a mammography case study, accuracy-maximizing thresholds carry up to four times the tail risk of CVaR-optimal thresholds. Insurance contracts create 7-8% gains for well-calibrated firms through risk transfer and regulatory capital relief, and up to 105% for poorly calibrated firms through mandatory recalibration, while partially aligned firms can lose under a strict mandate, calling for threshold flexibility in contract design. Unlike traditional insurance that merely transfers risk, algorithmic insurance can act as both a financial instrument and an operational governance mechanism, enabling risk transfer while improving AI safety.

cs.LG↗

SiGMA: Learning under Graph-Level Heterogeneity with Gated Message-Passing and Attention

In graph-level learning, graph neural networks are typically applied with a fixed computational template: the same depth, local message-passing operator, and global attention mechanism are used for every input graph. However, different graphs within the same dataset can vary substantially in prediction difficulty and may require different mixtures of local and non-local computation. We study this phenomenon as graph-level heterogeneity in both classification and regression tasks. To address it, we propose SiGMA (Sigmoid Gated Message-Passing and Attention), a gated hybrid graph neural network architecture that combines message-passing and attention heads within each layer. Inspired by the growing use of gated attention and gated computation in large language models and vision transformers, each head in SiGMA is modulated by a learned node-wise sigmoid gate, allowing the model to adaptively regulate which local or global mechanisms are active for different nodes, layers, and input graphs. The resulting architecture can emulate expressive message-passing backbones, admits near-identity computation paths that are not forced to over-smooth, and enables direct non-local communication beyond message-passing neighborhoods to mitigate over-squashing. Empirically, SiGMA achieves strong performance across graph-level benchmarks, consistently matching or improving over competitive graph transformer and hybrid baselines. Our results suggest that adapting computation to graph-level heterogeneity is a promising direction for building more flexible and effective graph learning architectures.

cs.LG↗

HOPSE: Scalable Higher-Order Positional and Structural Encoder for Combinatorial Representations

While Graph Neural Networks (GNNs) have proven highly effective at modeling relational data, pairwise connections cannot fully capture multi-way relationships naturally present in complex real-world systems. In response to this, Topological Deep Learning (TDL) leverages more general combinatorial representations--such as simplicial or cellular complexes--to accommodate higher-order interactions. Existing TDL methods often extend GNNs through Higher-Order Message Passing (HOMP), but face critical scalability challenges due to the steep complexity overhead of propagating messages through combinatorial structures. To overcome this limitation, we propose HOPSE (Higher-Order Positional and Structural Encoder), a framework free of learnable message passing layers that uses Hasse graph decompositions to derive efficient and expressive encodings over arbitrary higher-order domains. Notably, HOPSE training and inference scale linearly with the size of combinatorial representations while preserving the expressive power and permutation equivariance of the HOMP approaches. Experiments on molecular and topological benchmarks show that it matches or surpasses state-of-the-art performance while consistently achieving speedups over HOMP-based models, opening a new path for scalable TDL. The code is available at https://github.com/geometric-intelligence/topobench.

cs.LG↗