arXiv subjects

G{é}rard Biau

Publications and source records attributed to G{é}rard Biau.

3 recordsLinked to original sources

arXiv2026

Separators Make Carry Propagation Learnable:The Geometry of Latent Carry in a Multiplication Transformer

Transformers asked to multiply multi-digit numbers in a single forward pass often fail, and interpretability studies of pretrained language models find arithmetic solved by input-range heuristics rather than by an explicit carry. We train small Llama-style transformers from scratch on 4x4 multiplication without chain of thought and find that the input format is decisive: inserting a space token between digits raises exact-match accuracy from 1% to 89%. Output positions are learned in carry-chain order, with the middle digits, which have the longest-range dependencies, learned last. Inside the model, the separator token that predicts each digit (its prediction slot) encodes the carry-in as an angle on a ring in the residual stream; examples with more distinct carry values fill more of the ring. Activation patching between examples matched on the column sum shows that this state is causally used before the last layer: patching the prediction slot alone transfers the source carry in up to 84% of cases after block 4 for one middle column of our best model, while for other columns the carry is first assembled at the neighboring answer slot before reaching its own. Remaining errors are almost always off by one, consistent with a small error on the carry or on the circular digit code.

cs.LG↗

arXiv2026

Nonparametric Regression on Fixed-Cardinality Subsets:Minimax Risk and Random-Design Effects

We study regression with subsets as covariates. The response is an unknown function of the input subset, and observations consist of noisy evaluations at uniformly sampled subsets, each containing exactly \(k\) items from a ground set of size \(d\). This problem arises in combination screening, bundle preference modeling, and other settings in which outcomes depend on collections of prescribed size. The fixed-cardinality constraint couples the membership coordinates, so standard product-domain notions of interaction and smoothness cannot be imported unchanged. We use the harmonic decomposition of the Johnson graph to define an intrinsic Johnson--Sobolev scale and determine the exact dimensions of low-order interaction spaces. We derive finite-sample risk bounds for harmonic projection and establish a nonasymptotic minimax characterization when the Johnson--Sobolev energy budget is controlled relative to the noise variance. We also show that stable least squares removes the signal-dependent fluctuation of empirical projection when the empirical Gram matrix is sufficiently well conditioned. For arbitrary signal-to-noise ratios, a rank-deficiency analysis quantifies the components left unidentified by the random design. Together, the rank-deficiency analysis and a centered completion estimator yield minimax bounds that match up to constants whenever \(\min(k,d-k)\) and the smoothness order are fixed. Numerical experiments reported in the Supplementary Material illustrate estimation under different interaction profiles and distinguish the effects of observation noise, random-design fluctuation, and incomplete coverage.

math.ST↗

arXiv2026

Resolvable Fourier Degree and Minimax Estimation on theBoolean Hypercube

We study high-dimensional nonparametric regression on the Boolean hypercube \(\{0,1\}^d\), from a sample of size \(n\), under a Fourier--Sobolev budget, i.e., a bound \(B\) on the total quadratic influence of the regression function. The central object is the resolvable Fourier degree \(D^*(d,n)\), defined as the largest interaction order \(D\) such that the dimension of the degree-\(D\) Fourier--Walsh space does not exceed \(n\). We prove that the minimax risk over Fourier--Sobolev ellipsoids is of order \(B/D^*(d,n)\): estimation is possible up to the statistically visible interaction order, and the budget controls the residual mass carried by higher-order interactions. We then study monotone regression under the same budget. For monotone functions supported on a known set of \(s\) coordinates, with \(s\) of order \(\log n\), we obtain a matching minimax rate of order \(B/s\). Finally, we consider an unknown-support model in which the function is monotone and depends on at most \(s\) coordinates; its minimax risk is governed by the intrinsic dimension \(s\) together with the logarithmic cost of selecting the active support.

math.ST↗