Searcharxiv⌕ Search
Home/Archive/arXiv

arXiv · 2611.09110

Byzantine-Robust Distributed Sparse Softmax Regression

Yuxuan Wang·Yu Sun·Bowen Li·Kangqiang Li
View original source ↗Read source document ↗Export metadata ↓

Abstract

We study Byzantine-robust distributed sparse softmax regression in high-dimensional settings. We propose a communication-efficient distributed algorithm that employs robust aggregation methods---coordinate-wise trimmed mean and coordinate-wise median---to defend against Byzantine attacks. Under sub-Gaussian design assumptions with sparsity constraints, we establish that the distributed estimator achieves the convergence rate of $O_{P}(K^{\frac{5}{2}}\sqrt{s\log(Kd)/N})$ when the Byzantine fraction $α\asymp 1/\sqrt{m}$, matching the centralized oracle rate. Numerical experiments on synthetic data demonstrate the superior performance of the proposed method.

Explore related subjects

math.STstat.TH

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Yuxuan Wang, Yu Sun, Bowen Li, Kangqiang Li. 2026-11-06. Byzantine-Robust Distributed Sparse Softmax Regression. https://arxiv.org/abs/2611.09110

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Phase transition for conditional covariance matrices estimated by importance sampling, and implications for cross-entropy schemes in high dimension

Motivated by recent high-dimensional reliability algorithms, we study a random matrix model corresponding to the estimation by importance sampling of $Σ_A$, the covariance matrix of a Gaussian vector conditioned by a rare event $A$. For such models, we show that, in contrast to most random matrix models, the minimal sample size to ensure an accurate estimation of $Σ_A$ needs to grow superlinearly in the dimension. More precisely, if $n$ is the sample size and $d$ the dimension, we show that the estimation of $Σ_A$ is accurate if $n/d \gg L$ and not accurate if $n/d \ll L$ where $L$ is the maximum likelihood ratio. Moreover, we relate the asymptotic behavior of $L$ to $λ_1$, the smallest eigenvalue of the auxiliary covariance matrix, which suggests that $L$ (and thus the minimal sample size) will be larger for small values of $λ_1$. Numerical experiments confirm that this insight carries over to recent cross-entropy algorithms, with the best performing algorithms being also the ones with the largest smallest eigenvalue. Although backed up by our theoretical result, this result is surprising as these algorithms were not designed to act on the spectrum.

math.ST↗
arXiv2026

Closed-form fractional radial links for elliptical Mahalanobis discriminant analysis

We study binary classification when the two classes are elliptical with a shared radial generator $g$. The log-likelihood ratio is then an additive function of the two squared Mahalanobis radii, with radial link $φ=\log g$. For Gaussian classes $φ$ is affine and the rule reduces to quadratic discriminant analysis (QDA); otherwise the Bayes link is not the identity. Instead of fitting this link with a spline smoother, we derive its family from the within-class law of the radius and estimate a finite fractional-power projection. We prove that the link is identifiable from the radius law, that the plug-in estimator is $\sqrt{n}$-consistent and asymptotically normal under finite-moment conditions, and that the induced classifier is asymptotically Bayes-optimal in an iterated sieve limit. For $d\ge2$ and non-nested class parameters, the identity link suffices only for affine (Gaussian) generators. The bridge, GAM membership and the affine direction of this dichotomy are verified in Lean 4. We compare against the global Mahalanobis-GAM of Ghosh et al. (2025), re-implemented with mgcv REML splines at equal input budget. On two UCI benchmarks the derived link ties with the fitted GAM. On six financial series from three asset classes, with validation that respects temporal dependence, it is never significantly worse than the fitted GAM: better on CAD/USD and tied on the other five. It beats QDA on the three heaviest-tailed series (oil $[+0.023,+0.069]$, S&P 500 $[+0.039,+0.125]$, JPY/USD $[+0.009,+0.047]$) and ties elsewhere. A rate simulation with a closed-form ground truth reproduces the $\sqrt{n}$ rate and, under covariance heterogeneity, QDA's risk floor against the derived link's vanishing excess risk. The derived link thus matches a tuned GAM without smoothing-parameter selection and improves on QDA where the generator departs from the Gaussian.

math.ST↗
arXiv2026

Reweighting for Transfer Classification: Exact Risk Conversion and Minimax Adaptation

A source can make a training mixture easier to learn while making its accuracy less valuable for the target. We study how the training problem a source creates and the finite-sample error of learning it jointly determine target-risk guarantees under reweighting. For binary classification, for each pair of laws and each positive target weight, we derive the sharpest target excess hinge-risk bound implied by mixture excess alone, uniformly over clipped predictors. This exact conversion curve is determined by density-weighted local votes. A two-sample oracle inequality for weighted support vector machines supplies the mixture excess, and the composition becomes explicit for Gaussian kernels. On a tilted-boundary class, we establish the all-procedure minimax classification rate $[n+s_+^2/(m^{-1}+τ^2)]^{-1}$ in the transferable strength $s_+$ and the tilt $τ$, separating source sampling uncertainty ($m^{-1}$) from relation uncertainty ($τ^2$). A target-checked Gaussian-SVM selector adapts to unknown $s_+$ and $τ$ up to a logarithmic factor for fixed dimension, and retains the target-only rate up to the same factor against arbitrary independent sources under the stated target model. On a separate smooth opposition class, the minimax risk within the family of exact Gaussian SVMs fitted to the original labels with nonnegative weights is exponentially larger than the unrestricted minimax risk whenever the target sample is at most half the source sample, despite arbitrary data-dependent weighting and tuning. The value of a source thus depends on the training problem it creates, on what accuracy there means for the target, and on whether the permitted use of the source can implement the needed correction.

math.ST↗