arXiv subjects

Deanna Needell

Publications and source records attributed to Deanna Needell.

At least 19 recordsLinked to original sources

arXiv2026

Quantile Randomized Kaczmarz for Streaming Linear Systems with Massart Noise

Quantile randomized Kaczmarz (QRK) has proven to be an efficient solver for corrupted linear systems and has received much attention. It was recently shown by Cai et al. (SIAM J. Matrix Anal. Appl. 47(2):802-823, 2026) that using $O(\log T/\log(1/\beta))$ samples for computing the quantile is necessary and sufficient for QRK to converge linearly over $T$ iterations when solving linear systems with a $\beta$-fraction of arbitrary corruptions, as long as $\beta$ is small enough. However, it remains unclear how large the corruption level $\beta$ can be, and how to compute the required subsample size $D$ explicitly, without hidden constants. This paper studies streaming linear systems with Massart noise via QRK using an order-optimal batch size $D=O(\log T)$ in each update. The independence of samples from previous iterations in the streaming setting enables a sharper analysis, yielding explicit, computable bounds on both the tolerable corruption level and the required subsample size. In particular, we establish linear convergence for corruption levels of up to approximately 7%. We also discuss how the constants improve under oblivious noise.

math.NA↗

arXiv2026

RPLSS: A randomized projected linear systems solver

The projected linear system solver (PLSS), by incrementally appending columns to a random or deterministic sketching matrix, provides an attractive finite termination property for consistent linear systems. Nevertheless, a critical computational bottleneck of PLSS is accessing the whole coefficient matrix per iteration, making it prohibitive for extremely large-scale problems or applications with missing data. To alleviate this limitation, we propose a unified randomized PLSS (RPLSS) framework, built upon the tailored randomized row or column selection strategies that require only partial matrix information per iteration, for solving a general linear system, whether it is under- or overdetermined, and whether it is consistent or not. Within this framework, we develop a randomized Gaussian Kaczmarz method and its extended variant as row-action solvers, and randomized coordinate descent variants as column-action solvers. Theoretically, we prove that our methods inherit the finite termination property of PLSS, while achieving an exponential convergence rate, overcoming the sluggish convergence inherent in conventional randomized Kaczmarz and coordinate descent methods. Numerical experiments demonstrate the superiority of our method against state-of-the-art randomized methods, particularly in scenarios with large missing data.

math.NA↗

arXiv2026

Riemannian Archetypal Analysis: Interpretable non-linear data analysis on deformed star distributions

Classical archetypal analysis is appealing for its interpretability, but its linear geometry can limit performance on data with strongly non-linear structure; at the same time, existing neural extensions improve flexibility while often weakening the geometric meaning of archetypes and interpolations. In this work, we develop a Riemannian version of archetypal analysis based on data-driven pullback geometry for real-valued data, with the goal of combining the interpretability of classical archetypal analysis with the expressive power of modern non-linear models. We introduce a class of deformed star distributions together with associated pullback Riemannian geometry to provide a statistical interpretation of the resulting manifold mappings, define the Riemannian archetypal mapping (RAM) as a projection onto the manifold of geodesically convex combinations of archetypes, and propose a practical optimization scheme based on convex relaxation followed by non-convex refinement. We further propose a learning scheme that yields reasonable, albeit generally suboptimal, deformed star distributions from data. Experiments on synthetic examples and MNIST show that the resulting framework produces meaningful geodesics, useful denoising projections, and geometry-aware classifications, while also clarifying where current optimization limitations remain.

cs.LG↗

arXiv2026

Numerical Instabilities in the Kaczmarz Method and Stabilization by Iterative Refinement

The randomized Kaczmarz method and its accelerated variants are a powerful class of iterative methods for solving large-scale linear systems, offering guaranteed convergence with low per-iteration cost. However, their numerical stability remains poorly understood. In this work, we investigate the stability properties of both classical and accelerated randomized Kaczmarz methods, with an emphasis on how error propagates across iterations and interacts with acceleration. We show that both classical and accelerated randomized Kaczmarz fail to be forward stable. To address this issue, we propose the integration of iterative refinement into randomized Kaczmarz frameworks. We demonstrate that refinement can effectively control error accumulation and recover high-accuracy solutions, even when the system is ill-conditioned. Numerical experiments corroborate our theoretical findings and illustrate the practical benefits of combining refinement with both classical and accelerated Kaczmarz methods.

math.NA↗

arXiv2026

Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations

The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale inference. This has motivated a rapidly growing body of work aimed at accelerating attention through approximation and reformulation. In this survey, we revisit attention mechanisms through the lens of numerical analysis, with a particular emphasis on tools and perspectives from numerical linear algebra. Our goal is twofold: first, we aim to systematically review and classify fast approximation methods according to the numerical principles they exploit. These include sparsity and clustering approaches, low-rank and subspace projection techniques, randomized sketching methods, and tensor-based decompositions. We also discuss kernel-inspired reformulations of attention and recent architectural variants, such as Latent Attention, that modify the standard softmax formulation to improve efficiency. Second, by presenting these developments within a unified mathematical framework, we aim to bridge the gap between disciplines and highlight opportunities for further contributions from computational mathematics, particularly numerical linear algebra, to the design of scalable attention mechanisms.

math.NA↗

arXiv2026

Towards a Fairer Non-negative Matrix Factorization

There has been a recent critical need to study fairness and bias in machine learning (ML) algorithms. Since there is clearly no one-size-fits-all solution to fairness, ML methods should be developed alongside bias mitigation strategies that are practical and approachable to the practitioner. Motivated by recent work on ``fair" PCA, here we consider the more challenging method of non-negative matrix factorization (NMF) as both a showcasing example and a method that is important in its own right for both topic modeling tasks and feature extraction for other ML tasks. We demonstrate that a modification of the objective function, by using a min-max formulation, may \textit{sometimes} be able to offer an improvement in fairness for groups in the population. We derive two methods for the objective minimization, a multiplicative update rule as well as an alternating minimization scheme, and discuss implementation practicalities. We include a suite of synthetic and real experiments that show how the method may improve fairness while also highlighting the important fact that this may sometime increase error for some individuals and fairness is not a rigid definition and method choice should strongly depend on the application at hand.

cs.LG↗

arXiv2026

Manifold Learning with Normalizing Flows: Towards Regularity, Expressivity and Iso-Riemannian Geometry

Modern machine learning increasingly leverages the insight that high-dimensional data often lie near low-dimensional, non-linear manifolds, an idea known as the manifold hypothesis. By explicitly modeling the geometric structure of data through learning Riemannian geometry algorithms can achieve improved performance and interpretability in tasks like clustering, dimensionality reduction, and interpolation. In particular, learned pullback geometry has recently undergone transformative developments that now make it scalable to learn and scalable to evaluate, which further opens the door for principled non-linear data analysis and interpretable machine learning. However, there are still steps to be taken when considering real-world multi-modal data. This work focuses on addressing distortions and modeling errors that can arise in the multi-modal setting and proposes to alleviate both challenges through isometrizing the learned Riemannian structure and balancing regularity and expressivity of the diffeomorphism parametrization. We showcase the effectiveness of the synergy of the proposed approaches in several numerical experiments with both synthetic and real data.

cs.LG↗

arXiv2026

A Discrete-Time Model of the Academic Pipeline in Mathematical Sciences with Constrained Hiring in the United States

The field of the mathematical sciences relies on a continuous academic pipeline in which individuals progress from undergraduate study through graduate training and postdoctoral program to long term faculty employment. National statistics report trends in bachelor's, master's, and doctoral degree awards, but these data alone do not explain how individuals move through the academic system or how structural constraints shape downstream career outcomes. Persistent growth in postdoctoral appointments alongside relatively stable faculty employment indicates that degree production alone is insufficient to characterize workforce dynamics. In this study, we develop a discrete time compartmental model of the academic pipeline in the field of the mathematical sciences that links observed degree flows to latent population stocks. Undergraduate and graduate populations are reconstructed directly from nationally reported degree data, allowing postdoctoral and faculty dynamics to be examined under completion, exit, and hiring processes. Advancement to faculty positions is modeled as vacancy limited, with competition for permanent positions depending on downstream population size. Numerical simulations show that increases in degree inflow do not translate into proportional faculty growth when hiring is constrained by limited turnover. Instead, excess supply accumulates primarily at the postdoctoral stage, leading to sustained congestion and elevated competition. Sensitivity analyses indicate that long run workforce outcomes are governed mainly by faculty exit rates and hiring capacity rather than by degree production alone. These results demonstrate the central role of vacancy limited hiring in shaping academic career trajectories within the field of the mathematical sciences.

math.DS↗

arXiv2026

Harmful Overfitting in Sobolev Spaces

Motivated by recent work on benign overfitting in overparameterized machine learning, we study the generalization behavior of functions in Sobolev spaces $W^{k, p}(\mathbb{R}^d)$ that perfectly fit a noisy training data set. Under assumptions of label noise and sufficient regularity in the data distribution, we show that approximately norm-minimizing interpolators, which are canonical solutions selected by smoothness bias, exhibit harmful overfitting: even as the training sample size $n \to \infty$, the generalization error remains bounded below by a positive constant with high probability. Our results hold for arbitrary values of $p \in [1, \infty)$, in contrast to prior results studying the Hilbert space case ($p = 2$) using kernel methods. Our proof uses a geometric argument which identifies harmful neighborhoods of the training data using Sobolev inequalities.

stat.ML↗

arXiv2026

A dynamical model of the U.S. mathematics graduate degree pipeline

We present a latent-stock compartmental framework for modeling degree production systems when only completion flows, rather than enrollments, are observed. Applied to U.S.\ mathematics degrees from 1969 to 2017, the model treats master's and PhD populations as latent compartments -- unobserved state variables that are inferred indirectly because they generate the observed completion flows -- with time-varying routing fractions and completion hazards. Using information-criterion model comparison across a grid of specifications, we find strong support for smooth nonlinear time variation in routing fractions and hazards, while models with explicit international forcing are disfavored. The preferred model achieves a log-scale root mean squared error of approximately 0.036, corresponding to a typical multiplicative error of about 4\% in fitted degree counts, and highlights key structural shifts in the graduate pipeline: the master's pathway became increasingly central to PhD production through the late twentieth century before weakening, while direct bachelor's-to-PhD entry remained small but persistent. Estimated completion hazards for both degrees rise over time, indicating faster effective turnover in the graduate compartments. Methodologically, our main contribution is a latent stock dynamical approach that recasts linked degreecompletion time series as a coherent stock-flow system when intermediate enrollments are unobserved, making explicit both what features of pipeline dynamics are identifiable from completion data alone and what limitations such data impose.

math.DS↗

arXiv2026

Learn to Evolve: Self-supervised Neural JKO Operator for Wasserstein Gradient Flow

The Jordan-Kinderlehrer-Otto (JKO) scheme provides a stable variational framework for computing Wasserstein gradient flows, but its practical use is often limited by the high computational cost of repeatedly solving the JKO subproblems. We propose a self-supervised approach for learning a JKO solution operator without requiring numerical solutions of any JKO trajectories. The learned operator maps an input density directly to the minimizer of the corresponding JKO subproblem, and can be iteratively applied to efficiently generate the gradient-flow evolution. A key challenge is that only a number of initial densities are typically available for training. To address this, we introduce a Learn-to-Evolve algorithm that jointly learns the JKO operator and its induced trajectories by alternating between trajectory generation and operator updates. As training progresses, the generated data increasingly approximates true JKO trajectories. Meanwhile, this Learn-to-Evolve strategy serves as a natural form of data augmentation, significantly enhancing the generalization ability of the learned operator. Numerical experiments demonstrate the accuracy, stability, and robustness of the proposed method across various choices of energies and initial conditions.

cs.LG↗

arXiv2025

Are Greedy Task Orderings Better Than Random in Continual Linear Regression?

We analyze task orderings in continual learning for linear regression, assuming joint realizability of training data. We focus on orderings that greedily maximize dissimilarity between consecutive tasks, a concept briefly explored in prior work but still surrounded by open questions. Using tools from the Kaczmarz method literature, we formalize such orderings and develop geometric and algebraic intuitions around them. Empirically, we demonstrate that greedy orderings converge faster than random ones in terms of the average loss across tasks, both for linear regression with random data and for linear probing on CIFAR-100 classification tasks. Analytically, in a high-rank regression setting, we prove a loss bound for greedy orderings analogous to that of random ones. However, under general rank, we establish a repetition-dependent separation. Specifically, while prior work showed that for random orderings, with or without replacement, the average loss after $k$ iterations is bounded by $\mathcal{O}(1/\sqrt{k})$, we prove that single-pass greedy orderings may fail catastrophically, whereas those allowing repetition converge at rate $\mathcal{O}(1/\sqrt[3]{k})$. Overall, we reveal nuances within and between greedy and random orderings.

cs.LG↗

arXiv2025

Stochastic Gradient Descent for Incomplete Tensor Linear Systems

Solving large tensor linear systems poses significant challenges due to the high volume of data stored, and it only becomes more challenging when some of the data is missing. Recently, Ma et al. showed that this problem can be tackled using a stochastic gradient descent-based method, assuming that the missing data follows a uniform missing pattern. We adapt the technique by modifying the update direction, showing that the method is applicable under other missing data models. We prove convergence results and experimentally verify these results on synthetic data.

math.NA↗

arXiv2025

Stable Phase Retrieval: Optimal Rates in Poisson and Heavy-tailed Models

We investigate stable recovery guarantees for phase retrieval under two realistic and challenging noise models: the Poisson model and the heavy-tailed model. Our analysis covers both nonconvex least squares (NCVX-LS) and convex least squares (CVX-LS) estimators. For the Poisson model, we demonstrate that in the high-energy regime where the true signal $pmb{x}$ exceeds a certain energy threshold, both estimators achieve a signal-independent, minimax optimal error rate $\mathcal{O}(\sqrt{\frac{n}{m}})$, with $n$ denoting the signal dimension and $m$ the number of sampling vectors. In contrast, in the low-energy regime, the NCVX-LS estimator attains an error rate of $\mathcal{O}(\|\pmb{x}\|^{1/4}_2\cdot(\frac{n}{m})^{1/4})$, which decreases as the energy of signal $\pmb{x}$ diminishes and remains nearly optimal with respect to the oversampling ratio. This demonstrates a signal-energy-adaptive behavior in the Poisson setting. For the heavy-tailed model with noise having a finite $q$-th moment ($q>2$), both estimators attain the minimax optimal error rate $\mathcal{O}( \frac{\| ξ\|_{L_q}}{\| \pmb{x} \|_2} \cdot \sqrt{\frac{n}{m}} )$ in the high-energy regime, while the NCVX-LS estimator further achieves the minimax optimal rate $\mathcal{O}( \sqrt{\|ξ\|_{L_q}}\cdot (\frac{n}{m})^{1/4} )$ in the low-energy regime. Our analysis builds on two key ideas: the use of multiplier inequalities to handle noise that may exhibit dependence on the sampling vectors, and a novel interpretation of Poisson noise as sub-exponential in the high-energy regime yet heavy-tailed in the low-energy regime. These insights form the foundation of a unified analytical framework, which we further apply to a range of related problems, including sparse phase retrieval, low-rank PSD matrix recovery, and random blind deconvolution.

math.ST↗

arXiv2025

VDW-GNNs: Vector diffusion wavelets for geometric graph neural networks

We introduce vector diffusion wavelets (VDWs), a novel family of wavelets inspired by the vector diffusion maps algorithm that was introduced to analyze data lying in the tangent bundle of a Riemannian manifold. We show that these wavelets may be effectively incorporated into a family of geometric graph neural networks, which we refer to as VDW-GNNs. We demonstrate that such networks are effective on synthetic point cloud data, as well as on real-world data derived from wind field and neural activity measurements. Theoretically, we prove that these new wavelets have desirable frame theoretic properties, similar to traditional diffusion wavelets. Additionally, we prove that these wavelets have useful symmetries with respect to rotations and translations.

cs.LG↗

arXiv2025

Differentially Private Random Feature Model

Designing privacy-preserving machine learning algorithms has received great attention in recent years, especially in the setting when the data contains sensitive information. Differential privacy (DP) is a widely used mechanism for data analysis with privacy guarantees. In this paper, we produce a differentially private random feature model. Random features, which were proposed to approximate large-scale kernel machines, have been used to study privacy-preserving kernel machines as well. We consider the over-parametrized regime (more features than samples) where the non-private random feature model is learned via solving the min-norm interpolation problem, and then we apply output perturbation techniques to produce a private model. We show that our method preserves privacy and derive a generalization error bound for the method. To the best of our knowledge, we are the first to consider privacy-preserving random feature models in the over-parametrized regime and provide theoretical guarantees. We empirically compare our method with other privacy-preserving learning methods in the literature as well. Our results show that our approach is superior to the other methods in terms of generalization performance on synthetic data and benchmark data sets. Additionally, it was recently observed that DP mechanisms may exhibit and exacerbate disparate impact, which means that the outcomes of DP learning algorithms vary significantly among different groups. We show that both theoretically and empirically, random features have the potential to reduce disparate impact, and hence achieve better fairness.

cs.LG↗

arXiv2025

Cauchy Random Features for Operator Learning in Sobolev Space

Operator learning is the approximation of operators between infinite dimensional Banach spaces using machine learning approaches. While most progress in this area has been driven by variants of deep neural networks such as the Deep Operator Network and Fourier Neural Operator, the theoretical guarantees are often in the form of a universal approximation property. However, the existence theorems do not guarantee that an accurate operator network is obtainable in practice. Motivated by the recent kernel-based operator learning framework, we propose a random feature operator learning method with theoretical guarantees and error bounds. The random feature method can be viewed as a randomized approximation of a kernel method, which significantly reduces the computation requirements for training. We provide a generalization error analysis for our proposed random feature operator learning method along with comprehensive numerical results. Compared to kernel-based method and neural network methods, the proposed method can obtain similar or better test errors across benchmarks examples with significantly reduced training times. An additional advantages it that our implementation is simple and does require costly computational resources, such as GPU.

cs.LG↗

arXiv2025

Observational Multiplicity

Many prediction tasks can admit multiple models that can perform almost equally well. This phenomenon can can undermine interpretability and safety when competing models assign conflicting predictions to individuals. In this work, we study how arbitrariness can arise in probabilistic classification tasks as a result of an effect that we call \emph{observational multiplicity}. We discuss how this effect arises in a broad class of practical applications where we learn a classifier to predict probabilities $p_i \in [0,1]$ but are given a dataset of observations $y_i \in \{0,1\}$. We propose to evaluate the arbitrariness of individual probability predictions through the lens of \emph{regret}. We introduce a measure of regret for probabilistic classification tasks, which measures how the predictions of a model could change as a result of different training labels change. We present a general-purpose method to estimate the regret in a probabilistic classification task. We use our measure to show that regret is higher for certain groups in the dataset and discuss potential applications of regret. We demonstrate how estimating regret promote safety in real-world applications by abstention and data collection.

cs.LG↗