Searcharxiv⌕ Search

arXiv subjects

Deanna Needell

Publications and source records attributed to Deanna Needell.

At least 19 recordsLinked to original sources

OCP: a greedy projection method for row-sparse representations of minimum-norm solutions

In this paper, we propose a greedy projection method, termed orthogonal constraint pursuit (OCP), for identifying a small set of rows of a consistent linear system $Ax=b$ whose span contains the minimum-norm solution $x^*$, along with the coefficients of that representation. The method successively activates the most violated constraint, and then projects the iterate onto the intersection of all activated constraints. We prove that OCP is equivalent to OMP applied to the dictionary $A^T$ and the signal $x^*$, and that it terminates in at most rank(A) iterations. When $x^*$ is a linear combination of $s$ rows of a matrix whose rows are independent and uniformly distributed on the unit sphere, OCP identifies those rows in exactly $s$ iterations with probability $1-O(m^{-1})$, provided that $n\ge Cs\ln m$ for a sufficiently large constant $C$. We also derive a variant of OCP with constraint removal for hard-margin support vector machines with regularized bias, which returns the support vectors, along with the multipliers that satisfy the KKT conditions. Numerical results on both real and synthetic problems demonstrate the identification of representing rows and the regime in which it is competitive with Krylov subspace methods.

math.NA↗

A Geometric View of Adaptive Cross Approximation via Exterior Algebra

Adaptive cross approximation (ACA) constructs low-rank CUR approximations from selected rows and columns of a matrix, making it attractive when individual entries are inexpensive to query but forming or repeatedly multiplying by the full matrix is not. Its practical performance, however, is not well explained by existing worst-case bounds, which can be exponentially large and often fail to predict when greedy largest-entry pivoting performs poorly. By viewing the matrix as a cross-gram matrix, we reinterpret the classical determinantal formula for the CUR residual through the lens of exterior algebra. In this representation, each residual entry is a weighted inner product between two $(k+1)$-blades divided by the normalized volume of the pivot block. The formula separates three effects: the weighted magnitudes of singular-vector rows, their angular geometry, and the conditioning of the selected pivots. From this we recover a classical $σ_{k+1}$-type estimate, characterize when one pivot annihilates or nearly annihilates additional rows and columns, obtain a quasi-optimality estimate for positive-semidefinite matrices, and bound the rank-one update through an explicit angle-based formula. We use this framework to explain the failure of greedy ACA on an asymmetric spiral-kernel example and to motivate a geometry-aware weighted-mass pivoting rule for radial-basis Galerkin stiffness matrices. In the reported experiments, weighted-mass pivoting often reduces the Frobenius residual relative to greedy pivoting and randomly pivoted Cholesky while retaining comparable runtime on parallel hardware.

math.NA↗

Observational Multiplicity

Many prediction tasks can admit multiple models that can perform almost equally well. This phenomenon can undermine interpretability and safety when competing models assign conflicting predictions to individuals. In this work, we study how arbitrariness can arise in probabilistic classification tasks as a result of an effect that we call \emph{observational multiplicity}. We discuss how this effect arises in a broad class of practical applications where we learn a classifier to predict probabilities $p_i \in [0,1]$ but are given a dataset of observations $y_i \in \{0,1\}$. We propose to evaluate the arbitrariness of individual probability predictions through the lens of \emph{regret}. We introduce a measure of regret for probabilistic classification tasks, which measures how the predictions of a model could change as a result of different training labels. We present a general-purpose method to estimate the regret in a probabilistic classification task. We use our measure to show that regret is often higher for certain groups in the dataset and discuss potential applications of regret. We demonstrate how estimating regret can be used to promote safety in real-world applications by abstention and data collection.

cs.LG↗

Quantile Randomized Kaczmarz for Streaming Linear Systems with Massart Noise

Quantile randomized Kaczmarz (QRK) has proven to be an efficient solver for corrupted linear systems and has received much attention. It was recently shown by Cai et al. (SIAM J. Matrix Anal. Appl. 47(2):802-823, 2026) that using $O(\log T/\log(1/β))$ samples for computing the quantile is necessary and sufficient for QRK to converge linearly over $T$ iterations when solving linear systems with a $β$-fraction of arbitrary corruptions, as long as $β$ is small enough. However, it remains unclear how large the corruption level $β$ can be, and how to compute the required subsample size $D$ explicitly, without hidden constants. This paper studies streaming linear systems with Massart noise via QRK using an order-optimal batch size $D=O(\log T)$ in each update. The independence of samples from previous iterations in the streaming setting enables a sharper analysis, yielding explicit, computable bounds on both the tolerable corruption level and the required subsample size. In particular, we establish linear convergence for corruption levels of up to approximately 7%. We also discuss how the constants improve under oblivious noise.

math.NA↗

RPLSS: A randomized projected linear systems solver

The projected linear system solver (PLSS), by incrementally appending columns to a random or deterministic sketching matrix, provides an attractive finite termination property for consistent linear systems. Nevertheless, a critical computational bottleneck of PLSS is accessing the whole coefficient matrix per iteration, making it prohibitive for extremely large-scale problems or applications with missing data. To alleviate this limitation, we propose a unified randomized PLSS (RPLSS) framework, built upon the tailored randomized row or column selection strategies that require only partial matrix information per iteration, for solving a general linear system, whether it is under- or overdetermined, and whether it is consistent or not. Within this framework, we develop a randomized Gaussian Kaczmarz method and its extended variant as row-action solvers, and randomized coordinate descent variants as column-action solvers. Theoretically, we prove that our methods inherit the finite termination property of PLSS, while achieving an exponential convergence rate, overcoming the sluggish convergence inherent in conventional randomized Kaczmarz and coordinate descent methods. Numerical experiments demonstrate the superiority of our method against state-of-the-art randomized methods, particularly in scenarios with large missing data.

math.NA↗

VDW-GNNs: Vector diffusion wavelets for geometric graph neural networks

We introduce vector diffusion wavelets (VDWs), a novel family of wavelets inspired by the vector diffusion maps algorithm that was introduced to analyze data lying in the tangent bundle of a Riemannian manifold. We show that these wavelets may be effectively incorporated into a family of geometric graph neural networks, which we refer to as VDW-GNNs. We demonstrate that such networks are effective on synthetic point cloud data, as well as on real-world data derived from wind field and neural activity measurements. Theoretically, we prove that these new wavelets have desirable frame theoretic properties, similar to traditional diffusion wavelets. Additionally, we prove that these wavelets have useful symmetries with respect to rotations and translations.

cs.LG↗

Stochastic Gradient Descent for Incomplete Tensor Linear Systems

Solving large tensor linear systems poses significant challenges due to the high volume of data stored, and it only becomes more challenging when some of the data is missing. Recently, Ma et al. showed that this problem can be tackled using a stochastic gradient descent-based method, assuming that the missing data follows a uniform missing pattern. We adapt the technique by modifying the update direction, showing that the method is applicable under other missing data models. We prove convergence results and experimentally verify these results on synthetic data.

math.NA↗

Riemannian Archetypal Analysis: Interpretable non-linear data analysis on deformed star distributions

Classical archetypal analysis is appealing for its interpretability, but its linear geometry can limit performance on data with strongly non-linear structure; at the same time, existing neural extensions improve flexibility while often weakening the geometric meaning of archetypes and interpolations. In this work, we develop a Riemannian version of archetypal analysis based on data-driven pullback geometry for real-valued data, with the goal of combining the interpretability of classical archetypal analysis with the expressive power of modern non-linear models. We introduce a class of deformed star distributions together with associated pullback Riemannian geometry to provide a statistical interpretation of the resulting manifold mappings, define the Riemannian archetypal mapping (RAM) as a projection onto the manifold of geodesically convex combinations of archetypes, and propose a practical optimization scheme based on convex relaxation followed by non-convex refinement. We further propose a learning scheme that yields reasonable, albeit generally suboptimal, deformed star distributions from data. Experiments on synthetic examples and MNIST show that the resulting framework produces meaningful geodesics, useful denoising projections, and geometry-aware classifications, while also clarifying where current optimization limitations remain.

cs.LG↗

Numerical Instabilities in the Kaczmarz Method and Stabilization by Iterative Refinement

The randomized Kaczmarz method and its accelerated variants are a powerful class of iterative methods for solving large-scale linear systems, offering guaranteed convergence with low per-iteration cost. However, their numerical stability remains poorly understood. In this work, we investigate the stability properties of both classical and accelerated randomized Kaczmarz methods, with an emphasis on how error propagates across iterations and interacts with acceleration. We show that both classical and accelerated randomized Kaczmarz fail to be forward stable. To address this issue, we propose the integration of iterative refinement into randomized Kaczmarz frameworks. We demonstrate that refinement can effectively control error accumulation and recover high-accuracy solutions, even when the system is ill-conditioned. Numerical experiments corroborate our theoretical findings and illustrate the practical benefits of combining refinement with both classical and accelerated Kaczmarz methods.

math.NA↗

Attention Mechanisms Through the Lens of Numerical Methods: Approximation Methods and Alternative Formulations

The attention mechanism is the computational core of modern Transformer architectures, but its quadratic complexity in the input sequence length is the bottleneck for large-scale inference. This has motivated a rapidly growing body of work aimed at accelerating attention through approximation and reformulation. In this survey, we revisit attention mechanisms through the lens of numerical analysis, with a particular emphasis on tools and perspectives from numerical linear algebra. Our goal is twofold: first, we aim to systematically review and classify fast approximation methods according to the numerical principles they exploit. These include sparsity and clustering approaches, low-rank and subspace projection techniques, randomized sketching methods, and tensor-based decompositions. We also discuss kernel-inspired reformulations of attention and recent architectural variants, such as Latent Attention, that modify the standard softmax formulation to improve efficiency. Second, by presenting these developments within a unified mathematical framework, we aim to bridge the gap between disciplines and highlight opportunities for further contributions from computational mathematics, particularly numerical linear algebra, to the design of scalable attention mechanisms.

math.NA↗

Towards a Fairer Non-negative Matrix Factorization

There has been a recent critical need to study fairness and bias in machine learning (ML) algorithms. Since there is clearly no one-size-fits-all solution to fairness, ML methods should be developed alongside bias mitigation strategies that are practical and approachable to the practitioner. Motivated by recent work on ``fair" PCA, here we consider the more challenging method of non-negative matrix factorization (NMF) as both a showcasing example and a method that is important in its own right for both topic modeling tasks and feature extraction for other ML tasks. We demonstrate that a modification of the objective function, by using a min-max formulation, may \textit{sometimes} be able to offer an improvement in fairness for groups in the population. We derive two methods for the objective minimization, a multiplicative update rule as well as an alternating minimization scheme, and discuss implementation practicalities. We include a suite of synthetic and real experiments that show how the method may improve fairness while also highlighting the important fact that this may sometime increase error for some individuals and fairness is not a rigid definition and method choice should strongly depend on the application at hand.

cs.LG↗

Manifold Learning with Normalizing Flows: Towards Regularity, Expressivity and Iso-Riemannian Geometry

Modern machine learning increasingly leverages the insight that high-dimensional data often lie near low-dimensional, non-linear manifolds, an idea known as the manifold hypothesis. By explicitly modeling the geometric structure of data through learning Riemannian geometry algorithms can achieve improved performance and interpretability in tasks like clustering, dimensionality reduction, and interpolation. In particular, learned pullback geometry has recently undergone transformative developments that now make it scalable to learn and scalable to evaluate, which further opens the door for principled non-linear data analysis and interpretable machine learning. However, there are still steps to be taken when considering real-world multi-modal data. This work focuses on addressing distortions and modeling errors that can arise in the multi-modal setting and proposes to alleviate both challenges through isometrizing the learned Riemannian structure and balancing regularity and expressivity of the diffeomorphism parametrization. We showcase the effectiveness of the synergy of the proposed approaches in several numerical experiments with both synthetic and real data.

cs.LG↗

A Discrete-Time Model of the Academic Pipeline in Mathematical Sciences with Constrained Hiring in the United States

The field of the mathematical sciences relies on a continuous academic pipeline in which individuals progress from undergraduate study through graduate training and postdoctoral program to long term faculty employment. National statistics report trends in bachelor's, master's, and doctoral degree awards, but these data alone do not explain how individuals move through the academic system or how structural constraints shape downstream career outcomes. Persistent growth in postdoctoral appointments alongside relatively stable faculty employment indicates that degree production alone is insufficient to characterize workforce dynamics. In this study, we develop a discrete time compartmental model of the academic pipeline in the field of the mathematical sciences that links observed degree flows to latent population stocks. Undergraduate and graduate populations are reconstructed directly from nationally reported degree data, allowing postdoctoral and faculty dynamics to be examined under completion, exit, and hiring processes. Advancement to faculty positions is modeled as vacancy limited, with competition for permanent positions depending on downstream population size. Numerical simulations show that increases in degree inflow do not translate into proportional faculty growth when hiring is constrained by limited turnover. Instead, excess supply accumulates primarily at the postdoctoral stage, leading to sustained congestion and elevated competition. Sensitivity analyses indicate that long run workforce outcomes are governed mainly by faculty exit rates and hiring capacity rather than by degree production alone. These results demonstrate the central role of vacancy limited hiring in shaping academic career trajectories within the field of the mathematical sciences.

math.DS↗

Harmful Overfitting in Sobolev Spaces

Motivated by recent work on benign overfitting in overparameterized machine learning, we study the generalization behavior of functions in Sobolev spaces $W^{k, p}(\mathbb{R}^d)$ that perfectly fit a noisy training data set. Under assumptions of label noise and sufficient regularity in the data distribution, we show that approximately norm-minimizing interpolators, which are canonical solutions selected by smoothness bias, exhibit harmful overfitting: even as the training sample size $n \to \infty$, the generalization error remains bounded below by a positive constant with high probability. Our results hold for arbitrary values of $p \in [1, \infty)$, in contrast to prior results studying the Hilbert space case ($p = 2$) using kernel methods. Our proof uses a geometric argument which identifies harmful neighborhoods of the training data using Sobolev inequalities.

stat.ML↗

A dynamical model of the U.S. mathematics graduate degree pipeline

We present a latent-stock compartmental framework for modeling degree production systems when only completion flows, rather than enrollments, are observed. Applied to U.S.\ mathematics degrees from 1969 to 2017, the model treats master's and PhD populations as latent compartments -- unobserved state variables that are inferred indirectly because they generate the observed completion flows -- with time-varying routing fractions and completion hazards. Using information-criterion model comparison across a grid of specifications, we find strong support for smooth nonlinear time variation in routing fractions and hazards, while models with explicit international forcing are disfavored. The preferred model achieves a log-scale root mean squared error of approximately 0.036, corresponding to a typical multiplicative error of about 4\% in fitted degree counts, and highlights key structural shifts in the graduate pipeline: the master's pathway became increasingly central to PhD production through the late twentieth century before weakening, while direct bachelor's-to-PhD entry remained small but persistent. Estimated completion hazards for both degrees rise over time, indicating faster effective turnover in the graduate compartments. Methodologically, our main contribution is a latent stock dynamical approach that recasts linked degreecompletion time series as a coherent stock-flow system when intermediate enrollments are unobserved, making explicit both what features of pipeline dynamics are identifiable from completion data alone and what limitations such data impose.

math.DS↗

Learn to Evolve: Self-supervised Neural JKO Operator for Wasserstein Gradient Flow

The Jordan-Kinderlehrer-Otto (JKO) scheme provides a stable variational framework for computing Wasserstein gradient flows, but its practical use is often limited by the high computational cost of repeatedly solving the JKO subproblems. We propose a self-supervised approach for learning a JKO solution operator without requiring numerical solutions of any JKO trajectories. The learned operator maps an input density directly to the minimizer of the corresponding JKO subproblem, and can be iteratively applied to efficiently generate the gradient-flow evolution. A key challenge is that only a number of initial densities are typically available for training. To address this, we introduce a Learn-to-Evolve algorithm that jointly learns the JKO operator and its induced trajectories by alternating between trajectory generation and operator updates. As training progresses, the generated data increasingly approximates true JKO trajectories. Meanwhile, this Learn-to-Evolve strategy serves as a natural form of data augmentation, significantly enhancing the generalization ability of the learned operator. Numerical experiments demonstrate the accuracy, stability, and robustness of the proposed method across various choices of energies and initial conditions.

cs.LG↗

Are Greedy Task Orderings Better Than Random in Continual Linear Regression?

We analyze task orderings in continual learning for linear regression, assuming joint realizability of training data. We focus on orderings that greedily maximize dissimilarity between consecutive tasks, a concept briefly explored in prior work but still surrounded by open questions. Using tools from the Kaczmarz method literature, we formalize such orderings and develop geometric and algebraic intuitions around them. Empirically, we demonstrate that greedy orderings converge faster than random ones in terms of the average loss across tasks, both for linear regression with random data and for linear probing on CIFAR-100 classification tasks. Analytically, in a high-rank regression setting, we prove a loss bound for greedy orderings analogous to that of random ones. However, under general rank, we establish a repetition-dependent separation. Specifically, while prior work showed that for random orderings, with or without replacement, the average loss after $k$ iterations is bounded by $\mathcal{O}(1/\sqrt{k})$, we prove that single-pass greedy orderings may fail catastrophically, whereas those allowing repetition converge at rate $\mathcal{O}(1/\sqrt[3]{k})$. Overall, we reveal nuances within and between greedy and random orderings.

cs.LG↗

Stable Phase Retrieval: Optimal Rates in Poisson and Heavy-tailed Models

We investigate stable recovery guarantees for phase retrieval under two realistic and challenging noise models: the Poisson model and the heavy-tailed model. Our analysis covers both nonconvex least squares (NCVX-LS) and convex least squares (CVX-LS) estimators. For the Poisson model, we demonstrate that in the high-energy regime where the true signal $pmb{x}$ exceeds a certain energy threshold, both estimators achieve a signal-independent, minimax optimal error rate $\mathcal{O}(\sqrt{\frac{n}{m}})$, with $n$ denoting the signal dimension and $m$ the number of sampling vectors. In contrast, in the low-energy regime, the NCVX-LS estimator attains an error rate of $\mathcal{O}(\|\pmb{x}\|^{1/4}_2\cdot(\frac{n}{m})^{1/4})$, which decreases as the energy of signal $\pmb{x}$ diminishes and remains nearly optimal with respect to the oversampling ratio. This demonstrates a signal-energy-adaptive behavior in the Poisson setting. For the heavy-tailed model with noise having a finite $q$-th moment ($q>2$), both estimators attain the minimax optimal error rate $\mathcal{O}( \frac{\| ξ\|_{L_q}}{\| \pmb{x} \|_2} \cdot \sqrt{\frac{n}{m}} )$ in the high-energy regime, while the NCVX-LS estimator further achieves the minimax optimal rate $\mathcal{O}( \sqrt{\|ξ\|_{L_q}}\cdot (\frac{n}{m})^{1/4} )$ in the low-energy regime. Our analysis builds on two key ideas: the use of multiplier inequalities to handle noise that may exhibit dependence on the sampling vectors, and a novel interpretation of Poisson noise as sub-exponential in the high-energy regime yet heavy-tailed in the low-energy regime. These insights form the foundation of a unified analytical framework, which we further apply to a range of related problems, including sparse phase retrieval, low-rank PSD matrix recovery, and random blind deconvolution.

math.ST↗