arXiv subjects

Jeff Calder

Publications and source records attributed to Jeff Calder.

At least 19 recordsLinked to original sources

arXiv2026

On the continuum limit of t-SNE for data visualization

This work is concerned with the continuum limit of a graph-based data visualization technique called the t-Distributed Stochastic Neighbor Embedding (t-SNE), which is widely used for visualizing data in a variety of applications, but is still poorly understood from a theoretical standpoint. The t-SNE algorithm produces visualizations by minimizing the Kullback-Leibler divergence between similarity matrices representing the high dimensional data and its low dimensional representation. We prove that as the number of data points $n \to \infty$, after a natural rescaling and in applicable parameter regimes, the Kullback-Leibler divergence is consistent as the number of data points $n \to \infty$ and the similarity graph remains sparse with a continuum variational problem that involves a non-convex gradient regularization term and a penalty on the magnitude of the probability density function in the visualization space. These two terms represent the continuum limits of the attraction and repulsion forces in the t-SNE algorithm. Due to the lack of convexity in the continuum variational problem, the question of well-posedeness is only partially resolved. We show that when both dimensions are $1$, the problem admits a unique smooth minimizer, along with an infinite number of discontinuous minimizers (interpreted in a relaxed sense). This aligns well with the empirically observed ability of t-SNE to separate data in seemingly arbitrary ways in the visualization. The energy is also very closely related to the famously ill-posed Perona-Malik equation, which is used for denoising and simplifying images. We present numerical results validating the continuum limit, provide some preliminary results about the delicate nature of the limiting energetic problem in higher dimensions, and highlight several problems for future work.

stat.ML↗

arXiv2025

An Elementary Proof of a Minimax Theorem

Here, we give a self-contained and elementary proof of a minimax theorem due to Fan in a simplified setting that can be taught in an advanced undergraduate course. Our proof follows Nikaido's argument with some simplifications.

math.HO↗

arXiv2025

Energy Approach from $\varepsilon$-Graph to Continuum Diffusion Model with Connectivity Functional

We derive an energy-based continuum limit for $\varepsilon$-graphs endowed with a general connectivity functional. We prove that the discrete energy and its continuum counterpart differ by at most $O(\varepsilon)$; the prefactor involves only the $W^{1,1}$-norm of the connectivity density as $\varepsilon\to0$, so the error bound remains valid even when that density has strong local fluctuations. As an application, we introduce a neural-network procedure that reconstructs the connectivity density from edge-weight data and then embeds the resulting continuum model into a brain-dynamics framework. In this setting, the usual constant diffusion coefficient is replaced by the spatially varying coefficient produced by the learned density, yielding dynamics that differ significantly from those obtained with conventional constant-diffusion models.

math.NA↗

arXiv2025

Dynamical feedback control with operator learning for the Vlasov-Poisson system

To meet the demands of instantaneous control of instabilities over long time horizons in plasma fusion, we design a dynamic feedback control strategy for the Vlasov-Poisson system by constructing an operator that maps state perturbations to an external control field. In the first part of the paper, we propose learning such an operator using a neural network. Inspired by optimal control theory for linearized dynamics, we introduce a low-rank neural operator architecture and train it via adjoint state method. The resulting controller is effective at suppressing instabilities well beyond the training time horizon. To generalize control across varying initial data, we further introduce a novel cancellation-based control strategy that removes the destabilizing component of the electric field. This approach naturally defines an operator without requiring any training, ensures perturbation decay over infinite time, and demonstrates strong robustness under noisy feedback. Numerical experiments confirm the effectiveness of the method in both one- and multidimensional settings.

math.NA↗

arXiv2025

Controlling instability in the Vlasov-Poisson system through moment-based optimization

Controlling instability in plasma is one of the central challenges in fusion energy research. Among the various sources of instability, kinetic effects play a significant role. In this work, we aim to suppress the instability induced by kinetic effects by designing an external electric field. However, rather than directly solving the full kinetic Vlasov-Poisson system, we focus on a reduced-order model, specifically the moment-based system, to capture the underlying dynamics. This approach is motivated by the desire to reduce the computational cost associated with repeatedly solving the high-dimensional kinetic equations during the optimization of the electric field. Additionally, moment-based data is more readily accessible in practice, making a moment-based control framework more adaptable to data-driven scenarios. We investigate the effectiveness of moment-based control both analytically and numerically, by comparing it to control based on the full kinetic model.

math.NA↗

arXiv2025

Dynamics Over Landscape: The Emergence of Linear Separability via Spectral Alignment in Contrastive Learning

Contrastive learning effectively clusters data despite a loss landscape filled with poor solutions, a success that is heavily dependent on the choice of data augmentations. How optimization consistently finds meaningful patterns remains an open question. We show this success stems from training dynamics rather than the loss function alone. Crucially, under a highly specific structural assumption governing the connectivity and variance of the data augmentations, we prove that once a critical spectral alignment threshold is reached, data features inevitably and rapidly separate into distinct clusters. We establish this mechanism for both discrete datasets and the macroscopic continuum limit, modeling latent dynamics as a Wasserstein gradient flow to demonstrate that this separation persists as the number of data points approaches infinity. We hypothesize that natural training dynamics inherently drive the system toward this critical state. We extensively validate this empirically across four diverse domains (synthetic shapes, images, text, and PDEs). In every setting, a sharp increase in this spectral quantity consistently precedes clean data separation, revealing that contrastive learning's success is governed by a dynamically emerging trigger tightly coupled to the underlying augmentation structure.

math.NA↗

arXiv2025

Geometry-Preserving Encoder/Decoder in Latent Generative Models

Generative modeling aims to generate new data samples that resemble a given dataset. When using diffusion models for this task, one of the main challenges is solving the problem in the input space, which tends to be very high-dimensional. To address this, recent approaches solve diffusion models in the latent space through an encoder that maps from the data space to a lower-dimensional latent space, improving training efficiency and achieving state-of-the-art results. The variational autoencoder (VAE) is the most commonly used encoder/decoder framework in this domain, known for its ability to learn latent representations and generate data samples. In this paper, we introduce a novel encoder/decoder framework with theoretical properties distinct from those of the VAE, specifically designed to preserve the geometric structure of the data distribution. We demonstrate the significant advantages of this geometry-preserving encoder in the training process of both the encoder and decoder. Additionally, we provide theoretical results proving convergence of the training process, including convergence guarantees for encoder training, and results showing faster convergence of decoder training when using the geometry-preserving encoder.

math.NA↗

arXiv2024

GLL: A Differentiable Graph Learning Layer for Neural Networks

Standard deep learning architectures used for classification generate label predictions with a projection head and softmax activation function. Although successful, these methods fail to leverage the relational information between samples for generating label predictions. In recent works, graph-based learning techniques, namely Laplace learning, have been heuristically combined with neural networks for both supervised and semi-supervised learning (SSL) tasks. However, prior works approximate the gradient of the loss function with respect to the graph learning algorithm or decouple the processes; end-to-end integration with neural networks is not achieved. In this work, we derive backpropagation equations, via the adjoint method, for inclusion of a general family of graph learning layers into a neural network. The resulting method, distinct from graph neural networks, allows us to precisely integrate similarity graph construction and graph Laplacian-based label propagation into a neural network layer, replacing a projection head and softmax activation function for general classification task. Our experimental results demonstrate smooth label transitions across data, improved generalization and robustness to adversarial attacks, and improved training dynamics compared to a standard softmax-based approach.

cs.LG↗

arXiv2024

Attraction-Repulsion Swarming: A Generalized Framework of t-SNE via Force Normalization and Tunable Interactions

We propose a new method for data visualization based on attraction-repulsion swarming (ARS) dynamics, which we call ARS visualization. ARS is a generalized framework that is based on viewing the t-distributed stochastic neighbor embedding (t-SNE) visualization technique as a swarm of interacting agents driven by attraction and repulsion. Motivated by recent developments in swarming, we modify the t-SNE dynamics to include a normalization by the \emph{total influence}, which results in better posed dynamics in which we can use a data size independent time step (of $h=1$) and a simple iteration, without the need for the array of optimization tricks employed in t-SNE. ARS also includes the ability to separately tune the attraction and repulsion kernels, which gives the user control over the tightness within clusters and the spacing between them in the visualization. In contrast with t-SNE, our proposed ARS data visualization method is not gradient descent on the Kullback-Leibler divergence, and can be viewed solely as an interacting particle system driven by attraction and repulsion forces. We provide theoretical results illustrating how the choice of interaction kernel affects the dynamics, and experimental results to validate our method and compare to t-SNE on the MNIST and Cifar-10 data sets.

cs.LG↗

arXiv2024

En masse scanning and automated surfacing of small objects using Micro-CT

Modern archaeological methods increasingly utilize 3D virtual representations of objects, computationally intensive analyses, high resolution scanning, large datasets, and machine learning. With higher resolution scans, challenges surrounding computational power, memory, and file storage quickly arise. Processing and analyzing high resolution scans often requires memory-intensive workflows, which are infeasible for most computers and increasingly necessitate the use of super-computers or innovative methods for processing on standard computers. Here we introduce a novel protocol for en-masse micro-CT scanning of small objects with a {\em mostly-automated} processing workflow that functions in memory-limited settings. We scanned 1,112 animal bone fragments using just 10 micro-CT scans, which were post-processed into individual PLY files. Notably, our methods can be applied to any object (with discernible density from the packaging material) making this method applicable to a variety of inquiries and fields including paleontology, geology, electrical engineering, and materials science. Further, our methods may immediately be adopted by scanning institutes to pool customer orders together and offer more affordable scanning. The work presented herein is part of a larger program facilitated by the international and multi-disciplinary research consortium known as Anthropological and Mathematical Analysis of Archaeological and Zooarchaeological Evidence (AMAAZE). AMAAZE unites experts in anthropology, mathematics, and computer science to develop new methods for mass-scale virtual archaeological research. Overall, our new scanning method and processing workflows lay the groundwork and set the standard for future mass-scale, high resolution scanning studies.

cs.CV↗

arXiv2024

Continuous data assimilation for hydrodynamics: consistent discretization and application to moment recovery

Motivated by the challenge of moment recovery in hydrodynamic approximation in kinetic theory, we propose a data-driven approach for the hydrodynamic models. Inspired by continuous data assimilation, our method introduces a relaxation-based nudging system coupled with a novel discretization technique. This approach facilitates the simultaneous recovery of both the force term and a high-resolution solution from sparsely observed data. To address potential numerical artifacts, we use kernel regression to fit the observed data. We also analyze the convergence of the proposed nudging system under both full and partial data scenarios. When applied to moment systems, the source term involves the derivative of higher-order moments, our approach serves as a crucial step for data preparation in machine-learning based moment closure models. Multiple numerical experiments demonstrate the effectiveness of our algorithm, and we discuss its potential extension to high-dimensional systems.

math.NA↗

arXiv2024

Convergence rates for Poisson learning to a Poisson equation with measure data

In this paper we prove discrete to continuum convergence rates for Poisson Learning, a graph-based semi-supervised learning algorithm that is based on solving the graph Poisson equation with a source term consisting of a linear combination of Dirac deltas located at labeled points and carrying label information. The corresponding continuum equation is a Poisson equation with measure data in a Euclidean domain $\Omega \subset \mathbb{R}^d$. The singular nature of these equations is challenging and requires an approach with several distinct parts: (1) We prove quantitative error estimates when convolving the measure data of a Poisson equation with (approximately) radial function supported on balls. (2) We use quantitative variational techniques to prove discrete to continuum convergence rates on random geometric graphs with bandwidth $\varepsilon>0$ for bounded source terms. (3) We show how to regularize the graph Poisson equation via mollification with the graph heat kernel, and we study fine asymptotics of the heat kernel on random geometric graphs. Combining these three pillars we obtain $L^1$ convergence rates that scale, up to logarithmic factors, like $O(\varepsilon^{\frac{1}{d+2}})$ for general data distributions, and $O(\varepsilon^{\frac{2-\sigma}{d+4}})$ for uniformly distributed data, where $\sigma>0$. These rates are valid with high probability if $\varepsilon\gg\left({\log n}/{n}\right)^q$ where $n$ denotes the number of vertices of the graph and $q \approx \frac{1}{3d}$.

math.AP↗

arXiv2024

Numerical solution of a PDE arising from prediction with expert advice

This work investigates the online machine learning problem of prediction with expert advice in an adversarial setting through numerical analysis of, and experiments with, a related partial differential equation. The problem is a repeated two-person game involving decision-making at each step informed by $n$ experts in an adversarial environment. The continuum limit of this game over a large number of steps is a degenerate elliptic equation whose solution encodes the optimal strategies for both players. We develop numerical methods for approximating the solution of this equation in relatively high dimensions ($n\leq 10$) by exploiting symmetries in the equation and the solution to drastically reduce the size of the computational domain. Based on our numerical results we make a number of conjectures about the optimality of various adversarial strategies, in particular about the non-optimality of the COMB strategy.

math.NA↗

arXiv2024

Consistency of semi-supervised learning, stochastic tug-of-war games, and the p-Laplacian

In this paper we give a broad overview of the intersection of partial differential equations (PDEs) and graph-based semi-supervised learning. The overview is focused on a large body of recent work on PDE continuum limits of graph-based learning, which have been used to prove well-posedness of semi-supervised learning algorithms in the large data limit. We highlight some interesting research directions revolving around consistency of graph-based semi-supervised learning, and present some new results on the consistency of $p$-Laplacian semi-supervised learning using the stochastic tug-of-war game interpretation of the $p$-Laplacian. We also present the results of some numerical experiments that illustrate our results and suggest directions for future work.

math.ST↗

arXiv2023

Monotone discretizations of levelset convex geometric PDEs

We introduce a novel algorithm that converges to level-set convex viscosity solutions of high-dimensional Hamilton-Jacobi equations. The algorithm is applicable to a broad class of curvature motion PDEs, as well as a recently developed Hamilton-Jacobi equation for the Tukey depth, which is a statistical depth measure of data points. A main contribution of our work is a new monotone scheme for approximating the direction of the gradient, which allows for monotone discretizations of pure partial derivatives in the direction of, and orthogonal to, the gradient. We provide a convergence analysis of the algorithm on both regular Cartesian grids and unstructured point clouds in any dimension and present numerical experiments that demonstrate the effectiveness of the algorithm in approximating solutions of the affine flow in two dimensions and the Tukey depth measure of high-dimensional datasets such as MNIST and FashionMNIST.

math.NA↗

arXiv2023

Novel Batch Active Learning Approach and Its Application to Synthetic Aperture Radar Datasets

Active learning improves the performance of machine learning methods by judiciously selecting a limited number of unlabeled data points to query for labels, with the aim of maximally improving the underlying classifier's performance. Recent gains have been made using sequential active learning for synthetic aperture radar (SAR) data arXiv:2204.00005. In each iteration, sequential active learning selects a query set of size one while batch active learning selects a query set of multiple datapoints. While batch active learning methods exhibit greater efficiency, the challenge lies in maintaining model accuracy relative to sequential active learning methods. We developed a novel, two-part approach for batch active learning: Dijkstra's Annulus Core-Set (DAC) for core-set generation and LocalMax for batch sampling. The batch active learning process that combines DAC and LocalMax achieves nearly identical accuracy as sequential active learning but is more efficient, proportional to the batch size. As an application, a pipeline is built based on transfer learning feature embedding, graph learning, DAC, and LocalMax to classify the FUSAR-Ship and OpenSARShip datasets. Our pipeline outperforms the state-of-the-art CNN-based methods.

cs.LG↗

arXiv2022

Poisson Reweighted Laplacian Uncertainty Sampling for Graph-based Active Learning

We show that uncertainty sampling is sufficient to achieve exploration versus exploitation in graph-based active learning, as long as the measure of uncertainty properly aligns with the underlying model and the model properly reflects uncertainty in unexplored regions. In particular, we use a recently developed algorithm, Poisson ReWeighted Laplace Learning (PWLL) for the classifier and we introduce an acquisition function designed to measure uncertainty in this graph-based classifier that identifies unexplored regions of the data. We introduce a diagonal perturbation in PWLL which produces exponential localization of solutions, and controls the exploration versus exploitation tradeoff in active learning. We use the well-posed continuum limit of PWLL to rigorously analyze our method, and present experimental results on a number of graph-based image classification problems.

stat.ML↗

arXiv2022

Ratio convergence rates for Euclidean first-passage percolation: Applications to the graph infinity Laplacian

In this paper we prove the first quantitative convergence rates for the graph infinity Laplace equation for length scales at the connectivity threshold. In the graph-based semi-supervised learning community this equation is also known as Lipschitz learning. The graph infinity Laplace equation is characterized by the metric on the underlying space, and convergence rates follow from convergence rates for graph distances. At the connectivity threshold, this problem is related to Euclidean first passage percolation, which is concerned with the Euclidean distance function $d_{h}(x,y)$ on a homogeneous Poisson point process on $\mathbb{R}^d$, where admissible paths have step size at most $h>0$. Using a suitable regularization of the distance function and subadditivity we prove that ${d_{h_s}(0,se_1)}/ s \to \sigma$ as $s\to\infty$ almost surely where $\sigma \geq 1$ is a dimensional constant and $h_s\gtrsim \log(s)^\frac{1}{d}$. A convergence rate is not available due to a lack of approximate superadditivity when $h_s\to \infty$. Instead, we prove convergence rates for the ratio $\frac{d_{h}(0,se_1)}{d_{h}(0,2se_1)}\to \frac{1}{2}$ when $h$ is frozen and does not depend on $s$. Combining this with the techniques that we developed in (Bungert, Calder, Roith, IMA Journal of Numerical Analysis, 2022), we show that this notion of ratio convergence is sufficient to establish uniform convergence rates for solutions of the graph infinity Laplace equation at percolation length scales.

math.PR↗