arXiv subjects

Ben S. Southworth

Publications and source records attributed to Ben S. Southworth.

At least 19 recordsLinked to original sources

arXiv2026

Multiscale Graph Reduction for Heterogeneous and Anisotropic Discrete Diffusion Processes

We present multiscale graph-based reduction algorithms for upscaling heterogeneous and anisotropic diffusion problems. The proposed coarsening approaches begin by constructing a partition of the computational domain into a set of balanced local subdomains, resulting in a standard type of domain decomposition. Given this initial decomposition, general coarsening techniques based on spectral clustering are applied within each subgraph to accurately identify the key microscopic features of a given system. The spectral clustering algorithm is based on local generalized eigen-decompositions applied to the signed graph Laplacian. The resulting coarse-fine splittings are combined with two variants of energy-minimizing strategies for constructing coarse bases for diffusion problems. The first is an unconstrained minimization formulation in which local harmonic extensions are applied column-wise to construct multi-vector preserving interpolation in each region, whereas the second approach is a variant of the constrained energy minimization formulations derived in the context of non-local multi-continua upscaling techniques. We apply the resulting upscaling algorithms to a variety of tests arising from the graph Laplacian, including diffusion in the perforated domain, channelized media, highly anisotropic settings, and discrete pore network models to demonstrate the potential and robustness of the proposed coarsening approaches. We show numerically and theoretically that the proposed approaches lead to accurate coarse-scale models and approximations relative to the corresponding fine-scale discrete system and corresponding approximation. Initial results of a two-level solver applied to these same test problems that uses in addition a block smoother with coloring are also provided.

math.NA↗

arXiv2026

Preconditioning transformations of adjoint systems for evolution equations

Achieving robust control and optimization in high-fidelity physics simulations is extremely challenging, especially for evolutionary systems whose solutions span vast scales across space, time, and physical variables. In conjunction with gradient-based methods, adjoint systems are widely used in the optimization of systems subject to differential equation constraints. In optimization, gradient-based methods are often transformed using suitable preconditioners to accelerate the convergence of the optimization algorithm. Inspired by preconditioned gradient descent methods, we introduce a framework for the preconditioning of adjoint systems associated to evolution equations, which allows one to reshape the dynamics of the adjoint system. We develop two classes of adjoint preconditioning transformations: those that transform both the state dynamics and the adjoint equation and those that transform only the adjoint equation while leaving the state dynamics invariant. Both classes of transformations have the flexibility to include generally nonlinear state-dependent transformations. Using techniques from symplectic geometry and Hamiltonian mechanics, we further show that these preconditioned adjoint systems preserve the property that the adjoint system backpropagates the derivative of an objective function. We then apply this framework to the setting of coupled evolution equations, where we develop a notion of scale preconditioning of the adjoint equations when the state dynamics exhibit large scale-separation. We demonstrate the proposed scale preconditioning on an inverse problem for the radiation diffusion equations. Naive gradient descent is unstable for any practical gradient descent step size, whereas our proposed scale-preconditioned adjoint descent converges in 10-15 gradient-based optimization iterations, with highly accurate reproduction of the wavefront at the final time.

math.OC↗

arXiv2026

An Efficient Solver for Finite Element-based Constrained Transport in 3D Magnetohydrodynamics Applied to Magnetic Confinement Fusion

We present an efficient solver framework for the stiff magnetic wave coupling arising in resistive magnetohydrodynamics (MHD) on realistic tokamak geometries. The approach builds on an implicit-implicit (IMIM) time-splitting that separates fast magnetic waves and anisotropic heat transport from slower acoustic dynamics while retaining full coupling (Krzysik et al. 2026). Within this formulation, the magnetic wave subsystem appears as an anisotropic curl-curl operator, enabling the use of scalable auxiliary-space Maxwell (AMS) multigrid solvers. To exploit this structure at the discrete level, we employ curl-conforming finite element spaces for the magnetic field and design the velocity space to preserve the curl-curl structure induced by the Lorentz-force coupling. The resulting compatible discretization preserves the discrete magnetic divergence constraint while producing linear systems directly amenable to efficient AMS-based solvers. We demonstrate solver efficiency as well as the accuracy and stability of the resulting structure-preserving discretization on fully nonlinear three-dimensional tokamak test cases.

math.NA↗

arXiv2026

Shock solutions for the one-dimensional information geometric regularization of compressible flow

The information geometric regularization (IGR) is an inviscid regularization of the compressible Euler equations that alters the geometry of Lagrangian characteristics to prevent trajectories from crossing in finite time. Previous work on IGR established global strong solutions in one dimension, explored thermodynamic effects of the model, and enabled large-scale simulations of compressible flow. However, a fundamental question that remains is how this regularization alters the structure and regularity of a shock-like solution. We prove existence, uniqueness modulo translation, and regularity of transonic compressive IGR shock profiles in one spatial dimension. The analysis applies to the full thermodynamic compressible Euler--IGR model with a general equation of state, subject to mild convexity hypotheses. A traveling-wave ansatz reduces the Euler--IGR equations to a degenerate second-order scalar equation for the density profile. At the sonic crossing, the elliptic coefficient degenerates: the density profile remains continuous, but its derivative diverges. The profile is a classical solution away from this single point, while at the degeneracy it retains quantified Hölder and Sobolev regularity. We also analyze the vanishing-regularization limit, showing that the shock width scales like $\sqrt{α}$ and that the IGR profiles converge to the entropy-admissible Euler shock.

math.AP↗

arXiv2026

Broken-space Additive Schwarz Mass Inverse Approximations and (Block) Preconditioning

Finite-element mass matrix solves and approximate inverses arise often in explicit time integration as well as Schur-complement-based block preconditioning. A diagonal approximation of the mass matrix is cheap and widely used, but can be a poor approximation, particularly for high-order elements. This paper introduces a broken-space additive Schwarz (BRAS) mass inverse approximation, formed by applying exact element-local inverse mass matrices on the broken finite-element space and averaging the result back to the conforming space. The construction uses the same element matrices and local-to-global maps as standard mass assembly, has the same element-adjacency sparsity graph as the conforming mass matrix, and is symmetric positive definite for any conforming space, mesh geometry, and polynomial basis. We prove spectral bounds for the preconditioned mass matrix, and wide-ranging numerical experiments for $H^1$, $H(\operatorname{curl})$, and $H(\operatorname{div})$ finite elements on two- and three-dimensional simplicial meshes show that BRAS reduces spectral condition numbers, Krylov iterations, and solve times relative to diagonal preconditioning. For preconditioned conjugate gradient (CG), BRAS yields a 1.1--4.7$\times$ speedup over diagonal/Jacobi preconditioning across all cases tested over finite-element orders $p\in[1,4]$. Further, theory and numerical experiments show that in the block-preconditioning case, BRAS can improve Schur-complement approximations and reduce outer solve times. On mixed Poisson and biharmonic systems, BRAS yields a 1.5--3$\times$ speedup in time-to-solution over a standard diagonal-based preconditioning approach.

math.NA↗

arXiv2026

Causal Multi-fidelity Surrogate Forward and Inverse Models for ICF Implosions

Continued progress in inertial confinement fusion (ICF) requires solving inverse problems relating experimental observations to simulation input parameters, followed by design optimization. However, such high-dimensional dynamic PDE-constrained optimization problems are extremely challenging or even intractable. It has been recently shown that inverse problems can be solved by only considering certain robust features. Here we consider the ICF capsule's deuterium-tritium (DT) interface, and construct a causal, dynamic, multifidelity reduced-order surrogate that maps from a time-dependent radiation temperature drive to the interface's radius and velocity dynamics. The surrogate targets an ODE embedding of DT interface dynamics, and is constructed by learning a controller for a base analytical model using low- and high-fidelity simulation training data with respect to radiation energy group structure. After demonstrating excellent accuracy of the surrogate interface model, we use machine learning (ML) models with surrogate-generated data to solve inverse problems optimizing radiation temperature drive to reproduce observed interface dynamics. For sparse snapshots in time, the ML model further characterizes the most informative times at which to sample dynamics. Altogether we demonstrate how operator learning, causal architectures, and physical inductive bias can be integrated to accelerate discovery, design, and diagnostics in high-energy-density systems.

physics.comp-ph↗

arXiv2026

Muon in Vision Transformers: Optimizer-Recipe Interactions and Gradient Spectra

Muon is a recently developed matrix-aware optimizer that has shown strong results in transformer training, but its behavior in vision transformers (ViTs) is not yet well understood. We study Muon for ViT training, largely on ImageNet-100 and Pl@ntNet-300K, comparing against AdamW under standard vision recipes involving mixup, cutmix, smoothing, and random augmentation and erasing. Muon consistently outperforms AdamW, with especially large gains on long-tailed Pl@ntNet macro top-1. These gains are also recipe-dependent, where Muon benefits much more than AdamW from advanced and significant data augmentation techniques. To understand this interaction, we analyze the singular-value structure of matrix gradients throughout the ViT. Within Muon training runs, removing heavy data augmentation induces a late-training spectral concentration and mode collapse in gradient matrices, primarily in deep MLP-down blocks. Under a fixed "full" augmentation recipe, the clearest Muon-AdamW contrast appears instead in QKV gradients, where AdamW gradient energy remains concentrated in a much narrower basis while Muon spreads energy across substantially more singular modes. Muon in ViTs is therefore best understood as an optimizer-recipe interaction. Under a fixed recipe, Muon differs from AdamW most clearly in attention projections, where its gradients consist of a broader spectral basis. Within Muon, a full training recipe is important for preventing late spectral concentration and mode collapse in deep feedforward blocks. We further demonstrate efficacy in training ViTs on image segmentation and masked autoencoder models, where Muon outperforms AdamW in all settings considered.

cs.LG↗

arXiv2026

Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Training

Orthogonalized-momentum optimizers such as Muon improve transformer training by approximately whitening/orthogonalizing matrix-valued momentum updates via a short polar-decomposition iteration. However, polar-factor approximations typically require multiple large matrix multiplications, and the resulting overhead can be substantial and hardware-dependent. We introduce MUD (MomentUm Decorrelation), a complementary whitening approach that replaces Muon's polar update with a triangular (Cholesky-like) whitening surrogate inspired by classical Gram--Schmidt and Gauss-Seidel ideas. We show that row-orthonormal matrices are fixed points of the MUD map, relate the inner step to symmetric Gauss-Seidel preconditioning of the Gram matrix, and prove quadratic local convergence near the fixed point. In terms of time-to-perplexity, MUD yields consistent 10-50\% wall-clock improvements over tuned AdamW and Muon in time-to-perplexity, typically converging slightly slower per step than Muon but with substantially lower optimizer overhead -- relative to Muon, MUD improves peak tokens/s by roughly $1.3-2.6\times$ across most settings and up to nearly $3\times$ on GPT-2 large on an A100. We also demonstrate training a ESM-2 150M protein language model, where MUD matches Muon-level validation perplexity in significantly less wall-clock time.

cs.LG↗

arXiv2026

Multilevel Training for Kolmogorov Arnold Networks

Algorithmic speedup of training common neural architectures is made difficult by the lack of structure guaranteed by the function compositions inherent to such networks. In contrast to multilayer perceptrons (MLPs), Kolmogorov-Arnold networks (KANs) provide more structure by expanding learned activations in a specified basis. This paper exploits this structure to develop practical algorithms and theoretical insights, yielding training speedup via multilevel training for KANs. To do so, we first establish an equivalence between KANs with spline basis functions and multichannel MLPs with power ReLU activations through a linear change of basis. We then analyze how this change of basis affects the geometry of gradient-based optimization with respect to spline knots. The KANs change-of-basis motivates a multilevel training approach, where we train a sequence of KANs naturally defined through a uniform refinement of spline knots with analytic geometric interpolation operators between models. The interpolation scheme enables a ``properly nested hierarchy'' of architectures, ensuring that interpolation to a fine model preserves the progress made on coarse models, while the compact support of spline basis functions ensures complementary optimization on subsequent levels. Numerical experiments demonstrate that our multilevel training approach can achieve orders of magnitude improvement in accuracy over conventional methods to train comparable KANs or MLPs, particularly for physics informed neural networks. Finally, this work demonstrates how principled design of neural networks can lead to exploitable structure, and in this case, multilevel algorithms that can dramatically improve training performance.

cs.LG↗

arXiv2026

Implicit-Explicit Scheme with Multiscale Vanka Two-Grid Solver for Heterogeneous Unsaturated Poroelasticity

We consider a coupled nonlinear system of equations that describe unsaturated flow in heterogeneous poroelastic media. For the numerical solution, we use a finite element approximation in space and present an efficient multiscale two-grid solver for solving the coupled system of equations. The proposed two-grid solver contains two main parts: (i) accurate coarse grid approximation based on local spectral spaces and (ii) coupled smoothing iterations based on an overlapping multiscale Vanka method. A Vanka smoother and local spectral coarse grids come with significant computational cost in the setup phase. To avoid constructing a new solver for each time step and/or nonlinear iteration, we utilize an implicit-explicit integration scheme in time, where we partition the nonlinear operator as a sum of linear and nonlinear parts. In particular, we construct an implicit linear approximation of the stiff components that remains fixed across all time, while treating the remaining nonlinear residual explicitly. This allows us to construct a robust two-grid solver offline and utilize it for fast and efficient online time integration. A linear stability analysis of the proposed novel coupled scheme is presented based on the representation of the system as a two-step scheme. We show that the careful decomposition of linear and nonlinear parts guarantees a linearly stable scheme. A numerical study is presented for a two-dimensional nonlinear coupled test problem of unsaturated flow in heterogeneous poroelastic media. We demonstrate the robustness of the two-grid solver, particularly the efficacy of block smoothing compared with simple pointwise smoothing, and illustrate the accuracy and stability of implicit-explicit time integration.

math.NA↗

arXiv2026

Algebraic Multigrid with Overlapping Schwarz Smoothers and Local Spectral Coarse Grids for Least Squares Problems

This paper develops a new algebraic multigrid (AMG) method for sparse least-squares systems of the form $A=G^TG$ motivated by challenging applications in scientific computing where classical AMG methods fail. First we review and relate the use of local spectral problems in distinct fields of literature on AMG, domain decomposition (DD), and multiscale finite elements. We then propose a new approach blending aggregation-based coarsening, overlapping Schwarz smoothers, and locally constructed spectral coarse spaces. By exploiting the factorized structure of $A$, we construct an inexpensive symmetric positive semidefinite splitting that yields local generalized eigenproblems whose solutions define sparse, nonoverlapping coarse basis functions. This enables a fully algebraic and naturally recursive multilevel hierarchy that can either coarsen slowly to achieve AMG-like operator complexities, or coarsen aggressively-with correspondingly larger local spectral problems-to ensure robustness on problems that cannot be solved by existing AMG methods. The method requires no geometric information, avoids global eigenvalue solves, and maintains efficient parallelizable setup through localized operations. Numerical experiments demonstrate that the proposed least-squares AMG-DD method achieves convergence rates independent of anisotropy on rotated diffusion problems and remains scalable with problem size, while for small amounts of anisotropy we obtain convergence and operator complexities comparable with classical AMG methods. Most notably, for extremely anisotropic heat conduction operators arising in magnetic confinement fusion, where AMG and smoothed aggregation fail to reduce the residual even marginally, our method provides robust and efficient convergence across many orders of magnitude in anisotropy strength.

math.NA↗

arXiv2025

Hamiltonian Information Geometric Regularization of the Compressible Euler Equations

The recently proposed information geometric regularization (IGR) was the first inviscid regularization of the multi-dimensional compressible Euler equations, which enabled the simulation of realistic compressible fluid models at an unprecedented scale. However, the thermodynamic effects of this regularization have not yet been understood in a principled manner. To achieve a proper understanding of the thermodynamic aspects of the IGR, we decompose the regularization into its conservative dynamics, framed as a Hamiltonian subsystem, and its dissipative dynamics. In so doing, we further introduce two more models to compare to IGR, the Hamiltonian regularized Euler (HRE) model, which is the first multi-dimensional, non-dispersive Hamiltonian regularization of the compressible Euler equations with energy, as well as the Hamiltonian IGR (HIGR) model, which modifies the dissipation used in the IGR model to instead utilize a metriplectic dissipative force. Despite having many attractive features, the HRE and HIGR models exhibit notable defects in numerical tests on colliding shock problems, which preclude their use as computational tools without further study of dissipative weak solutions to these models. Additionally, our analysis presents new results on the IGR model itself, including its ability to conserve acoustic waves, as well as local energy transport laws and entropy production rates. By separating the conservative and dissipative dynamics of the IGR, our hope is that subsequent of analysis of the IGR model can benefit from this natural decomposition, such as, for example, rigorous proofs of strong solutions for multi-dimensional IGR for the compressible Euler system with thermodynamics.

math-ph↗

arXiv2025

Preconditioning a hybridizable discontinuous Galerkin method for Navier-Stokes at high Reynolds number

We introduce a preconditioner for a hybridizable discontinuous Galerkin discretization of the linearized Navier-Stokes equations at high Reynolds number. The preconditioner is based on an augmented Lagrangian approach of the full discretization. Unlike standard grad-div type augmentation, however, we consider augmentation based on divergence-conformity. With this augmentation we introduce two different, well-conditioned, and easy to solve matrices to approximate the trace pressure Schur complement. To introduce a completely algebraic solver, we propose to use multifrontal sparse LU solvers using butterfly compression to solve the trace velocity block. Numerical examples demonstrate that the trace pressure Schur complement is highly robust in mesh spacing and Reynolds number and that the multifrontal inexact LU performs well for a wide range of Reynolds numbers.

math.NA↗

arXiv2025

Galerkin Formulation of Path Integrals in Lattice Field Theory

We present a mathematical framework for Galerkin formulations of path integrals in lattice field theory. The framework is based on using the degrees of freedom associated to a Galerkin discretization as the fundamental lattice variables. We formulate standard concepts in lattice field theory, such as the partition function and correlation functions, in terms of the degrees of freedom. For example, using continuous finite element spaces, we show that the two-point spatial correlation function can be defined between any two points on the domain (as opposed to at just lattice sites) and furthermore, that this satisfies a weak propagator (or Green's function) identity, in analogy to the continuum case. Furthermore, this framework leads naturally to higher-order formulations of lattice field theories by considering higher-order finite element spaces for the Galerkin discretization. We consider analytical and numerical examples of scalar field theory to investigate how increasing the order of piecewise polynomial finite element spaces affect the approximation of lattice observables. Finally, we sketch an outline of this Galerkin framework in the context of gauge field theories.

hep-lat↗

arXiv2025

Physics consistent machine learning framework for inverse modeling with applications to ICF capsule implosions

In high energy density physics (HEDP) and inertial confinement fusion (ICF), predictive modeling is complicated by uncertainty in parameters that characterize various aspects of the modeled system, such as those characterizing material properties, equation of state (EOS), opacities, and initial conditions. Typically, however, these parameters are not directly observable. What is observed instead is a time sequence of radiographic projections using X-rays. In this work, we define a set of sparse hydrodynamic features derived from the outgoing shock profile and outer material edge, which can be obtained from radiographic measurements, to directly infer such parameters. Our machine learning (ML)-based methodology involves a pipeline of two architectures, a radiograph-to-features network (R2FNet) and a features-to-parameters network (F2PNet), that are trained independently and later combined to approximate a posterior distribution for the parameters from radiographs. We show that the estimated parameters can be used in a hydrodynamics code to obtain density fields and hydrodynamic shock and outer edge features that are consistent with the data. Finally, we demonstrate that features resulting from an unknown EOS model can be successfully mapped onto parameters of a chosen analytical EOS model, implying that network predictions are learning physics, with a degree of invariance to the underlying choice of EOS model.

physics.comp-ph↗

arXiv2025

Optimal transfer operators in algebraic two-level methods for nonsymmetric and indefinite problems

Consider an algebraic two-level method applied to the $n$-dimensional linear system $A \mathbf{x} = \mathbf{b}$ using fine-space preconditioner (i.e., ``relaxation'' or ``smoother'') $M$, with $M \approx A$, restriction and interpolation $R$ and $P$, and algebraic coarse-space operator ${A_c := R^*AP}$. Then, what are the the best possible transfer operators $R$ and $P$ of a given dimension $n_c < n$? Brannick et al. (2018) showed that when $A$ and $M$ are Hermitian positive definite (HPD), the optimal interpolation is such that its range contains the $n_c$ smallest generalized eigenvectors of the matrix pencil $(A, M)$. Recently, in Ali et al. (2025) we generalized this framework to the non-HPD setting, by considering both right (interpolation) and left (restriction) generalized eigenvectors of $(A, M)$ and defining corresponding nonsymmetric transfer operators $\{R_\#,P_\#\}$. Tight convergence bounds for $\{R_\#,P_\#\}$ are derived in spectral radius, as well as a proof of pseudo-optimality. Note, $\{R_\#,P_\#\}$ are typically complex valued, which is not practical for real-valued problems. Here we build on Ali et al. (2025), first characterizing all inner products in which the coarse-space correction defined by $\{R_\#,P_\#\}$ is orthogonal. We then develop tight two-level convergence bounds in these norms, and prove that the underlying transfer operators $\{R_\#,P_\#\}$ are genuinely optimal. As a special case, our theory both recovers and extends the HPD results from Brannick et al. (2018). Finally, we show how to construct optimal, real-valued transfer operators in the case of that $A$ and $M$ are real valued, but are not HPD. Numerical examples arising from discretized advection and wave-equation problems are used to verify and illustrate the theory.

math.NA↗

arXiv2025

Nonlinear Splitting for Gradient-Based Unconstrained and Adjoint Optimization

High dimensional and/or nonconvex optimization remains a challenging and important problem across a wide range of fields, such as machine learning, data assimilation, and partial differential equation (PDE) constrained optimization. Here we consider gradient-based methods for solving unconstrained and constrained optimization problems, and introduce the concept of nonlinear splitting to improve accuracy and efficiency. For unconstrained optimization, we consider splittings of the gradient to depend on two arguments, leading to semi-implicit gradient optimization algorithms. In the context of adjoint-based constrained optimization, we propose a splitting of the constraint $F(\mathbf{x},θ)$, effectively expanding the space on which we can evaluate the ``gradient''. In both cases, the formalism further allows natural coupling of nonlinearly split optimization methods with acceleration techniques, such as Nesterov or Anderson acceleration. The framework is demonstrated to outperform existing methods in terms of accuracy and/or runtime on a handful of diverse optimization problems. This includes low-dimensional analytic nonconvex functions, high-dimensional nonlinear least squares in quantum tomography, and PDE-constrained optimization of kinetic equations, where the total number of high-dimensional kinetic solves is reduced by a factor of three compared with standard adjoint optimization.

math.OC↗

arXiv2025

Moment-based adaptive time integration for thermal radiation transport

In this paper we develop a framework for moment-based adaptive time integration of deterministic multifrequency thermal radiation transpot (TRT). We generalize our recent semi-implicit-explicit (IMEX) integration framework for gray TRT to multifrequency TRT, and also introduce a semi-implicit variation that facilitates higher-order integration of TRT, where each stage is implicit in all components except opacities. To appeal to the broad literature on adaptivity with Runge--Kutta methods, we derive new embedded methods for four asymptotic preserving IMEX Runge--Kutta schemes we have found to be robust in our previous work on TRT and radiation hydrodynamics. We then use a moment-based high-order-low-order representation of the transport equations. Due to the high dimensionality, memory is always a concern in simulating TRT. We form error estimates and adaptivity in time purely based on temperature and radiation energy, for a trivial overhead in computational cost and memory usage compared with the base second order integrators. We then test the adaptivity in time on the tophat and Larsen problem, demonstrating the ability of the adaptive algorithm to naturally vary the timestep across 4--5 orders of magnitude, ranging from the dynamical timescales of the streaming regime to the thick diffusion limit.

math.NA↗