arXiv subjects

Joseph Hart

Publications and source records attributed to Joseph Hart.

At least 19 recordsLinked to original sources

arXiv2026

Hyper-differential sensitivity analysis with respect to model discrepancy: Sequential optimal experimental design

Large-scale optimization problems are ubiquitous in the physical sciences; yet, high-fidelity models can often be complex and computationally prohibitive for optimization. A practical alternative is to use a low-fidelity model to facilitate optimization. However, the discrepancy between the high- and low-fidelity models can lead to suboptimal solutions. To address this, we build on recent work in Hyper-Differential Sensitivity Analysis to leverage limited high-fidelity simulations to update the optimization solution. Our contributions in this article include: (i) incorporating pseudo-time continuation techniques to efficiently compute higher-accuracy optimal solution updates, and (ii) proposing a Bayesian framework for sequential data acquisition that strategically guides high-fidelity evaluations and reduces uncertainty in the model discrepancy estimation. Numerical results demonstrate that our framework delivers significant improvements to optimization solutions with only a few high-fidelity evaluations.

math.NA↗

arXiv2026

MACE-POLAR-1: A Polarisable Electrostatic Foundation Model for Molecular Chemistry

Accurate modelling of electrostatic interactions and charge transfer is fundamental to computational chemistry, yet most machine learning interatomic potentials (MLIPs) rely on local atomic descriptors that cannot capture long-range electrostatic effects. We present a new electrostatic foundation model for molecular chemistry that extends the MACE architecture with explicit treatment of long-range interactions and electrostatic induction. Our approach combines local many-body geometric features with a non-self-consistent field formalism that updates learnable charge and spin densities through polarisable iterations to model induction, followed by global charge equilibration via learnable Fukui functions to control total charge and total spin. This design enables an accurate and physical description of systems with varying charge and spin states while maintaining computational efficiency. Trained on the OMol25 dataset of 100 million hybrid DFT calculations, our models achieve chemical accuracy across diverse benchmarks, with accuracy competitive with hybrid DFT on thermochemistry, reaction barriers, conformational energies, and transition metal complexes. Notably, we demonstrate that the inclusion of long-range electrostatics leads to a large improvement in the description of non-covalent interactions and supramolecular complexes over non-electrostatic models, including sub-kcal/mol prediction of molecular crystal formation energy in the X23-DMC dataset and a fourfold improvement over short-ranged models on protein-ligand interactions. The model's ability to handle variable charge and spin states, respond to external fields, provide interpretable spin-resolved charge densities, and maintain accuracy from small molecules to protein-ligand complexes positions it as a versatile tool for computational molecular chemistry and drug discovery.

physics.chem-ph↗

arXiv2025

Cross Learning between Electronic Structure Theories for Unifying Molecular, Surface, and Inorganic Crystal Foundation Force Fields

Creating a single unified interatomic potential capable of attaining ab initio accuracy across all chemistry remains a long-standing challenge in computational chemistry and materials science. This work introduces a training protocol for foundation machine-learning interatomic potentials (MLIPs) that bridge molecular, surface, and materials chemistry through cross-domain learning. First, we introduce enhancements to the MACE architecture that improve its performance on chemically diverse databases by increasing weight sharing across chemical elements and introducing non-linear factors into the tensor decomposition of the product basis. Second, we develop a multi-head replay post-training methodology that enables efficient knowledge transfer across diverse chemical domains. By fine-tuning on datasets at different levels of electronic structure theory, including inorganic crystals, molecular systems, surface chemistry, and reactive organic chemistry, we demonstrate that a single unified model achieves state-of-the-art performance across several chemical domains. Comprehensive benchmarking reveals superior cross-domain transferability compared with existing specialised and multi-task models, with notable improvements in molecular and surface properties while maintaining state-of-the-art performance in materials-property prediction.

physics.chem-ph↗

arXiv2025

Toward real-time optimization through model reduction and model discrepancy sensitivities

Optimization problems arise in a range of scenarios, from optimal control to model parameter estimation. In many applications, such as the development of digital twins, it is essential to solve these optimization problems within wall-clock-time limitations. However, this is often unattainable for complex systems, such as those modeled by nonlinear partial differential equations. One strategy for mitigating this issue is to construct a reduced-order model (ROM) that enables more rapid optimization. In particular, the use of nonintrusive ROMs -- those that do not require access to the full-order model at evaluation time -- is popular because they facilitate optimization solutions can be computed within the wall-clock-time requirements. However, the optimization solution will be unreliable if the iterates move outside the ROM training data. This article proposes the use of hyper-differential sensitivity analysis with respect to model discrepancy (HDSA-MD) as a computationally efficient tool to augment ROM-constrained optimization and improve its reliability. The proposed approach consists of two phases: (i) an offline phase where several full-order model evaluations are computed to train the ROM, and (ii) an online phase where a ROM-constrained optimization problem is solved, $N=\mathcal{O}(1)$ full-order model evaluations are computed, and HDSA-MD is used to enhance the optimization solution using the full-order model data. Numerical results are demonstrated for two examples, atmospheric contaminant control and wildfire ignition location estimation, in which a ROM is trained offline using inaccurate atmospheric data. The HDSA-MD update yields a significant improvement in the ROM-constrained optimization solution using only one full-order model evaluation online with corrected atmospheric data.

math.OC↗

arXiv2025

Preconditioned pseudo-time continuation for parameterized inverse problems

We consider parameterized variational inverse problems that are constrained by partial differential equations (PDEs). We seek to efficiently compute the solution of the inverse problem when auxiliary model parameters, which appear in the governing PDE, are varied. Computing the solution of the inverse problem for different auxiliary parameter values is crucial for uncertainty quantification. This, however, is computationally challenging since it requires solving many optimization problems for different realizations of the auxiliary parameters. We leverage pseudo-time continuation and solve an initial value problem to evolve the optimal solution along an auxiliary parameter path. This article introduces the use of an adaptive quasi-Newton Hessian preconditioner to accelerate the computation. Our proposed preconditioner exploits properties of the pseudo-time continuation process to achieve reliable and efficient computation. We elaborate our proposed framework and elucidate its properties for two nonlinear inverse problems.

math.NA↗

arXiv2025

Hyper-differential sensitivity analysis with respect to model discrepancy: Prior distributions

Hyper-differential sensitivity analysis with respect to model discrepancy was recently developed to enable uncertainty quantification for optimization problems. The approach consists of two primary steps: (i) Bayesian calibration of the discrepancy between high- and low-fidelity models, and (ii) propagating the model discrepancy uncertainty through the optimization problem. When high-fidelity model evaluations are limited, as is common in practice, the prior discrepancy distribution plays a crucial role in the uncertainty analysis. However, specification of this prior is challenging due to its mathematical complexity and many hyper-parameters. This article presents a novel approach to specify the prior distribution. Our approach consists of two parts: (1) an algorithmic initialization of the prior hyper-parameters that uses existing data to initialize a hyper-parameter estimate, and (2) a visualization framework to systematically explore properties of the prior and guide tuning of the hyper-parameters to ensure that the prior captures the appropriate range of uncertainty. We provide detailed mathematical analysis and a collection of numerical examples that elucidate properties of the prior that are crucial to ensure uncertainty quantification.

math.NA↗

arXiv2024

Photonic frequency multiplexed next-generation reservoir computer

In this work, we introduce and experimentally demonstrate a photonic frequency-multiplexed next generation reservoir computer (FM-NGRC) capable of performing real-time inference at GHz speed. NGRCs apply a feed-forward architecture to produce a feature vector directly from the input data over a fixed number of time steps. This feature vector, analogous to the reservoir state in a conventional RC, is used to perform inference by applying a decision layer trained by linear regression. Photonic NGRC provides a flexible platform for real-time inference by forgoing the need for explicit feedback loops inherent to a physical reservoir. The FM-NGRC introduced here defines the memory structure using an optical frequency comb and dispersive fiber while the sinusoidal response of electro-optic Mach-Zehnder interferometers controls the nonlinear transform applied to elements of the feature vector. A programmable waveshaper modulates each comb tooth independently to apply the trained decision layer weights in the analog domain. We apply the FM-NGRC to solve the benchmark nonlinear channel equalization task; after theoretically determining feature vectors that enable high-accuracy distortion compensation, we construct an FM-NGRC that generates these vectors to experimentally demonstrate real-time channel equalization at 5 GS/s with a symbol error rate of $\sim 2\times 10^{-3}$.

physics.optics↗

arXiv2024

Photonic next-generation reservoir computer based on distributed feedback in optical fiber

Reservoir computing (RC) is a machine learning paradigm that excels at dynamical systems analysis. Photonic RCs, which perform implicit computation through optical interactions, have attracted increasing attention due to their potential for low latency predictions. However, most existing photonic RCs rely on a nonlinear physical cavity to implement system memory, limiting control over the memory structure and requiring long warm-up times to eliminate transients. In this work, we resolve these issues by demonstrating a photonic next-generation reservoir computer (NG-RC) using a fiber optic platform. Our photonic NG-RC eliminates the need for a cavity by generating feature vectors directly from nonlinear combinations of the input data with varying delays. Our approach uses Rayleigh backscattering to produce output feature vectors by an unconventional nonlinearity resulting from coherent, interferometric mixing followed by a quadratic readout. Performing linear optimization on these feature vectors, our photonic NG-RC demonstrates state-of-the-art performance for the observer (cross-prediction) task applied to the R\"ossler, Lorenz, and Kuramoto-Sivashinsky systems. In contrast to digital NG-RC implementations, this scheme is easily scalable to high-dimensional systems while maintaining low latency and low power consumption.

physics.optics↗

arXiv2024

Hyper-differential sensitivity analysis with respect to model discrepancy: Posterior Optimal Solution Sampling

Optimization constrained by high-fidelity computational models has potential for transformative impact. However, such optimization is frequently unattainable in practice due to the complexity and computational intensity of the model. An alternative is to optimize a low-fidelity model and use limited evaluations of the high-fidelity model to assess the quality of the solution. This article develops a framework to use limited high-fidelity simulations to update the optimization solution computed using the low-fidelity model. Building off a previous article [22], which introduced hyper-differential sensitivity analysis with respect to model discrepancy, this article provides novel extensions of the algorithm to enable uncertainty quantification of the optimal solution update via a Bayesian framework. Specifically, we formulate a Bayesian inverse problem to estimate the model discrepancy and propagate the posterior model discrepancy distribution through the post-optimality sensitivity operator for the low-fidelity optimization problem. We provide a rigorous treatment of the Bayesian formulation, a computationally efficient algorithm to compute posterior samples, a guide to specify and interpret the algorithm hyper-parameters, and a demonstration of the approach on three examples which highlight various types of discrepancy between low and high-fidelity models.

math.NA↗

arXiv2023

Solving High-Dimensional Inverse Problems with Auxiliary Uncertainty via Operator Learning with Limited Data

In complex large-scale systems such as climate, important effects are caused by a combination of confounding processes that are not fully observable. The identification of sources from observations of system state is vital for attribution and prediction, which inform critical policy decisions. The difficulty of these types of inverse problems lies in the inability to isolate sources and the cost of simulating computational models. Surrogate models may enable the many-query algorithms required for source identification, but data challenges arise from high dimensionality of the state and source, limited ensembles of costly model simulations to train a surrogate model, and few and potentially noisy state observations for inversion due to measurement limitations. The influence of auxiliary processes adds an additional layer of uncertainty that further confounds source identification. We introduce a framework based on (1) calibrating deep neural network surrogates to the flow maps provided by an ensemble of simulations obtained by varying sources, and (2) using these surrogates in a Bayesian framework to identify sources from observations via optimization. Focusing on an atmospheric dispersion exemplar, we find that the expressive and computationally efficient nature of the deep neural network operator surrogates in appropriately reduced dimension allows for source identification with uncertainty quantification using limited data. Introducing a variable wind field as an auxiliary process, we find that a Bayesian approximation error approach is essential for reliable source inversion when uncertainty due to wind stresses the algorithm.

stat.ML↗

arXiv2022

Hyper-differential sensitivity analysis in the context of Bayesian inference applied to ice-sheet problems

Inverse problems constrained by partial differential equations (PDEs) play a critical role in model development and calibration. In many applications, there are multiple uncertain parameters in a model which must be estimated. Although the Bayesian formulation is attractive for such problems, computational cost and high dimensionality frequently prohibit a thorough exploration of the parametric uncertainty. A common approach is to reduce the dimension by fixing some parameters (which we will call auxiliary parameters) to a best estimate and use techniques from PDE-constrained optimization to approximate properties of the Bayesian posterior distribution. For instance, the maximum a posteriori probability (MAP) and the Laplace approximation of the posterior covariance can be computed. In this article, we propose using hyper-differential sensitivity analysis (HDSA) to assess the sensitivity of the MAP point to changes in the auxiliary parameters. We establish an interpretation of HDSA as correlations in the posterior distribution. Our proposed framework is demonstrated on the inversion of bedrock topography for the Greenland ice sheet with uncertainties arising from the basal friction coefficient and climate forcing (ice accumulation rate)

stat.ME↗

arXiv2022

Hyper-differential sensitivity analysis with respect to model discrepancy: Mathematics and computation

Model discrepancy, defined as the difference between model predictions and reality, is ubiquitous in computational models for physical systems. It is common to derive partial differential equations (PDEs) from first principles physics, but make simplifying assumptions to produce tractable expressions for the governing equations or closure models. These PDEs are then used for analysis and design to achieve desirable performance. For instance, the end goal may be to solve a PDE-constrained optimization (PDECO) problem. This article considers the sensitivity of PDECO problems with respect to model discrepancy. We introduce a general representation of the discrepancy and apply post-optimality sensitivity analysis to derive an expression for the sensitivity of the optimal solution with respect to the discrepancy. An efficient algorithm is presented which combines the PDE discretization, post-optimality sensitivity operator, adjoint-based derivatives, and a randomized generalized singular value decomposition to enable scalable computation. Kronecker product structure in the underlying linear algebra and corresponding infrastructure in PDECO is exploited to yield a general purpose algorithm which is computationally efficient and portable across a range of applications. Known physics and problem specific characteristics of discrepancy are imposed through user specified weighting matrices. We demonstrate our proposed framework on two nonlinear PDECO problems to highlight its computational efficiency and rich insight.

math.NA↗

arXiv2022

Hyper-differential sensitivity analysis with respect to model discrepancy: Optimal solution updating

A common goal throughout science and engineering is to solve optimization problems constrained by computational models. However, in many cases a high-fidelity numerical emulation of systems cannot be optimized due to code complexity and computational costs which prohibit the use of intrusive and many query algorithms. Rather, lower-fidelity models are constructed to enable intrusive algorithms for large-scale optimization. As a result of the discrepancy between high and low-fidelity models, optimal solutions determined using low-fidelity models are frequently far from true optimality. In this article we introduce a novel approach that uses post-optimality sensitivities with respect to model discrepancy to update the optimization solution. Limited high-fidelity data is used to calibrate the model discrepancy in a Bayesian framework which in turn is propagated through post-optimality sensitivities of the low-fidelity optimization problem. Our formulation exploits structure in the post-optimality sensitivity operator to achieve computational scalability. Numerical results demonstrate how an optimal solution computed using a low-fidelity model may be significantly improved with limited evaluations of a high-fidelity model.

math.NA↗

arXiv2022

A new perspective on parameter study of optimization problems

We provide a new perspective on the study of parameterized optimization problems. Our approach combines methods for post-optimal sensitivity analysis and ordinary differential equations to quantify the uncertainty in the minimizer due to uncertain parameters in the optimization problem. We illustrate the proposed approach with a simple analytic example and an inverse problem governed by an advection diffusion equation.

math.OC↗

arXiv2022

Sensitivity driven experimental design to facilitate control of dynamical systems

Control of nonlinear dynamical systems is a complex and multifaceted process. Essential elements of many engineering systems include high fidelity physics-based modeling, offline trajectory planning, feedback control design, and data acquisition strategies to reduce uncertainties. This article proposes an optimization centric perspective which couples these elements in a cohesive framework. We introduce a novel use of hyper-differential sensitivity analysis to understand the sensitivity of feedback controllers to parametric uncertainty in physics-based models used for trajectory planning. These sensitivities provide a foundation to define an optimal experimental design which seeks to acquire data most relevant in reducing demand on the feedback controller. Our proposed framework is illustrated on the Zermelo navigation problem and a hypersonic trajectory control problem using data from NASA's X-43 hypersonic flight tests.

math.OC↗

arXiv2022

Hyper-differential sensitivity analysis for nonlinear Bayesian inverse problems

We consider hyper-differential sensitivity analysis (HDSA) of nonlinear Bayesian inverse problems governed by PDEs with infinite-dimensional parameters. In previous works, HDSA has been used to assess the sensitivity of the solution of deterministic inverse problems to additional model uncertainties and also different types of measurement data. In the present work, we extend HDSA to the class of Bayesian inverse problems governed by PDEs. The focus is on assessing the sensitivity of certain key quantities derived from the posterior distribution. Specifically, we focus on analyzing the sensitivity of the MAP point and the Bayes risk and make full use of the information embedded in the Bayesian inverse problem. After establishing our mathematical framework for HDSA of Bayesian inverse problems, we present a detailed computational approach for computing the proposed HDSA indices. We examine the effectiveness of the proposed approach on a model inverse problem governed by a PDE for heat conduction.

math.NA↗

arXiv2021

Enabling hyper-differential sensitivity analysis for ill-posed inverse problems

Inverse problems constrained by partial differential equations (PDEs) play a critical role in model development and calibration. In many applications, there are multiple uncertain parameters in a model that must be estimated. However, high dimensionality of the parameters and computational complexity of the PDE solves make such problems challenging. A common approach is to reduce the dimension by fixing some parameters (which we will call auxiliary parameters) to a best estimate and use techniques from PDE-constrained optimization to estimate the other parameters. In this article, hyper-differential sensitivity analysis (HDSA) is used to assess the sensitivity of the solution of the PDE-constrained optimization problem to changes in the auxiliary parameters. Foundational assumptions for HDSA require satisfaction of the optimality conditions which are not always practically feasible as a result of ill-posedness in the inverse problem. We introduce novel theoretical and computational approaches to justify and enable HDSA for ill-posed inverse problems by projecting the sensitivities on likelihood informed subspaces and defining a posteriori updates. Our proposed framework is demonstrated on a nonlinear multi-physics inverse problem motivated by estimation of spatially heterogenous material properties in the presence of spatially distributed parametric modeling uncertainties.

math.NA↗

arXiv2020

Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection

Deep neural networks (DNNs) have achieved state-of-the-art performance across a variety of traditional machine learning tasks, e.g., speech recognition, image classification, and segmentation. The ability of DNNs to efficiently approximate high-dimensional functions has also motivated their use in scientific applications, e.g., to solve partial differential equations (PDE) and to generate surrogate models. In this paper, we consider the supervised training of DNNs, which arises in many of the above applications. We focus on the central problem of optimizing the weights of the given DNN such that it accurately approximates the relation between observed input and target data. Devising effective solvers for this optimization problem is notoriously challenging due to the large number of weights, non-convexity, data-sparsity, and non-trivial choice of hyperparameters. To solve the optimization problem more efficiently, we propose the use of variable projection (VarPro), a method originally designed for separable nonlinear least-squares problems. Our main contribution is the Gauss-Newton VarPro method (GNvpro) that extends the reach of the VarPro idea to non-quadratic objective functions, most notably, cross-entropy loss functions arising in classification. These extensions make GNvpro applicable to all training problems that involve a DNN whose last layer is an affine mapping, which is common in many state-of-the-art architectures. In our four numerical experiments from surrogate modeling, segmentation, and classification GNvpro solves the optimization problem more efficiently than commonly-used stochastic gradient descent (SGD) schemes. Also, GNvpro finds solutions that generalize well, and in all but one example better than well-tuned SGD methods, to unseen data points.

cs.LG↗