arXiv subjects

Saad Qadeer

Publications and source records attributed to Saad Qadeer.

10 recordsLinked to original sources

arXiv2026

Mamba-Assisted Non-Markovian Closure for Reduced-Order Modeling

Reduced-order modeling of high-dimensional dynamical systems is often hindered by closure effects arising from unresolved variables, which can introduce non-Markovian dependence into the resolved dynamics. Motivated by the history-dependent memory term arising in the Mori--Zwanzig formalism, we recast non-Markovian closure modeling as a sequence modeling problem and propose the Mamba-Assisted Closure (MAC) framework. MAC employs a Mamba-based sequence model to predict the closure from the resolved trajectory and couples the learned closure with the reduced-order governing equations through a numerical integrator to advance the resolved variables in time. During training, the selective scan mechanism in Mamba enables efficient parallel sequence processing with linear scaling in sequence length, while autoregressive inference proceeds through recurrent state updates at essentially constant per-step cost. We evaluate MAC on four benchmark systems with complementary characteristics: the viscous Burgers' equation, the chaotic two-scale Lorenz '96 system, the 3-bus DeMarco--Zheng power-grid system, and the dispersive Korteweg--de Vries equation. Across these benchmarks, MAC consistently improves predictive accuracy and long-time rollout stability relative to the comparison models, demonstrating an effective and computationally scalable approach to non-Markovian closure modeling.

cs.LG↗

arXiv2026

Assessing Emulator Design and Training for Modal Aerosol Microphysics Parameterizations in E3SMv2

Toward the goal of using Scientific Machine Learning (SciML) emulators to improve the numerical representation of aerosol processes in global atmospheric models, we explore the emulation of aerosol microphysics processes under cloud-free conditions in the 4-mode Modal Aerosol Module (MAM4) within the Energy Exascale Earth System Model version 2 (E3SMv2). To develop an in-depth understanding of the challenges and opportunities in applying SciML to aerosol processes, we begin with a simple feedforward neural network architecture that has been used in earlier studies, but we systematically examine key emulator design choices, including architecture complexity and variable normalization, while closely monitoring training convergence behavior. Our results show that optimization convergence, scaling strategy, and network complexity strongly influence emulation accuracy. When effective scaling is applied and convergence is achieved, the relatively simple architecture, used together with a moderate network size, can reproduce key features of the microphysics-induced aerosol concentration changes with promising accuracy. These findings provide practical clues for the next stages of emulator development; they also provide general insights that are likely applicable to the emulation of other aerosol processes, as well as other atmospheric physics involving multi-scale variability.

physics.ao-ph↗

arXiv2026

Improving the accuracy of physics-informed neural networks via last-layer retraining

Physics-informed neural networks (PINNs) are a versatile tool in the burgeoning field of scientific machine learning for solving partial differential equations (PDEs). However, determining suitable training strategies for them is not obvious, with the result that they typically yield moderately accurate solutions. In this article, we propose a method for improving the accuracy of PINNs by coupling them with a post-processing step that seeks the best approximation in a function space associated with the network. We find that our method yields errors four to five orders of magnitude lower than those of the parent PINNs across architectures and dimensions. Moreover, we can reuse the basis functions for the linear space in more complex settings, such as time-dependent and nonlinear problems, allowing for transfer learning. Our approach also provides a residual-based metric that allows us to optimally choose the number of basis functions employed.

math.NA↗

arXiv2025

Stabilizing PDE--ML coupled systems

A long-standing obstacle in the use of machine-learnt surrogates with larger PDE systems is the onset of instabilities when solved numerically. Efforts towards ameliorating these have mostly concentrated on improving the accuracy of the surrogates or imbuing them with additional structure, and have garnered limited success. In this article, we study a prototype problem and draw insights that can help with more complex systems. In particular, we focus on a viscous Burgers'-ML system and, after identifying the cause of the instabilities, prescribe strategies to stabilize the coupled system. To improve the accuracy of the stabilized system, we next explore methods based on the Mori--Zwanzig formalism.

math.NA↗

arXiv2024

Efficient kernel surrogates for neural network-based regression

Despite their immense promise in performing a variety of learning tasks, a theoretical understanding of the limitations of Deep Neural Networks (DNNs) has so far eluded practitioners. This is partly due to the inability to determine the closed forms of the learned functions, making it harder to study their generalization properties on unseen datasets. Recent work has shown that randomly initialized DNNs in the infinite width limit converge to kernel machines relying on a Neural Tangent Kernel (NTK) with known closed form. These results suggest, and experimental evidence corroborates, that empirical kernel machines can also act as surrogates for finite width DNNs. The high computational cost of assembling the full NTK, however, makes this approach infeasible in practice, motivating the need for low-cost approximations. In the current work, we study the performance of the Conjugate Kernel (CK), an efficient approximation to the NTK that has been observed to yield fairly similar results. For the regression problem of smooth functions and logistic regression classification, we show that the CK performance is only marginally worse than that of the NTK and, in certain cases, is shown to be superior. In particular, we establish bounds for the relative test losses, verify them with numerical tests, and identify the regularity of the kernel as the key determinant of performance. In addition to providing a theoretical grounding for using CKs instead of NTKs, our framework suggests a recipe for improving DNN accuracy inexpensively. We present a demonstration of this on the foundation model GPT-2 by comparing its performance on a classification task using a conventional approach and our prescription. We also show how our approach can be used to improve physics-informed operator network training for regression tasks as well as convolutional neural network training for vision classification tasks.

cs.LG↗

arXiv2022

Vibrational Levels of a Generalized Morse Potential

A Generalized Morse Potential (GMP) is an extension of the Morse Potential (MP) with an additional exponential term and an additional parameter that compensate for MP's erroneous behavior in the long range part of the interaction potential. Because of the additional term and parameter, the vibrational levels of the GMP cannot be solved analytically, unlike the case for the MP. We present several numerical approaches for solving the vibrational problem of the GMP based on Galerkin methods, namely the Laguerre Polynomial Method (LPM), the Symmetrized Laguerre Polynomial Method (SLPM) and the Polynomial Expansion method (PEM) and apply them to the vibrational levels of the homonuclear diatomic molecules B$_2$, O$_2$ and F$_2$, for which high level theoretical Full CI potential energy surfaces and experimentally measured vibrational levels have been reported. Overall the LPM produces vibrational states for the GMP that are converged to within spectroscopic accuracy of 0.01 cm$^{-1}$ in between 1 and 2 orders of magnitude faster and with much fewer basis functions / grid points than the Colbert-Miller Discrete Variable Representation (CM-DVR) method for the three homonuclear diatomic molecules examined in this study.

physics.chem-ph↗

arXiv2021

The Projection Extension Method: A Spectrally Accurate Technique for Complex Domains

An essential ingredient of a spectral method is the choice of suitable bases for test and trial spaces. On complex domains, these bases are harder to devise, necessitating the use of domain partitioning techniques such as the spectral element method. In this study, we introduce the Projection Extension (PE) method, an approach that yields spectrally accurate solutions to various problems on complex geometries without requiring domain decomposition. This technique builds on the insights used by extension methodologies such as the immersed boundary smooth extension and Smooth Forcing Extension (SFE) methods that are designed to improve the order of accuracy of the immersed boundary method. In particular, it couples an accurate extension procedure, that functions on arbitrary domains regardless of connectedness or regularity, with a least squares minimization of the boundary conditions. The resulting procedure is stable under iterative application and straightforward to generalize to higher dimensions. Moreover, it rapidly and robustly yields exponentially convergent solutions to a number of challenging test problems, including elliptic, parabolic, Newtonian fluid flow, and viscoelastic problems.

math.NA↗

arXiv2021

Machine-learning custom-made basis functions for partial differential equations

Spectral methods are an important part of scientific computing's arsenal for solving partial differential equations (PDEs). However, their applicability and effectiveness depend crucially on the choice of basis functions used to expand the solution of a PDE. The last decade has seen the emergence of deep learning as a strong contender in providing efficient representations of complex functions. In the current work, we present an approach for combining deep neural networks with spectral methods to solve PDEs. In particular, we use a deep learning technique known as the Deep Operator Network (DeepONet), to identify candidate functions on which to expand the solution of PDEs. We have devised an approach which uses the candidate functions provided by the DeepONet as a starting point to construct a set of functions which have the following properties: i) they constitute a basis, 2) they are orthonormal, and 3) they are hierarchical i.e., akin to Fourier series or orthogonal polynomials. We have exploited the favorable properties of our custom-made basis functions to both study their approximation capability and use them to expand the solution of linear and nonlinear time-dependent PDEs.

math.NA↗

arXiv2021

The Smooth Forcing Extension Method: A High-Order Technique for Solving Elliptic Equations on Complex Domains

High-order numerical methods for solving elliptic equations over arbitrary domains typically require specialized machinery, such as high-quality conforming grids for finite elements method, and quadrature rules for boundary integral methods. These tools make it difficult to apply these techniques to higher dimensions. In contrast, fixed Cartesian grid methods, such as the immersed boundary (IB) method, are easy to apply and generalize, but typically are low-order accurate. In this study, we introduce the Smooth Forcing Extension (SFE) method, a fixed Cartesian grid technique that builds on the insights of the IB method, and allows one to obtain arbitrary orders of accuracy. Our approach relies on a novel Fourier continuation method to compute extensions of the inhomogeneous terms to any desired regularity. This is combined with the highly accurate Non-Uniform Fast Fourier Transform for interpolation operations to yield a fast and robust method. Numerical tests confirm that the technique performs precisely as expected on one-dimensional test problems. In higher dimensions, the performance is even better, in some cases yielding sub-geometric convergence. We also demonstrate how this technique can be applied to solving parabolic problems and for computing the eigenvalues of elliptic operators on general domains, in the process illustrating its stability and amenability to generalization.

math.NA↗

arXiv2018

Computing the Dirichlet-Neumann Operator on a Cylinder

The computation of the Dirichlet-Neumann operator for the Laplace equation is the primary challenge for the numerical simulation of the ideal fluid equations. The techniques used commonly for 2D fluids, such as conformal mapping and boundary integral methods, fail to generalize suitably to 3D. In this study, we address this problem by developing a Transformed Field Expansion method for computing the Dirichlet-Neumann operator in a cylindrical geometry with a variable upper boundary. This technique reduces the problem to a sequence of Poisson equations on a flat geometry. We design a fast and accurate solver for these sub-problems, a key ingredient being the use of Zernike polynomials for the circular cross-section instead of the traditional Bessel functions. This lends spectral accuracy to the method as well as allowing significant computational speed-up. We rigorously analyze the algorithm and prove its applicability to a wide class of problems before demonstrating its effectiveness numerically.

math.NA↗