arXiv subjects

Siu Wun Cheung

Publications and source records attributed to Siu Wun Cheung.

At least 19 recordsLinked to original sources

arXiv2026

Hyper-reduction methods for accelerating nonlinear finite element simulations: open source implementation and reproducible benchmarks

Hyper-reduction methods have gained increasing attention for their potential to accelerate reduced order models for nonlinear systems, yet their comparative accuracy and computational efficiency are not well understood. Motivated by this gap, we evaluate a range of hyper-reduction techniques for nonlinear finite element models across benchmark problems of varying complexity, assessing the inevitable tradeoff between accuracy and speedup. More specifically, we consider interpolation methods based on the gappy proper orthogonal decomposition as well as the empirical quadrature procedure (EQP), and apply them to the hyper-reduction of problems in nonlinear diffusion, nonlinear elasticity and Lagrangian hydrodynamics. Our numerical results are generated using the open source libROM, Laghos and MFEM numerical libraries. Our findings reveal that the comparative performance between hyper-reduction methods depends on both the problem and the choice of time integration method. The EQP method generally achieves lower relative errors than interpolation methods and is more efficient in terms of quadrature point usage, resulting in a lower wall time for the nonlinear diffusion and elasticity problems. However, its online computational cost is observed to be relatively high for Lagrangian hydrodynamics problems. Conversely, interpolation methods exhibit greater variability, especially with respect to the use of different time integration methods in the Lagrangian hydrodynamics problems. The presented results underscore the need for problem specific method selection to balance accuracy and efficiency, while also offering useful guidance for future comparisons and refinements of hyper-reduction techniques.

cs.MS↗

arXiv2026

WGFINNs: Weak formulation-based GENERIC formalism informed neural networks

Data-driven discovery of governing equations from noisy observations remains a fundamental challenge in scientific machine learning. While GENERIC formalism informed neural networks (GFINNs) provide a principled framework that enforces the laws of thermodynamics by construction, their reliance on strong-form loss formulations makes them highly sensitive to measurement noise. To address this limitation, we propose weak formulation-based GENERIC formalism informed neural networks (WGFINNs), which integrate the weak formulation of dynamical systems with the structure-preserving architecture of GFINNs. WGFINNs significantly enhance robustness to noisy data while retaining exact satisfaction of GENERIC degeneracy and symmetry conditions. We further incorporate a state-wise weighted loss and a residual-based attention mechanism to mitigate scale imbalance across state variables. Theoretical analysis contrasts quantitative differences between the strong-form and the weak-form estimators. Mainly, the strong-form estimator diverges as the time step decreases in the presence of noise, while the weak-form estimator can be accurate even with noisy data if test functions satisfy certain conditions. Numerical experiments demonstrate that WGFINNs consistently outperform GFINNs at varying noise levels, achieving more accurate predictions and reliable recovery of physical quantities.

cs.LG↗

arXiv2026

Machine-precision energy conservative reduced models for Lagrangian hydrodynamics by quadrature methods

We present an energy conservative, quadrature based model reduction framework for the compressible Euler equations of Lagrangian hydrodynamics. Building on a finite element discretization of the governing equations, we develop reduced models using data based reduced basis functions and the empirical quadrature procedure (EQP). We introduce a strongly energy conservative variant of EQP that enforces exact energy conservation in the reduction process. Numerical experiments for four benchmark problems -- Sedov blast, Gresho vortex, triple point and Taylor-Green vortex -- demonstrate that the numerical implementation of our proposed method conserves total energy to near machine precision, while maintaining accuracy comparable to the basic EQP formulation.

math.NA↗

arXiv2026

A Reduced Order Model approach for First-Principles Molecular Dynamics Computations

To leverage the redundancy between the electronic structure computed at each step of first-principles molecular dynamics, we present a data-driven modeling framework for Kohn-Sham Density Functional Theory that bypasses the explicit optimization of electronic wavefunctions. We sample a priori representative atomic configurations and construct a low-dimensional basis that efficiently approximates the electronic structure subspace. Subsequently, we employ this reduced basis in a direct solver for the electronic single particle density matrix, thereby enabling the efficient determination of ground state without iterative wavefunction optimization. We demonstrate the efficacy of our approach in a Born-Oppenheimer molecular dynamics of a water molecule, showing that the resulting simulations accurately reproduce key structural properties, such as bond lengths and bond angle, obtained from full first-principles molecular dynamics. This work highlights the potential of data-driven approaches to develop efficient electronic structure solvers for first-principles simulations.

math.NA↗

arXiv2026

Free-RBF-KAN: Kolmogorov-Arnold Networks with Adaptive Radial Basis Functions for Efficient Function Learning

Kolmogorov-Arnold Networks (KANs) offer a promising framework for approximating complex nonlinear functions, yet the original B-spline formulation suffers from significant computational overhead due to De Boor algorithm. While recent RBF-based variants improve efficiency, they often sacrifice the approximation accuracy inherent in the original spline-based design. To bridge this gap, we propose Free-RBF-KAN, an architecture that integrates adaptive learning grids and trainable smoothness parameters to enable expressive, high-resolution function approximation. Our method utilizes learnable RBF shapes that dynamically align with activation patterns, and we provide the first formal universal approximation proof for the RBF-KAN family. Empirical evaluations across multiscale regression, physics-informed PDEs, and operator learning demonstrate that Free-RBF-KAN can achieve accuracy comparable to its B-spline counterparts while delivering significantly faster training and inference. These results establish Free-RBF-KAN as an efficient and adaptive alternative for high-dimensional structured modeling tasks.

cs.LG↗

arXiv2025

Deep Global Model Reduction Learning in Porous Media Flow Simulation

In this paper, we combine deep learning concepts and some proper orthogonal decomposition (POD) model reduction methods for predicting flow in heterogeneous porous media. Nonlinear flow dynamics is studied, where the dynamics is regarded as a multi-layer network. The solution at the current time step is regarded as a multi-layer network of the solution at the initial time and input parameters. As for input, we consider various sources, which include source terms (well rates), permeability fields, and initial conditions. We consider the flow dynamics, where the solution is known at some locations and the data is integrated to the flow dynamics by modifying the reduced-order model. This approach allows modifying the reduced-order formulation of the problem. Because of the small problem size, limited observed data can be handled. We consider enriching the observed data using the computational data in deep learning networks. The basis functions of the global reduced order model are selected such that the degrees of freedom represent the solution at observation points. This way, we can avoid learning basis functions, which can also be done using neural networks. We present numerical results, where we consider channelized permeability fields, where the network is constructed for various channel configurations. Our numerical results show that one can achieve a good approximation using forward feed maps based on multi-layer networks.

math.NA↗

arXiv2025

Model Order Reduction for Quantum Molecular Dynamics

Molecular dynamics simulations are indispensable for exploring the behavior of atoms and molecules. Grounded in quantum mechanical principles, quantum molecular dynamics provides high predictive power but its computational cost is dominated by iterative high-fidelity electronic structure calculations. We propose a novel model order reduction approach as an alternative to high-fidelity electronic structure calculation. By learning a low-dimensional representation of the electronic solution manifold within the Kohn-Sham density functional theory framework, our model order reduction approach determines the ground state electronic density by projecting the problem onto a low-dimensional subspace, thereby avoiding the computationally expensive iterative optimization of electronic wavefunctions in the full space. We demonstrate the capability of our method on a water molecule, showing excellent agreement with high-fidelity simulations for both molecular geometry and dynamic properties, highlighting the generalizability through carefully designed parametrization and systematic sampling.

physics.chem-ph↗

arXiv2025

Theory and numerics of subspace approximation of eigenvalue problems

Large-scale eigenvalue problems arise in various fields of science and engineering and demand computationally efficient solutions. In this study, we investigate the subspace approximation for parametric linear eigenvalue problems, aiming to mitigate the computational burden associated with high-fidelity systems. We provide general error estimates under non-simple eigenvalue conditions, establishing some theoretical foundations for understanding the convergence behavior of subspace approximations. Numerical examples, including problems with one-dimensional to three-dimensional spatial domain and one-dimensional to two-dimensional parameter domain, are presented to demonstrate the efficacy of reduced basis method in handling parametric variations in boundary conditions and coefficient fields to achieve significant computational savings while maintaining high accuracy, making them promising tools for practical applications in large-scale eigenvalue computations.

math.NA↗

arXiv2025

Defining Foundation Models for Computational Science: A Call for Clarity and Rigor

The widespread success of foundation models in natural language processing and computer vision has inspired researchers to extend the concept to scientific machine learning and computational science. However, this position paper argues that as the term "foundation model" is an evolving concept, its application in computational science is increasingly used without a universally accepted definition, potentially creating confusion and diluting its precise scientific meaning. In this paper, we address this gap by proposing a formal definition of foundation models in computational science, grounded in the core values of generality, reusability, and scalability. We articulate a set of essential and desirable characteristics that such models must exhibit, drawing parallels with traditional foundational methods, like the finite element and finite volume methods. Furthermore, we introduce the Data-Driven Finite Element Method (DD-FEM), a framework that fuses the modular structure of classical FEM with the representational power of data-driven learning. We demonstrate how DD-FEM addresses many of the key challenges in realizing foundation models for computational science, including scalability, adaptability, and physics consistency. By bridging traditional numerical methods with modern AI paradigms, this work provides a rigorous foundation for evaluating and developing novel approaches toward future foundation models in computational science.

cs.LG↗

arXiv2024

tLaSDI: Thermodynamics-informed latent space dynamics identification

We propose a latent space dynamics identification method, namely tLaSDI, that embeds the first and second principles of thermodynamics. The latent variables are learned through an autoencoder as a nonlinear dimension reduction model. The latent dynamics are constructed by a neural network-based model that precisely preserves certain structures for the thermodynamic laws through the GENERIC formalism. An abstract error estimate is established, which provides a new loss formulation involving the Jacobian computation of autoencoder. The autoencoder and the latent dynamics are simultaneously trained to minimize the new loss. Computational examples demonstrate the effectiveness of tLaSDI, which exhibits robust generalization ability, even in extrapolation. In addition, an intriguing correlation is empirically observed between a quantity from tLaSDI in the latent space and the behaviors of the full-state solution.

cs.LG↗

arXiv2024

A Comprehensive Review of Latent Space Dynamics Identification Algorithms for Intrusive and Non-Intrusive Reduced-Order-Modeling

Numerical solvers of partial differential equations (PDEs) have been widely employed for simulating physical systems. However, the computational cost remains a major bottleneck in various scientific and engineering applications, which has motivated the development of reduced-order models (ROMs). Recently, machine-learning-based ROMs have gained significant popularity and are promising for addressing some limitations of traditional ROM methods, especially for advection dominated systems. In this chapter, we focus on a particular framework known as Latent Space Dynamics Identification (LaSDI), which transforms the high-fidelity data, governed by a PDE, to simpler and low-dimensional latent-space data, governed by ordinary differential equations (ODEs). These ODEs can be learned and subsequently interpolated to make ROM predictions. Each building block of LaSDI can be easily modulated depending on the application, which makes the LaSDI framework highly flexible. In particular, we present strategies to enforce the laws of thermodynamics into LaSDI models (tLaSDI), enhance robustness in the presence of noise through the weak form (WLaSDI), select high-fidelity training data efficiently through active learning (gLaSDI, GPLaSDI), and quantify the ROM prediction uncertainty through Gaussian processes (GPLaSDI). We demonstrate the performance of different LaSDI approaches on Burgers equation, a non-linear heat conduction problem, and a plasma physics problem, showing that LaSDI algorithms can achieve relative errors of less than a few percent and up to thousands of times speed-ups.

cs.CE↗

arXiv2023

AdjointBackMapV2: Precise Reconstruction of Arbitrary CNN Unit's Activation via Adjoint Operators

Adjoint operators have been found to be effective in the exploration of CNN's inner workings [1]. However, the previous no-bias assumption restricted its generalization. We overcome the restriction via embedding input images into an extended normed space that includes bias in all CNN layers as part of the extended space and propose an adjoint-operator-based algorithm that maps high-level weights back to the extended input space for reconstructing an effective hypersurface. Such hypersurface can be computed for an arbitrary unit in the CNN, and we prove that this reconstructed hypersurface, when multiplied by the original input (through an inner product), will precisely replicate the output value of each unit. We show experimental results based on the CIFAR-10 and CIFAR-100 data sets where the proposed approach achieves near 0 activation value reconstruction error.

cs.LG↗

arXiv2023

Data-scarce surrogate modeling of shock-induced pore collapse process

Understanding the mechanisms of shock-induced pore collapse is of great interest in various disciplines in sciences and engineering, including materials science, biological sciences, and geophysics. However, numerical modeling of the complex pore collapse processes can be costly. To this end, a strong need exists to develop surrogate models for generating economic predictions of pore collapse processes. In this work, we study the use of a data-driven reduced order model, namely dynamic mode decomposition, and a deep generative model, namely conditional generative adversarial networks, to resemble the numerical simulations of the pore collapse process at representative training shock pressures. Since the simulations are expensive, the training data are scarce, which makes training an accurate surrogate model challenging. To overcome the difficulties posed by the complex physics phenomena, we make several crucial treatments to the plain original form of the methods to increase the capability of approximating and predicting the dynamics. In particular, physics information is used as indicators or conditional inputs to guide the prediction. In realizing these methods, the training of each dynamic mode composition model takes only around 30 seconds on CPU. In contrast, training a generative adversarial network model takes 8 hours on GPU. Moreover, using dynamic mode decomposition, the final-time relative error is around 0.3% in the reproductive cases. We also demonstrate the predictive power of the methods at unseen testing shock pressures, where the error ranges from 1.3% to 5% in the interpolatory cases and 8% to 9% in extrapolatory cases.

physics.comp-ph↗

arXiv2022

Constraint Energy Minimizing Generalized Multiscale Finite Element Method for multi-continuum Richards equations

In fluid flow simulation, the multi-continuum model is a useful strategy. When the heterogeneity and contrast of coefficients are high, the system becomes multiscale, and some kinds of reduced-order methods are demanded. Combining these techniques with nonlinearity, we will consider in this paper a dual-continuum model which is generalized as a multi-continuum model for a coupled system of nonlinear Richards equations as unsaturated flows, in complex heterogeneous fractured porous media; and we will solve it by a novel multiscale approach utilizing the constraint energy minimizing generalized multiscale finite element method (CEM-GMsFEM). In particular, such a nonlinear system will be discretized in time and then linearized by Picard iteration (whose global convergence is proved theoretically). Subsequently, we tackle the resulting linearized equations by the CEM-GMsFEM and obtain proper offline multiscale basis functions to span the multiscale space (which contains the pressure solution). More specifically, we first introduce two new sources of samples, and the GMsFEM is used over each coarse block to build local auxiliary multiscale basis functions via solving local spectral problems, that are crucial for detecting high-contrast channels. Second, per oversampled coarse region, local multiscale basis functions are created through the CEM as constrainedly minimizing an energy functional. Various numerical tests for our approach reveal that the error converges with the coarse-grid size alone and that only a few oversampling layers, as well as basis functions, are needed.

math.NA↗

arXiv2022

S-OPT: A Points Selection Algorithm for Hyper-Reduction in Reduced Order Models

While projection-based reduced order models can reduce the dimension of full order solutions, the resulting reduced models may still contain terms that scale with the full order dimension. Hyper-reduction techniques are sampling-based methods that further reduce this computational complexity by approximating such terms with a much smaller dimension. The goal of this work is to introduce a points selection algorithm developed by Shin and Xiu [SIAM J. Sci. Comput., 38 (2016), pp. A385--A411], as a hyper-reduction method. The selection algorithm is originally proposed as a stochastic collocation method for uncertainty quantification. Since the algorithm aims at maximizing a quantity S that measures both the column orthogonality and the determinant, we refer to the algorithm as S-OPT. Numerical examples are provided to demonstrate the performance of S-OPT and to compare its performance with an over-sampled Discrete Empirical Interpolation (DEIM) algorithm. We found that using the S-OPT algorithm is shown to predict the full order solutions with higher accuracy for a given number of indices.

math.NA↗

arXiv2022

Local Lagrangian reduced-order modeling for Rayleigh-Taylor instability by solution manifold decomposition

Rayleigh-Taylor instability is a classical hydrodynamic instability of great interest in various disciplines of science and engineering, including astrophyics, atmospheric sciences and climate, geophysics, and fusion energy. Analytical methods cannot be applied to explain the long-time behavior of Rayleigh-Taylor instability, and therefore numerical simulation of the full problem is required. However, in order to capture the growth of amplitude of perturbations accurately, both the spatial and temporal discretization need to be extremely fine for traditional numerical methods, and the long-time simulation may become prohibitively expensive. In this paper, we propose efficient reduced order model techniques to accelerate the simulation of Rayleigh-Taylor instability in compressible gas dynamics. We introduce a general framework for decomposing the solution manifold to construct the temporal domain partition and temporally-local reduced order model construction with varying Atwood number. We propose two practical approaches in this framework, namely decomposition by using physical time and penetration distance respectively. Numerical results are presented to examine the performance of the proposed approaches.

math.NA↗

arXiv2021

Reduced order models for Lagrangian hydrodynamics

As a mathematical model of high-speed flow and shock wave propagation in a complex multimaterial setting, Lagrangian hydrodynamics is characterized by moving meshes, advection-dominated solutions, and moving shock fronts with sharp gradients. These challenges hinder the existing projection-based model reduction schemes from being practical. We develop several variations of projection-based reduced order model techniques for Lagrangian hydrodynamics by introducing three different reduced bases for position, velocity, and energy fields. A time-windowing approach is also developed to address the challenge imposed by the advection-dominated solutions. Lagrangian hydrodynamics is formulated as a nonlinear problem, which requires a proper hyper-reduction technique. Therefore, we apply the over-sampling DEIM and SNS approaches to reduce the complexity due to the nonlinear terms. Finally, we also present both a posteriori and a priori error bounds associated with our reduced order model. We compare the performance of the spatial and time-windowing reduced order modeling approaches in terms of accuracy and speed-up with respect to the corresponding full order model for several numerical examples, namely Sedov blast, Gresho vortices, Taylor-Green vortices, and triple-point problems.

math.NA↗

arXiv2021

Multiscale simulations for multi-continuum Richards equations

In this paper, we study a multiscale method for simulating a dual-continuum unsaturated flow problem within complex heterogeneous fractured porous media. Mathematically, each of the dual continua is modeled by a multiscale Richards equation (for pressure head), and these equations are coupled to one another by transfer terms. On its own, Richards equation is already a nonlinear partial differential equation, and it is exceedingly difficult to solve numerically due to the extra nonlinear dependencies involving the soil water. To deal with multiple scales, our strategy is that starting from a microscopic scale, we upscale the coupled system of dual-continuum Richards equations via homogenization by the two-scale asymptotic expansion, to obtain a homogenized system, at an intermediate scale (level). Based on a hierarchical approach, the homogenization's effective coefficients are computed through solving the arising cell problems. To tackle the nonlinearity, after time discretization, we use Picard iteration procedure for linearization of the homogenized Richards equations. At each Picard iteration, some degree of multiscale still remains from the intermediate level, so we utilize the generalized multiscale finite element method (GMsFEM) combining with a multi-continuum approach, to upscale the homogenized system to a macroscopic (coarse-grid) level. This scheme involves building uncoupled and coupled multiscale basis functions, which are used not only to construct coarse-grid solution approximation with high accuracy but also (with the coupled multiscale basis) to capture the interactions among continua. These prospects and convergence are demonstrated by several numerical results for the proposed method.

math.NA↗