arXiv subjects

Lizuo Liu

Publications and source records attributed to Lizuo Liu.

14 recordsLinked to original sources

arXiv2026

Time-Invariant Neural Operators with Applications in Solving Time-Dependent PDEs

The deep operator network (DeepONet) is one of the basic architectures for learning nonlinear operators with neural networks. However, for operators that describe the dynamic response of physical systems, DeepONet does not naturally respect fundamental time properties, including time causality and time invariance. We propose time-invariant neural operator (TINO) to overcome these limitations, by creating the connection between time-delay neural network (TDNN) and DeepONet. We then extend the proposal to solving time-dependent PDEs with initial values, which can be treated as truncated time-invariant systems, with spatial proper orthogonal decomposition (POD) implementation in the output function space. Various numerical tests comparing TINO, neural operators with time causality, and those without time properties justify the enhanced precision of the proposed neural operator frameworks.

math.NA↗

arXiv2026

Parametric Hyperbolic Conservation Laws: A Unified Framework for Conservation, Entropy Stability, and Hyperbolicity

We propose a parametric hyperbolic conservation law (SymCLaw) for learning hyperbolic systems directly from data while ensuring conservation, entropy stability, and hyperbolicity by design. Unlike existing approaches that typically enforce only conservation or rely on prior knowledge of the governing equations, our method parameterizes the flux functions in a form that guarantees real eigenvalues and complete eigenvectors of the flux Jacobian, thereby preserving hyperbolicity. At the same time, we embed entropy-stable design principles by jointly learning a convex entropy function and its associated flux potential, ensuring entropy dissipation and the selection of physically admissible weak solutions. A corresponding entropy-stable numerical flux scheme provides compatibility with standard discretizations, allowing seamless integration into classical solvers. Numerical experiments on benchmark problems, including Burgers, shallow water, Euler, and KPP equations, demonstrate that SymCLaw generalizes to unseen initial conditions, maintains stability under noisy training data, and achieves accurate long-time predictions, highlighting its potential as a principled foundation for data-driven modeling of hyperbolic conservation laws.

math.NA↗

arXiv2025

Non-intrusive structural-preserving sequential data assimilation

Data assimilation (DA) methods combine model predictions with observational data to improve state estimation in dynamical systems, inspiring their increasingly prominent role in geophysical and climate applications. Classical DA methods assume that the governing equations modeling the dynamics are known, which is unlikely for most real world applications. Machine learning (ML) provides a flexible alternative by learning surrogate models directly from data, but standard ML methods struggle in noisy and data-scarce environments, where meaningful extrapolation requires incorporating physical constraints. Recent advances in structure-preserving ML architectures, such as the development of the entropy-stable conservative flux form network (ESCFN), highlight the critical role of physical structure in improving learning stability and accuracy for unknown systems of conservation laws. Structural information has also been shown to improve DA performance. Gradient-based measures of spatial variability, in particular, can help refine ensemble updates in discontinuous systems. Motivated by both of these recent innovations, this investigation proposes a new non-intrusive, structure-preserving sequential data assimilation (NSSDA) framework that leverages structure at both the forecast and analysis stages. We use the ESCFN to construct a surrogate model to preserve physical laws during forecasting, and a structurally informed ensemble transform Kalman filter (SETKF) to embed local statistical structure into the assimilation step. Our method operates in a highly constrained environment, using only a single noisy trajectory for both training and assimilation. Numerical experiments where the unknown dynamics correspond respectively to the shallow water and Euler equations demonstrate significantly improved predictive accuracy.

math.NA↗

arXiv2025

Neural Entropy-stable conservative flux form neural networks for learning hyperbolic conservation laws

We propose a neural entropy-stable conservative flux form neural network (NESCFN) for learning hyperbolic conservation laws and their associated entropy functions directly from solution trajectories, without requiring any predefined numerical discretization. While recent neural network architectures have successfully integrated classical numerical principles into learned models, most rely on prior knowledge of the governing equations or assume a fixed discretization. Our approach removes this dependency by embedding entropy-stable design principles into the learning process itself, enabling the discovery of physically consistent dynamics in a fully data-driven setting. By jointly learning both the numerical flux function and a corresponding entropy, the proposed method ensures conservation and entropy dissipation, critical for long-term stability and fidelity in the system of hyperbolic conservation laws. Numerical results demonstrate that the method achieves stability and conservation over extended time horizons and accurately captures shock propagation speeds, even without oracle access to future-time solution profiles in the training data.

math.NA↗

arXiv2025

Post-Training Augmentation Invariance

This work develops a framework for post-training augmentation invariance, in which our goal is to add invariance properties to a pretrained network without altering its behavior on the original, non-augmented input distribution. We define this notion precisely and additionally introduce augmented encoders, which are probabilistic encoders that formalize augmentation-based encoding processes and that serve as our fundamental object of study. We introduce two losses for augmented encoders, namely, Markov-Wasserstein minimization and Wasserstein correlation maximization, and we demonstrate empirically that both losses can be used to train lightweight, one-hidden-layer MLP adapter networks $E_{\theta}$ that, when appended to the latent space of a pretrained network $F$, do indeed lead to (approximate) post-training augmentation invariance. For example, on STL10 with $F=\text{DINO}$ features, the composite network $C\circ E_{\theta}\circ F$, where $C$ is a linear classifier and where $E_{\theta}$ is one of our proposed adapter networks, achieves 94% classification accuracy on arbitrarily rotated images, whereas a network of the form $C\circ F$ without the adapter $E_{\theta}$ drops to 71% accuracy. Similarly, we can boost noise-invariant classification results from 58% up to 86%. Significantly, we obtain these results with no fine-tuning (the weights of $F$ remain frozen throughout), and our methods introduce little corruption to the original features, since $E_{\theta}$ acts nearly isometrically on the non-augmented latent distribution. In contrast, we show that adapter networks trained with alternative candidate losses, specifically SimCLR and HSIC maximization, produce uncompetitive classification results and fundamentally corrupt the original latent space. Code available at https://github.com/keenan-eikenberry/augmentation_invariance

cs.LG↗

arXiv2025

Multi-scale DeepOnet (Mscale-DeepOnet) for Mitigating Spectral Bias in Learning High Frequency Operators of Oscillatory Functions

In this paper, a multi-scale DeepOnet (Mscale-DeepOnet) is proposed to reduce the spectral bias of the DeepOnet in learning high-frequency mapping between highly oscillatory functions, with an application to the nonlinear mapping between the coefficient of the Helmholtz equation and its solution. The Mscale-DeepOnet introduces the multiscale neural network in the branch and trunk networks of the original DeepOnet, the resulting Mscale-DeepOnet is shown to be able to capture various high-frequency components of the mapping itself and its image. Numerical results demonstrate the substantial improvement of the Mscale-DeepOnet for the problem of wave scattering in the high-frequency regime over the normal DeepOnet with a similar number of network parameters.

cs.LG↗

arXiv2024

Entropy stable conservative flux form neural networks

We propose an entropy-stable conservative flux form neural network (CFN) that integrates classical numerical conservation laws into a data-driven framework using the entropy-stable, second-order, and non-oscillatory Kurganov-Tadmor (KT) scheme. The proposed entropy-stable CFN uses slope limiting as a denoising mechanism, ensuring accurate predictions in both noisy and sparse observation environments, as well as in both smooth and discontinuous regions. Numerical experiments demonstrate that the entropy-stable CFN achieves both stability and conservation while maintaining accuracy over extended time domains. Furthermore, it successfully predicts shock propagation speeds in long-term simulations, {\it without} oracle knowledge of later-time profiles in the training data.

math.NA↗

arXiv2023

On Spectral Bias Reduction of Multi-scale Neural Networks for Regression Problems

In this paper, we derive diffusion equation models in the spectral domain for the evolution of training errors of two-layer multi-scale deep neural networks (MscaleDNN) \cite{caixu2019,liu2020multi}, designed to reduce the spectral bias of fully connected deep neural networks in approximating oscillatory functions. The diffusion models are obtained from the spectral form of the error equation of the MscaleDNN, derived with a neural tangent kernel approach and gradient descent training and a sine activation function, assuming a vanishing learning rate and infinite network width and domain size. The involved diffusion coefficients are shown to have larger supports if more scales are used in the MscaleDNN, and thus, the proposed diffusion equation models in the frequency domain explain the MscaleDNN's spectral bias reduction capability. Numerical results of the diffusion models for a two-layer MscaleDNN training match with the error evolution of actual gradient descent training with a reasonably large network width, thus validating the effectiveness of the diffusion models. Meanwhile, the numerical results for MscaleDNN show error decay over a wide frequency range and confirm the advantage of using the MscaleDNN in approximating functions with a wide range of frequencies.

math.NA↗

arXiv2022

A Causality-DeepONet for Causal Responses of Linear Dynamical Systems

In this paper, we propose a DeepONet structure with causality to represent the causal linear operators between Banach spaces of time-dependent signals. The theorem of universal approximations to nonlinear operators proposed in \cite{tianpingchen1995} is extended to operators with causalities, and the proposed Causality-DeepONet implements the physical causality in its framework. The proposed Causality-DeepONet considers causality (the state of the system at the current time is not affected by that of the future, but only by its current state and past history) and uses a convolution-type weight in its design. To demonstrate its effectiveness in handling the causal response of a physical system, the Causality-DeepONet is applied to learn the operator representing the response of a building due to earthquake ground accelerations. Extensive numerical tests and comparisons with some existing variants of DeepONet are carried out, and the Causality-DeepONet clearly shows its unique capability to learn the retarded dynamic responses of the seismic response operator with good accuracy.

math.NA↗

arXiv2022

DeepPropNet -- A Recursive Deep Propagator Neural Network for Learning Evolution PDE Operators

In this paper, we propose a deep neural network approximation to the evolution operator for time dependent PDE systems over long time period by recursively using one single neural network propagator, in the form of POD-DeepONet with built-in causality feature, for a small time interval. The trained DeepPropNet of moderate size is shown to give accurate prediction of wave solutions over the whole time interval.

math.NA↗

arXiv2021

Multiscale DeepONet for Nonlinear Operators in Oscillatory Function Spaces for Building Seismic Wave Responses

In this paper, we propose a multiscale DeepONet to represent nonlinear operator between Banach spaces of highly oscillatory continuous functions. The multiscale deep neural network (DNN) utilizes a multiple scaling technique to convert high frequency function to lower frequency functions before using a DNN to learn a specific range of frequency of the function. The multi-scale concept is integrated into the DeepONet which is based on a universal approximation theory of nonlinear operators. The resulting multi-scale DeepONet is shown to be effective to represent building seismic response operator which maps oscillatory seismic excitation to the oscillatory building responses.

math.NA↗

arXiv2021

Linearized Learning Methods with Multiscale Deep Neural Networks for Stationary Navier-Stokes Equations with Oscillatory Solutions

In this paper, we present linearized learning methods to accelerate the convergence of training for stationary nonlinear Navier-Stokes equations. To solve the stationary nonlinear Navier-Stokes (NS) equation, we integrate the procedure of linearization of the nonlinear convection term in the NS equation into the training process of multi-scale deep neural network approximation of the NS solution. Four forms of linearizations are considered. After a benchmark problem, we solve the highly oscillating stationary flows utilizing the proposed linearized learning with multi-scale neural network for complex domains. The results show that multiscale deep neural network combining with the linearized schemes can be trained fast and accurately.

math.NA↗

arXiv2019

A Phase Shift Deep Neural Network for High Frequency Approximation and Wave Problems

In this paper, we propose a phase shift deep neural network (PhaseDNN), which provides a uniform wideband convergence in approximating high frequency functions and solutions of wave equations. The PhaseDNN makes use of the fact that common DNNs often achieve convergence in the low frequency range first, and a series of moderately-sized DNNs are constructed and trained for selected high frequency ranges. With the help of phase shifts in the frequency domain, each of the DNNs will be trained to approximate the function's higher frequency content over a specific range at the the speed of convergence as in the low frequency range. As a result, the proposed PhaseDNN is able to convert high frequency learning to low frequency one, allowing a uniform learning to wideband functions. The PhaseDNN will then be applied to find the solution of high frequency wave equations in inhomogeneous media through both differential and integral equation formulations with least square residual loss functions. Numerical results have demonstrated the capability of the PhaseDNN in learning high frequency functions and oscillatory solutions of interior and exterior Helmholtz equations.

cs.LG↗

arXiv2019

PhaseDNN - A Parallel Phase Shift Deep Neural Network for Adaptive Wideband Learning

In this paper, we propose a phase shift deep neural network (PhaseDNN) which provides a wideband convergence in approximating a high dimensional function during its training of the network. The PhaseDNN utilizes the fact that many DNN achieves convergence in the low frequency range first, thus, a series of moderately-sized of DNNs are constructed and trained in parallel for ranges of higher frequencies. With the help of phase shifts in the frequency domain, implemented through a simple phase factor multiplication on the training data, each DNN in the series will be trained to approximate the target function's higher frequency content over a specific range. Due to the phase shift, each DNN achieves the speed of convergence as in the low frequency range. As a result, the proposed PhaseDNN system is able to convert wideband frequency learning to low frequency learning, thus allowing a uniform learning to wideband high dimensional functions with frequency adaptive training. Numerical results have demonstrated the capability of PhaseDNN in learning information of a target function from low to high frequency uniformly.

eess.SP↗