arXiv subjects

Wing Tat Leung

Publications and source records attributed to Wing Tat Leung.

At least 37 records · Page 2Linked to original sources

arXiv2023

A discretization-invariant extension and analysis of some deep operator networks

We present a generalized version of the discretization-invariant neural operator and prove that the network is a universal approximation in the operator sense. Moreover, by incorporating additional terms in the architecture, we establish a connection between this discretization-invariant neural operator network and those discussed before. The discretization-invariance property of the operator network implies that different input functions can be sampled using various sensor locations within the same training and testing phases. Additionally, since the network learns a ``basis'' for the input and output function spaces, our approach enables the evaluation of input functions on different discretizations. To evaluate the performance of the proposed discretization-invariant neural operator, we focus on challenging examples from multiscale partial differential equations. Our experimental results indicate that the method achieves lower prediction errors compared to previous networks and benefits from its discretization-invariant property.

math.NA↗

arXiv2022

BelNet: Basis enhanced learning, a mesh-free neural operator

Operator learning trains a neural network to map functions to functions. An ideal operator learning framework should be mesh-free in the sense that the training does not require a particular choice of discretization for the input functions, allows for the input and output functions to be on different domains, and is able to have different grids between samples. We propose a mesh-free neural operator for solving parametric partial differential equations. The basis enhanced learning network (BelNet) projects the input function into a latent space and reconstructs the output functions. In particular, we construct part of the network to learn the ``basis'' functions in the training process. This generalized the networks proposed in Chen and Chen's universal approximation theory for the nonlinear operators to account for differences in input and output meshes. Through several challenging high-contrast and multiscale problems, we show that our approach outperforms other operator learning methods for these tasks and allows for more freedom in the sampling and/or discretization process.

math.NA↗

arXiv2022

Hybrid explicit-implicit learning for multiscale problems with time dependent source

The splitting method is a powerful method for solving partial differential equations. Various splitting methods have been designed to separate different physics, nonlinearities, and so on. Recently, a new splitting approach has been proposed where some degrees of freedom are handled implicitly while other degrees of freedom are handled explicitly. As a result, the scheme contains two equations, one implicit and the other explicit. The stability of this approach has been studied. It was shown that the time step scales as the coarse spatial mesh size, which can provide a significant computational advantage. However, the implicit solution part can still be expensive, especially for nonlinear problems. In this paper, we introduce modified partial machine learning algorithms to replace the implicit solution part of the algorithm. These algorithms are first introduced in arXiv:2109.02147, where a homogeneous source term is considered along with the Transformer, which is a neural network that can predict future dynamics. In this paper, we consider time-dependent source terms which is a generalization of the previous work. Moreover, we use the whole history of the solution to train the network. As the implicit part of the equations is more complicated to solve, we design a neural network to predict it based on training. Furthermore, we compute the explicit part of the solution using our splitting strategy. In addition, we use Proper Orthogonal Decomposition based model reduction in machine learning. The machine learning algorithms provide computational saving without sacrificing accuracy. We present three numerical examples which show that our machine learning scheme is stable and accurate.

math.NA↗

arXiv2022

Multicontinuum homogenization and its relation to nonlocal multicontinuum theories

In this paper, we present a general derivation of multicontinuum equations and discuss cell problems. We present constraint cell problem formulations in a representative volume element and oversampling techniques that allow reducing boundary effects. We discuss different choices of constraints for cell problems. We present numerical results that show how oversampling reduces boundary effects. Finally, we discuss the relation of the proposed methods to our previously developed methods, Nonlocal Multicontinuum Approaches.

math.NA↗

arXiv2022

AMS-Net: Adaptive Multiscale Sparse Neural Network with Interpretable Basis Expansion for Multiphase Flow Problems

In this work, we propose an adaptive sparse learning algorithm that can be applied to learn the physical processes and obtain a sparse representation of the solution given a large snapshot space. Assume that there is a rich class of precomputed basis functions that can be used to approximate the quantity of interest. We then design a neural network architecture to learn the coefficients of solutions in the spaces which are spanned by these basis functions. The information of the basis functions are incorporated in the loss function, which minimizes the differences between the downscaled reduced order solutions and reference solutions at multiple time steps. The network contains multiple submodules and the solutions at different time steps can be learned simultaneously. We propose some strategies in the learning framework to identify important degrees of freedom. To find a sparse solution representation, a soft thresholding operator is applied to enforce the sparsity of the output coefficient vectors of the neural network. To avoid over-simplification and enrich the approximation space, some degrees of freedom can be added back to the system through a greedy algorithm. In both scenarios, that is, removing and adding degrees of freedom, the corresponding network connections are pruned or reactivated guided by the magnitude of the solution coefficients obtained from the network outputs. The proposed adaptive learning process is applied to some toy case examples to demonstrate that it can achieve a good basis selection and accurate approximation. More numerical tests are performed on two-phase multiscale flow problems to show the capability and interpretability of the proposed method on complicated applications.

cs.LG↗

arXiv2022

Continuous data assimilation for two-phase flow: analysis and simulations

We propose, analyze, and test a novel continuous data assimilation two-phase flow algorithm for reservoir simulation. We show that the solutions of the algorithm, constructed using coarse mesh observations, converge at an exponential rate in time to the corresponding exact reference solution of the two-phase model. More precisely, we obtain a stability estimate which illustrates an exponential decay of the residual error between the reference and approximate solution, until the error hits a threshold depending on the order of data resolution. Numerical computations are included to demonstrate the effectiveness of this approach, as well as variants with data on sub-domains. In particular, we demonstrate numerically that synchronization is achieved for data collected from a small fraction of the domain.

math.NA↗

arXiv2022

Nonlocal transport equations in multiscale media. Modeling, dememorization, and discretizations

In this paper, we consider a class of convection-diffusion equations with memory effects. These equations arise as a result of homogenization or upscaling of linear transport equations in heterogeneous media and play an important role in many applications. First, we present a dememorization technique for these equations. We show that the convection-diffusion equations with memory effects can be written as a system of standard convection-diffusion-reaction equations. This allows removing the memory term and simplifying the computations. We consider a relation between dememorized equations and micro-scale equations, which do not contain memory terms. We note that dememorized equations differ from micro-scale equations and constitute a macroscopic model. Next, we consider both implicit and partially explicit methods. The latter is introduced for problems in multiscale media with high-contrast properties. Because of high contrast, explicit methods are restrictive and require time steps that are very small (scales as the inverse of the contrast). We show that, by appropriately decomposing the space, we can treat only a few degrees of freedom implicitly and the remaining degrees of freedom explicitly. We present a stability analysis. Numerical results are presented that confirm our theoretical findings of partially explicit schemes applied to dememorized systems of equations.

math.NA↗

arXiv2022

Multirate partially explicit scheme for multiscale flow problems

For time-dependent problems with high-contrast multiscale coefficients, the time step size for explicit methods is affected by the magnitude of the coefficient parameter. With a suitable construction of multiscale space, one can achieve a stable temporal splitting scheme where the time step size is independent of the contrast. Consider the parabolic equation with heterogeneous diffusion parameter, the flow rates vary significantly in different regions due to the high-contrast features of the diffusivity. In this work, we aim to introduce a multirate partially explicit splitting scheme to achieve efficient simulation with the desired accuracy. We first design multiscale subspaces to handle flow with different speed. For the fast flow, we obtain a low-dimensional subspace with respect to the high-diffusive component and adopt an implicit time discretization scheme. The other multiscale subspace will take care of the slow flow, and the corresponding degrees of freedom are treated explicitly. Then a multirate time stepping is introduced for the two parts. The stability of the multirate methods is analyzed for the partially explicit scheme. Moreover, we derive local error estimators corresponding to the two components of the solutions and provide an upper bound of the errors. An adaptive local temporal refinement framework is then proposed to achieve higher computational efficiency. Several numerical tests are presented to demonstrate the performance of the proposed method.

math.NA↗

arXiv2021

Partially Explicit Time Discretization for Nonlinear Time Fractional Diffusion Equations

Nonlinear time fractional partial differential equations are widely used in modeling and simulations. In many applications, there are high contrast changes in media properties. For solving these problems, one often uses coarse spatial grid for spatial resolution. For temporal discretization, implicit methods are often used. For implicit methods, though the time step can be relatively large, the equations are difficult to compute due to the nonlinearity and the fact that one deals with large-scale systems. On the other hand, the discrete system in explicit methods are easier to compute but it requires small time steps. In this work, we propose the partially explicit scheme following earlier works on developing partially explicit methods for nonlinear diffusion equations. In this scheme, the diffusion term is treated partially explicitly and the reaction term is treated fully explicitly. With the appropriate construction of spaces and stability analysis, we find that the required time step in our proposed scheme scales as the coarse mesh size, which creates a great saving in computing. The main novelty of this work is the extension of our earlier works for diffusion equations to time fractional diffusion equations. For the case of fractional diffusion equations, the constraints on time steps are more severe and the proposed methods alleviate this since the time step in partially explicit method scales as the coarse mesh size. We present stability results. Numerical results are presented where we compare our proposed partially explicit methods with a fully implicit approach. We show that our proposed approach provides similar results, while treating many degrees of freedom in nonlinear terms explicitly.

math.NA↗

arXiv2021

HEI: hybrid explicit-implicit learning for multiscale problems

Splitting is a method to handle application problems by splitting physics, scales, domain, and so on. Many splitting algorithms have been designed for efficient temporal discretization. In this paper, our goal is to use temporal splitting concepts in designing machine learning algorithms and, at the same time, help splitting algorithms by incorporating data and speeding them up. Since the spitting solution usually has an explicit and implicit part, we will call our method hybrid explicit-implict (HEI) learning. We will consider a recently introduced multiscale splitting algorithms. To approximate the dynamics, only a few degrees of freedom are solved implicitly, while others explicitly. In this paper, we use this splitting concept in machine learning and propose several strategies. First, the implicit part of the solution can be learned as it is more difficult to solve, while the explicit part can be computed. This provides a speed-up and data incorporation for splitting approaches. Secondly, one can design a hybrid neural network architecture because handling explicit parts requires much fewer communications among neurons and can be done efficiently. Thirdly, one can solve the coarse grid component via PDEs or other approximation methods and construct simpler neural networks for the explicit part of the solutions. We discuss these options and implement one of them by interpreting it as a machine translation task. This interpretation successfully enables us using the Transformer since it can perform model reduction for multiple time series and learn the connection. We also find that the splitting scheme is a great platform to predict the coarse solution with insufficient information of the target model: the target problem is partially given and we need to solve it through a known problem. We conduct four numerical examples and the results show that our method is stable and accurate.

math.NA↗

arXiv2021

NH-PINN: Neural homogenization based physics-informed neural network for multiscale problems

Physics-informed neural network (PINN) is a data-driven approach to solve equations. It is successful in many applications; however, the accuracy of the PINN is not satisfactory when it is used to solve multiscale equations. Homogenization is a way of approximating a multiscale equation by a homogenized equation without multiscale property; it includes solving cell problems and the homogenized equation. The cell problems are periodic; and we propose an oversampling strategy which greatly improves the PINN accuracy on periodic problems. The homogenized equation has constant or slow dependency coefficient and can also be solved by PINN accurately. We hence proposed a 3-step method to improve the PINN accuracy for solving multiscale problems with the help of the homogenization. We apply our method to solve three equations which represent three different homogenization. The results show that the proposed method greatly improves the PINN accuracy. Besides, we also find that the PINN aided homogenization may achieve better accuracy than the numerical methods driven homogenization; PINN hence is a potential alternative to implementing the homogenization.

math.NA↗

arXiv2021

Partially Explicit Time Discretization for Time Fractional Diffusion Equation

Time fractional PDEs have been used in many applications for modeling and simulations. Many of these applications are multiscale and contain high contrast variations in the media properties. It requires very small time step size to perform detailed computations. On the other hand, in the presence of small spatial grids, very small time step size is required for explicit methods. Explicit methods have many advantages as we discuss in the paper. In this paper, we propose a partial explicit method for time fractional PDEs. The approach solves the forward problem on a coarse computational grid, which is much larger than spatial heterogeneities, and requires only a few degrees of freedom to be treated implicitly. Via the construction of appropriate spaces and careful stability analysis, we can show that the time step can be chosen not to depend on the contrast or scale as the coarse mesh size. Thus, one can use larger time step size in an explicit approach. We present stability theory for our proposed method and our numerical results confirm the stability findings and demonstrate the performance of the approach.

math.NA↗

arXiv2021

Contrast-independent partially explicit time discretizations for nonlinear multiscale problems

This work continues a line of works on developing partially explicit methods for multiscale problems. In our previous works, we have considered linear multiscale problems, where the spatial heterogeneities are at subgrid level and are not resolved. In these works, we have introduced contrast-independent partially explicit time discretizations for linear equations. The contrast-independent partially explicit time discretization divides the spatial space into two components: contrast dependent (fast) and contrast independent (slow) spaces defined via multiscale space decomposition. Following this decomposition, temporal splitting is proposed that treats fast components implicitly and slow components explicitly. The space decomposition and temporal splitting are chosen such that it guarantees a stability and formulate a condition for the time stepping. This condition is formulated as a condition on slow spaces. In this paper, we extend this approach to nonlinear problems. We propose a splitting approach and derive a condition that guarantees stability. This condition requires some type of contrast-independent spaces for slow components of the solution. We present numerical results and show that the proposed methods provide results similar to implicit methods with the time step that is independent of the contrast.

math.NA↗

arXiv2021

Space-time Non-local multi-continua upscaling for parabolic equations with moving channelized media

In this paper, we consider a parabolic problem with time-dependent heterogeneous coefficients. Many applied problems have coupled space and time heterogeneities. Their homogenization or upscaling requires cell problems that are formulated in space-time representative volumes for problems with scale separation. In problems without scale separation, local problems include multiple macroscopic variables and oversampled local problems, where these macroscopic parameters are computed. These approaches, called Non-local multi-continua, are proposed for problems with complex spatial heterogeneities in a number of previous papers. In this paper, we extend this approach for space-time heterogeneities, by identifying macroscopic parameters in space-time regions. Our proposed method space-time Non-local multi-continua (space-time NLMC) is an efficient numerical solver to deal with time-dependent heterogeneous coefficients. It provides a flexible and systematic way to construct multiscale basis functions to approximate the solution. These multiscale basis functions are constructed by solving a local energy minimization problems in the oversampled space-time regions such that these multiscale basis functions decay exponentially outside the oversampled domain. Unlike the classical time-stepping methods combined with full-discretization technique, our space-time NLMC efficiently constructs the multiscale basis functions in a space-time domain and can provide a computational savings compared to space-only approaches as we discuss in the paper. We present two numerical experiments, which show that the proposed approach can provide a good accuracy.

math.NA↗

arXiv2021

Contrast-independent partially explicit time discretizations for multiscale wave problems

In this work, we design and investigate contrast-independent partially explicit time discretizations for wave equations in heterogeneous high-contrast media. We consider multiscale problems, where the spatial heterogeneities are at subgrid level and are not resolved. In our previous work, we have introduced contrast-independent partially explicit time discretizations and applied to parabolic equations. The main idea of contrast-independent partially explicit time discretization is to split the spatial space into two components: contrast dependent (fast) and contrast independent (slow) spaces defined via multiscale space decomposition. Using this decomposition, our goal is further appropriately to introduce time splitting such that the resulting scheme is stable and can guarantee contrast-independent discretization under some suitable (reasonable) conditions. In this paper, we propose contrast-independent partially explicitly scheme for wave equations. The splitting requires a careful design. We prove that the proposed splitting is unconditionally stable under some suitable conditions formulated for the second space (slow). This condition requires some type of non-contrast dependent space and is easier to satisfy in the "slow" space. We present numerical results and show that the proposed methods provide results similar to implicit methods with the time step that is independent of the contrast.

math.NA↗

arXiv2021

Contrast-independent partially explicit time discretizations for multiscale flow problems

Many multiscale problems have a high contrast, which is expressed as a very large ratio between the media properties. The contrast is known to introduce many challenges in the design of multiscale methods and domain decomposition approaches. These issues to some extend are analyzed in the design of spatial multiscale and domain decomposition approaches. However, some of these issues remain open for time dependent problems as the contrast affects the time scales, particularly, for explicit methods. For example, in parabolic equations, the time step is $dt=H^2/κ_{max}$, where $κ_{max}$ is the largest diffusivity. In this paper, we address this issue in the context of parabolic equation by designing a splitting algorithm. The proposed splitting algorithm treats dominant multiscale modes in the implicit fashion, while the rest in the explicit fashion. The unconditional stability of these algorithms require a special multiscale space design, which is the main purpose of the paper. We show that with an appropriate choice of multiscale spaces we can achieve an unconditional stability with respect to the contrast. This could provide computational savings as the time step in explicit methods is adversely affected by the contrast. We discuss some theoretical aspects of the proposed algorithms. Numerical results are presented.

math.NA↗

arXiv2020

Iterative Oversampling Technique for Constraint Energy Minimizing Generalized Multiscale Finite Element Method in the Mixed Formulation

In this paper, we develop an iterative scheme to construct multiscale basis functions within the framework of the Constraint Energy Minimizing Generalized Multiscale Finite Element Method (CEM-GMsFEM) for the mixed formulation. The iterative procedure starts with the construction of an energy minimizing snapshot space that can be used for approximating the solution of the model problem. A spectral decomposition is then performed on the snapshot space to form global multiscale space. Under this setting, each global multiscale basis function can be split into a non-decaying and a decaying parts. The non-decaying part of a global basis is localized and it is fixed during the iteration. Then, one can approximate the decaying part via a modified Richardson scheme with an appropriately defined preconditioner. Using this set of iterative-based multiscale basis functions, first-order convergence with respect to the coarse mesh size can be shown if sufficiently many times of iterations with regularization parameter being in an appropriate range are performed. Numerical results are presented to illustrate the effectiveness and efficiency of the proposed computational multiscale method.

math.NA↗

arXiv2020

Multi-agent Reinforcement Learning Accelerated MCMC on Multiscale Inversion Problem

In this work, we propose a multi-agent actor-critic reinforcement learning (RL) algorithm to accelerate the multi-level Monte Carlo Markov Chain (MCMC) sampling algorithms. The policies (actors) of the agents are used to generate the proposal in the MCMC steps; and the critic, which is centralized, is in charge of estimating the long term reward. We verify our proposed algorithm by solving an inverse problem with multiple scales. There are several difficulties in the implementation of this problem by using traditional MCMC sampling. Firstly, the computation of the posterior distribution involves evaluating the forward solver, which is very time consuming for a problem with heterogeneous. We hence propose to use the multi-level algorithm. More precisely, we use the generalized multiscale finite element method (GMsFEM) as the forward solver in evaluating a posterior distribution in the multi-level rejection procedure. Secondly, it is hard to find a function which can generate samplings which are meaningful. To solve this issue, we learn an RL policy as the proposal generator. Our experiments show that the proposed method significantly improves the sampling process

cs.LG↗