arXiv subjects

Wenrui Hao

Publications and source records attributed to Wenrui Hao.

At least 55 records · Page 3Linked to original sources

arXiv2023

Deep Learning via Neural Energy Descent

This paper proposes the Nerual Energy Descent (NED) via neural network evolution equations for a wide class of deep learning problems. We show that deep learning can be reformulated as the evolution of network parameters in an evolution equation and the steady state solution of the partial differential equation (PDE) provides a solution to deep learning. This equation corresponds to a gradient descent flow of a variational problem and hence the proposed time-dependent PDE solves an energy minimization problem to obtain a global minimizer of deep learning. This gives a novel interpretation and solution to deep learning optimization. The computational complexity of the proposed energy descent method can be enhanced by randomly sampling the spatial domain of the PDE leading to an efficient NED. Numerical examples are provided to demonstrate the numerical advantage of NED over stochastic gradient descent (SGD).

math.NA↗

arXiv2022

Bifurcation analysis of a free boundary model of vascular tumor growth with a necrotic core and chemotaxis

A considerable number of research works has been devoted to the study of tumor models. Several biophysical factors, such as cell proliferation, apoptosis, chemotaxis, angiogenesis and necrosis, have been discovered to have an impact on the complicated biological system of tumors. An indicator of the aggressiveness of tumor development is the instability of the shape of the tumor boundary. Complex patterns of tumor morphology have been explored by Lu, Min-Jhe et al. [Nonlinear simulation of vascular tumor growth with chemotaxis and the control of necrosis, Journal of Computational Physics 459 (2022): 111153]. In this paper, we continue to carry out a bifurcation analysis on such a vascular tumor model with a controlled necrotic core and chemotaxis. This bifurcation analysis, to the parameter of cell proliferation, is built on the explicit formulas of radially symmetric steady-state solutions. By perturbing the tumor free boundary and establishing rigorous estimates of the free boundary system, %applying the Hanzawa transformation, we prove the existence of the bifurcation branches with Crandall-Rabinowitz theorem. The parameter of chemotaxis is found to influence the monotonicity of the bifurcation point as the mode $l$ increases both theoretically and numerically.

math.AP↗

arXiv2022

Bifurcation Analysis Reveals Solution Structures of Phase Field Models

Phase field method is playing an increasingly important role in understanding and predicting morphological evolution in materials and biological systems. Here, we develop a new analytical approach based on bifurcation analysis to explore the mathematical solution structure of phase field models. Revealing such solution structures not only is of great mathematical interest but also may provide guidance to experimentally or computationally uncover new morphological evolution phenomena in materials undergoing electronic and structural phase transitions. To elucidate the idea, we apply this analytical approach to three representative phase field equations: Allen-Cahn equation, Cahn-Hilliard equation, and Allen-Cahn-Ohta-Kawasaki system. The solution structures of these three phase field equations are also verified numerically by the homotopy continuation method.

math.NA↗

arXiv2022

Stability Preserving Data-driven Models With Latent Dynamics

In this paper, we introduce a data-driven modeling approach for dynamics problems with latent variables. The state-space of the proposed model includes artificial latent variables, in addition to observed variables that can be fitted to a given data set. We present a model framework where the stability of the coupled dynamics can be easily enforced. The model is implemented by recurrent cells and trained using backpropagation through time. Numerical examples using benchmark tests from order reduction problems demonstrate the stability of the model and the efficiency of the recurrent cell implementation. As applications, two fluid-structure interaction problems are considered to illustrate the accuracy and predictive capability of the model.

math.OC↗

arXiv2022

Nonlinear simulation of vascular tumor growth with chemotaxis and the control of necrosis

In this paper, we develop a sharp interface tumor growth model in two dimensions to study the effect of both the intratumoral structure using a controlled necrotic core and the extratumoral nutrient supply from vasculature on tumor morphology. We first show that our model extends the benchmark results in the literature using linear stability analysis. Then we solve this generalized model numerically using a spectrally accurate boundary integral method in an evolving annular domain, not only with a Robin boundary condition on the outer boundary for the nutrient field which models tumor vasculature, but also with a static boundary condition on the inner boundary for pressure field which models the control of tumor necrosis. The discretized linear systems for both pressure and nutrient fields are shown to be well-conditioned through tracing GMRES iteration numbers. Our nonlinear simulations reveal the stabilizing effects of angiogenesis and the destabilizing ones of chemotaxis and necrosis in the development of tumor morphological instabilities if the necrotic core is fixed in a circular shape. When the necrotic core is controlled in a non-circular shape, the stabilizing effects of proliferation and the destabilizing ones of apoptosis are observed. Finally, the values of the nutrient concentration with its fluxes and the pressure level with its normal derivatives, which are solved accurately at the boundaries, help us to characterize the corresponding tumor morphology and the level of the biophysical quantities on interfaces required in keeping various shapes of the necrotic region of the tumor. Interestingly, we notice that when the necrotic region is fixed in a 3-fold non-circular shape, even if the initial shape of the tumor is circular, the tumor will evolve into a shape corresponding to the 3-fold symmetry of the shape of the fixed necrotic region.

math.NA↗

arXiv2021

Learn bifurcations of nonlinear parametric systems via equation-driven neural networks

Nonlinear parametric systems have been widely used in modeling nonlinear dynamics in science and engineering. Bifurcation analysis of these nonlinear systems on the parameter space are usually used to study the solution structure such as the number of solutions and the stability. In this paper, we develop a new machine learning approach to compute the bifurcations via so-called equation-driven neural networks (EDNNs). The EDNNs consist of a two-step optimization: the first step is to approximate the solution function of the parameter by training empirical solution data; the second step is to compute bifurcations by using the approximated neural network obtained in the first step. Both theoretical convergence analysis and numerical implementation on several examples have been performed to demonstrate the feasibility of the proposed method.

math.NA↗

arXiv2021

Two neural-network-based methods for solving obstacle problems

Two neural-network-based numerical schemes are proposed to solve the classical obstacle problems. The schemes are based on the universal approximation property of neural networks, and the cost functions are taken as the energy minimization of the obstacle problems. We rigorously prove the convergence of the two schemes and derive the convergence rates with the number of neurons $N$. In the simulations, we use two example problems (1-D & 2-D) to verify the convergence rate of the methods and the quality of the results.

math.NA↗

arXiv2021

Projection based model reduction for the immersed boundary method

Fluid-structure interactions are central to many bio-molecular processes, and they impose a great challenge for computational and modeling methods. In this paper, we consider the immersed boundary method (IBM) for biofluid systems, and to alleviate the computational cost, we apply reduced-order techniques to eliminate the degrees of freedom associated with a large number of fluid variables. We show how reduced models can be derived using Petrov-Galerkin projection and subspaces that maintain the incompressibility condition. More importantly, the reduced-order model is shown to preserve the Lyapunov stability. We also address the practical issue of computing coefficient matrices in the reduced-order model using an interpolation technique. The efficiency and robustness of the proposed formulation are examined with test examples from various applications.

math.NA↗

arXiv2021

A Weight Initialization Based on the Linear Product Structure for Neural Networks

Weight initialization plays an important role in training neural networks and also affects tremendous deep learning applications. Various weight initialization strategies have already been developed for different activation functions with different neural networks. These initialization algorithms are based on minimizing the variance of the parameters between layers and might still fail when neural networks are deep, e.g., dying ReLU. To address this challenge, we study neural networks from a nonlinear computation point of view and propose a novel weight initialization strategy that is based on the linear product structure (LPS) of neural networks. The proposed strategy is derived from the polynomial approximation of activation functions by using theories of numerical algebraic geometry to guarantee to find all the local minima. We also provide a theoretical analysis that the LPS initialization has a lower probability of dying ReLU comparing to other existing initialization strategies. Finally, we test the LPS initialization algorithm on both fully connected neural networks and convolutional neural networks to show its feasibility, efficiency, and robustness on public datasets.

cs.LG↗

arXiv2021

Power Series Expansion Neural Network

In this paper, we develop a new neural network family based on power series expansion, which is proved to achieve a better approximation accuracy in comparison with existing neural networks. This new set of neural networks embeds the power series expansion (PSE) into the neural network structure. Then it can improve the representation ability while preserving comparable computational cost by increasing the degree of PSE instead of increasing the depth or width. Both theoretical approximation and numerical results show the advantages of this new neural network.

math.NA↗

arXiv2021

A stochastic homotopy tracking algorithm for parametric systems of nonlinear equations

The homotopy continuation method has been widely used in solving parametric systems of nonlinear equations. But it can be very expensive and inefficient due to singularities during the tracking even though both start and end points are non-singular. The current tracking algorithms focus on the adaptivity of the stepsize by estimating the distance to the singularities but cannot avoid these singularities during the tracking. We present a stochastic homotopy tracking algorithm that perturbs the original parametric system randomly each step to avoid the singularities. We then prove that the stochastic solution path introduced by this new method is still closed to the original solution path theoretically. Moreover, several homotopy examples have been tested to show the efficiency of the stochastic homotopy tracking method.

math.NA↗

arXiv2020

Convergence analysis of neural networks for solving a free boundary system

Free boundary problems deal with systems of partial differential equations, where the domain boundaries are apriori unknown. Due to this special characteristic, it is challenging to solve free boundary problems either theoretically or numerically. In this paper, we develop a novel approach for solving a modified Hele-Shaw problem based on neural network discretization. The existence of the numerical solution with this discretization is established theoretically. We also numerically verify this approach by computing the symmetry-breaking solutions which are guided by the bifurcation analysis near the radially-symmetric branch. Moreover, we further verify the capability of this approach by computing some non-radially symmetric solutions which are not characterized by any theorems.

math.NA↗

arXiv2020

Bifurcation analysis of a free boundary model of plaque formation associated with the cholesterol ratio

The low-density lipoprotein (LDL)/high-density lipoprotein (HDL)-cholesterol ratio has been shown a high correlation with the cardiovascular risk assessment. Is it possible to quantify the correlation mathematically? In this paper, we develop a bifurcation analysis for a mathematical model of the plaque formation with a free boundary in the early stage of atherosclerosis. This bifurcation analysis, to the ratio of LDL/HDL, is based on the explicit formulation of radially symmetric steady-state solutions. By performing the perturbation analysis to these solutions, we establish the existence of bifurcation branches and derive a theoretical condition that a bifurcation occurs for different modes. Then we also analyze the stability of radially symmetric steady-state solutions and conduct numerical simulations to verify all the theoretical results.

math.AP↗

arXiv2020

An adaptive homotopy method for computing bifurcations of nonlinear parametric systems

In this paper, we present an adaptive step-size homotopy tracking method for computing bifurcation points of nonlinear systems. There are four components in this new method: 1) an adaptive tracking technique is developed near bifurcation points; 2) an inflation technique is backed up when the adaptive tracking fails; 3) Puiseux series interpolation is used to compute bifurcation points; and 4)the tangent cone structure of the bifurcation point is approximated numerically to compute solutions on different branches. Various numerical examples of nonlinear systems are given to illustrate the efficiency of this new approach. This new adaptive homotopy tracking method is also applied to a system of nonlinear PDEs and shows robustness and efficiency for large-scale nonlinear discretized systems.

math.NA↗

arXiv2019

A randomized Newton's method for solving differential equations based on the neural network discretization

We develop a randomized Newton's method for solving differential equations, based on a fully connected neural network discretization. In particular, the randomized Newton's method randomly chooses equations from the overdetermined nonlinear system resulting from the neural network discretization and solves the nonlinear system adaptively. We prove theoretically that the randomized Newton's method has a quadratic convergence locally. We also apply this new method to various numerical examples, from one- to high-dimensional differential equations, in order to verify its feasibility and efficiency. Moreover, the randomized Newton's method can allow the neural network to "learn" multiple solutions for nonlinear systems of differential equations, such as pattern formation problems, and provides an alternative way to study the solution structure of nonlinear differential equations overall.

math.NA↗

arXiv2019

A machine learning method correlating pulse pressure wave data with pregnancy

Pulse feeling, representing the tactile arterial palpation of the heartbeat, has been widely used in traditional Chinese medicine (TCM) to diagnose various diseases. The quantitative relationship between the pulse wave and health conditions however has not been investigated in modern medicine. In this paper, we explored the correlation between pulse pressure wave (PPW), rather than the pulse key features in TCM, and pregnancy by using deep learning technology. This computational approach shows that the accuracy of pregnancy detection by the PPW is 84% with an AUC of 91%. Our study is a proof of concept of pulse diagnosis and will also motivate further sophisticated investigations on pulse waves.

q-bio.QM↗

arXiv2019

A homotopy training algorithm for fully connected neural networks

In this paper, we present a Homotopy Training Algorithm (HTA) to solve optimization problems arising from fully connected neural networks with complicated structures. The HTA dynamically builds the neural network starting from a simplified version to the fully connected network via adding layers and nodes adaptively. Therefore, the corresponding optimization problem is easy to solve at the beginning and connects to the original model via a continuous path guided by the HTA, which provides a high probability to get a global minimum. By gradually increasing the complexity of model along the continuous path, the HTA gets a rather good solution to the original loss function. This is confirmed by various numerical results including VGG models on CIFAR-10. For example, on the VGG13 model with batch normalization, HTA reduces the error rate by 11.86% on test dataset comparing with the traditional method. Moreover, the HTA also allows us to find the optimal structure for a fully connected neural network by building the neutral network adaptively.

math.OC↗

arXiv2018

An Equation-By-Equation Method for Solving the Multidimensional Moment Constrained Maximum Entropy Problem

An equation-by-equation (EBE) method is proposed to solve a system of nonlinear equations arising from the moment constrained maximum entropy problem of multidimensional variables. The design of the EBE method combines ideas from homotopy continuation and Newton's iterative methods. Theoretically, we establish the local convergence under appropriate conditions and show that the proposed method, geometrically, finds the solution by searching along the surface corresponding to one component of the nonlinear problem. We will demonstrate the robustness of the method on various numerical examples, including: (1) A six-moment one-dimensional entropy problem with an explicit solution that contains components of order $10^0-10^3$ in magnitude; (2) Four-moment multidimensional entropy problems with explicit solutions where the resulting systems to be solved ranging from $70-310$ equations; (3) Four- to eight-moment of a two-dimensional entropy problem, which solutions correspond to the densities of the two leading EOFs of the wind stress-driven large-scale oceanic model. In this case, we find that the EBE method is more accurate compared to the classical Newton's method, the MATLAB generic solver, and the previously developed BFGS-based method, which was also tested on this problem. (4) Four-moment constrained of up to five-dimensional entropy problems which solutions correspond to multidimensional densities of the components of the solutions of the Kuramoto-Sivashinsky equation. For the higher dimensional cases of this example, the EBE method is superior because it automatically selects a subset of the prescribed moment constraints from which the maximum entropy solution can be estimated within the desired tolerance. This selection feature is particularly important since the moment constrained maximum entropy problems do not necessarily have solutions in general.

math.NA↗