arXiv subjects

Wang Kong

Publications and source records attributed to Wang Kong.

3 recordsLinked to original sources

arXiv2024

Physics-Informed Tailored Finite Point Operator Network for Parametric Interface Problems

Learning operators for parametric partial differential equations (PDEs) using neural networks has gained significant attention in recent years. However, standard approaches like Deep Operator Networks (DeepONets) require extensive labeled data, and physics-informed DeepONets encounter training challenges. In this paper, we introduce a novel physics-informed tailored finite point operator network (PI-TFPONet) method to solve parametric interface problems without the need for labeled data. Our method fully leverages the prior physical information of the problem, eliminating the need to include the PDE residual in the loss function, thereby avoiding training challenges. The PI-TFPONet is specifically designed to address certain properties of the problem, allowing us to naturally obtain an approximate solution that closely matches the exact solution. Our method is theoretically proven to converge if the local mesh size is sufficiently small and the training loss is minimized. Notably, our approach is uniformly convergent for singularly perturbed interface problems. Extensive numerical studies show that our unsupervised PI-TFPONet is comparable to or outperforms existing state-of-the-art supervised deep operator networks in terms of accuracy and versatility.

math.NA↗

arXiv2024

Convergence Analysis of Natural Gradient Descent for Over-parameterized Physics-Informed Neural Networks

In the context of over-parameterization, there is a line of work demonstrating that randomly initialized (stochastic) gradient descent (GD) converges to a globally optimal solution at a linear convergence rate for the quadratic loss function. However, the learning rate of GD for training two-layer neural networks exhibits poor dependence on the sample size and the Gram matrix, leading to a slow training process. In this paper, we show that for training two-layer $\text{ReLU}^3$ Physics-Informed Neural Networks (PINNs), the learning rate can be improved from $\mathcal{O}(\lambda_0)$ to $\mathcal{O}(1/\|\bm{H}^{\infty}\|_2)$, implying that GD actually enjoys a faster convergence rate. Despite such improvements, the convergence rate is still tied to the least eigenvalue of the Gram matrix, leading to slow convergence. We then develop the positive definiteness of Gram matrices with general smooth activation functions and provide the convergence analysis of natural gradient descent (NGD) in training two-layer PINNs, demonstrating that the learning rate can be $\mathcal{O}(1)$ and at this rate, the convergence rate is independent of the Gram matrix. In particular, for smooth activation functions, the convergence rate of NGD is quadratic. Numerical experiments are conducted to verify our theoretical results.

cs.LG↗

arXiv2024

Convergence of Implicit Gradient Descent for Training Two-Layer Physics-Informed Neural Networks

The optimization algorithms are crucial in training physics-informed neural networks (PINNs), as unsuitable methods may lead to poor solutions. Compared to the common gradient descent (GD) algorithm, implicit gradient descent (IGD) outperforms it in handling certain multi-scale problems. In this paper, we provide convergence analysis for the IGD in training over-parameterized two-layer PINNs. We first derive the training dynamics of IGD in training two-layer PINNs. Then, over-parameterization allows us to prove that the randomly initialized IGD converges to a globally optimal solution at a linear convergence rate. Moreover, due to the distinct training dynamics of IGD compared to GD, the learning rate can be selected independently of the sample size and the least eigenvalue of the Gram matrix. Additionally, the novel approach used in our convergence analysis imposes a milder requirement on the network width. Finally, empirical results validate our theoretical findings.

cs.LG↗