arXiv subjects

Qin Li

Publications and source records attributed to Qin Li.

At least 163 records · Page 9Linked to original sources

arXiv2021

On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime

Finding the optimal configuration of parameters in ResNet is a nonconvex minimization problem, but first-order methods nevertheless find the global optimum in the overparameterized regime. We study this phenomenon with mean-field analysis, by translating the training process of ResNet to a gradient-flow partial differential equation (PDE) and examining the convergence properties of this limiting process. The activation function is assumed to be $2$-homogeneous or partially $1$-homogeneous; the regularized ReLU satisfies the latter condition. We show that if the ResNet is sufficiently large, with depth and width depending algebraically on the accuracy and confidence levels, first-order optimization methods can find global minimizers that fit the training data.

cs.LG↗

arXiv2021

Overparameterization of deep ResNet: zero loss and mean-field analysis

Finding parameters in a deep neural network (NN) that fit training data is a nonconvex optimization problem, but a basic first-order optimization method (gradient descent) finds a global optimizer with perfect fit (zero-loss) in many practical situations. We examine this phenomenon for the case of Residual Neural Networks (ResNet) with smooth activation functions in a limiting regime in which both the number of layers (depth) and the number of weights in each layer (width) go to infinity. First, we use a mean-field-limit argument to prove that the gradient descent for parameter training becomes a gradient flow for a probability distribution that is characterized by a partial differential equation (PDE) in the large-NN limit. Next, we show that under certain assumptions, the solution to the PDE converges in the training time to a zero-loss solution. Together, these results suggest that the training of the ResNet gives a near-zero loss if the ResNet is large enough. We give estimates of the depth and width needed to reduce the loss below a given threshold, with high probability.

cs.LG↗

arXiv2021

Manifold Learning and Nonlinear Homogenization

We describe an efficient domain decomposition-based framework for nonlinear multiscale PDE problems. The framework is inspired by manifold learning techniques and exploits the tangent spaces spanned by the nearest neighbors to compress local solution manifolds. Our framework is applied to a semilinear elliptic equation with oscillatory media and a nonlinear radiative transfer equation; in both cases, significant improvements in efficacy are observed. This new method does not rely on detailed analytical understanding of the multiscale PDEs, such as their asymptotic limits, and thus is more versatile for general multiscale problems.

math.NA↗

arXiv2021

Quasi-linear analysis of dispersion relation preservation for nonlinear schemes

In numerical simulations of complex flows with discontinuities, it is necessary to use nonlinear schemes. The spectrum of the scheme used have a significant impact on the resolution and stability of the computation. Based on the approximate dispersion relation method, we combine the corresponding spectral property with the dispersion relation preservation proposed by De and Eswaran (J. Comput. Phys. 218 (2006) 398-416) and propose a quasi-linear dispersion relation preservation (QL-DRP) analysis method, through which the group velocity of the nonlinear scheme can be determined. In particular, we derive the group velocity property when a high-order Runge-Kutta scheme is used and compare the performance of different time schemes with QL-DRP. The rationality of the QL-DRP method is verified by a numerical simulation and the discrete Fourier transform method. To further evaluate the performance of a nonlinear scheme in finding the group velocity, new hyperbolic equations are designed. The validity of QL-DRP and the group velocity preservation of several schemes are investigated using two examples of the equation for one-dimensional wave propagation and the new hyperbolic equations. The results show that the QL-DRP method integrated with high-order time schemes can determine the group velocity for nonlinear schemes and evaluate their performance reasonably and efficiently.

math.NA↗

arXiv2021

Constrained Ensemble Langevin Monte Carlo

The classical Langevin Monte Carlo method looks for samples from a target distribution by descending the samples along the gradient of the target distribution. The method enjoys a fast convergence rate. However, the numerical cost is sometimes high because each iteration requires the computation of a gradient. One approach to eliminate the gradient computation is to employ the concept of ``ensemble." A large number of particles are evolved together so the neighboring particles provide gradient information to each other. In this article, we discuss two algorithms that integrate the ensemble feature into LMC and the associated properties. In particular, we find that if one directly surrogates the gradient using the ensemble approximation, the algorithm, termed Ensemble Langevin Monte Carlo, is unstable due to a high variance term. If the gradients are replaced by the ensemble approximations only in a constrained manner, to protect from the unstable points, the algorithm, termed Constrained Ensemble Langevin Monte Carlo, resembles the classical LMC up to an ensemble error but removes most of the gradient computation.

stat.ML↗

arXiv2021

Improved cooperation by balancing exploration and exploitation in intertemporal social dilemma tasks

When an individual's behavior has rational characteristics, this may lead to irrational collective actions for the group. A wide range of organisms from animals to humans often evolve the social attribute of cooperation to meet this challenge. Therefore, cooperation among individuals is of great significance for allowing social organisms to adapt to changes in the natural environment. Based on multi-agent reinforcement learning, we propose a new learning strategy for achieving coordination by incorporating a learning rate that can balance exploration and exploitation. We demonstrate that agents that use the simple strategy improve a relatively collective return in a decision task called the intertemporal social dilemma, where the conflict between the individual and the group is particularly sharp. We also explore the effects of the diversity of learning rates on the population of reinforcement learning agents and show that agents trained in heterogeneous populations develop particularly coordinated policies relative to those trained in homogeneous populations.

cs.MA↗

arXiv2021

Langevin Monte Carlo: random coordinate descent and variance reduction

Langevin Monte Carlo (LMC) is a popular Bayesian sampling method. For the log-concave distribution function, the method converges exponentially fast, up to a controllable discretization error. However, the method requires the evaluation of a full gradient in each iteration, and for a problem on $\mathbb{R}^d$, this amounts to $d$ times partial derivative evaluations per iteration. The cost is high when $d\gg1$. In this paper, we investigate how to enhance computational efficiency through the application of RCD (random coordinate descent) on LMC. There are two sides of the theory: 1 By blindly applying RCD to LMC, one surrogates the full gradient by a randomly selected directional derivative per iteration. Although the cost is reduced per iteration, the total number of iteration is increased to achieve a preset error tolerance. Ultimately there is no computational gain; 2 We then incorporate variance reduction techniques, such as SAGA (stochastic average gradient) and SVRG (stochastic variance reduced gradient), into RCD-LMC. It will be proved that the cost is reduced compared with the classical LMC, and in the underdamped case, convergence is achieved with the same number of iterations, while each iteration requires merely one-directional derivative. This means we obtain the best possible computational cost in the underdamped-LMC framework.

stat.ML↗

arXiv2021

Unique Reconstruction of the Heat-Reflection Indices at Solid Interfaces

We show the unique reconstruction of the heat-reflection coefficients in a phonon transport equation. This is a mathematical model used to characterize the dynamics of heat-conducting phonons in multiple layers of media, commonly composed of metals and silicon. In experiments, the heat-reflection indices are inferred by measuring the temperature at the surface of the exterior metal after applying heat sources. In this article, we rigorously justify the unique reconstruction of these indices by using such procedures.

math.AP↗

arXiv2021

I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modeling

Reconstructing a high-precision and high-fidelity 3D human hand from a color image plays a central role in replicating a realistic virtual hand in human-computer interaction and virtual reality applications. The results of current methods are lacking in accuracy and fidelity due to various hand poses and severe occlusions. In this study, we propose an I2UV-HandNet model for accurate hand pose and shape estimation as well as 3D hand super-resolution reconstruction. Specifically, we present the first UV-based 3D hand shape representation. To recover a 3D hand mesh from an RGB image, we design an AffineNet to predict a UV position map from the input in an image-to-image translation fashion. To obtain a higher fidelity shape, we exploit an additional SRNet to transform the low-resolution UV map outputted by AffineNet into a high-resolution one. For the first time, we demonstrate the characterization capability of the UV-based hand shape representation. Our experiments show that the proposed method achieves state-of-the-art performance on several challenging benchmarks.

cs.CV↗

arXiv2021

Third-order scale-independent WENO-Z scheme achieving optimal order at critical points

As we found previously, when critical points occur within grid intervals, the accuracy relations of smoothness indicators of WENO-JS would differ from that assuming critical points occurring on grid nodes, and accordingly the global smoothness indicator in WENO-Z scheme will differ from the original one. Based on above understandings, we first discuss several issues regarding current third-order WENO-Z improvements (e.g. WENO-NP3, -F3, -NN3, -PZ3 and -P+3), i.e. the numerical results with scale dependency, the validity of analysis assuming critical points occurring on nodes, and the sensitivity regarding computational time step and initial condition in order convergence studies. By analyses and numerical validations, the defections of present improvements are demonstrated, either scale-dependency of results or failure to recover optimal order when critical points occurring at half nodes, then a generic analysis is provided which considers the first-order critical point occurring within grid intervals. Based on achieved theoretical outcomes, two scale-independent, third-order WENO-Z schemes are proposed which could truly recover the optimal order at critical points: the first one is acquired by limited expansion of grid stencil, deriving new global smoothness indicator and incorporating with the mapping function; the second one is achieved by further expanding grid stencil and employing a different global smoothness indicator. For validating, typical 1-D scalar advection problems, 1-D and 2-D problems by Euler equations are chosen and tested. The consequences verify the optimal order recovery at critical points by proposed schemes and show that: the first scheme outperforms aforementioned third-order WENO-Z improvements in terms of numerical resolution, while the second scheme indicates weak numerical robustness in spite of improved resolution and is mainly of theoretical significance

math.NA↗

arXiv2021

Tracing Halpha Fibrils through Bayesian Deep Learning

We present a new deep learning method, dubbed FibrilNet, for tracing chromospheric fibrils in Halpha images of solar observations. Our method consists of a data pre-processing component that prepares training data from a threshold-based tool, a deep learning model implemented as a Bayesian convolutional neural network for probabilistic image segmentation with uncertainty quantification to predict fibrils, and a post-processing component containing a fibril-fitting algorithm to determine fibril orientations. The FibrilNet tool is applied to high-resolution Halpha images from an active region (AR 12665) collected by the 1.6 m Goode Solar Telescope (GST) equipped with high-order adaptive optics at the Big Bear Solar Observatory (BBSO). We quantitatively assess the FibrilNet tool, comparing its image segmentation algorithm and fibril-fitting algorithm with those employed by the threshold-based tool. Our experimental results and major findings are summarized as follows. First, the image segmentation results (i.e., detected fibrils) of the two tools are quite similar, demonstrating the good learning capability of FibrilNet. Second, FibrilNet finds more accurate and smoother fibril orientation angles than the threshold-based tool. Third, FibrilNet is faster than the threshold-based tool and the uncertainty maps produced by FibrilNet not only provide a quantitative way to measure the confidence on each detected fibril, but also help identify fibril structures that are not detected by the threshold-based tool but are inferred through machine learning. Finally, we apply FibrilNet to full-disk Halpha images from other solar observatories and additional high-resolution Halpha images collected by BBSO/GST, demonstrating the tool's usability in diverse datasets.

astro-ph.SR↗

arXiv2021

Quantization of Kähler manifolds

This is a survey on our recent works which reveal new relationships among deformation quantization, geometric quantization, Berezin-Toeplitz quantization and BV quantization on Kähler manifolds.

math.DG↗

arXiv2021

Dynamic Routing for Traffic Flow through Multi-agent Systems

Routing strategies for traffics and vehicles have been historically studied. However, in the absence of considering drivers' preferences, current route planning algorithms are developed under ideal situations where all drivers are expected to behave rationally and properly. Especially, for jumbled urban road networks, drivers' actual routing strategies deteriorated to a series of empirical and selfish decisions that result in congestion. Self-evidently, if minimum mobility can be kept, traffic congestion is avoidable by traffic load dispersing. In this paper, we establish a novel dynamic routing method catering drivers' preferences and retaining maximum traffic mobility simultaneously through multi-agent systems (MAS). Modeling human-drivers' behavior through agents' dynamics, MAS can analyze the global behavior of the entire traffic flow. Therefore, regarding agents as particles in smoothed particles hydrodynamics (SPH), we can enforce the traffic flow to behave like a real flow. Thereby, with the characteristic of distributing itself uniformly in road networks, our dynamic routing method realizes traffic load balancing without violating the individual time-saving motivation. Moreover, as a discrete control mechanism, our method is robust to chaos meaning driver's disobedience can be tolerated. As controlled by SPH based density, the only intelligent transportation system (ITS) we require is the location-based service (LBS). A mathematical proof is accomplished to scrutinize the stability of the proposed control law. Also, multiple testing cases are built to verify the effectiveness of the proposed dynamic routing algorithm.

cs.RO↗

arXiv2021

Bridging and Improving Theoretical and Computational Electric Impedance Tomography via Data Completion

In computational PDE-based inverse problems, a finite amount of data is collected to infer unknown parameters in the PDE. In order to obtain accurate inferences, the collected data must be informative about the unknown parameters. How to decide which data is most informative and how to efficiently sample it, is the notoriously challenging task of optimal experimental design (OED). In this context, the best, and often infeasible, scenario is when the full input-to-output (ItO) map, i.e., an infinite amount of data, is available: This is the typical setting in many theoretical inverse problems, which is used to guarantee the unique parameter reconstruction. These two different settings have created a gap between computational and theoretical inverse problems. In this manuscript we aim to bridge this gap while circumventing the OED task. This is achieved by exploiting the structures of the ItO data from the underlying inverse problem, using the electrical impedance tomography (EIT) problem as an example. We leverage the rank-structure of the EIT model, and formulate the discretized ItO map, as an H-matrix. This suggests that one can recover the full ItO matrix, with high probability, from a subset of its entries sampled following the rank structure: The data in the diagonal blocks is informative thus fully sampled, while data in the off-diagonal blocks can be sub-sampled. This recovered ItO matrix is then utilized to represent the full ItO map, allowing us to connect with the problem in the theoretical setting where the unique reconstruction is guaranteed. This strategy achieves two goals: I) it bridges the gap between the settings for the numerical and theoretical inverse problems and II) it improves the quality of computational inverse solutions. We detail the theory for the EIT model, and provide numerical verification to both EIT and optical tomography problems

math.NA↗

arXiv2021

Dynamical low-rank integrator for the linear Boltzmann equation: error analysis in the diffusion limit

Dynamical low-rank algorithms are a class of numerical methods that compute low-rank approximations of dynamical systems. This is accomplished by projecting the dynamics onto a low-dimensional manifold and writing the solution directly in terms of the low-rank factors. The approach has been successfully applied to many types of differential equations. Recently, efficient dynamical low-rank algorithms have been applied to treat kinetic equations, including the Vlasov--Poisson and the Boltzmann equation, where it was demonstrated that the methods are able to capture the low-rank structure of the solution and significantly reduce numerical cost, while often maintaining high accuracy. However, no numerical analysis is currently available. In this paper, we investigate the error analysis for a dynamical low-rank algorithm applied to the multi-scale linear Boltzmann equation (a classical model in kinetic theory) to showcase the validity of the application of dynamical low-rank algorithms to kinetic theory. The equation, in its parabolic regime, is known to be rank one theoretically, and we will prove that the scheme can dynamically and automatically capture this low-rank structure. This work thus serves as the first mathematical error analysis for a dynamical low-rank approximation applied to a kinetic problem.

math.NA↗

arXiv2021

A simple and robust method for noise variance estimation for time-varying signals

In this brief paper, we present a simple approach to estimate the variance of measurement noise with time-varying 1-D signals. The proposed approach exploits the relationship between the noise variance and the variance of the prediction errors (or innovation sequence) of a linear estimator, the idea that was pioneered by [9] and [2]. Compared with the classic and more recent methods in the same category, the proposed method can render more robust estimates with the presence of unmodelled dynamics and outliers in the measurement.

eess.SP↗

arXiv2021

Local Well-posedness of Vlasov-Poisson-Boltzmann Equation with Generalized Diffuse Boundary Condition

The Vlasov-Poisson-Boltzmann equation is a classical equation governing the dynamics of charged particles with the electric force being self-imposed. We consider the system in a convex domain with the Cercignani-Lampis boundary condition. We construct a uniqueness local-in-time solution based on an $L^\infty$-estimate and $W^{1,p}$-estimate. In particular, we develop a new iteration scheme along the characteristic with the Cercignani-Lampis boundary for the $L^\infty$-estimate, and an intrinsic decomposition of boundary integral for $W^{1,p}$-estimate.

math.AP↗

arXiv2021

Blind quantum computation for a user who only performs single-qubit gates

Blind quantum computation (BQC) allows a user who has limited quantum capability to complete a quantum computational task with the aid of a remote quantum server, such that the user's input, output, and even the algorithm can be kept hidden from the server. Up to now, there are mainly two models of BQC. One is that the client just needs the ability to prepare single qubits initiated by Broadbent, Fitzsimons, and Kashefi, and the other is that the client only needs perform single-qubit measurements first given by Morimae. In this paper, we put forward a new model of BQC in which a user only requires implementing a few single-qubit gates. We also propose a specific BQC protocol where a user only needs to implement two kinds of single-qubit gates to show the feasibility of the presented model. This circuit model is quite flexible since various users with the ability to perform different single-qubit gates may all have the chance to achieve BQC. Furthermore, compared with the other two models, it may be more suitable for practical implementation in some experimental setups such as trapped ions and superconducting systems since the single-qubit gates are the most exact operations in such systems.

quant-ph↗