arXiv subjects

Xu Qian

Publications and source records attributed to Xu Qian.

23 records · Page 2Linked to original sources

arXiv2020

Channel-wise Hessian Aware trace-Weighted Quantization of Neural Networks

Second-order information has proven to be very effective in determining the redundancy of neural network weights and activations. Recent paper proposes to use Hessian traces of weights and activations for mixed-precision quantization and achieves state-of-the-art results. However, prior works only focus on selecting bits for each layer while the redundancy of different channels within a layer also differ a lot. This is mainly because the complexity of determining bits for each channel is too high for original methods. Here, we introduce Channel-wise Hessian Aware trace-Weighted Quantization (CW-HAWQ). CW-HAWQ uses Hessian trace to determine the relative sensitivity order of different channels of activations and weights. What's more, CW-HAWQ proposes to use deep Reinforcement learning (DRL) Deep Deterministic Policy Gradient (DDPG)-based agent to find the optimal ratios of different quantization bits and assign bits to channels according to the Hessian trace order. The number of states in CW-HAWQ is much smaller compared with traditional AutoML based mix-precision methods since we only need to search ratios for the quantization bits. Compare CW-HAWQ with state-of-the-art shows that we can achieve better results for multiple networks.

cs.CV↗

arXiv2020

Regularized finite difference methods for the logarithmic Klein-Gordon equation

We propose and analyze two regularized finite difference methods for the logarithmic Klein-Gordon equation (LogKGE). Due to the blowup phenomena caused by the logarithmic nonlinearity of the LogKGE, it is difficult to construct numerical schemes and establish their error bounds. In order to avoid singularity, we present a regularized logarithmic Klein-Gordon equation (RLogKGE) with a small regularized parameter $0<\varepsilon\ll1$. Besides, two finite difference methods are adopted to solve the regularized logarithmic Klein-Gordon equation (RLogKGE) and rigorous error bounds are estimated in terms of the mesh size $h$, time step $τ$, and the small regularized parameter $\varepsilon$. Finally, numerical experiments are carried out to verify our error estimates of the two numerical methods and the convergence results from the LogKGE to the RLogKGE with the linear convergence order $O(\varepsilon)$.

math.AP↗

arXiv2020

Two regularized energy-preserving finite difference methods for the logarithmic Klein-Gordon equation

We present and analyze two regularized finite difference methods which preserve energy of the logarithmic Klein-Gordon equation (LogKGE). In order to avoid singularity caused by the logarithmic nonlinearity of the LogKGE, we propose a regularized logarithmic Klein-Gordon equation (RLogKGE) with a small regulation parameter $0<\varepsilon\ll1$ to approximate the LogKGE with the convergence order $O(\varepsilon)$. By adopting the energy method, the inverse inequality, and the cut-off technique of the nonlinearity to bound the numerical solution, the error bound $O(h^{2}+\frac{τ^{2}}{\varepsilon^{2}})$ of the two schemes with the mesh size $h$, the time step $τ$ and the parameter $\varepsilon$. Numerical results are reported to support our conclusions.

math.AP↗

arXiv2019

Mass and energy conservative high order diagonally implicit Runge--Kutta schemes for nonlinear Schrödinger equation in one and two dimensions

We present and analyze a series of conservative diagonally implicit Runge--Kutta schemes for the nonlinear Schrödiner equation. With the application of the newly developed invariant energy quadratization approach, these schemes possess not only high accuracy , high order convergence (up to fifth order) and efficiency due to diagonally implicity but also mass and energy conservative properties. Both theoretical analysis and numerical experiments of one- and two-dimensional dynamics are carried out to verify the invariant conservative properties, convergence orders and longtime simulation stability.

math.NA↗

arXiv2019

SAGA with Arbitrary Sampling

We study the problem of minimizing the average of a very large number of smooth functions, which is of key importance in training supervised learning models. One of the most celebrated methods in this context is the SAGA algorithm. Despite years of research on the topic, a general-purpose version of SAGA---one that would include arbitrary importance sampling and minibatching schemes---does not exist. We remedy this situation and propose a general and flexible variant of SAGA following the {\em arbitrary sampling} paradigm. We perform an iteration complexity analysis of the method, largely possible due to the construction of new stochastic Lyapunov functions. We establish linear convergence rates in the smooth and strongly convex regime, and under a quadratic functional growth condition (i.e., in a regime not assuming strong convexity). Our rates match those of the primal-dual method Quartz for which an arbitrary sampling analysis is available, which makes a significant step towards closing the gap in our understanding of complexity of primal and dual methods for finite sum problems.

cs.LG↗