arXiv subjects

Haochuan Li

Publications and source records attributed to Haochuan Li.

24 records · Page 2Linked to original sources

arXiv2021

Complexity Lower Bounds for Nonconvex-Strongly-Concave Min-Max Optimization

We provide a first-order oracle complexity lower bound for finding stationary points of min-max optimization problems where the objective function is smooth, nonconvex in the minimization variable, and strongly concave in the maximization variable. We establish a lower bound of $Ω\left(\sqrtκε^{-2}\right)$ for deterministic oracles, where $ε$ defines the level of approximate stationarity and $κ$ is the condition number. Our analysis shows that the upper bound achieved in (Lin et al., 2020b) is optimal in the $ε$ and $κ$ dependence up to logarithmic factors. For stochastic oracles, we provide a lower bound of $Ω\left(\sqrtκε^{-2} + κ^{1/3}ε^{-4}\right)$. It suggests that there is a significant gap between the upper bound $\mathcal{O}(κ^3 ε^{-4})$ in (Lin et al., 2020a) and our lower bound in the condition number dependence.

math.OC↗

arXiv2021

The Stellar Distribution Function and Local Vertical Potential from Gaia DR2

We develop a novel method to simultaneously determine the vertical potential, force and stellar $z-v_z$ phase space distribution function (DF) in our local patch of the Galaxy. We assume that the Solar Neighborhood can be treated as a one-dimensional system in dynamical equilibrium and directly fit the number density in the $z-v_z$ plane to what we call the Rational Linear DF (RLDF) model. This model can be regarded as a continuous sum of isothermal DFs though it has only one more parameter than the isothermal model. We apply our method to a sample of giant stars from Gaia Data Release 2 and show that the RLDF provides an excellent fit to the data. The well-known phase space spiral emerges in the residual map of the $z-v_z$ plane. We use the best-fit potential to plot the residuals in terms of the frequency and angle of vertical oscillations and show that the spiral maps into a straight line. From its slope, we estimate that the phase spirals were generated by a perturbation $\sim540$ Myr years ago. We also determine the differential surface density as a function of vertical velocity dispersion, a.k.a. the vertical temperature distribution. The result is qualitatively similar to what was previously found for SDSS/SEGUE G dwarfs. Finally, we address parameter degeneracies and the validity of the 1D approximation. Particularly, the mid-plane density derived from a cold subsample, where the 1D approximation is more secure, is closer to literature values than that derived from the sample as a whole.

astro-ph.GA↗

arXiv2019

Convergence of Adversarial Training in Overparametrized Neural Networks

Neural networks are vulnerable to adversarial examples, i.e. inputs that are imperceptibly perturbed from natural data and yet incorrectly classified by the network. Adversarial training, a heuristic form of robust optimization that alternates between minimization and maximization steps, has proven to be among the most successful methods to train networks to be robust against a pre-defined family of perturbations. This paper provides a partial answer to the success of adversarial training, by showing that it converges to a network where the surrogate loss with respect to the the attack algorithm is within $ε$ of the optimal robust loss. Then we show that the optimal robust loss is also close to zero, hence adversarial training finds a robust classifier. The analysis technique leverages recent work on the analysis of neural networks via Neural Tangent Kernel (NTK), combined with motivation from online-learning when the maximization is solved by a heuristic, and the expressiveness of the NTK kernel in the $\ell_\infty$-norm. In addition, we also prove that robust interpolation requires more model capacity, supporting the evidence that adversarial training requires wider networks.

cs.LG↗

arXiv2019

Gradient Descent Finds Global Minima of Deep Neural Networks

Gradient descent finds a global minimum in training deep neural networks despite the objective function being non-convex. The current paper proves gradient descent achieves zero training loss in polynomial time for a deep over-parameterized neural network with residual connections (ResNet). Our analysis relies on the particular structure of the Gram matrix induced by the neural network architecture. This structure allows us to show the Gram matrix is stable throughout the training process and this stability implies the global optimality of the gradient descent algorithm. We further extend our analysis to deep residual convolutional neural networks and obtain a similar convergence result.

cs.LG↗

arXiv2018

Two Types of Long-duration Quasi-static Evolution of Solar Filaments

In this Letter, we investigate the long-duration quasi-static evolution of 12 pre-eruptive filaments (4 active region and 8 quiescent filaments), mainly focusing on the evolution of the filament height in three dimension (3D) and the decay index of the background magnetic field. The filament height in 3D is derived through two-perspective observations of \textit{Solar Dynamics Observatory} and \textit{Solar TErrestrial RElations Observatory}. The coronal magnetic field is reconstructed using the potential field source surface model. A new finding is that the filaments we studied show two types of long-duration evolution: one type is comprised of a long-duration static phase and a short slow rise phase with a duration of less than 12 hours and a speed of 0.1--0.7 km s$^{-1}$, while the other one only presents a slow rise phase but with an extremely long duration of more than 60 hours and a smaller speed of 0.01--0.2 km s$^{-1}$. At the moment approaching the eruption, the decay index of the background magnetic field at the filament height is similar for both active region and quiescent filaments. The average value and upper limit are $\sim$0.9 and $\sim$1.4, close to the critical index of torus instability. Moreover, the filament height and background magnetic field strength are also found to be linearly and exponentially related with the filament length, respectively.

astro-ph.SR↗

arXiv2018

Randomness in Deconvolutional Networks for Visual Representation

Toward a deeper understanding on the inner work of deep neural networks, we investigate CNN (convolutional neural network) using DCN (deconvolutional network) and randomization technique, and gain new insights for the intrinsic property of this network architecture. For the random representations of an untrained CNN, we train the corresponding DCN to reconstruct the input images. Compared with the image inversion on pre-trained CNN, our training converges faster and the yielding network exhibits higher quality for image reconstruction. It indicates there is rich information encoded in the random features; the pre-trained CNN may discard information irrelevant for classification and encode relevant features in a way favorable for classification but harder for reconstruction. We further explore the property of the overall random CNN-DCN architecture. Surprisingly, images can be inverted with satisfactory quality. Extensive empirical evidence as well as theoretical analysis are provided.

cs.CV↗