arXiv subjects

Ziang Long

Publications and source records attributed to Ziang Long.

5 recordsLinked to original sources

arXiv2022

Existence of an effective burning velocity in cellular flow for curvature G-equation via game analysis

G-equation is a popular level set model in turbulent combustion, and becomes an advective mean curvature type evolution equation when curvature of a moving flame in a fluid flow is considered: $$ G_t + \left(1-d\, \mathrm{Div}{\frac{DG}{|DG|}}\right)_+|DG|+V(x)\cdot DG=0. $$ Here $d>0$ is the Markstein number and the positive part $()_+$ is imposed to avoid a non-physical negative laminar flame speed. For simplicity of presentation, we focus mainly on the case when $V:\mathbb{R}^2\to \mathbb{R}^2$ is the two dimensional cellular flow with Hamiltonian $H = \sin x_1 \, \sin x_2$ and amplitude $A$. Our main result is that for any unit vector $p\in \mathbb{R}^2$, there exists a positive number $\overline H(p)$ such that if $G(x,0)=p\cdot x$, then $$ \left|G(x,t)-p\cdot x+\overline H(p)t\right|\leq C \quad \text{in $\mathbb{R}^2\times [0,\infty)$} $$ for a constant $C$ depending only on the Markstein number $d$ and the cellular flow amplitude $A$. The number $\overline H(p)$ corresponds to the effective burning velocity in the physics literature. The non-coercivity encountered here is one of the major difficulties for homogenization of the mean curvature-type equations. To overcome it, we introduce a new approach that combines PDE methods with a dynamical analysis of the Kohn-Serfaty deterministic game characterization of the curvature G-equation utilizing the streamline structure of cellular flows. Extension to general two-dimensional incompressible flows is also discussed. In three dimensional incompressible flows, the existence of $\overline H(p)$ might fail when the flow intensity exceeds a bifurcation value even for simple shear flows [32].

math.AP↗

arXiv2022

Enhancing Zero-Shot Many to Many Voice Conversion with Self-Attention VAE

Variational auto-encoder (VAE) is an effective neural network architecture to disentangle a speech utterance into speaker identity and linguistic content latent embeddings, then generate an utterance for a target speaker from that of a source speaker. This is possible by concatenating the identity embedding of the target speaker and the content embedding of the source speaker uttering a desired sentence. In this work, we propose to improve VAE models with self-attention and structural regularization (RGSM). Specifically, we found a suitable location of VAE's decoder to add a self-attention layer for incorporating non-local information in generating a converted utterance and hiding the source speaker's identity. We applied relaxed group-wise splitting method (RGSM) to regularize network weights and remarkably enhance generalization performance. In experiments of zero-shot many-to-many voice conversion task on VCTK data set, with the self-attention layer and relaxed group-wise splitting method, our model achieves a gain of speaker classification accuracy on unseen speakers by 28.3\% while slightly improved conversion voice quality in terms of MOSNet scores. Our encouraging findings point to future research on integrating more variety of attention structures in VAE framework while controlling model size and overfitting for advancing zero-shot many-to-many voice conversions.

cs.SD↗

arXiv2020

Recurrence of Optimum for Training Weight and Activation Quantized Networks

Deep neural networks (DNNs) are quantized for efficient inference on resource-constrained platforms. However, training deep learning models with low-precision weights and activations involves a demanding optimization task, which calls for minimizing a stage-wise loss function subject to a discrete set-constraint. While numerous training methods have been proposed, existing studies for full quantization of DNNs are mostly empirical. From a theoretical point of view, we study practical techniques for overcoming the combinatorial nature of network quantization. Specifically, we investigate a simple yet powerful projected gradient-like algorithm for quantizing two-linear-layer networks, which proceeds by repeatedly moving one step at float weights in the negation of a heuristic \emph{fake} gradient of the loss function (so-called coarse gradient) evaluated at quantized weights. For the first time, we prove that under mild conditions, the sequence of quantized weights recurrently visits the global optimum of the discrete minimization problem for training fully quantized network. We also show numerical evidence of the recurrence phenomenon of weight evolution in training quantized deep networks.

cs.LG↗

arXiv2020

Learning Quantized Neural Nets by Coarse Gradient Method for Non-linear Classification

Quantized or low-bit neural networks are attractive due to their inference efficiency. However, training deep neural networks with quantized activations involves minimizing a discontinuous and piecewise constant loss function. Such a loss function has zero gradients almost everywhere (a.e.), which makes the conventional gradient-based algorithms inapplicable. To this end, we study a novel class of \emph{biased} first-order oracle, termed coarse gradient, for overcoming the vanished gradient issue. A coarse gradient is generated by replacing the a.e. zero derivatives of quantized (i.e., stair-case) ReLU activation composited in the chain rule with some heuristic proxy derivative called straight-through estimator (STE). Although having been widely used in training quantized networks empirically, fundamental questions like when and why the ad-hoc STE trick works, still lacks theoretical understanding. In this paper, we propose a class of STEs with certain monotonicity, and consider their applications to the training of a two-linear-layer network with quantized activation functions for non-linear multi-category classification. We establish performance guarantees for the proposed STEs by showing that the corresponding coarse gradient methods converge to the global minimum, which leads to a perfect classification. Lastly, we present experimental results on synthetic data as well as MNIST dataset to verify our theoretical findings and demonstrate the effectiveness of our proposed STEs.

cs.LG↗

arXiv2020

Global Convergence and Geometric Characterization of Slow to Fast Weight Evolution in Neural Network Training for Classifying Linearly Non-Separable Data

In this paper, we study the dynamics of gradient descent in learning neural networks for classification problems. Unlike in existing works, we consider the linearly non-separable case where the training data of different classes lie in orthogonal subspaces. We show that when the network has sufficient (but not exceedingly large) number of neurons, (1) the corresponding minimization problem has a desirable landscape where all critical points are global minima with perfect classification; (2) gradient descent is guaranteed to converge to the global minima. Moreover, we discovered a geometric condition on the network weights so that when it is satisfied, the weight evolution transitions from a slow phase of weight direction spreading to a fast phase of weight convergence. The geometric condition says that the convex hull of the weights projected on the unit sphere contains the origin.

cs.LG↗