Searcharxiv⌕ Search

arXiv subjects

Yulan Liu

Publications and source records attributed to Yulan Liu.

34 records · Page 2Linked to original sources
arXiv2022

Calmness of partial perturbation to composite rank constraint systems and its applications

This paper is concerned with the calmness of a partial perturbation to the composite rank constraint system, an intersection of the rank constraint set and a general closed set, which is shown to be equivalent to a local Lipschitz-type error bound and also a global Lipschitz-type error bound under a certain compactness. Based on its lifted formulation, we derive two criteria for identifying those closed sets such that the associated partial perturbation possesses the calmness, and provide a collection of examples to demonstrate that the criteria are satisfied by common nonnegative and positive semidefinite rank constraint sets. Then, we use the calmness of this perturbation to obtain several global exact penalties for rank constrained optimization problems, and a family of equivalent DC surrogates for rank regularized problems.

math.OC↗
arXiv2021

Bootstrap an end-to-end ASR system by multilingual training, transfer learning, text-to-text mapping and synthetic audio

Bootstrapping speech recognition on limited data resources has been an area of active research for long. The recent transition to all-neural models and end-to-end (E2E) training brought along particular challenges as these models are known to be data hungry, but also came with opportunities around language-agnostic representations derived from multilingual data as well as shared word-piece output representations across languages that share script and roots. We investigate here the effectiveness of different strategies to bootstrap an RNN-Transducer (RNN-T) based automatic speech recognition (ASR) system in the low resource regime, while exploiting the abundant resources available in other languages as well as the synthetic audio from a text-to-speech (TTS) engine. Our experiments demonstrate that transfer learning from a multilingual model, using a post-ASR text-to-text mapping and synthetic audio deliver additive improvements, allowing us to bootstrap a model for a new language with a fraction of the data that would otherwise be needed. The best system achieved a 46% relative word error rate (WER) reduction compared to the monolingual baseline, among which 25% relative WER improvement is attributed to the post-ASR text-to-text mappings and the TTS synthetic data.

eess.AS↗
arXiv2021

SynthASR: Unlocking Synthetic Data for Speech Recognition

End-to-end (E2E) automatic speech recognition (ASR) models have recently demonstrated superior performance over the traditional hybrid ASR models. Training an E2E ASR model requires a large amount of data which is not only expensive but may also raise dependency on production data. At the same time, synthetic speech generated by the state-of-the-art text-to-speech (TTS) engines has advanced to near-human naturalness. In this work, we propose to utilize synthetic speech for ASR training (SynthASR) in applications where data is sparse or hard to get for ASR model training. In addition, we apply continual learning with a novel multi-stage training strategy to address catastrophic forgetting, achieved by a mix of weighted multi-style training, data augmentation, encoder freezing, and parameter regularization. In our experiments conducted on in-house datasets for a new application of recognizing medication names, training ASR RNN-T models with synthetic audio via the proposed multi-stage training improved the recognition performance on new application by more than 65% relative, without degradation on existing general applications. Our observations show that SynthASR holds great promise in training the state-of-the-art large-scale E2E ASR models for new applications while reducing the costs and dependency on production data.

cs.LG↗
arXiv2021

Streaming Multi-speaker ASR with RNN-T

Recent research shows end-to-end ASR systems can recognize overlapped speech from multiple speakers. However, all published works have assumed no latency constraints during inference, which does not hold for most voice assistant interactions. This work focuses on multi-speaker speech recognition based on a recurrent neural network transducer (RNN-T) that has been shown to provide high recognition accuracy at a low latency online recognition regime. We investigate two approaches to multi-speaker model training of the RNN-T: deterministic output-target assignment and permutation invariant training. We show that guiding separation with speaker order labels in the former case enhances the high-level speaker tracking capability of RNN-T. Apart from that, with multistyle training on single- and multi-speaker utterances, the resulting models gain robustness against ambiguous numbers of speakers during inference. Our best model achieves a WER of 10.2% on simulated 2-speaker LibriSpeech data, which is competitive with the previously reported state-of-the-art nonstreaming model (10.3%), while the proposed model could be directly applied for streaming applications.

eess.AS↗
arXiv2021

Using Synthetic Audio to Improve The Recognition of Out-Of-Vocabulary Words in End-To-End ASR Systems

Today, many state-of-the-art automatic speech recognition (ASR) systems apply all-neural models that map audio to word sequences trained end-to-end along one global optimisation criterion in a fully data driven fashion. These models allow high precision ASR for domains and words represented in the training material but have difficulties recognising words that are rarely or not at all represented during training, i.e. trending words and new named entities. In this paper, we use a text-to-speech (TTS) engine to provide synthetic audio for out-of-vocabulary (OOV) words. We aim to boost the recognition accuracy of a recurrent neural network transducer (RNN-T) on OOV words by using the extra audio-text pairs, while maintaining the performance on the non-OOV words. Different regularisation techniques are explored and the best performance is achieved by fine-tuning the RNN-T on both original training data and extra synthetic data with elastic weight consolidation (EWC) applied on the encoder. This yields a 57% relative word error rate (WER) reduction on utterances containing OOV words without any degradation on the whole test set.

eess.AS↗
arXiv2019

Second-order optimality conditions for SDCMPCC and application to rank optimization problems

This paper is concerned with second-order optimality conditions for the mathematical program with semidefinite cone complementarity constraints (SDCMPCC).To achieve this goal, we first provide an exact characterization on the second-order tangent set to the semidefinite cone complementarity (SDCC) set in terms of the second-order directional derivative of the projection operator onto the SDCC set, and then use the second-order tangent set to the SDCC set to derive second-order necessary and sufficient conditions for SDCMPCC under suitable subregularity constraint qualifications. An application is also illustrated in characterizing the second-order necessary optimality condition for the semidefinite rank regularized problem.

math.OC↗
arXiv2019

Subregularity of subdifferential mappings relative to the critical set and KL property of exponent 1/2

For a proper extended real-valued function, this work focuses on the relationship between the subregularity of its subdifferential mapping relative to the critical set and its KL property of exponent 1/2. When the function is lsc convex, we establish the equivalence between them under the continuous assumption on the critical set. Then, for the uniformly prox-regular function, under its continuity on the local minimum set, the KL property of exponent 1/2 on the local minimum set is shown to be equivalent to the subregularity of its subdifferential relative to this set. Moreover, for this class of nonconvex functions, under a separation assumption of stationary values, we show that the subregularity of its subdifferential relative to the critical set also implies its KL property of exponent $1/2$. These results provide a bridge for the two kinds of regularity, and their application is illustrated by examples.

math.OC↗
arXiv2019

Several classes of stationary points for rank regularized minimization problems

For the rank regularized minimization problem, we introduce several kinds of stationary points by the problem itself and its equivalent reformulations including the mathematical program with an equilibrium constraint (MPEC), the global exact penalty of the MPEC,the surrogate yielded by eliminating the dual part in the exact penalty. A clear relation chart is established for these stationary points, which guides the user to choose an appropriate reformulation for seeking a low-rank solution. As a byproduct, we also provide a weaker condition for a local minimizer of the MPEC to be the M-stationary point by characterizing the directional limiting normal cone to the graph of the normal cone mapping of the positive semidefinite (PSD) cone.

math.OC↗
arXiv2018

Computation of graphical derivatives of normal cone maps to a class of conic constraint sets

This paper concerns with the graphical derivative of the normals to the conic constraint $g(x)\in\!K$, where $g\!:\mathbb{X}\to\mathbb{Y}$ is a twice continuously differentiable mapping and $K\subseteq\mathbb{Y}$ is a nonempty closed convex set assumed to be $C^2$-cone reducible. Such a generalized derivative plays a crucial role in characterizing isolated calmness of the solution maps to generalized equations whose multivalued parts are modeled via the normals to the nonconvex set $Γ=g^{-1}(K)$. The main contribution of this paper is to provide an exact characterization for the graphical derivative of the normals to this class of nonconvex conic constraints under an assumption without requiring the nondegeneracy of the reference point as the papers \cite{Gfrerer17,Mordu15,Mordu151} do.

math.OC↗
arXiv2018

Equivalent Lipschitz surrogates for zero-norm and rank optimization problems

This paper proposes a mechanism to produce equivalent Lipschitz surrogates for zero-norm and rank optimization problems by means of the global exact penalty for their equivalent mathematical programs with an equilibrium constraint (MPECs). Specifically, we reformulate these combinatorial problems as equivalent MPECs by the variational characterization of the zero-norm and rank function, show that their penalized problems, yielded by moving the equilibrium constraint into the objective, are the global exact penalization, and obtain the equivalent Lipschitz surrogates by eliminating the dual variable in the global exact penalty. These surrogates, including the popular SCAD function in statistics, are also difference of two convex functions (D.C.) if the function and constraint set involved in zero-norm and rank optimization problems are convex. We illustrate an application by designing a multi-stage convex relaxation approach to the rank plus zero-norm regularized minimization problem.

stat.ML↗
arXiv2018

Comment on "Piezoelectricity in planar boron nitride via a geometric phase"

Using the strain-dependent effective Hamiltonian and the geometric phase, Droth et al. [Phys. Rev. B 94, 075404 (2016)] obtain an analytical expression for the electronic contribution to the piezoelectricity of planar hexagonal boron nitride (h -BN). Their analytical results of piezoelectric constants for h -BN are invalid because of the mistakes in constructing the adiabatic process of the piezoelectricity. In this comment, we reconstruct a proper adiabatic process for piezoelectricity and formulate a general Berry phase expression for the piezoelectric coefficients of two-dimensional piezoelectric crystals by means of the modern theory of polarization. The corrected analytical results of the piezoelectric constants are in complete consistency with the first-principles calculations and hence manifest the validity and generality of the Berry phase expression of piezoelectric coefficients.

cond-mat.mes-hall↗
arXiv2018

Tight-binding Piezoelectric Theory and Electromechanical Coupling Correlations for Transition Metal Dichalcogenide Monolayers

The lack of inversion symmetry in semiconducting transition metal dichalcogenide monolayers (TMDMs) enables a considerable molecular-level intrinsic piezoelectricity, which opens prospects for atomically-thin piezotronics and optoelectronics. Here, based on the tight-binding (TB) approach and Berry phase polarization theory, we establish an atomic-scale TB theory for demonstrating piezoelectric physics in TMDMs. Using the TB piezoelectric theory, we predict their electronic Grüneisen parameters (EGP) which measure the electron-phonon couplings. By virtue of the constructed analytical piezoelectric model, we further reveal the correlation between the electronic contribution to piezoelectric coefficients and strain-induced pseudomagnetic gauge field (PMF). These predicted EGP and PMF for TMDMs are experimentally testable, and hence the TB piezoelectric model is an alternative theoretical framework for calculating electron-phonon interactions and PMF.

cond-mat.mes-hall↗
arXiv2018

Strong calmness of perturbed KKT system for a class of conic programming with degenerate solutions

This paper is concerned with the strong calmness of the KKT solution mapping for a class of canonically perturbed conic programming, which plays a central role in achieving fast convergence under situations when the Lagrange multiplier associated to a solution of these conic optimization problems is not unique. We show that the strong calmness of the KKT solution mapping is equivalent to a local error bound for solutions of perturbed KKT system, and is also equivalent to the pseudo-isolated calmness of the stationary point mapping along with the calmness of the multiplier set map at the corresponding reference point. Sufficient conditions are also provided for the strong calmness by establishing the pseudo-isolated calmness of the stationary point mapping in terms of the noncriticality of the associated multiplier, and the calmness of the multiplier set mapping in terms of a relative interior condition for the multiplier set. These results cover and extend the existing ones in \cite{Hager99,Izmailov12} for nonlinear programming and in \cite{Cui16,Zhang17} for semidefinite programming.

math.OC↗
arXiv2016

Locally upper Lipschitz of the perturbed KKT system of Ky Fan $k$-norm matrix conic optimization problems

This note is concerned with the nonlinear Ky Fan $k$-norm matrix conic optimization problems, which include the nuclear norm regularized minimization problem as a special case. For this class of nonpolyhedral matrix conic optimization problems, under the assumption that a stationary solution satisfies the second-order sufficient condition and the associated Lagrange multiplier satisfies the strict Robinson's CQ, we show that two classes of perturbed KKT systems are locally upper Lipschitz at the origin, which implies a local error bound for the distance from any point in a neighborhood of the corresponding KKT point to the whole set of KKT points.

math.OC↗
arXiv2015

The 2015 Sheffield System for Transcription of Multi-Genre Broadcast Media

We describe the University of Sheffield system for participation in the 2015 Multi-Genre Broadcast (MGB) challenge task of transcribing multi-genre broadcast shows. Transcription was one of four tasks proposed in the MGB challenge, with the aim of advancing the state of the art of automatic speech recognition, speaker diarisation and automatic alignment of subtitles for broadcast media. Four topics are investigated in this work: Data selection techniques for training with unreliable data, automatic speech segmentation of broadcast media shows, acoustic modelling and adaptation in highly variable environments, and language modelling of multi-genre shows. The final system operates in multiple passes, using an initial unadapted decoding stage to refine segmentation, followed by three adapted passes: a hybrid DNN pass with input features normalised by speaker-based cepstral normalisation, another hybrid stage with input features normalised by speaker feature-MLLR transformations, and finally a bottleneck-based tandem stage with noise and speaker factorisation. The combination of these three system outputs provides a final error rate of 27.5% on the official development set, consisting of 47 multi-genre shows.

cs.CL↗