Searcharxiv⌕ Search

arXiv subjects

Yuan Cao

Publications and source records attributed to Yuan Cao.

At least 217 records · Page 12Linked to original sources
arXiv2019

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models. Lingvo models are composed of modular building blocks that are flexible and easily extensible, and experiment configurations are centralized and highly customizable. Distributed training and quantized inference are supported directly within the framework, and it contains existing implementations of a large number of utilities, helper functions, and the newest research ideas. Lingvo has been used in collaboration by dozens of researchers in more than 20 papers over the last two years. This document outlines the underlying design of Lingvo and serves as an introduction to the various pieces of the framework, while also offering examples of advanced features that showcase the capabilities of the framework.

cs.LG↗
arXiv2019

Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation

End-to-end Speech Translation (ST) models have many potential advantages when compared to the cascade of Automatic Speech Recognition (ASR) and text Machine Translation (MT) models, including lowered inference latency and the avoidance of error compounding. However, the quality of end-to-end ST is often limited by a paucity of training data, since it is difficult to collect large parallel corpora of speech and translated transcript pairs. Previous studies have proposed the use of pre-trained components and multi-task learning in order to benefit from weakly supervised training data, such as speech-to-transcript or text-to-foreign-text pairs. In this paper, we demonstrate that using pre-trained MT or text-to-speech (TTS) synthesis models to convert weakly supervised data into speech-to-translation pairs for ST training can be more effective than multi-task learning. Furthermore, we demonstrate that a high quality end-to-end ST model can be trained using only weakly supervised datasets, and that synthetic data sourced from unlabeled monolingual text or speech can be used to improve performance. Finally, we discuss methods for avoiding overfitting to synthetic speech with a quantitative ablation study.

cs.CL↗
arXiv2019

Strange metal in magic-angle graphene with near Planckian dissipation

Recent experiments on magic-angle twisted bilayer graphene have discovered correlated insulating behavior and superconductivity at a fractional filling of an isolated narrow band. In this paper we show that magic-angle bilayer graphene exhibits another hallmark of strongly correlated systems --- a broad regime of $T-$linear resistivity above a small, density dependent, crossover temperature--- for a range of fillings near the correlated insulator. We also extract a transport "scattering rate", which satisfies a near Planckian form that is universally related to the ratio of $(k_BT/\hbar)$. Our results establish magic-angle bilayer graphene as a highly tunable platform to investigate strange metal behavior, which could shed light on this mysterious ubiquitous phase of correlated matter.

cond-mat.str-el↗
arXiv2018

Hierarchical Generative Modeling for Controllable Speech Synthesis

This paper proposes a neural sequence-to-sequence text-to-speech (TTS) model which can control latent attributes in the generated speech that are rarely annotated in the training data, such as speaking style, accent, background noise, and recording conditions. The model is formulated as a conditional generative model based on the variational autoencoder (VAE) framework, with two levels of hierarchical latent variables. The first level is a categorical variable, which represents attribute groups (e.g. clean/noisy) and provides interpretability. The second level, conditioned on the first, is a multivariate Gaussian variable, which characterizes specific attribute configurations (e.g. noise level, speaking rate) and enables disentangled fine-grained control over these attributes. This amounts to using a Gaussian mixture model (GMM) for the latent distribution. Extensive evaluation demonstrates its ability to control the aforementioned attributes. In particular, we train a high-quality controllable TTS model on real found data, which is capable of inferring speaker and style attributes from a noisy utterance and use it to synthesize clean speech with controllable speaking style.

cs.CL↗
arXiv2018

Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks

We study the problem of training deep neural networks with Rectified Linear Unit (ReLU) activation function using gradient descent and stochastic gradient descent. In particular, we study the binary classification problem and show that for a broad family of loss functions, with proper random weight initialization, both gradient descent and stochastic gradient descent can find the global minima of the training loss for an over-parameterized deep ReLU network, under mild assumption on the training data. The key idea of our proof is that Gaussian random initialization followed by (stochastic) gradient descent produces a sequence of iterates that stay inside a small perturbation region centering around the initial weights, in which the empirical loss function of deep ReLU networks enjoys nice local curvature properties that ensure the global convergence of (stochastic) gradient descent. Our theoretical results shed light on understanding the optimization for deep learning, and pave the way for studying the optimization dynamics of training modern deep neural networks.

cs.LG↗
arXiv2018

Giant intrinsic photoresponse in pristine graphene

When the Fermi level matches the Dirac point in graphene, the reduced charge screening can dramatically enhance electron-electron (e-e) scattering to produce a strongly interacting Dirac liquid. While the dominance of e-e scattering already leads to novel behaviors, such as electron hydrodynamic flow, further exotic phenomena have been predicted to arise specifically from the unique kinematics of e-e scattering in massless Dirac systems. Here, we use optoelectronic probes, which are highly sensitive to the kinematics of electron scattering, to uncover a giant intrinsic photocurrent response in pristine graphene. This photocurrent emerges exclusively at the charge neutrality point and vanishes abruptly at non-zero charge densities. Moreover, it is observed at places with broken reflection symmetry, and it is selectively enhanced at free graphene edges with sharp bends. Our findings reveal that the photocurrent relaxation is strongly suppressed by a drastic change of fast photocarrier kinematics in graphene when its Fermi level matches the Dirac point. The emergence of robust photocurrents in neutral Dirac materials promises new energy-harvesting functionalities and highlights intriguing electron dynamics in the optoelectronic response of Dirac fluids.

cond-mat.mes-hall↗
arXiv2018

Nearly Flat Chern Bands in Moiré Superlattices

Topology and electron interactions are two central themes in modern condensed matter physics. Here we propose graphene based systems where both the band topology and interaction effects can be simply controlled with electric fields. We study a number of systems of twisted double layers with small twist angle where a moiré super-lattice is formed. Each layer is chosen to be either AB stacked bilayer graphene (BG), ABC stacked trilayer graphene (TG), or hexagonal boron nitride (h-BN). In these systems a vertical applied electric field enables control of the bandwidth, and interestingly also the Chern number. We find that the Chern numbers of the bands associated with each of the two microscopic valleys can be $\pm 0,\pm 1,\pm 2, \pm 3$ depending on the specific system and vertical electrical field. We show that these graphene moiré super-lattices are promising platforms to realize a number of fascinating many-body phenomena, including (Fractional) Quantum Anomalous Hall Effects. We also discuss conceptual similarities and implications for modeling twisted bilayer graphene systems.

cond-mat.str-el↗
arXiv2018

An explicit representation and enumeration for self-dual cyclic codes over $\mathbb{F}_{2^m}+u\mathbb{F}_{2^m}$ of length $2^s$

Let $\mathbb{F}_{2^m}$ be a finite field of cardinality $2^m$ and $s$ a positive integer. Using properties for Kronecker product of matrices and calculation for linear equations over $\mathbb{F}_{2^m}$, an efficient method for the construction of all distinct self-dual cyclic codes with length $2^s$ over the finite chain ring $\mathbb{F}_{2^m}+u\mathbb{F}_{2^m}$ $(u^2=0)$ is provided. On that basis, an explicit representation for every self-dual cyclic code of length $2^s$ over $\mathbb{F}_{2^m}+u\mathbb{F}_{2^m}$ and an exact formula to count the number of all these self-dual cyclic codes are given.

cs.IT↗
arXiv2018

A class of repeated-root constacyclic codes over $\mathbb{F}_{p^m}[u]/\langle u^e\rangle$ of Type $2$

Let $\mathbb{F}_{p^m}$ be a finite field of cardinality $p^m$ where $p$ is an odd prime, $n$ be a positive integer satisfying ${\rm gcd}(n,p)=1$, and denote $R=\mathbb{F}_{p^m}[u]/\langle u^e\rangle$ where $e\geq 4$ be an even integer. Let $δ,α\in \mathbb{F}_{p^m}^{\times}$. Then the class of $(δ+αu^2)$-constacyclic codes over $R$ is a significant subclass of constacyclic codes over $R$ of Type 2. For any integer $k\geq 1$, an explicit representation and a complete description for all distinct $(δ+αu^2)$-constacyclic codes over $R$ of length $np^k$ and their dual codes are given. Moreover, formulas for the number of codewords in each code and the number of all such codes are provided respectively. In particular, all distinct $(δ+αu^2)$-contacyclic codes over $\mathbb{F}_{p^m}[u]/\langle u^{e}\rangle$ of length $p^k$ and their dual codes are presented precisely.

cs.IT↗
arXiv2018

Electrically Tunable Low Density Superconductivity in a Monolayer Topological Insulator

The capability to switch electrically between superconducting and insulating states of matter represents a novel paradigm in the state-of-the-art engineering of correlated electronic systems. An exciting possibility is to turn on superconductivity in a topologically non-trivial insulator, which provides a route to search for non-Abelian topological states. However, existing demonstrations of superconductor-insulator switches have involved only topologically trivial systems, and even those are rare due to the stringent requirement to tune the carrier density over a wide range. Here we report reversible, in-situ electrostatic on off switching of superconductivity in a recently established quantum spin Hall insulator, namely monolayer tungsten ditelluride (WTe2). Fabricated into a van der Waals field effect transistor, the monolayer's ground state can be continuously gate-tuned from the topological insulating to the superconducting state, with critical temperatures Tc up to ~ 1 Kelvin. The critical density for the onset of superconductivity is estimated to be ~ 5 x 10^12 cm^-2, among the lowest for two-dimensional (2D) superconductors. Our results establish monolayer WTe2 as a material platform for engineering novel superconducting nanodevices and topological phases of matter.

cond-mat.mes-hall↗
arXiv2018

Training Deeper Neural Machine Translation Models with Transparent Attention

While current state-of-the-art NMT models, such as RNN seq2seq and Transformers, possess a large number of parameters, they are still shallow in comparison to convolutional models used for both text and vision applications. In this work we attempt to train significantly (2-3x) deeper Transformer and Bi-RNN encoders for machine translation. We propose a simple modification to the attention mechanism that eases the optimization of deeper models, and results in consistent gains of 0.7-1.1 BLEU on the benchmark WMT'14 English-German and WMT'15 Czech-English tasks for both architectures.

cs.CL↗
arXiv2018

An ultra-sensitive balanced detector with low noise for continuous-variable quantum key distribution

A well-balanced detector with high sensitivity and low noise is presented in this paper. The two-stage amplification structure is used to increase electronic gain while keeping an effective bandwidth of about 70 MHz. In order to further reduce electronic noise, a junction field-effect transistor(JFET) is connected between photodiodes and transimpedance amplifier to reduce the impact of amplifier leakage current. Benefit from these designs, the root-mean-square(RMS) of noise voltage is about 6 mV with a gain of 3.2E5 V/W, and it means an ultra-low noise equivalent power density of 2.2E-12 W/rtHz, only half of common low-noise commercial detectors. In addition, two photodiodes in similar frequency response are selected for detector and make the common mode rejection ratio(CMRR) of detector reached 53 dB, about 13 dB higher than commercial detectors. Further tests indicate that 16.8 dB shot-noise to electronic-noise ratio is measured in our detector, which is better than most high speed balanced detectors.

physics.ins-det↗
arXiv2018

Magic-angle graphene superlattices: a new platform for unconventional superconductivity

The understanding of strongly-correlated materials, and in particular unconventional superconductors, has puzzled physicists for decades. Such difficulties have stimulated new research paradigms, such as ultra-cold atom lattices for simulating quantum materials. Here we report on the realization of intrinsic unconventional superconductivity in a 2D superlattice created by stacking two graphene sheets with a small twist angle. For angles near $1.1^\circ$, the first `magic' angle, twisted bilayer graphene (TBG) exhibits ultra-flat bands near charge neutrality, which lead to correlated insulating states at half-filling. Upon electrostatic doping away from these correlated insulating states, we observe tunable zero-resistance states with a critical temperature $T_c$ up to 1.7 K. The temperature-density phase diagram shows similarities with that of the cuprates, including superconducting domes. Moreover, quantum oscillations indicate small Fermi surfaces near the correlated insulating phase, in analogy with under-doped cuprates. The relative high $T_c$, given such small Fermi surface (corresponding to a record-low 2D carrier density of $10^{11} \textrm{cm}^{-2}$ , renders TBG among the strongest coupling superconductors, in a regime close to the BCS-BEC crossover. These novel results establish TBG as the first purely carbon-based 2D superconductor and as a highly tunable platform to investigate strongly-correlated phenomena, which could lead to insights into the physics of high-$T_c$ superconductors and quantum spin liquids.

cond-mat.mes-hall↗
arXiv2018

Intelligent Identification of Two-Dimensional Structure by Machine-Learning Optical Microscopy

Two-dimensional (2D) materials and their heterostructures, with wafer-scale synthesis methods and fascinating properties, have attracted numerous interest and triggered revolutions of corresponding device applications. However, facile methods to realize accurate, intelligent and large-area characterizations of these 2D structures are still highly desired. Here, we report a successful application of machine-learning strategy in the optical identification of 2D structure. The machine-learning optical identification method (MOI method) endows optical microscopy with intelligent insight into the characteristic colour information in the optical photograph. Experimental results indicate that the MOI method enables accurate, intelligent and large-area characterizations of graphene, molybdenum disulphide (MoS2) and their heterostructures, including identifications of the thickness, the existence of impurities, and even the stacking order. Thanks to the convergence of artificial intelligence and nanoscience, this intelligent identification method can certainly promote the fundamental research and wafer-scale device application of 2D structures.

cond-mat.mtrl-sci↗
arXiv2018

Matrix-product structure of constacyclic codes over finite chain rings $\mathbb{F}_{p^m}[u]/\langle u^e\rangle$

Let $m,e$ be positive integers, $p$ a prime number, $\mathbb{F}_{p^m}$ be a finite field of $p^m$ elements and $R=\mathbb{F}_{p^m}[u]/\langle u^e\rangle$ which is a finite chain ring. For any $ω\in R^\times$ and positive integers $k, n$ satisfying ${\rm gcd}(p,n)=1$, we prove that any $(1+ωu)$-constacyclic code of length $p^kn$ over $R$ is monomially equivalent to a matrix-product code of a nested sequence of $p^k$ cyclic codes with length $n$ over $R$ and a $p^k\times p^k$ matrix $A_{p^k}$ over $\mathbb{F}_p$. Using the matrix-product structures, we give an iterative construction of every $(1+ωu)$-constacyclic code by $(1+ωu)$-constacyclic codes of shorter lengths over $R$.

cs.IT↗
arXiv2018

Negacyclic codes over the local ring $\mathbb{Z}_4[v]/\langle v^2+2v\rangle$ of oddly even length and their Gray images

Let $R=\mathbb{Z}_{4}[v]/\langle v^2+2v\rangle=\mathbb{Z}_{4}+v\mathbb{Z}_{4}$ ($v^2=2v$) and $n$ be an odd positive integer. Then $R$ is a local non-principal ideal ring of $16$ elements and there is a $\mathbb{Z}_{4}$-linear Gray map from $R$ onto $\mathbb{Z}_{4}^2$ which preserves Lee distance and orthogonality. First, a canonical form decomposition and the structure for any negacyclic code over $R$ of length $2n$ are presented. From this decomposition, a complete classification of all these codes is obtained. Then the cardinality and the dual code for each of these codes are given, and self-dual negacyclic codes over $R$ of length $2n$ are presented. Moreover, all $23\cdot(4^p+5\cdot 2^p+9)^{\frac{2^{p}-2}{p}}$ negacyclic codes over $R$ of length $2M_p$ and all $3\cdot(4^p+5\cdot 2^p+9)^{\frac{2^{p-1}-1}{p}}$ self-dual codes among them are presented precisely, where $M_p=2^p-1$ is a Mersenne prime. Finally, $36$ new and good self-dual $2$-quasi-twisted linear codes over $\mathbb{Z}_4$ with basic parameters $(28,2^{28}, d_L=8,d_E=12)$ and of type $2^{14}4^7$ and basic parameters $(28,2^{28}, d_L=6,d_E=12)$ and of type $2^{16}4^6$ which are Gray images of self-dual negacyclic codes over $R$ of length $14$ are listed.

cs.IT↗
arXiv2018

Correlated Insulator Behaviour at Half-Filling in Magic Angle Graphene Superlattices

Van der Waals (vdW) heterostructures are an emergent class of metamaterials comprised of vertically stacked two-dimensional (2D) building blocks, which provide us with a vast tool set to engineer their properties on top of the already rich tunability of 2D materials. One of the knobs, the twist angle between different layers, plays a crucial role in the ultimate electronic properties of a vdW heterostructure and does not have a direct analog in other systems such as MBE-grown semiconductor heterostructures. For small twist angles, the moiré pattern produced by the lattice misorientation creates a long-range modulation. So far, the study of the effect of twist angles in vdW heterostructures has been mostly concentrated in graphene/hexagonal boron nitride (h-BN) twisted structures, which exhibit relatively weak interlayer interaction due to the presence of a large bandgap in h-BN. Here we show that when two graphene sheets are twisted by an angle close to the theoretically predicted 'magic angle', the resulting flat band structure near charge neutrality gives rise to a strongly-correlated electronic system. These flat bands exhibit half-filling insulating phases at zero magnetic field, which we show to be a Mott-like insulator arising from electrons localized in the moiré superlattice. These unique properties of magic-angle twisted bilayer graphene (TwBLG) open up a new playground for exotic many-body quantum phases in a 2D platform made of pure carbon and without magnetic field. The easy accessibility of the flat bands, the electrical tunability, and the bandwidth tunability though twist angle may pave the way towards more exotic correlated systems, such as unconventional superconductors or quantum spin liquids.

cond-mat.mes-hall↗