arXiv subjects

Xiao Chen

Publications and source records attributed to Xiao Chen.

At least 271 records · Page 15Linked to original sources

arXiv2020

The number of Dirac-weighted eigenvalues of Sturm-Liouville equations with integrable potentials and an application to inverse problems

In this paper, we further Meirong Zhang, et al.'s work by computing the number of weighted eigenvalues for Sturm-Liouville equations, equipped with general integrable potentials and Dirac weights, under Dirichlet boundary condition. We show that, for a Sturm-Liouville equation with a general integrable potential, if its weight is a positive linear combination of $n$ Dirac Delta functions, then it has at most $n$ (may be less than $n$, or even be $0$) distinct real Dirichlet eigenvalues, or every complex number is a Dirichlet eigenvalue; in particular, under some sharp condition, the number of Dirichlet eigenvalues is exactly $n$. Our main method is to introduce the concepts of characteristics matrix and characteristics polynomial for Sturm-Liouville problem with Dirac weights, and put forward a general and direct algorithm used for computing eigenvalues. As an application, a class of inverse Dirichelt problems for Sturm-Liouville equations involving single Dirac distribution weights is studied.

math.CA↗

arXiv2020

Nematicity and superconductivity: Competition versus cooperation

Electronic nematic behavior has been identified and studied in iron-based superconductors for some time, particularly in the well-known BaFe$_2$As$_2$ system, where it is well-known to compete with superconductivity. On the other hand, it has been shown recently that FeSe displays a negligible effect of nematicity on superconductivity near the superconducting transition, and actual cooperation between the two orders when the system is doped with S. Recently it has also been proposed that LiFeAs undergoes a nematic transition in the superconducting state itself. Generally, we expect superconductivity to be anisotropic when it coexists with nematic order, but it is not clear under what circumstances the two orders compete or cooperate, nor how the anisotropy of the superconducting state correlates with that in the nematic state. To address this, we study a simple mean field model of a $d$-wave Pomeranchuk instability together with a mixed $s,d$ pairing interaction, and identify when nematicity is enhanced or suppressed by superconductivity. We show that the competition or cooperation depends significantly on the distortion of the Fermi surface due to nematicity relative to the anisotropy of the superconducting gap function. Further, we discuss the implications of our results for FeSe and LiFeAs.

cond-mat.supr-con↗

arXiv2020

Improving Task-Agnostic BERT Distillation with Layer Mapping Search

Knowledge distillation (KD) which transfers the knowledge from a large teacher model to a small student model, has been widely used to compress the BERT model recently. Besides the supervision in the output in the original KD, recent works show that layer-level supervision is crucial to the performance of the student BERT model. However, previous works designed the layer mapping strategy heuristically (e.g., uniform or last-layer), which can lead to inferior performance. In this paper, we propose to use the genetic algorithm (GA) to search for the optimal layer mapping automatically. To accelerate the search process, we further propose a proxy setting where a small portion of the training corpus are sampled for distillation, and three representative tasks are chosen for evaluation. After obtaining the optimal layer mapping, we perform the task-agnostic BERT distillation with it on the whole corpus to build a compact student model, which can be directly fine-tuned on downstream tasks. Comprehensive experiments on the evaluation benchmarks demonstrate that 1) layer mapping strategy has a significant effect on task-agnostic BERT distillation and different layer mappings can result in quite different performances; 2) the optimal layer mapping strategy from the proposed search process consistently outperforms the other heuristic ones; 3) with the optimal layer mapping, our student model achieves state-of-the-art performance on the GLUE tasks.

cs.CL↗

arXiv2020

Generating private data with user customization

Personal devices such as mobile phones can produce and store large amounts of data that can enhance machine learning models; however, this data may contain private information specific to the data owner that prevents the release of the data. We want to reduce the correlation between user-specific private information and the data while retaining the useful information. Rather than training a large model to achieve privatization from end to end, we first decouple the creation of a latent representation, and then privatize the data that allows user-specific privatization to occur in a setting with limited computation and minimal disturbance on the utility of the data. We leverage a Variational Autoencoder (VAE) to create a compact latent representation of the data that remains fixed for all devices and all possible private labels. We then train a small generative filter to perturb the latent representation based on user specified preferences regarding the private and utility information. The small filter is trained via a GAN-type robust optimization that can take place on a distributed device such as a phone or tablet. Under special conditions of our linear filter, we disclose the connections between our generative approach and renyi differential privacy. We conduct experiments on multiple datasets including MNIST, UCI-Adult, and CelebA, and give a thorough evaluation including visualizing the geometry of the latent embeddings and estimating the empirical mutual information to show the effectiveness of our approach.

eess.SP↗

arXiv2020

Entanglement Negativity at Measurement-Induced Criticality

We propose entanglement negativity as a fine-grained probe of measurement-induced criticality. We motivate this proposal in stabilizer states, where for two disjoint subregions, comparing their "mutual negativity" and their mutual information leads to a precise distinction between bipartite and multipartite entanglement. In a measurement-only stabilizer circuit that maps exactly to two-dimensional critical percolation, we show that the mutual information and the mutual negativity are governed by boundary conformal fields of different scaling dimensions at long distances. We then consider a class of "hybrid" circuit models obtained by perturbing the measurement-only circuit with unitary gates of progressive levels of complexity. While other critical exponents vary appreciably for different choices of unitary gate ensembles at their respective critical points, the mutual negativity has scaling dimension 3 across remarkably many of the hybrid circuits, which is notably different from that in percolation. We contrast our results with limiting cases where a geometrical minimal-cut picture is available.

cond-mat.stat-mech↗

arXiv2020

Data-driven quantitative modeling of bacterial active nematics

Active matter comprises individual units that convert energy into mechanical motion. In many examples, such as bacterial systems and biofilament assays, constituent units are elongated and can give rise to local nematic orientational order. Such `active nematics' systems have attracted much attention from both theorists and experimentalists. However, despite intense research efforts, data-driven quantitative modeling has not been achieved, a situation mainly due to the lack of systematic experimental data and to the large number of parameters of current models. Here we introduce a new active nematics system made of swarming filamentous bacteria. We simultaneously measure orientation and velocity fields and show that the complex spatiotemporal dynamics of our system can be quantitatively reproduced by a new type of microscopic model for active suspensions whose important parameters are all estimated from comprehensive experimental data. This provides unprecedented access to key effective parameters and mechanisms governing active nematics. Our approach is applicable to different types of dense suspensions and shows a path towards more quantitative active matter research.

cond-mat.soft↗

arXiv2020

Hydrodynamics in lattice models with continuous non-Abelian symmetries

We develop a systematic effective field theory of hydrodynamics for many-body systems on the lattice with global continuous non-Abelian symmetries. Models with continuous non-Abelian symmetries are ubiquitous in physics, arising in diverse settings ranging from hot nuclear matter to cold atomic gases and quantum spin chains. In every dimension and for every flavor symmetry group, the low energy theory is a set of coupled noisy diffusion equations. Independence of the physics on the choice of canonical or microcanonical ensemble is manifest in our hydrodynamic expansion, even though the ensemble choice causes an apparent shift in quasinormal mode spectra. We use our formalism to explain why flavor symmetry is qualitatively different from hydrodynamics with other non-Abelian conservation laws, including angular momentum and charge multipoles. As a significant application of our framework, we study spin and energy diffusion in classical one-dimensional SU(2)-invariant spin chains, including the Heisenberg model along with multiple generalizations. We argue based on both numerical simulations and our effective field theory framework that non-integrable spin chains on a lattice exhibit conventional spin diffusion, in contrast to some recent predictions that diffusion constants grow logarithmically at late times. We show that the apparent enhancement of diffusion is due to slow equilibration caused by (non-Abelian) hydrodynamic fluctuations.

cond-mat.stat-mech↗

arXiv2020

Exact Self-Consistent Effective Hamiltonian Theory

We propose a general variational fermionic many-body wavefunction that generates an effective Hamiltonian in a quadratic form, which can then be exactly solved. The theory can be constructed within the density functional theory framework, and a self-consistent scheme is proposed for solving the exact density functional theory. We apply the theory to structurally-disordered systems, symmetric and asymmetric Hubbard dimers, and the corresponding lattice models. The single fermion excitation spectra show a persistent gap due to the fermionic-entanglement-induced pairing condensate. For disordered systems, the density of states at the edge of the gap diverges in the thermodynamic limit, suggesting a topologically ordered phase. A sharp resonance is predicted as the gap is not dependent on the temperature of the system. For the symmetric Hubbard model, the gap for both half-filling and doped case suggests that the quantum phase transition between the antiferromagnetic and superconducting phases is continuous.

cond-mat.str-el↗

arXiv2020

The HUAWEI Speaker Diarisation System for the VoxCeleb Speaker Diarisation Challenge

This paper describes system setup of our submission to speaker diarisation track (Track 4) of VoxCeleb Speaker Recognition Challenge 2020. Our diarisation system consists of a well-trained neural network based speech enhancement model as pre-processing front-end of input speech signals. We replace conventional energy-based voice activity detection (VAD) with a neural network based VAD. The neural network based VAD provides more accurate annotation of speech segments containing only background music, noise, and other interference, which is crucial to diarisation performance. We apply agglomerative hierarchical clustering (AHC) of x-vectors and variational Bayesian hidden Markov model (VB-HMM) based iterative clustering for speaker clustering. Experimental results demonstrate that our proposed system achieves substantial improvements over the baseline system, yielding diarisation error rate (DER) of 10.45%, and Jacard error rate (JER) of 22.46% on the evaluation set.

cs.SD↗

arXiv2020

TinyBERT: Distilling BERT for Natural Language Understanding

Language model pre-training, such as BERT, has significantly improved the performances of many natural language processing tasks. However, pre-trained language models are usually computationally expensive, so it is difficult to efficiently execute them on resource-restricted devices. To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models. By leveraging this new KD method, the plenty of knowledge encoded in a large teacher BERT can be effectively transferred to a small student Tiny-BERT. Then, we introduce a new two-stage learning framework for TinyBERT, which performs Transformer distillation at both the pretraining and task-specific learning stages. This framework ensures that TinyBERT can capture he general-domain as well as the task-specific knowledge in BERT. TinyBERT with 4 layers is empirically effective and achieves more than 96.8% the performance of its teacher BERTBASE on GLUE benchmark, while being 7.5x smaller and 9.4x faster on inference. TinyBERT with 4 layers is also significantly better than 4-layer state-of-the-art baselines on BERT distillation, with only about 28% parameters and about 31% inference time of them. Moreover, TinyBERT with 6 layers performs on-par with its teacher BERTBASE.

cs.CL↗

arXiv2020

GFB-MRF: A parallel spatial and Bloch manifold regularized iterative reconstruction method for MR Fingerprinting

Magnetic Resonance Fingerprinting (MRF) reconstructs tissue maps based on a sequence of very highly undersampled images. In order to be able to perform MRF reconstruction, state-of-the-art MRF methods rely on priors such as the MR physics (Bloch equations) and might also use some additional low-rank or spatial regularization. However to our knowledge these three regularizations are not applied together in a joint reconstruction. The reason is that it is indeed challenging to incorporate effectively multiple regularizations in a single MRF optimization algorithm. As a result most of these methods are not robust to noise especially when the sequence length is short. In this paper, we propose a family of new methods where spatial and low-rank regularizations, in addition to the Bloch manifold regularization, are applied on the images. We show on digital phantom and NIST phantom scans, as well as volunteer scans that the proposed methods bring significant improvement in the quality of the estimated tissue maps.

eess.IV↗

arXiv2020

Many-body quantum dynamics slows down at low density

We study quantum many-body systems with a global U(1) conservation law, focusing on a theory of $N$ interacting fermions with charge conservation, or $N$ interacting spins with one conserved component of total spin. We define an effective operator size at finite chemical potential through suitably regularized out-of-time-ordered correlation functions. The growth rate of this density-dependent operator size vanishes algebraically with charge density; hence we obtain new bounds on Lyapunov exponents and butterfly velocities in charged systems at a given density, which are parametrically stronger than any Lieb-Robinson bound. We argue that the density dependence of our bound on the Lyapunov exponent is saturated in the charged Sachdev-Ye-Kitaev model. We also study random automaton quantum circuits and Brownian Sachdev-Ye-Kitaev models, each of which exhibit a different density dependence for the Lyapunov exponent, and explain the discrepancy. We propose that our results are a cartoon for understanding Planckian-limited energy-conserving dynamics at finite temperature.

quant-ph↗

arXiv2020

TernaryBERT: Distillation-aware Ultra-low Bit BERT

Transformer-based pre-training models like BERT have achieved remarkable performance in many natural language processing tasks.However, these models are both computation and memory expensive, hindering their deployment to resource-constrained devices. In this work, we propose TernaryBERT, which ternarizes the weights in a fine-tuned BERT model. Specifically, we use both approximation-based and loss-aware ternarization methods and empirically investigate the ternarization granularity of different parts of BERT. Moreover, to reduce the accuracy degradation caused by the lower capacity of low bits, we leverage the knowledge distillation technique in the training process. Experiments on the GLUE benchmark and SQuAD show that our proposed TernaryBERT outperforms the other BERT quantization methods, and even achieves comparable performance as the full-precision model while being 14.9x smaller.

cs.CL↗

arXiv2020

DynaBERT: Dynamic BERT with Adaptive Width and Depth

The pre-trained language models like BERT, though powerful in many natural language processing tasks, are both computation and memory expensive. To alleviate this problem, one approach is to compress them for specific tasks before deployment. However, recent works on BERT compression usually compress the large BERT model to a fixed smaller size. They can not fully satisfy the requirements of different edge devices with various hardware performances. In this paper, we propose a novel dynamic BERT model (abbreviated as DynaBERT), which can flexibly adjust the size and latency by selecting adaptive width and depth. The training process of DynaBERT includes first training a width-adaptive BERT and then allowing both adaptive width and depth, by distilling knowledge from the full-sized model to small sub-networks. Network rewiring is also used to keep the more important attention heads and neurons shared by more sub-networks. Comprehensive experiments under various efficiency constraints demonstrate that our proposed dynamic BERT (or RoBERTa) at its largest size has comparable performance as BERT-base (or RoBERTa-base), while at smaller widths and depths consistently outperforms existing BERT compression methods. Code is available at https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/DynaBERT.

cs.CL↗

arXiv2020

A General Framework for Adaptation of Neural Machine Translation to Simultaneous Translation

Despite the success of neural machine translation (NMT), simultaneous neural machine translation (SNMT), the task of translating in real time before a full sentence has been observed, remains challenging due to the syntactic structure difference and simultaneity requirements. In this paper, we propose a general framework for adapting neural machine translation to translate simultaneously. Our framework contains two parts: prefix translation that utilizes a consecutive NMT model to translate source prefixes and a stopping criterion that determines when to stop the prefix translation. Experiments on three translation corpora and two language pairs show the efficacy of the proposed framework on balancing the quality and latency in adapting NMT to perform simultaneous translation.

cs.CL↗

arXiv2020

Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation

Cardiac motion estimation plays a key role in MRI cardiac feature tracking and function assessment such as myocardium strain. In this paper, we propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation. We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field. We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance. Our teacher model provides more accurate motion estimation as supervision through progressive motion compensations. Our student model learns from the teacher model to estimate motion in a single step while maintaining accuracy. The teacher-student knowledge distillation is performed in a cyclic way for a further performance boost. Our proposed method outperforms a strong baseline model on two public available clinical datasets significantly, evaluated by a variety of metrics and the inference time. New evaluation metrics are also proposed to represent errors in a clinically meaningful manner.

eess.IV↗

arXiv2020

An Investigation of Few-Shot Learning in Spoken Term Classification

In this paper, we investigate the feasibility of applying few-shot learning algorithms to a speech task. We formulate a user-defined scenario of spoken term classification as a few-shot learning problem. In most few-shot learning studies, it is assumed that all the N classes are new in a N-way problem. We suggest that this assumption can be relaxed and define a N+M-way problem where N and M are the number of new classes and fixed classes respectively. We propose a modification to the Model-Agnostic Meta-Learning (MAML) algorithm to solve the problem. Experiments on the Google Speech Commands dataset show that our approach outperforms the conventional supervised learning approach and the original MAML.

cs.CL↗

arXiv2020

Real-Time Cardiac Cine MRI with Residual Convolutional Recurrent Neural Network

Real-time cardiac cine MRI does not require ECG gating in the data acquisition and is more useful for patients who can not hold their breaths or have abnormal heart rhythms. However, to achieve fast image acquisition, real-time cine commonly acquires highly undersampled data, which imposes a significant challenge for MRI image reconstruction. We propose a residual convolutional RNN for real-time cardiac cine reconstruction. To the best of our knowledge, this is the first work applying deep learning approach to Cartesian real-time cardiac cine reconstruction. Based on the evaluation from radiologists, our deep learning model shows superior performance than compressed sensing.

eess.IV↗