Searcharxiv⌕ Search

arXiv subjects

Jing Li

Publications and source records attributed to Jing Li.

At least 595 records · Page 33Linked to original sources
arXiv2021

On the Transformer Growth for Progressive BERT Training

Due to the excessive cost of large-scale language model pre-training, considerable efforts have been made to train BERT progressively -- start from an inferior but low-cost model and gradually grow the model to increase the computational complexity. Our objective is to advance the understanding of Transformer growth and discover principles that guide progressive training. First, we find that similar to network architecture search, Transformer growth also favors compound scaling. Specifically, while existing methods only conduct network growth in a single dimension, we observe that it is beneficial to use compound growth operators and balance multiple dimensions (e.g., depth, width, and input length of the model). Moreover, we explore alternative growth operators in each dimension via controlled comparison to give operator selection practical guidance. In light of our analyses, the proposed method speeds up BERT pre-training by 73.6% and 82.2% for the base and large models respectively, while achieving comparable performances

cs.CL↗
arXiv2021

Oriental Language Recognition (OLR) 2020: Summary and Analysis

The fifth Oriental Language Recognition (OLR) Challenge focuses on language recognition in a variety of complex environments to promote its development. The OLR 2020 Challenge includes three tasks: (1) cross-channel language identification, (2) dialect identification, and (3) noisy language identification. We choose Cavg as the principle evaluation metric, and the Equal Error Rate (EER) as the secondary metric. There were 58 teams participating in this challenge and one third of the teams submitted valid results. Compared with the best baseline, the Cavg values of Top 1 system for the three tasks were relatively reduced by 82%, 62% and 48%, respectively. This paper describes the three tasks, the database profile, and the final results. We also outline the novel approaches that improve the performance of language recognition systems most significantly, such as the utilization of auxiliary information.

cs.SD↗
arXiv2021

Tidal effects in 4D Einstein-Gauss-Bonnet Black Hole Spacetime

We have investigated tidal forces and geodesic deviation motion in the 4D-Einstein-Gauss-Bonnet spacetime. Our results show that tidal force and geodesic deviation motion depend sharply on the sign of Gauss-Bonnet coupling constant. Comparing with Schwarzschild spacetime, the strength of tidal force becomes stronger for the negative Gauss-Bonnet coupling constant, but is weaker for the positive one. Moreover, tidal force behaves like those in the Schwarzschild spacetime as the coupling constant is negative, and like those in Reissner-Nordström black hole as the constant is positive. We also present the change of geodesic deviation vector with Gauss-Bonnet coupling constant under two kinds of initial conditions.

gr-qc↗
arXiv2021

An Integrated Framework for Two-pass Personalized Voice Trigger

In this paper, we present the XMUSPEECH system for Task 1 of 2020 Personalized Voice Trigger Challenge (PVTC2020). Task 1 is a joint wake-up word detection with speaker verification on close talking data. The whole system consists of a keyword spotting (KWS) sub-system and a speaker verification (SV) sub-system. For the KWS system, we applied a Temporal Depthwise Separable Convolution Residual Network (TDSC-ResNet) to improve the system's performance. For the SV system, we proposed a multi-task learning network, where phonetic branch is trained with the character label of the utterance, and speaker branch is trained with the label of the speaker. Phonetic branch is optimized with connectionist temporal classification (CTC) loss, which is treated as an auxiliary module for speaker branch. Experiments show that our system gets significant improvements compared with baseline system.

eess.AS↗
arXiv2021

Adiabatic critical quantum metrology cannot reach the Heisenberg limit even when shortcuts to adiabaticity are applied

We show that the quantum Fisher information attained in an adiabatic approach to critical quantum metrology cannot lead to the Heisenberg limit of precision and therefore regular quantum metrology under optimal settings is always superior. Furthermore, we argue that even though shortcuts to adiabaticity can arbitrarily decrease the time of preparing critical ground states, they cannot be used to achieve or overcome the Heisenberg limit for quantum parameter estimation in adiabatic critical quantum metrology. As case studies, we explore the application of counter-diabatic driving to the Landau-Zener model and the quantum Rabi model.

quant-ph↗
arXiv2021

Existence and Exponential Growth of Global Classical Solutions to the Compressible Navier-Stokes Equations with Slip Boundary Conditions in 3D Bounded Domains

We investigate the barotropic compressible Navier-Stokes equations with slip boundary conditions in a three-dimensional (3D) simply connected bounded domain, whose smooth boundary has a finite number of two-dimensional connected components. For any adiabatic exponent bigger than one, after discovering some new estimates on boundary integrals related to the slip boundary condition, we prove that both the weak and classical solutions to the initial-boundary-value problem of this system exist globally in time provided the initial energy is suitably small. Moreover, the density has large oscillations and contains vacuum states. Finally, it is also shown that for the classical solutions, the oscillation of the density will grow unboundedly in the long run with an exponential rate provided vacuum appears (even at a point) initially. This is the first result concerning the global existence of classical solutions to the compressible Navier-Stokes equations with density containing vacuum states initially for general 3D bounded smooth domains.

math.AP↗
arXiv2021

Continuity of Topic, Interaction, and Query: Learning to Quote in Online Conversations

Quotations are crucial for successful explanations and persuasions in interpersonal communications. However, finding what to quote in a conversation is challenging for both humans and machines. This work studies automatic quotation generation in an online conversation and explores how language consistency affects whether a quotation fits the given context. Here, we capture the contextual consistency of a quotation in terms of latent topics, interactions with the dialogue history, and coherence to the query turn's existing content. Further, an encoder-decoder neural framework is employed to continue the context with a quotation via language generation. Experiment results on two large-scale datasets in English and Chinese demonstrate that our quotation generation model outperforms the state-of-the-art models. Further analysis shows that topic, interaction, and query consistency are all helpful to learn how to quote in online conversations.

cs.CL↗
arXiv2021

Quantum Gaussian process regression

In this paper, a quantum algorithm based on gaussian process regression model is proposed. The proposed quantum algorithm consists of three sub-algorithms. One is the first quantum subalgorithm to efficiently generate mean predictor. The improved HHL algorithm is proposed to obtain the sign of outcomes. Therefore, the terrible situation that results is ambiguous in terms of original HHL algorithm is avoided, which makes whole algorithm more clear and exact. The other is to product covariance predictor with same method. Thirdly, the squared exponential covariance matrices are prepared that annihilation operator and generation operator are simulated by the unitary linear decomposition Hamiltonian simulation and kernel function vectors is generated with blocking coding techniques on covariance matrices. In addition, it is shown that the proposed quantum gaussian process regression algorithm can achieve quadratic faster over the classical counterpart.

quant-ph↗
arXiv2021

II-VI Organic-Inorganic Hybrid Nanostructures with Greatly Enhanced Optoelectronic Properties, Perfectly Ordered Structures, and Over 15-Year Shelf Stability

Organic-inorganic hybrids may offer material properties not available from their inorganic components. However, they are typically less stable and disordered. Long-term stability study of the hybrid materials, over the anticipated lifespan of a real-world electronic device, is practically nonexistent. Disordering, prevalent in most nanostructure assemblies, is a prominent adversary to quantum coherence. A family of perfectly ordered II-VI based hybrid nanostructures has been shown to possess a number of unusual properties and potential applications. Here, using a prototype structure ZnTe(en)0.5 - a hybrid superlattice, and applying an array of optical, structural, surface, thermal, and electrical characterization techniques in conjunction with density-functional theory calculations, we have performed a comprehensive and correlative study of the crystalline quality, structural degradation, electronic, optical, and transport properties on samples from over 15-years old to the recently synthesized. The findings show that not only do they exhibit an exceptionally high level of crystallinity in both macroscopic and microscopic scale, comparable to high-quality binary semiconductors; and greatly enhanced material properties, compared to those of the inorganic constituents; but also, some of them over 15-years old remain as good in structure and property as freshly made ones. This study reveals (1) what level of structural perfectness is achievable in a complex organic-inorganic hybrid structure or a man-made superlattice, suggesting a non-traditional strategy to make periodically stacked heterostructures with abrupt interfaces; and (2) how the stability of a hybrid material is affected differently by its intrinsic attributes, primarily formation energy, and extrinsic factors, such as surface and defects.

cond-mat.mtrl-sci↗
arXiv2021

Disintegration of Long-Period Comet C/2019 Y4 (ATLAS): I. Hubble Space Telescope Observations

Near-Sun Comet C/2019 Y4 (ATLAS) is the first member of a long-period comet group observed to disintegrate well before perihelion. Here we present our investigation into this disintegration event using images obtained in a 3-day {\it Hubble Space Telescope} (\hst) campaign. We identify two fragment clusters produced by the initial disintegration event, corresponding to fragments C/2019 Y4-A and C/2019 Y4-B identified in ground-based data. These two clusters started with similar integrated brightness, but exhibit different evolutionary behavior. C/2019 Y4-A was much shorter-lived compared to C/2019 Y4-B, and showed signs of significant mass-loss and changes in size distribution throughout the 3-day campaign. The cause of the initial fragmentation is undetermined by the limited evidence but crudely compatible with either the spin-up disruption of the nucleus or runaway sublimation of sub-surface supervolatile ices, either of which would lead to the release of a large amount of gas as inferred from the significant bluing of the comet observed shortly before the disintegration. Gas can only be produced by the sublimation of volatile ices, which must have survived at least one perihelion passage at a perihelion distance of $q=0.25$~au. We speculate that Comet ATLAS is derived from the ice-rich interior of a non-uniform, kilometer-wide progenitor that split during its previous perihelion. This suggests that comets down to a few kilometers in diameter can still possess complex, non-uniform interiors that can protect ices against intense solar heating.

astro-ph.EP↗
arXiv2021

Who Responded to Whom: The Joint Effects of Latent Topics and Discourse in Conversation Structure

Numerous online conversations are produced on a daily basis, resulting in a pressing need to conversation understanding. As a basis to structure a discussion, we identify the responding relations in the conversation discourse, which link response utterances to their initiations. To figure out who responded to whom, here we explore how the consistency of topic contents and dependency of discourse roles indicate such interactions, whereas most prior work ignore the effects of latent factors underlying word occurrences. We propose a model to learn latent topics and discourse in word distributions, and predict pairwise initiation-response links via exploiting topic consistency and discourse dependency. Experimental results on both English and Chinese conversations show that our model significantly outperforms the previous state of the arts, such as 79 vs. 73 MRR on Chinese customer service dialogues. We further probe into our outputs and shed light on how topics and discourse indicate conversational user interactions.

cs.CL↗
arXiv2021

Forecasting Irreversible Disease via Progression Learning

Forecasting Parapapillary atrophy (PPA), i.e., a symptom related to most irreversible eye diseases, provides an alarm for implementing an intervention to slow down the disease progression at early stage. A key question for this forecast is: how to fully utilize the historical data (e.g., retinal image) up to the current stage for future disease prediction? In this paper, we provide an answer with a novel framework, namely \textbf{D}isease \textbf{F}orecast via \textbf{P}rogression \textbf{L}earning (\textbf{DFPL}), which exploits the irreversibility prior (i.e., cannot be reversed once diagnosed). Specifically, based on this prior, we decompose two factors that contribute to the prediction of the future disease: i) the current disease label given the data (retinal image, clinical attributes) at present and ii) the future disease label given the progression of the retinal images that from the current to the future. To model these two factors, we introduce the current and progression predictors in DFPL, respectively. In order to account for the degree of progression of the disease, we propose a temporal generative model to accurately generate the future image and compare it with the current one to get a residual image. The generative model is implemented by a recurrent neural network, in order to exploit the dependency of the historical data. To verify our approach, we apply it to a PPA in-house dataset and it yields a significant improvement (\textit{e.g.}, \textbf{4.48\%} of accuracy; \textbf{3.45\%} of AUC) over others. Besides, our generative model can accurately localize the disease-related regions.

cs.CV↗
arXiv2021

KAM theorem with large perturbation and application to network of Duffing oscillators

We prove that there is an invariant torus with given Diophantine frequency vector for a class of Hamiltonian systems defined by an integrable large Hamiltonian function with a large non-autonomous Hamiltonian perturbation. As for application, we prove that a finite network of Duffing oscillators with periodic exterior forces possesses Lagrangian stability for almost all initial data.

math.DS↗
arXiv2021

Causal Hidden Markov Model for Time Series Disease Forecasting

We propose a causal hidden Markov model to achieve robust prediction of irreversible disease at an early stage, which is safety-critical and vital for medical treatment in early stages. Specifically, we introduce the hidden variables which propagate to generate medical data at each time step. To avoid learning spurious correlation (e.g., confounding bias), we explicitly separate these hidden variables into three parts: a) the disease (clinical)-related part; b) the disease (non-clinical)-related part; c) others, with only a),b) causally related to the disease however c) may contain spurious correlations (with the disease) inherited from the data provided. With personal attributes and the disease label respectively provided as side information and supervision, we prove that these disease-related hidden variables can be disentangled from others, implying the avoidance of spurious correlation for generalization to medical data from other (out-of-) distributions. Guaranteed by this result, we propose a sequential variational auto-encoder with a reformulated objective function. We apply our model to the early prediction of peripapillary atrophy and achieve promising results on out-of-distribution test data. Further, the ablation study empirically shows the effectiveness of each component in our method. And the visualization shows the accurate identification of lesion regions from others.

cs.CV↗
arXiv2021

SuperDTI: Ultrafast diffusion tensor imaging and fiber tractography with deep learning

Purpose: To propose a deep learning-based reconstruction framework for ultrafast and robust diffusion tensor imaging and fiber tractography. Methods: We propose SuperDTI to learn the nonlinear relationship between diffusion-weighted images (DWIs) and the corresponding tensor-derived quantitative maps as well as the fiber tractography. Super DTI bypasses the tensor fitting procedure, which is well known to be highly susceptible to noise and motion in DWIs. The network is trained and tested using datasets from Human Connectome Project and patients with ischemic stroke. SuperDTI is compared against the state-of-the-art methods for diffusion map reconstruction and fiber tracking. Results: Using training and testing data both from the same protocol and scanner, SuperDTI is shown to generate fractional anisotropy and mean diffusivity maps, as well as fiber tractography, from as few as six raw DWIs. The method achieves a quantification error of less than 5% in all regions of interest in white matter and gray matter structures. We also demonstrate that the trained neural network is robust to noise and motion in the testing data, and the network trained using healthy volunteer data can be directly applied to stroke patient data without compromising the lesion detectability. Conclusion: This paper demonstrates the feasibility of superfast diffusion tensor imaging and fiber tractography using deep learning with as few as six DWIs directly, bypassing tensor fitting. Such a significant reduction in scan time may allow the inclusion of DTI into the clinical routine for many potential applications.

eess.IV↗
arXiv2021

Dilated SpineNet for Semantic Segmentation

Scale-permuted networks have shown promising results on object bounding box detection and instance segmentation. Scale permutation and cross-scale fusion of features enable the network to capture multi-scale semantics while preserving spatial resolution. In this work, we evaluate this meta-architecture design on semantic segmentation - another vision task that benefits from high spatial resolution and multi-scale feature fusion at different network stages. By further leveraging dilated convolution operations, we propose SpineNet-Seg, a network discovered by NAS that is searched from the DeepLabv3 system. SpineNet-Seg is designed with a better scale-permuted network topology with customized dilation ratios per block on a semantic segmentation task. SpineNet-Seg models outperform the DeepLabv3/v3+ baselines at all model scales on multiple popular benchmarks in speed and accuracy. In particular, our SpineNet-S143+ model achieves the new state-of-the-art on the popular Cityscapes benchmark at 83.04% mIoU and attained strong performance on the PASCAL VOC2012 benchmark at 85.56% mIoU. SpineNet-Seg models also show promising results on a challenging Street View segmentation dataset. Code and checkpoints will be open-sourced.

cs.CV↗
arXiv2021

Valley relaxation of resident electrons and holes in a monolayer semiconductor: Dependence on carrier density and the role of substrate-induced disorder

Using time-resolved optical Kerr rotation, we measure the low temperature valley dynamics of resident electrons and holes in exfoliated WSe$_2$ monolayers as a systematic function of carrier density. In an effort to reconcile the many disparate timescales of carrier valley dynamics in monolayer semiconductors reported to date, we directly compare the doping-dependent valley relaxation in two electrostatically-gated WSe$_2$ monolayers having different dielectric environments. In a fully-encapsulated structure (hBN/WSe$_2$/hBN, where hBN is hexagonal boron nitride), valley relaxation is found to be monoexponential. The valley relaxation time $τ_v$ is quite long ($\sim$10~$μ$s) at low carrier densities, but decreases rapidly to less than 100~ns at high electron or hole densities $\gtrsim$2 $\times 10^{12}$~cm$^{-2}$. In contrast, in a partially-encapsulated WSe$_2$ monolayer placed directly on silicon dioxide (hBN/WSe$_2$/SiO$_2$), carrier valley relaxation is multi-exponential at low carrier densities. The difference is attributed to environmental disorder from the SiO$_2$ substrate. Unexpectedly, very small out-of-plane magnetic fields can increase $τ_v$, especially in the hBN/WSe$_2$/SiO$_2$ structure, suggesting that localized states induced by disorder can play an important role in depolarizing spins and mediating the valley relaxation of resident carriers in monolayer transition metal-dichalcogenide semiconductors.

cond-mat.mtrl-sci↗