arXiv subjects

Jian Luo

Publications and source records attributed to Jian Luo.

At least 55 records · Page 3Linked to original sources

arXiv2022

Adaptive Activation Network For Low Resource Multilingual Speech Recognition

Low resource automatic speech recognition (ASR) is a useful but thorny task, since deep learning ASR models usually need huge amounts of training data. The existing models mostly established a bottleneck (BN) layer by pre-training on a large source language, and transferring to the low resource target language. In this work, we introduced an adaptive activation network to the upper layers of ASR model, and applied different activation functions to different languages. We also proposed two approaches to train the model: (1) cross-lingual learning, replacing the activation function from source language to target language, (2) multilingual learning, jointly training the Connectionist Temporal Classification (CTC) loss of each language and the relevance of different languages. Our experiments on IARPA Babel datasets demonstrated that our approaches outperform the from-scratch training and traditional bottleneck feature based methods. In addition, combining the cross-lingual learning and multilingual learning together could further improve the performance of multilingual speech recognition.

cs.CL↗

arXiv2022

Speech Augmentation Based Unsupervised Learning for Keyword Spotting

In this paper, we investigated a speech augmentation based unsupervised learning approach for keyword spotting (KWS) task. KWS is a useful speech application, yet also heavily depends on the labeled data. We designed a CNN-Attention architecture to conduct the KWS task. CNN layers focus on the local acoustic features, and attention layers model the long-time dependency. To improve the robustness of KWS model, we also proposed an unsupervised learning method. The unsupervised loss is based on the similarity between the original and augmented speech features, as well as the audio reconstructing information. Two speech augmentation methods are explored in the unsupervised learning: speed and intensity. The experiments on Google Speech Commands V2 Dataset demonstrated that our CNN-Attention model has competitive results. Moreover, the augmentation based unsupervised learning could further improve the classification accuracy of KWS task. In our experiments, with augmentation based unsupervised learning, our KWS model achieves better performance than other unsupervised methods, such as CPC, APC, and MPC.

cs.SD↗

arXiv2022

Atomic-scale origin of the low grain-boundary resistance in perovskite solid electrolytes

Oxide solid electrolytes (OSEs) have the potential to achieve improved safety and energy density for lithium-ion batteries, but their high grain-boundary (GB) resistance is a general bottleneck. In the most well studied perovskite OSE, Li3xLa2/3-xTiO3 (LLTO), the ionic conductivity of GBs is about three orders of magnitude lower than that of the bulk. In contrast, the related Li0.375Sr0.4375Ta0.75Zr0.25O3 (LSTZ0.75) perovskite exhibits low GB resistance for reasons yet unknown. Here, we used aberration-corrected scanning transmission electron microscopy and spectroscopy, along with an active learning moment tensor potential, to reveal the atomic scale structure and composition of LSTZ0.75 GBs. Vibrational electron energy loss spectroscopy is applied for the first time to characterize the otherwise unmeasurable Li distribution in GBs of LSTZ0.75. We found that Li depletion, which is a major reason for the low GB ionic conductivity of LLTO, is absent for the GBs of LSTZ0.75. Instead, the low GB resistivity of LSTZ0.75 is attributed to the formation of a unique defective cubic perovskite interfacial structure that contained abundant vacancies. Our study provides insights into the atomic scale mechanisms of low GB resistivity and sheds light on possible paths for designing OSEs with high total ionic conductivity.

cond-mat.mtrl-sci↗

arXiv2022

21-Component Compositionally Complex Ceramics: Discovery of Ultrahigh-Entropy Weberite and Fergusonite Phases and a Pyrochlore-Weberite Transition

Two new high-entropy ceramics (HECs) in the weberite and fergusonite structures, along with unexpected formation of ordered pyrochlore phases with ultrahigh-entropy compositions and an abrupt pyrochlore-weberite transition, are discovered in a 21-component oxide system. While the Gibbs phase rule allows 21 equilibrium phases, nine out of the 13 compositions examined possess single HEC phases (with ultrahigh ideal configurational entropies: ~2.7kB per cation or higher on one sublattice in most cases). Notably, (15RE1/15)(Nb1/2Ta1/2)O4 possess a single monoclinic fergusonite (C2/c) phase and (15RE1/15)3(Nb1/2Ta1/2)1O7 form a single orthorhombic (C2221) weberite phase, where 15RE1/15 represents Sc1/15Y1/15La1/15Pr1/15Nd1/15Sm1/15Eu1/15Gd1/15Tb1/15Dy1/15Ho1/15Er1/15Tm1/15Yb1/15Lu1/15. Moreover, a series of eight (15RE1/15)2+x(Ti1/4Zr1/4Ce1/4Hf1/4)2-2x(Nb1/2Ta1/2)xO7 specimens all exhibit single phases, where a pyrochlore-weberite transition occurs within 0.75 < x < 0.8125. This cubic-to-orthorhombic transition does not change the temperature-dependent thermal conductivity appreciably, as the amorphous limit may have already been achieved in the ultrahigh-entropy 21-component oxides. These discoveries expand the diversity and complexity of HECs, towards many-component compositionally complex ceramics (CCCs) and ultrahigh-entropy ceramics.

cond-mat.mtrl-sci↗

arXiv2022

Grain Boundary Segregation Transitions and Critical Phenomena in Binary Regular Solutions: A Systematics of Complexion Diagrams with Universal Characters

A systematics of grain boundary (GB) segregation transitions and critical phenomena has been derived to expand the classical GB segregation theory. Using twist GBs as an example, this study uncovers when GB layering vs. prewetting transitions should occur and how they are related to one another. Moreover, a novel descriptor, normalized segregation strength (phi_seg), is introduced. It can represent several factors that control GB segregation, including strain and bond energies, as well as misorientation for small-angle GBs (in a mean-field approximation), which had to be treated separately in prior models. In a strong segregation system with a large phi_seg, first-order layering transitions occur at low temperatures and become continuous above GB roughing temperatures. With reducing phi_seg, the layering transitions gradually merge and finally lump into prewetting transitions without quantized layer numbers, akin to Cahn's critical-point wetting model. Furthermore, GB complexion diagrams with universal characters are constructed as the GB counterpart to the classical exemplar of Pelton-Thompson regular-solution binary bulk phase diagrams.

cond-mat.mtrl-sci↗

arXiv2022

Computing Grain Boundary Diagrams of Thermodynamic and Mechanical Properties

Computing the grain boundary (GB) counterparts to bulk phase diagrams represents an emerging research direction. Using a classical embrittlement model system Ga-doped Al alloy, this study demonstrates the feasibility of computing temperature- and composition-dependent GB diagrams to represent not only equilibrium thermodynamic and structural characters, but also mechanical properties. Specifically, hybrid Monte Carlo and molecular dynamics (MC/MD) simulations are used to obtain the equilibrium GB structure as a function of temperature and composition. Simulated GB structures are validated by aberration-corrected scanning transmission electron microscopy. Subsequently, MD tensile tests are performed on the simulated equilibrium GB structures. GB diagrams are computed for not only GB adsorption and structural disorder, but also interfacial structural and chemical widths, MD ultimate tensile strength, and MD tensile toughness. This study suggests a research direction to investigate GB composition-structure-property relationships via computing GB diagrams of thermodynamic, structural, and mechanical (or potentially other) properties.

cond-mat.mtrl-sci↗

arXiv2021

A New Type of Compositionally Complex M5Si3 Silicides: Cation Ordering and Unexpected Phase Stability

A new type of compositionally complex (medium- or high-entropy) M5Si3 silicides is synthesized. Both (V1/5Cr1/5Nb1/5Ta1/5W1/5)5Si3 and (Ti1/5Zr1/5Nb1/5Mo1/5Hf1/5)5Si3 form single-phase homogenous solid solutions. Notably, (V1/5Cr1/5Nb1/5Ta1/5W1/5)5Si3 forms the hexagonal gamma (D88) phase, while all its five constituent binary silicides, V5Si3, Cr5Si3, Nb5Si3, Ta5Si3, and W5Si3, are stable in the tetragonal alpha (D8l) or beta (D8m) phases. Annealing at 1600C demonstrates that this hexagonal gamma phase is stable. Comparison of the experimental and calculated X-ray diffraction patterns, Rietveld refinements, and analysis of aberration-corrected scanning transmission electron microscopy high-angle annular dark-field images suggest cation ordering, which reduces the configurational entropy. This work expands the field of high-entropy and compositional complex ceramics by not only discovering a new compositional complex silicide phase but also demonstrating the cation ordering and unusual phase stability. These compositionally complex silicides can be combined with refractory high-entropy alloys to make the high-entropy counterparts to the Nb-silicide and Mo-Si-B composites.

cond-mat.mtrl-sci↗

arXiv2021

Discovery of a reversible redox-induced order-disorder transition in a 10-cation compositionally complex ceramic

This study discovers a reversible order-disorder transition (ODT) in a 10-cation compositionally complex ceramic, (Nd0.15Pr0.15Dy0.8Ho0.8Er0.8Ti0.2Yb0.1Hf0.1Zr0.1Nb0.8)O7-delta, induced via annealing in oxidized vs. reduced environments at 1600C. Notably, the 10-cation oxide remains a homogenous single-phase high-entropy solid solution before and after the ODT in the pyrochlore vs. fluorite structure that can be quenched. In-situ neutron diffraction reveals the oxygen vacancy formation and atomic displacement during this ODT. The temperature dependence of thermal conductivity is altered by the ODT. This study reveals a new pathway to induce ODT via a redox transition to tailor the properties of compositionally complex fluorite-based oxides.

cond-mat.mtrl-sci↗

arXiv2021

Emergence of near-boundary segregation zones in face-centered cubic multi-principal element alloys

Grain boundaries have been shown to dramatically influence the behavior of relatively simple materials such as monatomic metals and binary alloys. The increased chemical complexity associated with multi-principal element alloys is hypothesized to lead to new grain boundary phenomena. To explore the relationship between grain boundary structure and chemistry in these materials, hybrid molecular dynamics/Monte Carlo simulations of a faceted Σ11 <110> tilt boundary, chosen to sample both high- and low-energy boundary configurations, are performed in face-centered cubic CrFeCoNiCu and CrFeCoNi equiatomic alloys. Unexpected enrichment of Fe is discovered in the face-centered cubic regions adjacent to the interface and found to be correlated with a structurally-distinct region of reduced atomic volume. Comparison with the boundary of the same type in monatomic Cu demonstrates that altered near-boundary regions exist in simpler systems as well, with the chemical complexity of the multi-principal element alloys highlighting its existence and importance.

cond-mat.mtrl-sci↗

arXiv2021

Dropout Regularization for Self-Supervised Learning of Transformer Encoder Speech Representation

Predicting the altered acoustic frames is an effective way of self-supervised learning for speech representation. However, it is challenging to prevent the pretrained model from overfitting. In this paper, we proposed to introduce two dropout regularization methods into the pretraining of transformer encoder: (1) attention dropout, (2) layer dropout. Both of the two dropout methods encourage the model to utilize global speech information, and avoid just copying local spectrum features when reconstructing the masked frames. We evaluated the proposed methods on phoneme classification and speaker recognition tasks. The experiments demonstrate that our dropout approaches achieve competitive results, and improve the performance of classification accuracy on downstream tasks.

eess.AS↗

arXiv2021

Loss Prediction: End-to-End Active Learning Approach For Speech Recognition

End-to-end speech recognition systems usually require huge amounts of labeling resource, while annotating the speech data is complicated and expensive. Active learning is the solution by selecting the most valuable samples for annotation. In this paper, we proposed to use a predicted loss that estimates the uncertainty of the sample. The CTC (Connectionist Temporal Classification) and attention loss are informative for speech recognition since they are computed based on all decoding paths and alignments. We defined an end-to-end active learning pipeline, training an ASR/LP (Automatic Speech Recognition/Loss Prediction) joint model. The proposed approach was validated on an English and a Chinese speech recognition task. The experiments show that our approach achieves competitive results, outperforming random selection, least confidence, and estimated loss method.

eess.AS↗

arXiv2021

Accelerating Materials Discovery with Bayesian Optimization and Graph Deep Learning

Machine learning (ML) models utilizing structure-based features provide an efficient means for accurate property predictions across diverse chemical spaces. However, obtaining equilibrium crystal structures typically requires expensive density functional theory (DFT) calculations, which limits ML-based exploration to either known crystals or a small number of hypothetical crystals. Here, we demonstrate that the application of Bayesian optimization with symmetry constraints using a graph deep learning energy model can be used to perform "DFT-free" relaxations of crystal structures. Using this approach to significantly improve the accuracy of ML-predicted formation energies and elastic moduli of hypothetical crystals, two novel ultra-incompressible hard materials MoWC2 (P63/mmc) and ReWB (Pca21) were identified and successfully synthesized via in-situ reactive spark plasma sintering from a screening of 399,960 transition metal borides and carbides. This work addresses a critical bottleneck to accurate property predictions for hypothetical materials, paving the way to ML-accelerated discovery of new materials with exceptional properties.

cond-mat.mtrl-sci↗

arXiv2021

Decoding Grain Boundary Thermodynamics in High-Entropy Alloys in a 5D Space: Coupled Segregation and Disordering

Grain boundaries (GBs) can critically influence the microstructural evolution and various materials properties. However, a fundamental understanding of GBs in high-entropy alloys (HEAs) is lacking because of the complex couplings of the segregations of multiple elements and interfacial disordering, which can generate new phenomena and challenge the classical theories. Here, by combining large-scale atomistic simulations and machine learning, we demonstrate the feasibility of predicting the GB properties as functions of four independent compositional degrees of freedoms and temperature in a 5D space. Subsequently, GB counterparts to bulk phase diagrams are constructed for the first time for quinary HEAs. A data-driven discovery further reveals new coupled segregation and disordering effects in HEAs. Notably, an analysis of a large dataset discovers a critical compensation temperature at which the segregations of all elements virtually vanish simultaneously. While the machine learning model can predict GB properties via a black-box approach, a surrogate data-based analytical model (DBAM) is constructed to provide more physics insights and better transferability, with good accuracies. This study not only provides a new paradigm enabling prediction of GB properties in a 5D space, but also uncovers new GB segregation phenomena in HEAs beyond the classical GB segregation models.

cond-mat.mtrl-sci↗

arXiv2021

Bulk High-Entropy Hexaborides

For the first time, a group of CaB6-typed cubic rare earth high-entropy hexaborides have been successfully fabricated into dense bulk pellets (>98.5% in relative densities). The specimens are prepared from elemental precursors via in-situ metal-boron reactive spark plasma sintering. The sintered bulk pellets are determined to be single-phase without any detectable oxides or other secondary phases. The homogenous elemental distributions have been confirmed at both microscale and nanoscale. The Vickers microhardness are measured to be 16-18 GPa at a standard indentation load of 9.8 N. The nanoindentation hardness and Young's moduli have been measured to be 19-22 GPa and 190-250 GPa, respectively, by nanoindentation test using a maximum load of 500 mN. The material work functions are determined to be 3.7-4.0 eV by ultraviolet photoelectron spectroscopy characterizations, which are significantly higher than that of LaB6.

cond-mat.mtrl-sci↗

arXiv2021

Short-Range Order and Origin of the Low Thermal Conductivity in Compositionally Complex Rare-Earth Niobates and Tantalates

Rare-earth niobates and tantalates possess low thermal conductivities, which can be further reduced in high-entropy compositions. Here, a large number of 40 compositions are synthesized to investigate the origin of low thermal conductivity. Amongst, 29 possess single (nominally cubic) fluorite phases and most of them are new compositionally complex (medium- or high-entropy) compositions. One new finding is that doping 2 % of light element cations can further reduce thermal conductivity. This large data set enables the discovery of a negative correlation between the thermal conductivity and averaged radius ratio of the 3+/5+ cations. While this ratio is still below the threshold for forming long-range ordered weberite phases, this correlation suggests the reduced thermal conductivity is related to short-range weberite order, which is indeed revealed by diffuse scattering in X-ray and neutron diffraction. Specifically, neutron diffraction is used characterize five selected specimens. A better fit to a weberite structure is found at nanoscale (~1 nm). The characteristic length (domain size) is smaller but with stronger short-range ordering in more insulative materials. As it approaches the Ioffe-Regel limit, the phonon limit breaks down and "diffusons" give rise to the observed amorphous-like thermal conductivity. Disordered oxygen sublattices are also confirmed by neutron diffraction.

cond-mat.mtrl-sci↗

arXiv2021

Discovery of Electrochemically Induced Grain Boundary Transitions

Electric fields and currents, which are used in innovative materials processing and electrochemical energy conversion, can often alter microstructures in unexpected ways. However, little is known about the underlying mechanisms. Using ZnO-Bi2O3 as a model system, this study uncovers how an applied electric current can change the microstructural evolution through an electrochemically induced grain boundary (GB) transition. By combining aberration-corrected electron microscopy, photoluminescence spectroscopy, first-principles calculations, a generalizable thermodynamic model, and ab initio molecular dynamics, this study reveals that electrochemical reduction can cause a GB disorder-to-order transition to markedly increase GB diffusivities and mobilities. Consequently, abruptly enhanced or abnormal grain growth takes place. These findings advance our fundamental knowledge of GB complexion (phase-like) transitions and electric field effects on microstructural stability and evolution, with broad scientific and technological impacts. A new method to tailor the GB structures and properties, as well as the microstructures, electrochemically can also be envisioned.

cond-mat.mtrl-sci↗

arXiv2021

Unidirectional Memory-Self-Attention Transducer for Online Speech Recognition

Self-attention models have been successfully applied in end-to-end speech recognition systems, which greatly improve the performance of recognition accuracy. However, such attention-based models cannot be used in online speech recognition, because these models usually have to utilize a whole acoustic sequences as inputs. A common method is restricting the field of attention sights by a fixed left and right window, which makes the computation costs manageable yet also introduces performance degradation. In this paper, we propose Memory-Self-Attention (MSA), which adds history information into the Restricted-Self-Attention unit. MSA only needs localtime features as inputs, and efficiently models long temporal contexts by attending memory states. Meanwhile, recurrent neural network transducer (RNN-T) has proved to be a great approach for online ASR tasks, because the alignments of RNN-T are local and monotonic. We propose a novel network structure, called Memory-Self-Attention (MSA) Transducer. Both encoder and decoder of the MSA Transducer contain the proposed MSA unit. The experiments demonstrate that our proposed models improve WER results than Restricted-Self-Attention models by $13.5 on WSJ and $7.1 on SWBD datasets relatively, and without much computation costs increase.

eess.AS↗

arXiv2021

Low Latency Scheduling Algorithms for Full-Duplex V2X Networks

Vehicular communication systems have been an active subject of research for many years and are important technologies in the 5G and the post-5G era. One important use case is platooning which is seemingly the first step towards fully autonomous driving systems. Furthermore, a key performance parameter in all vehicular communication systems is the end-to-end packet latency. Towards this goal, full-duplex (FD) transceivers can potentially be an efficient and practical solution towards reducing the delay in platooning networks. In this paper, we study the delay performance of dynamic and TDMAbased scheduling algorithms and assess the effect of FD-enabled vehicles with imperfect self-interference cancellation (SIC). By simulations, we demonstrate the performance-complexity tradeoff of these algorithms and show that a TDMA centralized scheme with low-complexity and overhead can achieve comparable performance with a fully-dynamic, centralized algorithm.

cs.IT↗