arXiv subjects

Jichen Yang

Publications and source records attributed to Jichen Yang.

33 records · Page 2Linked to original sources

arXiv2023

Key-Sparse Transformer for Multimodal Speech Emotion Recognition

Speech emotion recognition is a challenging research topic that plays a critical role in human-computer interaction. Multimodal inputs further improve the performance as more emotional information is used. However, existing studies learn all the information in the sample while only a small portion of it is about emotion. The redundant information will become noises and limit the system performance. In this paper, a key-sparse Transformer is proposed for efficient emotion recognition by focusing more on emotion related information. The proposed method is evaluated on the IEMOCAP and LSSED. Experimental results show that the proposed method achieves better performance than the state-of-the-art approaches.

cs.SD↗

arXiv2022

Deep learning-based algorithm for assessment of knee osteoarthritis severity in radiographs matches performance of radiologists

A fully-automated deep learning algorithm matched performance of radiologists in assessment of knee osteoarthritis severity in radiographs using the Kellgren-Lawrence grading system. To develop an automated deep learning-based algorithm that jointly uses Posterior-Anterior (PA) and Lateral (LAT) views of knee radiographs to assess knee osteoarthritis severity according to the Kellgren-Lawrence grading system. We used a dataset of 9739 exams from 2802 patients from Multicenter Osteoarthritis Study (MOST). The dataset was divided into a training set of 2040 patients, a validation set of 259 patients and a test set of 503 patients. A novel deep learning-based method was utilized for assessment of knee OA in two steps: (1) localization of knee joints in the images, (2) classification according to the KL grading system. Our method used both PA and LAT views as the input to the model. The scores generated by the algorithm were compared to the grades provided in the MOST dataset for the entire test set as well as grades provided by 5 radiologists at our institution for a subset of the test set. The model obtained a multi-class accuracy of 71.90% on the entire test set when compared to the ratings provided in the MOST dataset. The quadratic weighted Kappa coefficient for this set was 0.9066. The average quadratic weighted Kappa between all pairs of radiologists from our institution who took a part of study was 0.748. The average quadratic-weighted Kappa between the algorithm and the radiologists at our institution was 0.769. The proposed model performed demonstrated equivalency of KL classification to MSK radiologists, but clearly superior reproducibility. Our model also agreed with radiologists at our institution to the same extent as the radiologists with each other. The algorithm could be used to provide reproducible assessment of knee osteoarthritis severity.

eess.IV↗

arXiv2022

Knee arthritis severity measurement using deep learning: a publicly available algorithm with a multi-institutional validation showing radiologist-level performance

The assessment of knee osteoarthritis (KOA) severity on knee X-rays is a central criteria for the use of total knee arthroplasty. However, this assessment suffers from imprecise standards and a remarkably high inter-reader variability. An algorithmic, automated assessment of KOA severity could improve overall outcomes of knee replacement procedures by increasing the appropriateness of its use. We propose a novel deep learning-based five-step algorithm to automatically grade KOA from posterior-anterior (PA) views of radiographs: (1) image preprocessing (2) localization of knees joints in the image using the YOLO v3-Tiny model, (3) initial assessment of the severity of osteoarthritis using a convolutional neural network-based classifier, (4) segmentation of the joints and calculation of the joint space narrowing (JSN), and (5), a combination of the JSN and the initial assessment to determine a final Kellgren-Lawrence (KL) score. Furthermore, by displaying the segmentation masks used to make the assessment, our algorithm demonstrates a higher degree of transparency compared to typical "black box" deep learning classifiers. We perform a comprehensive evaluation using two public datasets and one dataset from our institution, and show that our algorithm reaches state-of-the art performance. Moreover, we also collected ratings from multiple radiologists at our institution and showed that our algorithm performs at the radiologist level. The software has been made publicly available at https://github.com/MaciejMazurowski/osteoarthritis-classification.

eess.IV↗

arXiv2021

Geophysical fluid models with simple energy backscatter: explicit flows and unbounded exponential growth

Motivated by numerical schemes for large scale geophysical flow, we consider the rotating shallow water and Boussinesq equations on the whole space with horizontal kinetic energy backscatter source terms built from negative viscosity and stabilising hyperviscosity with constant parameters. We study the impact of this energy input through various explicit flows, which are simultaneously solving the nonlinear equations and the linear equations that arise upon dropping the transport nonlinearity, i.e. the linearisation in the zero state. These include barotropic, parallel and Kolmogorov flows as well as monochromatic inertia gravity waves. With focus on stable stratification we find that the backscatter generates numerous solutions of this type that grow exponentially and unboundedly, also with vertical structure. This signifies the possibility of undesired energy concentration into specific modes due to the backscatter. Families of steady state flows of this type arise as well and superposition principles in the nonlinear equations provide explicit sufficient conditions for instability of some of these. For certain steady barotropic flows of this type we provide numerical evidence of eigenmodes whose growth rates are proportional to the amplitude factor of the flow. For all other arising steady solutions we prove this is not possible.

physics.flu-dyn↗

arXiv2021

Data Augmentation with Signal Companding for Detection of Logical Access Attacks

The recent advances in voice conversion (VC) and text-to-speech (TTS) make it possible to produce natural sounding speech that poses threat to automatic speaker verification (ASV) systems. To this end, research on spoofing countermeasures has gained attention to protect ASV systems from such attacks. While the advanced spoofing countermeasures are able to detect known nature of spoofing attacks, they are not that effective under unknown attacks. In this work, we propose a novel data augmentation technique using a-law and mu-law based signal companding. We believe that the proposed method has an edge over traditional data augmentation by adding small perturbation or quantization noise. The studies are conducted on ASVspoof 2019 logical access corpus using light convolutional neural network based system. We find that the proposed data augmentation technique based on signal companding outperforms the state-of-the-art spoofing countermeasures showing ability to handle unknown nature of attacks.

eess.AS↗

arXiv2021

Reaction-subdiffusion systems and memory: spectra, Turing instability and decay estimates

The modelling of linear and nonlinear reaction-subdiffusion processes is more subtle than normal diffusion and causes different phenomena. The resulting equations feature a spatial Laplacian with a temporal memory term through a time fractional derivative. It is known that the precise form depends on the interaction of dispersal and reaction, and leads to qualitative differences. We refine these results by defining generalised spectra through dispersion relations, which allows us to examine the onset of instability and in particular inspect Turing type instabilities. These results are numerically illustrated. Moreover, we prove expansions that imply for one class of subdiffusion reaction equations algebraic decay for stable spectrum, whereas for another class this is exponential.

math.AP↗

arXiv2021

GridTracer: Automatic Mapping of Power Grids using Deep Learning and Overhead Imagery

Energy system information valuable for electricity access planning such as the locations and connectivity of electricity transmission and distribution towers, termed the power grid, is often incomplete, outdated, or altogether unavailable. Furthermore, conventional means for collecting this information is costly and limited. We propose to automatically map the grid in overhead remotely sensed imagery using deep learning. Towards this goal, we develop and publicly-release a large dataset ($263km^2$) of overhead imagery with ground truth for the power grid, to our knowledge this is the first dataset of its kind in the public domain. Additionally, we propose scoring metrics and baseline algorithms for two grid mapping tasks: (1) tower recognition and (2) power line interconnection (i.e., estimating a graph representation of the grid). We hope the availability of the training data, scoring metrics, and baselines will facilitate rapid progress on this important problem to help decision-makers address the energy needs of societies around the world.

cs.CV↗

arXiv2020

HLT-NUS Submission for NIST 2019 Multimedia Speaker Recognition Evaluation

This work describes the speaker verification system developed by Human Language Technology Laboratory, National University of Singapore (HLT-NUS) for 2019 NIST Multimedia Speaker Recognition Evaluation (SRE). The multimedia research has gained attention to a wide range of applications and speaker recognition is no exception to it. In contrast to the previous NIST SREs, the latest edition focuses on a multimedia track to recognize speakers with both audio and visual information. We developed separate systems for audio and visual inputs followed by a score level fusion of the systems from the two modalities to collectively use their information. The audio systems are based on x-vector based speaker embedding, whereas the face recognition systems are based on ResNet and InsightFace based face embeddings. With post evaluation studies and refinements, we obtain an equal error rate (EER) of 0.88% and an actual detection cost function (actDCF) of 0.026 on the evaluation set of 2019 NIST multimedia SRE corpus.

eess.AS↗

arXiv2020

Light Convolutional Neural Network with Feature Genuinization for Detection of Synthetic Speech Attacks

Modern text-to-speech (TTS) and voice conversion (VC) systems produce natural sounding speech that questions the security of automatic speaker verification (ASV). This makes detection of such synthetic speech very important to safeguard ASV systems from unauthorized access. Most of the existing spoofing countermeasures perform well when the nature of the attacks is made known to the system during training. However, their performance degrades in face of unseen nature of attacks. In comparison to the synthetic speech created by a wide range of TTS and VC methods, genuine speech has a more consistent distribution. We believe that the difference between the distribution of synthetic and genuine speech is an important discriminative feature between the two classes. In this regard, we propose a novel method referred to as feature genuinization that learns a transformer with convolutional neural network (CNN) using the characteristics of only genuine speech. We then use this genuinization transformer with a light CNN classifier. The ASVspoof 2019 logical access corpus is used to evaluate the proposed method. The studies show that the proposed feature genuinization based LCNN system outperforms other state-of-the-art spoofing countermeasures, depicting its effectiveness for detection of synthetic speech attacks.

eess.AS↗

arXiv2020

Multi-modal Attention for Speech Emotion Recognition

Emotion represents an essential aspect of human speech that is manifested in speech prosody. Speech, visual, and textual cues are complementary in human communication. In this paper, we study a hybrid fusion method, referred to as multi-modal attention network (MMAN) to make use of visual and textual cues in speech emotion recognition. We propose a novel multi-modal attention mechanism, cLSTM-MMA, which facilitates the attention across three modalities and selectively fuse the information. cLSTM-MMA is fused with other uni-modal sub-networks in the late fusion. The experiments show that speech emotion recognition benefits significantly from visual and textual cues, and the proposed cLSTM-MMA alone is as competitive as other fusion methods in terms of accuracy, but with a much more compact network structure. The proposed hybrid network MMAN achieves state-of-the-art performance on IEMOCAP database for emotion recognition.

eess.AS↗

arXiv2020

The impact of advection on the stability of stripes on lattices near planar Turing instabilities

Striped patterns are known to bifurcate in reaction-diffusion systems with differential isotropic diffusions at a supercritical Turing instability. In this paper we study the impact of weak anisotropy by directional advection on the stability of stripes with respect to various lattice modes, and the role of quadratic terms therein. We focus on the generic form of planar reaction-diffusion systems with two components near such a bifurcation. Using centre manifold reduction we derive a rigorous parameter expansion for the critical eigenvalues for lattice mode perturbations, specifically nearly square and nearly hexagonal ones. This provides detailed formulae for the loci of stability boundaries under the influence of the advection and quadratic terms. In particular, the well known destabilising effect of quadratic terms can be counterbalanced by advection, which leads to intriguing arrangements of stability boundaries. We illustrate these results numerically by a specific example. Finally, we show numerical computations of these stability boundaries in the extended Klausmeier model for vegetation patterns and show stripes bifurcate stably in the presence of sufficiently strong advection.

math.AP↗

arXiv2020

PDA: Progressive Data Augmentation for General Robustness of Deep Neural Networks

Adversarial images are designed to mislead deep neural networks (DNNs), attracting great attention in recent years. Although several defense strategies achieved encouraging robustness against adversarial samples, most of them fail to improve the robustness on common corruptions such as noise, blur, and weather/digital effects (e.g. frost, pixelate). To address this problem, we propose a simple yet effective method, named Progressive Data Augmentation (PDA), which enables general robustness of DNNs by progressively injecting diverse adversarial noises during training. In other words, DNNs trained with PDA are able to obtain more robustness against both adversarial attacks as well as common corruptions than the recent state-of-the-art methods. We also find that PDA is more efficient than prior arts and able to prevent accuracy drop on clean samples without being attacked. Furthermore, we theoretically show that PDA can control the perturbation bound and guarantee better generalization ability than existing work. Extensive experiments on many benchmarks such as CIFAR-10, SVHN, and ImageNet demonstrate that PDA significantly outperforms its counterparts in various experimental setups.

cs.LG↗

arXiv2019

The impact of advection on large-wavelength stability of stripes near planar Turing instabilities

It is well known that for reaction-diffusion systems with differential isotropic diffusions, a Turing instability yields striped solutions. In this paper we study the impact of weak anisotropy by directional advection on such solutions, and the role of quadratic terms. We focus on the generic form of planar reaction-diffusion systems with two components near such a bifurcation. Using Lyapunov-Schmidt reduction and Floquet-Bloch decomposition we derive a rigorous parameter expansion for existence and stability against large wavelength perturbations. This provides detailed formulae for the loci of bifurcations and so-called Eckhaus as well as zigzag stability boundaries under the influence of the advection and quadratic terms. In particular, while destabilisation of the background state is through modes perpendicular to the advection (Squire-theorem), we show that stripes can bifurcate zigzag unstably. We illustrate these results numerically by an example. Finally, we show numerical computations of these stability boundaries in the extended Klausmeier model for vegetation patterns and show stripes bifurcate stably in the presence of advection.

math.AP↗

arXiv2019

I4U Submission to NIST SRE 2018: Leveraging from a Decade of Shared Experiences

The I4U consortium was established to facilitate a joint entry to NIST speaker recognition evaluations (SRE). The latest edition of such joint submission was in SRE 2018, in which the I4U submission was among the best-performing systems. SRE'18 also marks the 10-year anniversary of I4U consortium into NIST SRE series of evaluation. The primary objective of the current paper is to summarize the results and lessons learned based on the twelve sub-systems and their fusion submitted to SRE'18. It is also our intention to present a shared view on the advancements, progresses, and major paradigm shifts that we have witnessed as an SRE participant in the past decade from SRE'08 to SRE'18. In this regard, we have seen, among others, a paradigm shift from supervector representation to deep speaker embedding, and a switch of research challenge from channel compensation to domain adaptation.

eess.AS↗

arXiv2018

Generative x-vectors for text-independent speaker verification

Speaker verification (SV) systems using deep neural network embeddings, so-called the x-vector systems, are becoming popular due to its good performance superior to the i-vector systems. The fusion of these systems provides improved performance benefiting both from the discriminatively trained x-vectors and generative i-vectors capturing distinct speaker characteristics. In this paper, we propose a novel method to include the complementary information of i-vector and x-vector, that is called generative x-vector. The generative x-vector utilizes a transformation model learned from the i-vector and x-vector representations of the background data. Canonical correlation analysis is applied to derive this transformation model, which is later used to transform the standard x-vectors of the enrollment and test segments to the corresponding generative x-vectors. The SV experiments performed on the NIST SRE 2010 dataset demonstrate that the system using generative x-vectors provides considerably better performance than the baseline i-vector and x-vector systems. Furthermore, the generative x-vectors outperform the fusion of i-vector and x-vector systems for long-duration utterances, while yielding comparable results for short-duration utterances.

eess.AS↗