Searcharxiv⌕ Search

arXiv subjects

Chao Wang

Publications and source records attributed to Chao Wang.

At least 577 records · Page 32Linked to original sources
arXiv2020

Intra-Utterance Similarity Preserving Knowledge Distillation for Audio Tagging

Knowledge Distillation (KD) is a popular area of research for reducing the size of large models while still maintaining good performance. The outputs of larger teacher models are used to guide the training of smaller student models. Given the repetitive nature of acoustic events, we propose to leverage this information to regulate the KD training for Audio Tagging. This novel KD method, "Intra-Utterance Similarity Preserving KD" (IUSP), shows promising results for the audio tagging task. It is motivated by the previously published KD method: "Similarity Preserving KD" (SP). However, instead of preserving the pairwise similarities between inputs within a mini-batch, our method preserves the pairwise similarities between the frames of a single input utterance. Our proposed KD method, IUSP, shows consistent improvements over SP across student models of different sizes on the DCASE 2019 Task 5 dataset for audio tagging. There is a 27.1% to 122.4% percent increase in improvement of micro AUPRC over the baseline relative to SP's improvement of over the baseline.

eess.AS↗
arXiv2020

Tail risk forecasting using Bayesian realized EGARCH models

This paper develops a Bayesian framework for the realized exponential generalized autoregressive conditional heteroskedasticity (realized EGARCH) model, which can incorporate multiple realized volatility measures for the modelling of a return series. The realized EGARCH model is extended by adopting a standardized Student-t and a standardized skewed Student-t distribution for the return equation. Different types of realized measures, such as sub-sampled realized variance, sub-sampled realized range, and realized kernel, are considered in the paper. The Bayesian Markov chain Monte Carlo (MCMC) estimation employs the robust adaptive Metropolis algorithm (RAM) in the burn in period and the standard random walk Metropolis in the sample period. The Bayesian estimators show more favourable results than maximum likelihood estimators in a simulation study. We test the proposed models with several indices to forecast one-step-ahead Value at Risk (VaR) and Expected Shortfall (ES) over a period of 1000 days. Rigorous tail risk forecast evaluations show that the realized EGARCH models employing the standardized skewed Student-t distribution and incorporating sub-sampled realized range are favored, compared to a range of models.

q-fin.RM↗
arXiv2020

A Joint Framework for Audio Tagging and Weakly Supervised Acoustic Event Detection Using DenseNet with Global Average Pooling

This paper proposes a network architecture mainly designed for audio tagging, which can also be used for weakly supervised acoustic event detection (AED). The proposed network consists of a modified DenseNet as the feature extractor, and a global average pooling (GAP) layer to predict frame-level labels at inference time. This architecture is inspired by the work proposed by Zhou et al., a well-known framework using GAP to localize visual objects given image-level labels. While most of the previous works on weakly supervised AED used recurrent layers with attention-based mechanism to localize acoustic events, the proposed network directly localizes events using the feature map extracted by DenseNet without any recurrent layers. In the audio tagging task of DCASE 2017, our method significantly outperforms the state-of-the-art method in F1 score by 5.3% on the dev set, and 6.0% on the eval set in terms of absolute values. For weakly supervised AED task in DCASE 2018, our model outperforms the state-of-the-art method in event-based F1 by 8.1% on the dev set, and 0.5% on the eval set in terms of absolute values, by using data augmentation and tri-training to leverage unlabeled data.

eess.AS↗
arXiv2020

Fast Accurate Point of Care COVID-19 Pandemic Diagnosis Enabled Through Advanced Lab-on-a-Chip Optical Biosensors: Opportunities and Challenges

The sudden rise of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic early 2020 throughout the world has called into drastic action measures to do instant detection and reduce the spread rate. The common diagnostics testing methods has been only partially effective in satisfying the booming demand for fast detection methods to contain the further spread. However, the point-of-risk accurate diagnosis of this new emerging viral infection is paramount as simultaneous normal working operation and dealing with symptoms of SARS-CoV-2 can become the norm for years to come. Sensitive cost-effective biosensor with mass production capability is crucial throughout the world until a universal vaccination become available. Optical label-free biosensors can provide a non-invasive, extremely sensitive rapid detection technique up to ~1 fM concentration along with few minutes sensing. These biosensors can be manufactured on a mass-scale (billions) to detect the COVID-19 viral load in nasal, saliva, urinal, and serological samples even if the infected person is asymptotic. Methods investigated here are the most advanced available platforms for biosensing optical devices resulted from the integration of state-of-the-art designs and materials. These approaches are including but not limited to integrated optical devices, plasmonic resonance and also emerging nanomaterial biosensors. The lab-on-a-chip platforms examined here are suitable not only for SARS-CoV-2 spike protein detection but also other contagious virions such as influenza, and middle east respiratory syndrome (MERS).

physics.ins-det↗
arXiv2020

Semi-supervised ASR by End-to-end Self-training

While deep learning based end-to-end automatic speech recognition (ASR) systems have greatly simplified modeling pipelines, they suffer from the data sparsity issue. In this work, we propose a self-training method with an end-to-end system for semi-supervised ASR. Starting from a Connectionist Temporal Classification (CTC) system trained on the supervised data, we iteratively generate pseudo-labels on a mini-batch of unsupervised utterances with the current model, and use the pseudo-labels to augment the supervised data for immediate model update. Our method retains the simplicity of end-to-end ASR systems, and can be seen as performing alternating optimization over a well-defined learning objective. We also perform empirical investigations of our method, regarding the effect of data augmentation, decoding beamsize for pseudo-label generation, and freshness of pseudo-labels. On a commonly used semi-supervised ASR setting with the WSJ corpus, our method gives 14.4% relative WER improvement over a carefully-trained base system with data augmentation, reducing the performance gap between the base system and the oracle system by 50%.

eess.AS↗
arXiv2020

Data Techniques For Online End-to-end Speech Recognition

Practitioners often need to build ASR systems for new use cases in a short amount of time, given limited in-domain data. While recently developed end-to-end methods largely simplify the modeling pipelines, they still suffer from the data sparsity issue. In this work, we explore a few simple-to-implement techniques for building online ASR systems in an end-to-end fashion, with a small amount of transcribed data in the target domain. These techniques include data augmentation in the target domain, domain adaptation using models previously trained on a large source domain, and knowledge distillation on non-transcribed target domain data, using an adapted bi-directional model as the teacher; they are applicable in real scenarios with different types of resources. Our experiments demonstrate that each technique is independently useful in the improvement of the online ASR performance in the target domain.

eess.AS↗
arXiv2020

Deep Learning Based Equalizer for MIMO-OFDM Systems with Insufficient Cyclic Prefix

In this paper, we study the equalization design for multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) systems with insufficient cyclic prefix (CP). In particular, the signal detection performance is severely impaired by inter-carrier interference (ICI) and inter-symbol interference (ISI) when the multipath delay spread exceeding the length of CP. To tackle this problem, a deep learning-based equalizer is proposed for approximating the maximum likelihood detection. Inspired by the dependency between the adjacent subcarriers, a computationally efficient joint detection scheme is developed. Employing the proposed equalizer, an iterative receiver is also constructed and the detection performance is evaluated through simulations over measured multipath channels. Our results reveal that the proposed receiver can achieve significant performance improvement compared to two traditional baseline schemes.

cs.IT↗
arXiv2020

DiffRNN: Differential Verification of Recurrent Neural Networks

Recurrent neural networks (RNNs) such as Long Short Term Memory (LSTM) networks have become popular in a variety of applications such as image processing, data classification, speech recognition, and as controllers in autonomous systems. In practical settings, there is often a need to deploy such RNNs on resource-constrained platforms such as mobile phones or embedded devices. As the memory footprint and energy consumption of such components become a bottleneck, there is interest in compressing and optimizing such networks using a range of heuristic techniques. However, these techniques do not guarantee the safety of the optimized network, e.g., against adversarial inputs, or equivalence of the optimized and original networks. To address this problem, we propose DIFFRNN, the first differential verification method for RNNs to certify the equivalence of two structurally similar neural networks. Existing work on differential verification for ReLUbased feed-forward neural networks does not apply to RNNs where nonlinear activation functions such as Sigmoid and Tanh cannot be avoided. RNNs also pose unique challenges such as handling sequential inputs, complex feedback structures, and interactions between the gates and states. In DIFFRNN, we overcome these challenges by bounding nonlinear activation functions with linear constraints and then solving constrained optimization problems to compute tight bounding boxes on nonlinear surfaces in a high-dimensional space. The soundness of these bounding boxes is then proved using the dReal SMT solver. We demonstrate the practical efficacy of our technique on a variety of benchmarks and show that DIFFRNN outperforms state-of-the-art RNN verification tools such as POPQORN.

cs.LG↗
arXiv2020

Field-free Deterministic Magnetization Switching Induced by Interlaced Spin-Orbit Torques

Spin-orbit torque (SOT) based magnetic random access memory (MRAM) is envisioned as an emerging non-volatile memory due to its ultra-high speed and low power consumption. The field-free switching schema in SOT devices is of great interest to both academia and industry. Here we propose a novel field-free deterministic magnetization switching in a regular magnetic tunnel junction (MTJ) by using two currents sequentially passing interlaced paths, with less requirements of manufacturing process or additional physical effects. The switching is bipolar since the final magnetization state depends on the combination of current paths. The functionality and robustness of the proposed schema is validated through both macrospin and micromagnetic simulation. The influences of field-like torque and Dzyaloshinskii-Moriya interaction (DMI) effect are further researched. Our proposed schema shows good scalability and is expected to realize novel digital logic and even computing-in-memory platform.

physics.app-ph↗
arXiv2020

Will Dependency Conflicts Affect My Program's Semantics?

Java projects are often built on top of various third-party libraries. If multiple versions of a library exist on the classpath, JVM will only load one version and shadow the others, which we refer to as dependency conflicts. This would give rise to semantic conflict (SC) issues, if the library APIs referenced by a project have identical method signatures but inconsistent semantics across the loaded and shadowed versions of libraries. SC issues are difficult for developers to diagnose in practice, since understanding them typically requires domain knowledge. Although adapting the existing test generation technique for dependency conflict issues, Riddle, to detect SC issues is feasible, its effectiveness is greatly compromised. This is mainly because Riddle randomly generates test inputs, while the SC issues typically require specific arguments in the tests to be exposed. To address that, we conducted an empirical study of 75 real SC issues to understand the characteristics of such specific arguments in the test cases that can capture the SC issues. Inspired by our empirical findings, we propose an automated testing technique Sensor, which synthesizes test cases using ingredients from the project under test to trigger inconsistent behaviors of the APIs with the same signatures in conflicting library versions. Our evaluation results show that \textsc{Sensor} is effective and useful: it achieved a $Precision$ of 0.803 and a $Recall$ of 0.760 on open-source projects and a $Precision$ of 0.821 on industrial projects; it detected 150 semantic conflict issues in 29 projects, 81.8\% of which had been confirmed as real bugs.

cs.SE↗
arXiv2020

Cross-Channel Intragroup Sparsity Neural Network

Modern deep neural networks rely on overparameterization to achieve state-of-the-art generalization. But overparameterized models are computationally expensive. Network pruning is often employed to obtain less demanding models for deployment. Fine-grained pruning removes individual weights in parameter tensors and can achieve a high model compression ratio with little accuracy degradation. However, it introduces irregularity into the computing dataflow and often does not yield improved model inference efficiency in practice. Coarse-grained model pruning, while realizing satisfactory inference speedup through removal of network weights in groups, e.g. an entire filter, often lead to significant accuracy degradation. This work introduces the cross-channel intragroup (CCI) sparsity structure, which can prevent the inference inefficiency of fine-grained pruning while maintaining outstanding model performance. We then present a novel training algorithm designed to perform well under the constraint imposed by the CCI-Sparsity. Through a series of comparative experiments we show that our proposed CCI-Sparsity structure and the corresponding pruning algorithm outperform prior art in inference efficiency by a substantial margin given suited hardware acceleration in the future.

cs.LG↗
arXiv2020

Point Spread Function Engineering for 3D Imaging of Space Debris using a Continuous Exact l0 Penalty (CEL0) Based Algorithm

We consider three-dimensional (3D) localization and imaging of space debris from only one two-dimensional (2D) snapshot image. The technique involves an optical imager that exploits off-center image rotation to encode both the lateral and depth coordinates of point sources, with the latter being encoded in the angle of rotation of the PSF. We formulate 3D localization into a large-scale sparse 3D inverse problem in the discretized form. A recently developed penalty called continuous exact l0 (CEL0) is applied in this problem for the Gaussian noise model. Numerical experiments and comparisons illustrate the efficiency of the algorithm.

eess.IV↗
arXiv2020

Bootstrapping Named Entity Recognition in E-Commerce with Positive Unlabeled Learning

Named Entity Recognition (NER) in domains like e-commerce is an understudied problem due to the lack of annotated datasets. Recognizing novel entity types in this domain, such as products, components, and attributes, is challenging because of their linguistic complexity and the low coverage of existing knowledge resources. To address this problem, we present a bootstrapped positive-unlabeled learning algorithm that integrates domain-specific linguistic features to quickly and efficiently expand the seed dictionary. The model achieves an average F1 score of 72.02% on a novel dataset of product descriptions, an improvement of 3.63% over a baseline BiLSTM classifier, and in particular exhibits better recall (4.96% on average).

cs.CL↗
arXiv2020

An Artificial-intelligence/Statistics Solution to Quantify Material Distortion for Thermal Compensation in Additive Manufacturing

In this paper, we introduce a probabilistic statistics solution or artificial intelligence (AI) approach to identify and quantify permanent (non-zero strain) continuum/material deformation only based on the scanned material data in the spatial configuration and the shape of the initial design configuration or the material configuration. The challenge of this problem is that we only know the scanned material data in the spatial configuration and the shape of the design configuration of three-dimensional (3D) printed products, whereas for a specific scanned material point we do not know its corresponding material coordinates in the initial or designed referential configuration, provided that we do not know the detailed information on actual physical deformation process. Different from physics-based modeling, the method developed here is a data-driven artificial intelligence method, which solves the problem with incomplete deformation data or with missing information of actual physical deformation process. We coined the method is an AI-based material deformation finding algorithm. This method has practical significance and important applications in finding and designing thermal compensation configuration of a 3D printed product in additive manufacturing, which is at the heart of the cutting edge 3D printing technology. In this paper, we demonstrate that the proposed AI continuum/material deformation finding approach can accurately find permanent thermal deformation configuration for a complex 3D printed structure component, and hence to identify the thermal compensation design configuration in order to minimizing the impact of temperature fluctuations on 3D printed structure components that are sensitive to changes of temperature.

cs.CE↗
arXiv2020

LSHR-Net: a hardware-friendly solution for high-resolution computational imaging using a mixed-weights neural network

Recent work showed neural-network-based approaches to reconstructing images from compressively sensed measurements offer significant improvements in accuracy and signal compression. Such methods can dramatically boost the capability of computational imaging hardware. However, to date, there have been two major drawbacks: (1) the high-precision real-valued sensing patterns proposed in the majority of existing works can prove problematic when used with computational imaging hardware such as a digital micromirror sampling device and (2) the network structures for image reconstruction involve intensive computation, which is also not suitable for hardware deployment. To address these problems, we propose a novel hardware-friendly solution based on mixed-weights neural networks for computational imaging. In particular, learned binary-weight sensing patterns are tailored to the sampling device. Moreover, we proposed a recursive network structure for low-resolution image sampling and high-resolution reconstruction scheme. It reduces both the required number of measurements and reconstruction computation by operating convolution on small intermediate feature maps. The recursive structure further reduced the model size, making the network more computationally efficient when deployed with the hardware. Our method has been validated on benchmark datasets and achieved the state of the art reconstruction accuracy. We tested our proposed network in conjunction with a proof-of-concept hardware setup.

eess.IV↗
arXiv2020

Versatile security analysis of measurement-device-independent quantum key distribution

Measurement-device-independent quantum key distribution (MDI-QKD) is the only known QKD scheme that can completely overcome the problem of detection side-channel attacks. Yet, despite its practical importance, there is no standard approach towards proving the security of MDI-QKD. Here, we present a simple numerical method that can efficiently compute almost-tight security bounds for any discretely modulated MDI-QKD protocol. To demonstrate the broad utility of our method, we use it to analyze the security of coherent-state MDI-QKD, decoy-state MDI-QKD with leaky sources, and a variant of twin-field QKD called phase-matching QKD. In all of the numerical simulations (using realistic detection models) we find that our method gives significantly higher secret key rates than those obtained with current security proof techniques. Interestingly, we also find that phase-matching QKD using only two coherent test states is enough to overcome the fundamental rate-distance limit of QKD. Taken together, these findings suggest that our security proof method enables a versatile, fast, and possibly optimal approach towards the security validation of practical MDI-QKD systems.

quant-ph↗
arXiv2020

Improving Recommendation Diversity by Highlighting the ExTrA Fabricated Experts

Nowadays, recommender systems (RSes) are becoming increasingly important to individual users and business marketing, especially in the online e-commerce scenarios. However, while the majority of recommendation algorithms proposed in the literature have focused their efforts on improving prediction accuracy, other important aspects of recommendation quality, such as diversity of recommendations, have been more or less overlooked. In the latest decade, recommendation diversity has drawn more research attention, especially in the models based on user-item bipartite networks. In this paper, we introduce a family of approaches to extract fabricated experts from users in RSes, named as the Expert Tracking Approaches (ExTrA for short), and explore the capability of these fabricated experts in improving the recommendation diversity, by highlighting them in a well-known bipartite network-based method, called the Mass Diffusion (MD for short) model. These ExTrA-based models are compared with two state-of-the-art MD-improved models HHP and BHC, with respect to recommendation accuracy and diversity. Comprehensive empirical results on three real-world datasets MovieLens, Netflix and RYM show that, our proposed ExTrA-based models can achieve significant diversity gain while maintain comparable level of recommendation accuracy.

cs.SI↗