arXiv subjects

Yi Pan

Publications and source records attributed to Yi Pan.

At least 109 records · Page 6Linked to original sources

arXiv2021

Infant Cry Classification with Graph Convolutional Networks

We propose an approach of graph convolutional networks for robust infant cry classification. We construct non-fully connected graphs based on the similarities among the relevant nodes in both supervised and semi-supervised node classification with convolutional neural networks to consider the short-term and long-term effects of infant cry signals related to inner-class and inter-class messages. The approach captures the diversity of variations within infant cries, especially for limited training samples. The effectiveness of this approach is evaluated on Baby Chillanto Database and Baby2020 database. With as limited as 20% of labeled training data, our model outperforms that of CNN model with 80% labeled training data and the accuracy stably improves as the number of labeled training samples increases. The best results give significant improvements of 7.36% and 3.59% compared with the results of the CNN models on Baby Chillanto database and Baby2020 database respectively.

eess.AS↗

arXiv2020

Acquisition and Analysis of Scanning Tunneling Spectroscopy Data $-$ WSe$_2$ Monolayer

Acquisition and analysis is described for scanning tunneling spectroscopy data acquired from a monolayer of WSe$_2$ grown on epitaxial graphene on SiC. Curve fitting of the data is performed, in order to deduce band-edge energies. In addition to describing the details of the theoretical curves used for the fitting, the acquisition and analysis methods are also discussed within the larger context of the historical development of scanning tunneling spectroscopy techniques.

cond-mat.mes-hall↗

arXiv2020

PK-GCN: Prior Knowledge Assisted Image Classification using Graph Convolution Networks

Deep learning has gained great success in various classification tasks. Typically, deep learning models learn underlying features directly from data, and no underlying relationship between classes are included. Similarity between classes can influence the performance of classification. In this article, we propose a method that incorporates class similarity knowledge into convolutional neural networks models using a graph convolution layer. We evaluate our method on two benchmark image datasets: MNIST and CIFAR10, and analyze the results on different data and model sizes. Experimental results show that our model can improve classification accuracy, especially when the amount of available data is small.

cs.CV↗

arXiv2020

Deep Neural Networks with Short Circuits for Improved Gradient Learning

Deep neural networks have achieved great success both in computer vision and natural language processing tasks. However, mostly state-of-art methods highly rely on external training or computing to improve the performance. To alleviate the external reliance, we proposed a gradient enhancement approach, conducted by the short circuit neural connections, to improve the gradient learning of deep neural networks. The proposed short circuit is a unidirectional connection that single back propagates the sensitive from the deep layer to the shallows. Moreover, the short circuit formulates to be a gradient truncation of its crossing layers which can plug into the backbone deep neural networks without introducing external training parameters. Extensive experiments demonstrate deep neural networks with our short circuit gain a large margin over the baselines on both computer vision and natural language processing tasks.

cs.LG↗

arXiv2020

Graph Convolution Networks Using Message Passing and Multi-Source Similarity Features for Predicting circRNA-Disease Association

Graphs can be used to effectively represent complex data structures. Learning these irregular data in graphs is challenging and still suffers from shallow learning. Applying deep learning on graphs has recently showed good performance in many applications in social analysis, bioinformatics etc. A message passing graph convolution network is such a powerful method which has expressive power to learn graph structures. Meanwhile, circRNA is a type of non-coding RNA which plays a critical role in human diseases. Identifying the associations between circRNAs and diseases is important to diagnosis and treatment of complex diseases. However, there are limited number of known associations between them and conducting biological experiments to identify new associations is time consuming and expensive. As a result, there is a need of building efficient and feasible computation methods to predict potential circRNA-disease associations. In this paper, we propose a novel graph convolution network framework to learn features from a graph built with multi-source similarity information to predict circRNA-disease associations. First we use multi-source information of circRNA similarity, disease and circRNA Gaussian Interaction Profile (GIP) kernel similarity to extract the features using first graph convolution. Then we predict disease associations for each circRNA with second graph convolution. Proposed framework with five-fold cross validation on various experiments shows promising results in predicting circRNA-disease association and outperforms other existing methods.

cs.LG↗

arXiv2020

A Novel Ensemble Deep Learning Model for Stock Prediction Based on Stock Prices and News

In recent years, machine learning and deep learning have become popular methods for financial data analysis, including financial textual data, numerical data, and graphical data. This paper proposes to use sentiment analysis to extract useful information from multiple textual data sources and a blending ensemble deep learning model to predict future stock movement. The blending ensemble model contains two levels. The first level contains two Recurrent Neural Networks (RNNs), one Long-Short Term Memory network (LSTM) and one Gated Recurrent Units network (GRU), followed by a fully connected neural network as the second level model. The RNNs, LSTM, and GRU models can effectively capture the time-series events in the input data, and the fully connected neural network is used to ensemble several individual prediction results to further improve the prediction accuracy. The purpose of this work is to explain our design philosophy and show that ensemble deep learning technologies can truly predict future stock price trends more effectively and can better assist investors in making the right investment decision than other traditional methods.

q-fin.ST↗

arXiv2020

Improved Detection of Adversarial Images Using Deep Neural Networks

Machine learning techniques are immensely deployed in both industry and academy. Recent studies indicate that machine learning models used for classification tasks are vulnerable to adversarial examples, which limits the usage of applications in the fields with high precision requirements. We propose a new approach called Feature Map Denoising to detect the adversarial inputs and show the performance of detection on the mixed dataset consisting of adversarial examples generated by different attack algorithms, which can be used to associate with any pre-trained DNNs at a low cost. Wiener filter is also introduced as the denoise algorithm to the defense model, which can further improve performance. Experimental results indicate that good accuracy of detecting the adversarial examples can be achieved through our Feature Map Denoising algorithm.

cs.CV↗

arXiv2020

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

Convolutional Neural Networks (CNN) have gained great success in many artificial intelligence tasks. However, finding a good set of hyperparameters for a CNN remains a challenging task. It usually takes an expert with deep knowledge, and trials and errors. Genetic algorithms have been used in hyperparameter optimizations. However, traditional genetic algorithms with fixed-length chromosomes may not be a good fit for optimizing deep learning hyperparameters, because deep learning models have variable number of hyperparameters depending on the model depth. As the depth increases, the number of hyperparameters grows exponentially, and searching becomes exponentially harder. It is important to have an efficient algorithm that can find a good model in reasonable time. In this article, we propose to use a variable length genetic algorithm (GA) to systematically and automatically tune the hyperparameters of a CNN to improve its performance. Experimental results show that our algorithm can find good CNN hyperparameters efficiently. It is clear from our experiments that if more time is spent on optimizing the hyperparameters, better results could be achieved. Theoretically, if we had unlimited time and CPU power, we could find the optimized hyperparameters and achieve the best results in the future.

cs.NE↗

arXiv2020

Gradient Amplification: An efficient way to train deep neural networks

Improving performance of deep learning models and reducing their training times are ongoing challenges in deep neural networks. There are several approaches proposed to address these challenges one of which is to increase the depth of the neural networks. Such deeper networks not only increase training times, but also suffer from vanishing gradients problem while training. In this work, we propose gradient amplification approach for training deep learning models to prevent vanishing gradients and also develop a training strategy to enable or disable gradient amplification method across several epochs with different learning rates. We perform experiments on VGG-19 and resnet (Resnet-18 and Resnet-34) models, and study the impact of amplification parameters on these models in detail. Our proposed approach improves performance of these deep learning models even at higher learning rates, thereby allowing these models to achieve higher performance with reduced training time.

cs.LG↗

arXiv2020

Flatbands and Mechanical Deformation Effects in the Moiré Superlattice of MoS$_2$-WSe$_2$ Heterobilayers

It has recently been shown that quantum-confined states can appear in epitaxially grown van der Waals material heterobilayers without a rotational misalignment ($θ=0^\circ$), associated with flat bands in the Brillouin zone of the moiré pattern formed due to the lattice mismatch of the two layers. Peaks in the local density of states and confinement in a MoS$_2$/WSe$_2$ system was qualitatively described only considering local stacking arrangements, which cause band edge energies to vary spatially. In this work, we report the presence of large in-plane strain variation across the moiré unit cell of a $θ=0^\circ$ MoS$_2$/WSe$_2$ heterobilayer, and show that inclusion of strain variation and out-of-plane displacement in density functional theory calculations greatly improves their agreement with the experimental data. We further explore the role of twist-angle by showing experimental data for a twisted MoS$_2$/WSe$_2$ heterobilayer structure with twist angle of $θ=15^\circ$, that exhibits a moiré pattern but no confinement.

cond-mat.mtrl-sci↗

arXiv2018

Advancing the State of the Art in Open Domain Dialog Systems through the Alexa Prize

Building open domain conversational systems that allow users to have engaging conversations on topics of their choice is a challenging task. Alexa Prize was launched in 2016 to tackle the problem of achieving natural, sustained, coherent and engaging open-domain dialogs. In the second iteration of the competition in 2018, university teams advanced the state of the art by using context in dialog models, leveraging knowledge graphs for language understanding, handling complex utterances, building statistical and hierarchical dialog managers, and leveraging model-driven signals from user responses. The 2018 competition also included the provision of a suite of tools and models to the competitors including the CoBot (conversational bot) toolkit, topic and dialog act detection models, conversation evaluators, and a sensitive content detection model so that the competing teams could focus on building knowledge-rich, coherent and engaging multi-turn dialog systems. This paper outlines the advances developed by the university teams as well as the Alexa Prize team to achieve the common goal of advancing the science of Conversational AI. We address several key open-ended problems such as conversational speech recognition, open domain natural language understanding, commonsense reasoning, statistical dialog management, and dialog evaluation. These collaborative efforts have driven improved experiences by Alexa users to an average rating of 3.61, the median duration of 2 mins 18 seconds, and average turns to 14.6, increases of 14%, 92%, 54% respectively since the launch of the 2018 competition. For conversational speech recognition, we have improved our relative Word Error Rate by 55% and our relative Entity Error Rate by 34% since the launch of the Alexa Prize. Socialbots improved in quality significantly more rapidly in 2018, in part due to the release of the CoBot toolkit.

cs.CL↗

arXiv2018

Reconstruction of Hidden Representation for Robust Feature Extraction

This paper aims to develop a new and robust approach to feature representation. Motivated by the success of Auto-Encoders, we first theoretical summarize the general properties of all algorithms that are based on traditional Auto-Encoders: 1) The reconstruction error of the input can not be lower than a lower bound, which can be viewed as a guiding principle for reconstructing the input. Additionally, when the input is corrupted with noises, the reconstruction error of the corrupted input also can not be lower than a lower bound. 2) The reconstruction of a hidden representation achieving its ideal situation is the necessary condition for the reconstruction of the input to reach the ideal state. 3) Minimizing the Frobenius norm of the Jacobian matrix of the hidden representation has a deficiency and may result in a much worse local optimum value. We believe that minimizing the reconstruction error of the hidden representation is more robust than minimizing the Frobenius norm of the Jacobian matrix of the hidden representation. Based on the above analysis, we propose a new model termed Double Denoising Auto-Encoders (DDAEs), which uses corruption and reconstruction on both the input and the hidden representation. We demonstrate that the proposed model is highly flexible and extensible and has a potentially better capability to learn invariant and robust feature representations. We also show that our model is more robust than Denoising Auto-Encoders (DAEs) for dealing with noises or inessential features. Furthermore, we detail how to train DDAEs with two different pre-training methods by optimizing the objective function in a combined and separate manner, respectively. Comparative experiments illustrate that the proposed model is significantly better for representation learning than the state-of-the-art models.

cs.LG↗

arXiv2018

Quantum-Confined Electronic States arising from Moiré Pattern of MoS2-WSe2 Hetero-bilayers

A two-dimensional (2D) hetero-bilayer system consisting of MoS2 on WSe2, deposited on epitaxial graphene, is studied by scanning tunneling microscopy and spectroscopy at temperatures of 5 and 80 K. A moiré pattern is observed, arising from lattice mismatch of 3.7% between the MoS2 and WSe2. Significant energy shifts are observed in tunneling spectra observed at the maxima of the moiré corrugation, as compared with spectra obtained at corrugation minima, consistent with prior work. Furthermore, at the minima of the moiré corrugation, sharp peaks in the spectra at energies near the band edges are observed, for spectra acquired at 5 K. The peaks correspond to discrete states that are confined within the moiré unit cells. Conductance mapping is employed to reveal the detailed structure of the wave functions of the states. For measurements at 80 K, the sharp peaks in the spectra are absent, and conductance maps of the band edges reveal little structure.

cond-mat.mes-hall↗

arXiv2018

Conversational AI: The Science Behind the Alexa Prize

Conversational agents are exploding in popularity. However, much work remains in the area of social conversation as well as free-form conversation over a broad range of domains and topics. To advance the state of the art in conversational AI, Amazon launched the Alexa Prize, a 2.5-million-dollar university competition where sixteen selected university teams were challenged to build conversational agents, known as socialbots, to converse coherently and engagingly with humans on popular topics such as Sports, Politics, Entertainment, Fashion and Technology for 20 minutes. The Alexa Prize offers the academic community a unique opportunity to perform research with a live system used by millions of users. The competition provided university teams with real user conversational data at scale, along with the user-provided ratings and feedback augmented with annotations by the Alexa team. This enabled teams to effectively iterate and make improvements throughout the competition while being evaluated in real-time through live user interactions. To build their socialbots, university teams combined state-of-the-art techniques with novel strategies in the areas of Natural Language Understanding, Context Modeling, Dialog Management, Response Generation, and Knowledge Acquisition. To support the efforts of participating teams, the Alexa Prize team made significant scientific and engineering investments to build and improve Conversational Speech Recognition, Topic Tracking, Dialog Evaluation, Voice User Experience, and tools for traffic management and scalability. This paper outlines the advances created by the university teams as well as the Alexa Prize team to achieve the common goal of solving the problem of Conversational AI.

cs.AI↗

arXiv2017

Three-Stream Convolutional Networks for Video-based Person Re-Identification

This paper aims to develop a new architecture that can make full use of the feature maps of convolutional networks. To this end, we study a number of methods for video-based person re-identification and make the following findings: 1) Max-pooling only focuses on the maximum value of a receptive field, wasting a lot of information. 2) Networks with different streams even including the one with the worst performance work better than networks with same streams, where each one has the best performance alone. 3) A full connection layer at the end of convolutional networks is not necessary. Based on these studies, we propose a new convolutional architecture termed Three-Stream Convolutional Networks (TSCN). It first uses different streams to learn different aspects of feature maps for attentive spatio-temporal fusion of video, and then merges them together to study some union features. To further utilize the feature maps, two architectures are designed by using the strategies of multi-scale and upsampling. Comparative experiments on iLIDS-VID, PRID-2011 and MARS datasets illustrate that the proposed architectures are significantly better for feature extraction than the state-of-the-art models.

cs.CV↗

arXiv2016

Parallel Large-Scale Attribute Reduction on Cloud Systems

The rapid growth of emerging information technologies and application patterns in modern society, e.g., Internet, Internet of Things, Cloud Computing and Tri-network Convergence, has caused the advent of the era of big data. Big data contains huge values, however, mining knowledge from big data is a tremendously challenging task because of data uncertainty and inconsistency. Attribute reduction (also known as feature selection) can not only be used as an effective preprocessing step, but also exploits the data redundancy to reduce the uncertainty. However, existing solutions are designed 1) either for a single machine that means the entire data must fit in the main memory and the parallelism is limited; 2) or for the Hadoop platform which means that the data have to be loaded into the distributed memory frequently and therefore become inefficient. In this paper, we overcome these shortcomings for maximum efficiency possible, and propose a unified framework for Parallel Large-scale Attribute Reduction, termed PLAR, for big data analysis. PLAR consists of three components: 1) Granular Computing (GrC)-based initialization: it converts a decision table (i.e., original data representation) into a granularity representation which reduces the amount of space and hence can be easily cached in the distributed memory: 2) model-parallelism: it simultaneously evaluates all feature candidates and makes attribute reduction highly parallelizable; 3) data-parallelism: it computes the significance of an attribute in parallel using a MapReduce-style manner. We implement PLAR with four representative heuristic feature selection algorithms on Spark, and evaluate them on various huge datasets, including UCI and astronomical datasets, finding our method's advantages beyond existing solutions.

cs.DC↗

arXiv2015

Numerical relativity reaching into post-Newtonian territory: a compact-object binary simulation spanning 350 gravitational-wave cycles

We present the first numerical-relativity simulation of a compact-object binary whose gravitational waveform is long enough to cover the entire frequency band of advanced gravitational-wave detectors, such as LIGO, Virgo and KAGRA, for mass ratio 7 and total mass as low as $45.5\,M_\odot$. We find that effective-one-body models, either uncalibrated or calibrated against substantially shorter numerical-relativity waveforms at smaller mass ratios, reproduce our new waveform remarkably well, with a negligible loss in detection rate due to modeling error. In contrast, post-Newtonian inspiral waveforms and existing calibrated phenomenological inspiral-merger-ringdown waveforms display greater disagreement with our new simulation. The disagreement varies substantially depending on the specific post-Newtonian approximant used.

gr-qc↗

arXiv2014

Impact of Higher Harmonics in Searching for Gravitational Waves from Non-Spinning Binary Black Holes

Current searches for gravitational waves from coalescing binary black holes (BBH) use templates that only include the dominant harmonic. In this study we use effective-one-body multipolar waveforms calibrated to numerical-relativity simulations to quantify the effect of neglecting sub-dominant harmonics on the sensitivity of searches. We consider both signal-to-noise ratio (SNR) and the signal-based vetoes that are used to re-weight SNR. We find that neglecting sub-dominant modes when searching for non-spinning BBHs with component masses $3\,M_{\odot} \leq m_1, m_2 \leq 200\,M_{\odot}$ and total mass $M < 360\,M_{\odot}$ in advanced LIGO results in a negligible reduction of the re-weighted SNR at detection thresholds. Sub-dominant modes therefore have no effect on the detection rates predicted for advanced LIGO. Furthermore, we find that if sub-dominant modes are included in templates the sensitivity of the search becomes worse if we use current search priors, due to an increase in false alarm probability. Templates would need to be weighted differently than what is currently done to compensate for the increase in false alarms. If we split the template bank such that sub-dominant modes are only used when $M \gtrsim 100\,M_{\odot}$ and mass ratio $q \gtrsim 4$, we find that the sensitivity does improve for these intermediate mass-ratio BBHs, but the sensitive volume associated with these systems is still small compared to equal-mass systems. Using sub-dominant modes is therefore unlikely to substantially increase the probability of detecting gravitational waves from non-spinning BBH signals unless there is a relatively large population of intermediate mass-ratio BBHs in the universe.

gr-qc↗