Searcharxiv⌕ Search

arXiv subjects

Wei Wei

Publications and source records attributed to Wei Wei.

At least 397 records · Page 22Linked to original sources
arXiv2021

Graph Classification Based on Skeleton and Component Features

Most existing popular methods for learning graph embedding only consider fixed-order global structural features and lack structures hierarchical representation. To address this weakness, we propose a novel graph embedding algorithm named GraphCSC that realizes classification based on skeleton information using fixed-order structures learned in anonymous random walks manner, and component information using different size subgraphs. Two graphs are similar if their skeletons and components are both similar, thus in our model, we integrate both of them together into embeddings as graph homogeneity characterization. We demonstrate our model on different datasets in comparison with a comprehensive list of up-to-date state-of-the-art baselines, and experiments show that our work is superior in real-world graph classification tasks.

cs.LG↗
arXiv2021

Representation Learning of Reconstructed Graphs Using Random Walk Graph Convolutional Network

Graphs are often used to organize data because of their simple topological structure, and therefore play a key role in machine learning. And it turns out that the low-dimensional embedded representation obtained by graph representation learning are extremely useful in various typical tasks, such as node classification, content recommendation and link prediction. However, the existing methods mostly start from the microstructure (i.e., the edges) in the graph, ignoring the mesoscopic structure (high-order local structure). Here, we propose wGCN -- a novel framework that utilizes random walk to obtain the node-specific mesoscopic structures of the graph, and utilizes these mesoscopic structures to reconstruct the graph And organize the characteristic information of the nodes. Our method can effectively generate node embeddings for previously unseen data, which has been proven in a series of experiments conducted on citation networks and social networks (our method has advantages over baseline methods). We believe that combining high-order local structural information can more efficiently explore the potential of the network, which will greatly improve the learning efficiency of graph neural network and promote the establishment of new learning models.

cs.LG↗
arXiv2021

A Robust and Domain-Adaptive Approach for Low-Resource Named Entity Recognition

Recently, it has attracted much attention to build reliable named entity recognition (NER) systems using limited annotated data. Nearly all existing works heavily rely on domain-specific resources, such as external lexicons and knowledge bases. However, such domain-specific resources are often not available, meanwhile it's difficult and expensive to construct the resources, which has become a key obstacle to wider adoption. To tackle the problem, in this work, we propose a novel robust and domain-adaptive approach RDANER for low-resource NER, which only uses cheap and easily obtainable resources. Extensive experiments on three benchmark datasets demonstrate that our approach achieves the best performance when only using cheap and easily obtainable resources, and delivers competitive results against state-of-the-art methods which use difficultly obtainable domainspecific resources. All our code and corpora can be found on https://github.com/houking-can/RDANER.

cs.CL↗
arXiv2020

Contextual Temperature for Language Modeling

Temperature scaling has been widely used as an effective approach to control the smoothness of a distribution, which helps the model performance in various tasks. Current practices to apply temperature scaling assume either a fixed, or a manually-crafted dynamically changing schedule. However, our studies indicate that the individual optimal trajectory for each class can change with the context. To this end, we propose contextual temperature, a generalized approach that learns an optimal temperature trajectory for each vocabulary over the context. Experimental results confirm that the proposed method significantly improves state-of-the-art language models, achieving a perplexity of 55.31 and 62.89 on the test set of Penn Treebank and WikiText-2, respectively. In-depth analyses show that the behaviour of the learned temperature schedules varies dramatically by vocabulary, and that the optimal schedules help in controlling the uncertainties. These evidences further justify the need for the proposed method and its advantages over fixed temperature schedules.

cs.CL↗
arXiv2020

Self-attention Comparison Module for Boosting Performance on Retrieval-based Open-Domain Dialog Systems

Since the pre-trained language models are widely used, retrieval-based open-domain dialog systems, have attracted considerable attention from researchers recently. Most of the previous works select a suitable response only according to the matching degree between the query and each individual candidate response. Although good performance has been achieved, these recent works ignore the comparison among the candidate responses, which could provide rich information for selecting the most appropriate response. Intuitively, better decisions could be made when the models can get access to the comparison information among all the candidate responses. In order to leverage the comparison information among the candidate responses, in this paper, we propose a novel and plug-in Self-attention Comparison Module for retrieval-based open-domain dialog systems, called SCM. Extensive experiment results demonstrate that our proposed self-attention comparison module effectively boosts the performance of the existing retrieval-based open-domain dialog systems. Besides, we have publicly released our source codes for future research.

cs.CL↗
arXiv2020

Ultra-Fast, Low-Storage, Highly Effective Coarse-grained Selection in Retrieval-based Chatbot by Using Deep Semantic Hashing

We study the coarse-grained selection module in retrieval-based chatbot. Coarse-grained selection is a basic module in a retrieval-based chatbot, which constructs a rough candidate set from the whole database to speed up the interaction with customers. So far, there are two kinds of approaches for coarse-grained selection module: (1) sparse representation; (2) dense representation. To the best of our knowledge, there is no systematic comparison between these two approaches in retrieval-based chatbots, and which kind of method is better in real scenarios is still an open question. In this paper, we first systematically compare these two methods from four aspects: (1) effectiveness; (2) index stoarge; (3) search time cost; (4) human evaluation. Extensive experiment results demonstrate that dense representation method significantly outperforms the sparse representation, but costs more time and storage occupation. In order to overcome these fatal weaknesses of dense representation method, we propose an ultra-fast, low-storage, and highly effective Deep Semantic Hashing Coarse-grained selection method, called DSHC model. Specifically, in our proposed DSHC model, a hashing optimizing module that consists of two autoencoder models is stacked on a trained dense representation model, and three loss functions are designed to optimize it. The hash codes provided by hashing optimizing module effectively preserve the rich semantic and similarity information in dense vectors. Extensive experiment results prove that, our proposed DSHC model can achieve much faster speed and lower storage than sparse representation, with limited performance loss compared with dense representation. Besides, our source codes have been publicly released for future research.

cs.CL↗
arXiv2020

Spatiotemporal Graph Neural Network based Mask Reconstruction for Video Object Segmentation

This paper addresses the task of segmenting class-agnostic objects in semi-supervised setting. Although previous detection based methods achieve relatively good performance, these approaches extract the best proposal by a greedy strategy, which may lose the local patch details outside the chosen candidate. In this paper, we propose a novel spatiotemporal graph neural network (STG-Net) to reconstruct more accurate masks for video object segmentation, which captures the local contexts by utilizing all proposals. In the spatial graph, we treat object proposals of a frame as nodes and represent their correlations with an edge weight strategy for mask context aggregation. To capture temporal information from previous frames, we use a memory network to refine the mask of current frame by retrieving historic masks in a temporal graph. The joint use of both local patch details and temporal relationships allow us to better address the challenges such as object occlusion and missing. Without online learning and fine-tuning, our STG-Net achieves state-of-the-art performance on four large benchmarks (DAVIS, YouTube-VOS, SegTrack-v2, and YouTube-Objects), demonstrating the effectiveness of the proposed approach.

cs.CV↗
arXiv2020

Com-DDPG: A Multiagent Reinforcement Learning-based Offloading Strategy for Mobile Edge Computing

The development of mobile services has impacted a variety of computation-intensive and time-sensitive applications, such as recommendation systems and daily payment methods. However, computing task competition involving limited resources increases the task processing latency and energy consumption of mobile devices, as well as time constraints. Mobile edge computing (MEC) has been widely used to address these problems. However, there are limitations to existing methods used during computation offloading. On the one hand, they focus on independent tasks rather than dependent tasks. The challenges of task dependency in the real world, especially task segmentation and integration, remain to be addressed. On the other hand, the multiuser scenarios related to resource allocation and the mutex access problem must be considered. In this paper, we propose a novel offloading approach, Com-DDPG, for MEC using multiagent reinforcement learning to enhance the offloading performance. First, we discuss the task dependency model, task priority model, energy consumption model, and average latency from the perspective of server clusters and multidependence on mobile tasks. Our method based on these models is introduced to formalize communication behavior among multiple agents; then, reinforcement learning is executed as an offloading strategy to obtain the results. Because of the incomplete state information, long short-term memory (LSTM) is employed as a decision-making tool to assess the internal state. Moreover, to optimize and support effective action, we consider using a bidirectional recurrent neural network (BRNN) to learn and enhance features obtained from agents' communication. Finally, we simulate experiments on the Alibaba cluster dataset. The results show that our method is better than other baselines in terms of energy consumption, load status and latency.

cs.DC↗
arXiv2020

Meta-Generating Deep Attentive Metric for Few-shot Classification

Learning to generate a task-aware base learner proves a promising direction to deal with few-shot learning (FSL) problem. Existing methods mainly focus on generating an embedding model utilized with a fixed metric (eg, cosine distance) for nearest neighbour classification or directly generating a linear classier. However, due to the limited discriminative capacity of such a simple metric or classifier, these methods fail to generalize to challenging cases appropriately. To mitigate this problem, we present a novel deep metric meta-generation method that turns to an orthogonal direction, ie, learning to adaptively generate a specific metric for a new FSL task based on the task description (eg, a few labelled samples). In this study, we structure the metric using a three-layer deep attentive network that is flexible enough to produce a discriminative metric for each task. Moreover, different from existing methods that utilize an uni-modal weight distribution conditioned on labelled samples for network generation, the proposed meta-learner establishes a multi-modal weight distribution conditioned on cross-class sample pairs using a tailored variational autoencoder, which can separately capture the specific inter-class discrepancy statistics for each class and jointly embed the statistics for all classes into metric generation. By doing this, the generated metric can be appropriately adapted to a new FSL task with pleasing generalization performance. To demonstrate this, we test the proposed method on four benchmark FSL datasets and gain surprisingly obvious performance improvement over state-of-the-art competitors, especially in the challenging cases, eg, improve the accuracy from 26.14% to 46.69% in the 20-way 1-shot task on miniImageNet, while improve the accuracy from 45.2% to 68.72% in the 5-way 1-shot task on FC100. Code is available: https://github.com/NWPUZhoufei/DAM.

cs.CV↗
arXiv2020

Unsupervised Alternating Optimization for Blind Hyperspectral Imagery Super-resolution

Despite the great success of deep model on Hyperspectral imagery (HSI) super-resolution(SR) for simulated data, most of them function unsatisfactory when applied to the real data, especially for unsupervised HSI SR methods. One of the main reason comes from the fact that the predefined degeneration models (e.g. blur in spatial domain) utilized by most HSI SR methods often exist great discrepancy with the real one, which results in these deep models overfit and ultimately degrade their performance on real data. To well mitigate such a problem, we explore the unsupervised blind HSI SR method. Specifically, we investigate how to effectively obtain the degeneration models in spatial and spectral domain, respectively, and makes them can well compatible with the fusion based SR reconstruction model. To this end, we first propose an alternating optimization based deep framework to estimate the degeneration models and reconstruct the latent image, with which the degeneration models estimation and HSI reconstruction can mutually promotes each other. Then, a meta-learning based mechanism is further proposed to pre-train the network, which can effectively improve the speed and generalization ability adapting to different complex degeneration. Experiments on three benchmark HSI SR datasets report an excellent superiority of the proposed method on handling blind HSI fusion problem over other competing methods.

eess.IV↗
arXiv2020

Robust Processing-In-Memory Neural Networks via Noise-Aware Normalization

Analog computing hardwares, such as Processing-in-memory (PIM) accelerators, have gradually received more attention for accelerating the neural network computations. However, PIM accelerators often suffer from intrinsic noise in the physical components, making it challenging for neural network models to achieve the same performance as on the digital hardware. Previous works in mitigating intrinsic noise assumed the knowledge of the noise model, and retraining the neural networks accordingly was required. In this paper, we propose a noise-agnostic method to achieve robust neural network performance against any noise setting. Our key observation is that the degradation of performance is due to the distribution shifts in network activations, which are caused by the noise. To properly track the shifts and calibrate the biased distributions, we propose a "noise-aware" batch normalization layer, which is able to align the distributions of the activations under variational noise inherent in the analog environments. Our method is simple, easy to implement, general to various noise settings, and does not need to retrain the models. We conduct experiments on several tasks in computer vision, including classification, object detection and semantic segmentation. The results demonstrate the effectiveness of our method, achieving robust performance under a wide range of noise settings, more reliable than existing methods. We believe that our simple yet general method can facilitate the adoption of analog computing devices for neural networks.

cs.CV↗
arXiv2020

Remix: Rebalanced Mixup

Deep image classifiers often perform poorly when training data are heavily class-imbalanced. In this work, we propose a new regularization technique, Remix, that relaxes Mixup's formulation and enables the mixing factors of features and labels to be disentangled. Specifically, when mixing two samples, while features are mixed in the same fashion as Mixup, Remix assigns the label in favor of the minority class by providing a disproportionately higher weight to the minority class. By doing so, the classifier learns to push the decision boundaries towards the majority classes and balance the generalization error between majority and minority classes. We have studied the state-of-the art regularization techniques such as Mixup, Manifold Mixup and CutMix under class-imbalanced regime, and shown that the proposed Remix significantly outperforms these state-of-the-arts and several re-weighting and re-sampling techniques, on the imbalanced datasets constructed by CIFAR-10, CIFAR-100, and CINIC-10. We have also evaluated Remix on a real-world large-scale imbalanced dataset, iNaturalist 2018. The experimental results confirmed that Remix provides consistent and significant improvements over the previous methods.

cs.CV↗
arXiv2020

Compactness Theorem of Complete k-Curvature Manifolds with Isolated Singularities

In this paper we prove that the set of metrics conformal to the standard metric on $\mathbb{S}^{n}\backslash\{p_{1},\cdots,p_{l}\}$ is locally compact in $C^{m,α}$ topology for any $m>0$, whenever the metrics have constant $σ_{k}$ curvature and the $k$-Dilational Pohozaev invariants have positive lower bound for $k<n/2$. Here the $k$-Dilational Pohozaev invariants come from the Kazdan-Warner type identity for the $σ_{k}$ curvature, which is derived by Viaclovsky \cite{Viac2000} and Han \cite{H1}. When $k=1$, Pollack \cite{Pollack} proved the compactness results for the complete metrics of constant positive scalar curvature on $\mathbb{S}^{n}\backslash\{p_{1},\cdots,p_{l}\}$.

math.DG↗
arXiv2020

User-based Network Embedding for Collective Opinion Spammer Detection

Due to the huge commercial interests behind online reviews, a tremendousamount of spammers manufacture spam reviews for product reputation manipulation. To further enhance the influence of spam reviews, spammers often collaboratively post spam reviewers within a short period of time, the activities of whom are called collective opinion spam campaign. As the goals and members of the spam campaign activities change frequently, and some spammers also imitate normal purchases to conceal identity, which makes the spammer detection challenging. In this paper, we propose an unsupervised network embedding-based approach to jointly exploiting different types of relations, e.g., direct common behaviour relation and indirect co-reviewed relation to effectively represent the relevances of users for detecting the collective opinion spammers. The average improvements of our method over the state-of-the-art solutions on dataset AmazonCn and YelpHotel are [14.09%,12.04%] and [16.25%,12.78%] in terms of AP and AUC, respectively.

cs.IR↗
arXiv2020

A Survey on Recent Advances in Sequence Labeling from Deep Learning Models

Sequence labeling (SL) is a fundamental research problem encompassing a variety of tasks, e.g., part-of-speech (POS) tagging, named entity recognition (NER), text chunking, etc. Though prevalent and effective in many downstream applications (e.g., information retrieval, question answering, and knowledge graph embedding), conventional sequence labeling approaches heavily rely on hand-crafted or language-specific features. Recently, deep learning has been employed for sequence labeling tasks due to its powerful capability in automatically learning complex features of instances and effectively yielding the stat-of-the-art performances. In this paper, we aim to present a comprehensive review of existing deep learning-based sequence labeling models, which consists of three related tasks, e.g., part-of-speech tagging, named entity recognition, and text chunking. Then, we systematically present the existing approaches base on a scientific taxonomy, as well as the widely-used experimental datasets and popularly-adopted evaluation metrics in the SL domain. Furthermore, we also present an in-depth analysis of different SL models on the factors that may affect the performance and future directions in the SL domain.

cs.CL↗
arXiv2020

The study of calibration for the hybrid pixel detector with single photon counting in HEPS-BPIX

The calibration process for the hybrid array pixel detector designed for High Energy Photon Source in China, we called HEPS-BPIX, is presented in this paper. Based on the threshold scanning, the relationship between energy and threshold is quantified for the threshold calibration. For the threshold trimming, the precise algorithm basing on LDAC characteristic and fast algorithm basing on LDAC scanning are proposed in this paper to study the performance of the threshold DACs which will be applied to the pixel. The threshold dispersion has been reduced from 46.28 mV without algorithm to 6.78 mV with the precise algorithm, whereas it is 7.61 mV with fast algorithm. For the temperature from 5 to 60 , the threshold dispersion of precise algorithm varies in the range of about 5.69 mV, whereas it is about 33.21 mV with the fast algorithm which can be re-corrected to 1.49 mV. The measurement results show that the fast algorithm could get the applicable threshold dispersion for a silicon pixel module and take a shorter time, while the precise algorithm could get better threshold dispersion, but time consuming. The temperature dependence of the silicon pixel module noise is also studied to assess the detector working status. The minimum detection energy can be reduced about 0.83 keV at a 20 lower temperature.

physics.ins-det↗
arXiv2020

Deep Learning Ensemble for Real-time Gravitational Wave Detection of Spinning Binary Black Hole Mergers

We introduce the use of deep learning ensembles for real-time, gravitational wave detection of spinning binary black hole mergers. This analysis consists of training independent neural networks that simultaneously process strain data from multiple detectors. The output of these networks is then combined and processed to identify significant noise triggers. We have applied this methodology in O2 and O3 data finding that deep learning ensembles clearly identify binary black hole mergers in open source data available at the Gravitational-Wave Open Science Center. We have also benchmarked the performance of this new methodology by processing 200 hours of open source, advanced LIGO noise from August 2017. Our findings indicate that our approach identifies real gravitational wave sources in advanced LIGO data with a false positive rate of 1 misclassification for every 2.7 days of searched data. A follow up of these misclassifications identified them as glitches. Our deep learning ensemble represents the first class of neural network classifiers that are trained with millions of modeled waveforms that describe quasi-circular, spinning, non-precessing, binary black hole mergers. Once fully trained, our deep learning ensemble processes advanced LIGO strain data faster than real-time using 4 NVIDIA V100 GPUs.

gr-qc↗