arXiv subjects

Xiang Ren

Publications and source records attributed to Xiang Ren.

At least 181 records · Page 10Linked to original sources

arXiv2020

Structured Distributed Compressive Channel Estimation over Doubly Selective Channels

For an orthogonal frequency-division multiplexing (OFDM) system over a doubly selective (DS) channel, a large number of pilot subcarriers are needed to estimate the numerous channel parameters, resulting in low spectral efficiency. In this paper, by exploiting temporal correlation of practical wireless channels, we propose a highly efficient structured distributed compressive sensing (SDCS) based joint multi-symbol channel estimation scheme. Specifically, by using the complex exponential basis expansion model (CE-BEM) and exploiting the sparsity in the delay domain within multiple OFDM symbols, we turn to estimate jointly sparse CE-BEM coefficient vectors rather than numerous channel taps. Then a sparse pilot pattern within multiple OFDM symbols is designed to obtain an ICI-free structure and transform the channel estimation problem into a joint-block-sparse model. Next, a novel block-based simultaneous orthogonal matching pursuit (BSOMP) algorithm is proposed to jointly recover coefficient vectors accurately. Finally, to reduce the CE-BEM modeling error, we carry out smoothing treatments of already estimated channel taps via piecewise linear approximation.Simulation results demonstrate that the proposed channel estimation scheme can achieve higher estimation accuracy than conventional schemes, although with a smaller number of pilot subcarriers.

eess.SP↗

arXiv2020

Block Distributed Compressive Sensing Based Doubly Selective Channel Estimation and Pilot Design for Large-Scale MIMO Systems

The doubly selective (DS) channel estimation in the large-scale multiple-input multiple-output (MIMO) systems is a challenging problem due to the large number of the channel coefficients to be estimated, which requires unaffordable and prohibitive pilot overhead. In this paper, firstly we conduct the analysis about the common sparsity of the basis expansion model (BEM) coefficients among all the BEM orders and all the transmit-receive antenna pairs. Then a novel pilot pattern is proposed, which inserts the guard pilots to deal with the inter carrier interference (ICI) under the superimposed pilot pattern. Moreover, by exploiting the common sparsity of the BEM coefficients among different BEM orders and different antennas, we propose a block distributed compressive sensing (BDCS) based DS channel estimator for the large-scale MIMO systems. Its structured sparsity leads to the reduction of the pilot overhead under the premise of guaranteeing the accuracy of the estimation. Furthermore, taking consideration of the block structure, a pilot design algorithm referred to as block discrete stochastic optimization (BDSO) is proposed. It optimizes the pilot positions by reducing the coherence among different blocks of the measurement matrix. Besides, a linear smoothing method is extended to large-scale MIMO systems to improve the accuracy of the estimation. Simulation results verify the performance gains of our proposed estimator and the pilot design algorithm compared with the existing schemes.

eess.SP↗

arXiv2020

Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling

Sequence labeling is a fundamental framework for various natural language processing problems. Its performance is largely influenced by the annotation quality and quantity in supervised learning scenarios, and obtaining ground truth labels is often costly. In many cases, ground truth labels do not exist, but noisy annotations or annotations from different domains are accessible. In this paper, we propose a novel framework Consensus Network (ConNet) that can be trained on annotations from multiple sources (e.g., crowd annotation, cross-domain data...). It learns individual representation for every source and dynamically aggregates source-specific knowledge by a context-aware attention module. Finally, it leads to a model reflecting the agreement (consensus) among multiple sources. We evaluate the proposed framework in two practical settings of multi-source learning: learning with crowd annotations and unsupervised cross-domain model adaptation. Extensive experimental results show that our model achieves significant improvements over existing methods in both settings. We also demonstrate that the method can apply to various tasks and cope with different encoders.

cs.CL↗

arXiv2020

LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation

Successfully training a deep neural network demands a huge corpus of labeled data. However, each label only provides limited information to learn from and collecting the requisite number of labels involves massive human effort. In this work, we introduce LEAN-LIFE, a web-based, Label-Efficient AnnotatioN framework for sequence labeling and classification tasks, with an easy-to-use UI that not only allows an annotator to provide the needed labels for a task, but also enables LearnIng From Explanations for each labeling decision. Such explanations enable us to generate useful additional labeled data from unlabeled instances, bolstering the pool of available training data. On three popular NLP tasks (named entity recognition, relation extraction, sentiment analysis), we find that using this enhanced supervision allows our models to surpass competitive baseline F1 scores by more than 5-10 percentage points, while using 2X times fewer labeled instances. Our framework is the first to utilize this enhanced supervision technique and does so for three important tasks -- thus providing improved annotation recommendations to users and an ability to build datasets of (data, label, explanation) triples instead of the regular (data, label) pair.

cs.CL↗

arXiv2020

Generating Natural Language Adversarial Examples on a Large Scale with Generative Models

Today text classification models have been widely used. However, these classifiers are found to be easily fooled by adversarial examples. Fortunately, standard attacking methods generate adversarial texts in a pair-wise way, that is, an adversarial text can only be created from a real-world text by replacing a few words. In many applications, these texts are limited in numbers, therefore their corresponding adversarial examples are often not diverse enough and sometimes hard to read, thus can be easily detected by humans and cannot create chaos at a large scale. In this paper, we propose an end to end solution to efficiently generate adversarial texts from scratch using generative models, which are not restricted to perturbing the given texts. We call it unrestricted adversarial text generation. Specifically, we train a conditional variational autoencoder (VAE) with an additional adversarial loss to guide the generation of adversarial examples. Moreover, to improve the validity of adversarial texts, we utilize discrimators and the training framework of generative adversarial networks (GANs) to make adversarial texts consistent with real data. Experimental results on sentiment analysis demonstrate the scalability and efficiency of our method. It can attack text classification models with a higher success rate than existing methods, and provide acceptable quality for humans in the meantime.

cs.CL↗

arXiv2020

Compressed Channel Estimation with Position-Based ICI Elimination for High-Mobility SIMO-OFDM Systems

Orthogonal frequency-division multiplexing (OFDM) is widely adopted for providing reliable and high data rate communication in high-speed train systems. However, with the increasing train mobility, the resulting large Doppler shift introduces intercarrier interference (ICI) in OFDM systems and greatly degrades the channel estimation accuracy. Therefore, it is necessary and important to investigate reliable channel estimation and ICI mitigation methods in high-mobility environments. In this paper, we consider a typical HST communication system and show that the ICI caused by the large Doppler shift can be mitigated by exploiting the train position information as well as the sparsity of the conventional basis expansion model (BEM) based channel model. Then, we show that for the complex-exponential BEM (CE-BEM) based channel model, the ICI can be completely eliminated to get the ICI-free pilots at each receive antenna. After that, we propose a new pilot pattern design algorithm to reduce the system coherence and hence can improve the compressed sensing (CS) based channel estimation accuracy. The proposed optimal pilot pattern is independent of the number of receive antennas, the Doppler shifts, the train position, or the train speed. Simulation results confirms the performance merits of the proposed scheme in high-mobility environments. In addition, it is also shown that the proposed scheme is robust to the respect of high mobility.

eess.SP↗

arXiv2020

Position Based Compressed Channel Estimation and Pilot Design for High Mobility OFDM Systems

With the development of high speed trains (HST) in many countries, providing broadband wireless services in HSTs is becoming crucial. Orthogonal frequency-division multiplexing (OFDM) has been widely adopted for broadband wireless communications due to its high spectral efficiency. However, OFDM is sensitive to the time selectivity caused by high-mobility channels, which costs large spectrum or time resources to obtain the accurate channel state information (CSI). Therefore, the channel estimation in high-mobility OFDM systems has been a long-standing challenge. In this paper, we first propose a new position-based high-mobility channel model,in which the HST's position information and Doppler shift are utilized to determine the positions of the dominant channel coefficients. %In this way, we can reduce the estimation complexity and to design the transmitted pilot.Then, we propose a joint pilot placement and pilot symbol design algorithm for compressed channel estimation. It aims to reduce the coherence between the pilot signal and the proposed channel model, and hence can improve the channel estimation accuracy. Simulation results demonstrate that the proposed method achieves better performances than existing channel estimation methods over high-mobility channels. Furthermore, we give an example of the designed pilot codebook to show the practical applicability of the proposed scheme.

eess.SP↗

arXiv2020

Position-Based Interference Elimination for High Mobility OFDM Channel Estimation in Multi-cell Systems

Orthogonal frequency-division multiplexing (OFD-M) and multi-cell architecture are widely adopted in current high speed train (HST) systems for providing high data rate wireless communications. In this paper, a typical multi-antenna OFDM HST communication system with multi-cell architecture is considered, where the inter-carrier interference (ICI) caused by high mobility and multi-cell interference (MCI) are both taken into consideration. By exploiting the train position information, a new position-based interference elimination method is proposed to eliminate both the MCI and ICI for a general basis expansion model (BEM). We show that the MCI and ICI can be completely eliminated by the proposed method to get the ICI-free pilots at each receive antenna. In addition, for the considered multi-cell HST system, we develop a low-complexity compressed channel estimation method and consider the optimal pilot pattern design. Both the proposed interference elimination method and the optimal pilot pattern are robust to the train speed and position,as well as the multi-cell multi-antenna system. Simulation results demonstrate the benefits and robustness of the proposed method in the multi-cell HST system.

eess.SP↗

arXiv2020

Improving BERT Fine-tuning with Embedding Normalization

Large pre-trained sentence encoders like BERT start a new chapter in natural language processing. A common practice to apply pre-trained BERT to sequence classification tasks (e.g., classification of sentences or sentence pairs) is by feeding the embedding of [CLS] token (in the last layer) to a task-specific classification layer, and then fine tune the model parameters of BERT and classifier jointly. In this paper, we conduct systematic analysis over several sequence classification datasets to examine the embedding values of [CLS] token before the fine tuning phase, and present the biased embedding distribution issue---i.e., embedding values of [CLS] concentrate on a few dimensions and are non-zero centered. Such biased embedding brings challenge to the optimization process during fine-tuning as gradients of [CLS] embedding may explode and result in degraded model performance. We further propose several simple yet effective normalization methods to modify the [CLS] embedding during the fine-tuning. Compared with the previous practice, neural classification model with the normalized embedding shows improvements on several text classification tasks, demonstrates the effectiveness of our method.

cs.CL↗

arXiv2020

Learning from Explanations with Neural Execution Tree

While deep neural networks have achieved impressive performance on a range of NLP tasks, these data-hungry models heavily rely on labeled data, which restricts their applications in scenarios where data annotation is expensive. Natural language (NL) explanations have been demonstrated very useful additional supervision, which can provide sufficient domain knowledge for generating more labeled data over new instances, while the annotation time only doubles. However, directly applying them for augmenting model learning encounters two challenges: (1) NL explanations are unstructured and inherently compositional, which asks for a modularized model to represent their semantics, (2) NL explanations often have large numbers of linguistic variants, resulting in low recall and limited generalization ability. In this paper, we propose a novel Neural Execution Tree (NExT) framework to augment training data for text classification using NL explanations. After transforming NL explanations into executable logical forms by semantic parsing, NExT generalizes different types of actions specified by the logical forms for labeling data instances, which substantially increases the coverage of each NL explanation. Experiments on two NLP tasks (relation extraction and sentiment analysis) demonstrate its superiority over baseline methods. Its extension to multi-hop question answering achieves performance gain with light annotation effort.

cs.CL↗

arXiv2020

NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction

Deep neural models for relation extraction tend to be less reliable when perfectly labeled data is limited, despite their success in label-sufficient scenarios. Instead of seeking more instance-level labels from human annotators, here we propose to annotate frequent surface patterns to form labeling rules. These rules can be automatically mined from large text corpora and generalized via a soft rule matching mechanism. Prior works use labeling rules in an exact matching fashion, which inherently limits the coverage of sentence matching and results in the low-recall issue. In this paper, we present a neural approach to ground rules for RE, named NERO, which jointly learns a relation extraction module and a soft matching module. One can employ any neural relation extraction models as the instantiation for the RE module. The soft matching module learns to match rules with semantically similar sentences such that raw corpora can be automatically labeled and leveraged by the RE module (in a much better coverage) as augmented supervision, in addition to the exactly matched sentences. Extensive experiments and analysis on two public and widely-used datasets demonstrate the effectiveness of the proposed NERO framework, comparing with both rule-based and semi-supervised methods. Through user studies, we find that the time efficiency for a human to annotate rules and sentences are similar (0.30 vs. 0.35 min per label). In particular, NERO's performance using 270 rules is comparable to the models trained using 3,000 labeled sentences, yielding a 9.5x speedup. Moreover, NERO can predict for unseen relations at test time and provide interpretable predictions. We release our code to the community for future research.

cs.CL↗

arXiv2019

Mining News Events from Comparable News Corpora: A Multi-Attribute Proximity Network Modeling Approach

We present ProxiModel, a novel event mining framework for extracting high-quality structured event knowledge from large, redundant, and noisy news data sources. The proposed model differentiates itself from other approaches by modeling both the event correlation within each individual document as well as across the corpus. To facilitate this, we introduce the concept of a proximity-network, a novel space-efficient data structure to facilitate scalable event mining. This proximity network captures the corpus-level co-occurence statistics for candidate event descriptors, event attributes, as well as their connections. We probabilistically model the proximity network as a generative process with sparsity-inducing regularization. This allows us to efficiently and effectively extract high-quality and interpretable news events. Experiments on three different news corpora demonstrate that the proposed method is effective and robust at generating high-quality event descriptors and attributes. We briefly detail many interesting applications from our proposed framework such as news summarization, event tracking and multi-dimensional analysis on news. Finally, we explore a case study on visualizing the events for a Japan Tsunami news corpus and demonstrate ProxiModel's ability to automatically summarize emerging news events.

cs.LG↗

arXiv2019

SetExpan: Corpus-Based Set Expansion via Context Feature Selection and Rank Ensemble

Corpus-based set expansion (i.e., finding the "complete" set of entities belonging to the same semantic class, based on a given corpus and a tiny set of seeds) is a critical task in knowledge discovery. It may facilitate numerous downstream applications, such as information extraction, taxonomy induction, question answering, and web search. To discover new entities in an expanded set, previous approaches either make one-time entity ranking based on distributional similarity, or resort to iterative pattern-based bootstrapping. The core challenge for these methods is how to deal with noisy context features derived from free-text corpora, which may lead to entity intrusion and semantic drifting. In this study, we propose a novel framework, SetExpan, which tackles this problem, with two techniques: (1) a context feature selection method that selects clean context features for calculating entity-entity distributional similarity, and (2) a ranking-based unsupervised ensemble method for expanding entity set based on denoised context features. Experiments on three datasets show that SetExpan is robust and outperforms previous state-of-the-art methods in terms of mean average precision.

cs.CL↗

arXiv2019

HiExpan: Task-Guided Taxonomy Construction by Hierarchical Tree Expansion

Taxonomies are of great value to many knowledge-rich applications. As the manual taxonomy curation costs enormous human effects, automatic taxonomy construction is in great demand. However, most existing automatic taxonomy construction methods can only build hypernymy taxonomies wherein each edge is limited to expressing the "is-a" relation. Such a restriction limits their applicability to more diverse real-world tasks where the parent-child may carry different relations. In this paper, we aim to construct a task-guided taxonomy from a domain-specific corpus and allow users to input a "seed" taxonomy, serving as the task guidance. We propose an expansion-based taxonomy construction framework, namely HiExpan, which automatically generates key term list from the corpus and iteratively grows the seed taxonomy. Specifically, HiExpan views all children under each taxonomy node forming a coherent set and builds the taxonomy by recursively expanding all these sets. Furthermore, HiExpan incorporates a weakly-supervised relation extraction module to extract the initial children of a newly-expanded node and adjusts the taxonomy tree by optimizing its global structure. Our experiments on three real datasets from different domains demonstrate the effectiveness of HiExpan for building task-guided taxonomies.

cs.CL↗

arXiv2019

Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction

In recent years there is a surge of interest in applying distant supervision (DS) to automatically generate training data for relation extraction (RE). In this paper, we study the problem what limits the performance of DS-trained neural models, conduct thorough analyses, and identify a factor that can influence the performance greatly, shifted label distribution. Specifically, we found this problem commonly exists in real-world DS datasets, and without special handing, typical DS-RE models cannot automatically adapt to this shift, thus achieving deteriorated performance. To further validate our intuition, we develop a simple yet effective adaptation method for DS-trained models, bias adjustment, which updates models learned over the source domain (i.e., DS training set) with a label distribution estimated on the target domain (i.e., test set). Experiments demonstrate that bias adjustment achieves consistent performance gains on DS-trained models, especially on neural models, with an up to 23% relative F1 improvement, which verifies our assumptions. Our code and data can be found at \url{https://github.com/INK-USC/shifted-label-distribution}.

cs.CL↗

arXiv2019

Learning Dynamic Context Augmentation for Global Entity Linking

Despite of the recent success of collective entity linking (EL) methods, these "global" inference methods may yield sub-optimal results when the "all-mention coherence" assumption breaks, and often suffer from high computational cost at the inference stage, due to the complex search space. In this paper, we propose a simple yet effective solution, called Dynamic Context Augmentation (DCA), for collective EL, which requires only one pass through the mentions in a document. DCA sequentially accumulates context information to make efficient, collective inference, and can cope with different local EL models as a plug-and-enhance module. We explore both supervised and reinforcement learning strategies for learning the DCA model. Extensive experiments show the effectiveness of our model with different learning settings, base models, decision orders and attention mechanisms.

cs.CL↗

arXiv2019

Reporting the Unreported: Event Extraction for Analyzing the Local Representation of Hate Crimes

Official reports of hate crimes in the US are under-reported relative to the actual number of such incidents. Further, despite statistical approximations, there are no official reports from a large number of US cities regarding incidents of hate. Here, we first demonstrate that event extraction and multi-instance learning, applied to a corpus of local news articles, can be used to predict instances of hate crime. We then use the trained model to detect incidents of hate in cities for which the FBI lacks statistics. Lastly, we train models on predicting homicide and kidnapping, compare the predictions to FBI reports, and establish that incidents of hate are indeed under-reported, compared to other types of crimes, in local press.

cs.CL↗

arXiv2019

KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning

Commonsense reasoning aims to empower machines with the human ability to make presumptions about ordinary situations in our daily life. In this paper, we propose a textual inference framework for answering commonsense questions, which effectively utilizes external, structured commonsense knowledge graphs to perform explainable inferences. The framework first grounds a question-answer pair from the semantic space to the knowledge-based symbolic space as a schema graph, a related sub-graph of external knowledge graphs. It represents schema graphs with a novel knowledge-aware graph network module named KagNet, and finally scores answers with graph representations. Our model is based on graph convolutional networks and LSTMs, with a hierarchical path-based attention mechanism. The intermediate attention scores make it transparent and interpretable, which thus produce trustworthy inferences. Using ConceptNet as the only external resource for Bert-based models, we achieved state-of-the-art performance on the CommonsenseQA, a large-scale dataset for commonsense reasoning.

cs.CL↗