arXiv subjects

Xiaoya Li

Publications and source records attributed to Xiaoya Li.

49 records · Page 3Linked to original sources

arXiv2021

Multiple Intelligent Reflecting Surfaces Assisted Cell-Free MIMO Communications

In this paper, we investigate an intelligent reflecting surface (IRS) assisted cell-free multiple input multiple output (MIMO) communication system, where distributed multiple IRSs are dedicated deployed to assist distributed multiple base stations (BSs) for cooperative transmission. Our objective is to maximize the achievable sum-rate of the cell-free system by jointly optimizing the active transmit beamforming matrices at BSs and the passive reflecting beamforming matrices at IRSs, while the constraints on the maximum transmit power of each BS and the phase shift of each IRS element are satisfied. We propose an efficient framework to jointly design the BSs, the IRSs, and the user equipment (UEs). As a compromise approach, we first transform the non-convex problem into an equivalent form based on the fractional programming methods and then decompose the reformulated problem into two subproblems and solve them alternately. Particularly, we propose a Lagrangian dual sub-gradient based algorithm to solve the subproblem of optimizing the active transmit beamforming with nearly closed-form solutions. We reformulate the subproblem of optimizing the passive reflecting beamforming as a constant modulus constrained quadratic programming (CMC-QP) problem. We first provide two feasible solutions by proposing a pair of relaxation-based algorithms. We also develop a low-complexity alternating sequential optimization (ASO) algorithm to obtain closed-form solutions. All three algorithms are guaranteed to converge to locally optimal solutions. Simulation results demonstrate that the proposed algorithms achieve considerable performance improvements compared with the benchmark schemes.

cs.IT↗

arXiv2021

Progressive Domain Expansion Network for Single Domain Generalization

Single domain generalization is a challenging case of model generalization, where the models are trained on a single domain and tested on other unseen domains. A promising solution is to learn cross-domain invariant representations by expanding the coverage of the training domain. These methods have limited generalization performance gains in practical applications due to the lack of appropriate safety and effectiveness constraints. In this paper, we propose a novel learning framework called progressive domain expansion network (PDEN) for single domain generalization. The domain expansion subnetwork and representation learning subnetwork in PDEN mutually benefit from each other by joint learning. For the domain expansion subnetwork, multiple domains are progressively generated in order to simulate various photometric and geometric transforms in unseen domains. A series of strategies are introduced to guarantee the safety and effectiveness of the expanded domains. For the domain invariant representation learning subnetwork, contrastive learning is introduced to learn the domain invariant representation in which each class is well clustered so that a better decision boundary can be learned to improve it's generalization. Extensive experiments on classification and segmentation have shown that PDEN can achieve up to 15.28% improvement compared with the state-of-the-art single-domain generalization methods.

cs.CV↗

arXiv2020

SAC: Accelerating and Structuring Self-Attention via Sparse Adaptive Connection

While the self-attention mechanism has been widely used in a wide variety of tasks, it has the unfortunate property of a quadratic cost with respect to the input length, which makes it difficult to deal with long inputs. In this paper, we present a method for accelerating and structuring self-attentions: Sparse Adaptive Connection (SAC). In SAC, we regard the input sequence as a graph and attention operations are performed between linked nodes. In contrast with previous self-attention models with pre-defined structures (edges), the model learns to construct attention edges to improve task-specific performances. In this way, the model is able to select the most salient nodes and reduce the quadratic complexity regardless of the sequence length. Based on SAC, we show that previous variants of self-attention models are its special cases. Through extensive experiments on neural machine translation, language modeling, graph representation learning and image classification, we demonstrate SAC is competitive with state-of-the-art models while significantly reducing memory cost.

cs.CL↗

arXiv2020

Improving Robustness and Generality of NLP Models Using Disentangled Representations

Supervised neural networks, which first map an input $x$ to a single representation $z$, and then map $z$ to the output label $y$, have achieved remarkable success in a wide range of natural language processing (NLP) tasks. Despite their success, neural models lack for both robustness and generality: small perturbations to inputs can result in absolutely different outputs; the performance of a model trained on one domain drops drastically when tested on another domain. In this paper, we present methods to improve robustness and generality of NLP models from the standpoint of disentangled representation learning. Instead of mapping $x$ to a single representation $z$, the proposed strategy maps $x$ to a set of representations $\{z_1,z_2,...,z_K\}$ while forcing them to be disentangled. These representations are then mapped to different logits $l$s, the ensemble of which is used to make the final prediction $y$. We propose different methods to incorporate this idea into currently widely-used models, including adding an $L$2 regularizer on $z$s or adding Total Correlation (TC) under the framework of variational information bottleneck (VIB). We show that models trained with the proposed criteria provide better robustness and domain adaptation ability in a wide range of supervised learning tasks.

cs.CL↗

arXiv2020

Analyzing COVID-19 on Online Social Media: Trends, Sentiments and Emotions

At the time of writing, the ongoing pandemic of coronavirus disease (COVID-19) has caused severe impacts on society, economy and people's daily lives. People constantly express their opinions on various aspects of the pandemic on social media, making user-generated content an important source for understanding public emotions and concerns. In this paper, we perform a comprehensive analysis on the affective trajectories of the American people and the Chinese people based on Twitter and Weibo posts between January 20th, 2020 and May 11th 2020. Specifically, by identifying people's sentiments, emotions (i.e., anger, disgust, fear, happiness, sadness, surprise) and the emotional triggers (e.g., what a user is angry/sad about) we are able to depict the dynamics of public affect in the time of COVID-19. By contrasting two very different countries, China and the Unites States, we reveal sharp differences in people's views on COVID-19 in different cultures. Our study provides a computational approach to unveiling public emotions and concerns on the pandemic in real-time, which would potentially help policy-makers better understand people's need and thus make optimal policy.

cs.SI↗

arXiv2020

LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention

Non-autoregressive translation (NAT) models generate multiple tokens in one forward pass and is highly efficient at inference stage compared with autoregressive translation (AT) methods. However, NAT models often suffer from the multimodality problem, i.e., generating duplicated tokens or missing tokens. In this paper, we propose two novel methods to address this issue, the Look-Around (LA) strategy and the Vocabulary Attention (VA) mechanism. The Look-Around strategy predicts the neighbor tokens in order to predict the current token, and the Vocabulary Attention models long-term token dependencies inside the decoder by attending the whole vocabulary for each position to acquire knowledge of which token is about to generate. %We also propose a dynamic bidirectional decoding approach to accelerate the inference process of the LAVA model while preserving the high-quality of the generated output. Our proposed model uses significantly less time during inference compared with autoregressive models and most other NAT models. Our experiments on four benchmarks (WMT14 En$\rightarrow$De, WMT14 De$\rightarrow$En, WMT16 Ro$\rightarrow$En and IWSLT14 De$\rightarrow$En) show that the proposed model achieves competitive performance compared with the state-of-the-art non-autoregressive and autoregressive models while significantly reducing the time cost in inference phase.

cs.CL↗

arXiv2019

Query-Based Named Entity Recognition

In this paper, we propose a new strategy for the task of named entity recognition (NER). We cast the task as a query-based machine reading comprehension task: e.g., the task of extracting entities with PER is formalized as answering the question of "which person is mentioned in the text ?". Such a strategy comes with the advantage that it solves the long-standing issue of handling overlapping or nested entities (the same token that participates in more than one entity categories) with sequence-labeling techniques for NER. Additionally, since the query encodes informative prior knowledge, this strategy facilitates the process of entity extraction, leading to better performances. We experiment the proposed model on five widely used NER datasets on English and Chinese, including MSRA, Resume, OntoNotes, ACE04 and ACE05. The proposed model sets new SOTA results on all of these datasets.

cs.CL↗

arXiv2019

Is Word Segmentation Necessary for Deep Learning of Chinese Representations?

Segmenting a chunk of text into words is usually the first step of processing Chinese text, but its necessity has rarely been explored. In this paper, we ask the fundamental question of whether Chinese word segmentation (CWS) is necessary for deep learning-based Chinese Natural Language Processing. We benchmark neural word-based models which rely on word segmentation against neural char-based models which do not involve word segmentation in four end-to-end NLP benchmark tasks: language modeling, machine translation, sentence matching/paraphrase and text classification. Through direct comparisons between these two types of models, we find that char-based models consistently outperform word-based models. Based on these observations, we conduct comprehensive experiments to study why word-based models underperform char-based models in these deep learning-based NLP tasks. We show that it is because word-based models are more vulnerable to data sparsity and the presence of out-of-vocabulary (OOV) words, and thus more prone to overfitting. We hope this paper could encourage researchers in the community to rethink the necessity of word segmentation in deep learning-based Chinese Natural Language Processing. \footnote{Yuxian Meng and Xiaoya Li contributed equally to this paper.}

cs.CL↗

arXiv2019

Large-scale Pretraining for Neural Machine Translation with Tens of Billions of Sentence Pairs

In this paper, we investigate the problem of training neural machine translation (NMT) systems with a dataset of more than 40 billion bilingual sentence pairs, which is larger than the largest dataset to date by orders of magnitude. Unprecedented challenges emerge in this situation compared to previous NMT work, including severe noise in the data and prohibitively long training time. We propose practical solutions to handle these issues and demonstrate that large-scale pretraining significantly improves NMT performance. We are able to push the BLEU score of WMT17 Chinese-English dataset to 32.3, with a significant performance boost of +3.2 over existing state-of-the-art results.

cs.CL↗

arXiv2019

DSReg: Using Distant Supervision as a Regularizer

In this paper, we aim at tackling a general issue in NLP tasks where some of the negative examples are highly similar to the positive examples, i.e., hard-negative examples. We propose the distant supervision as a regularizer (DSReg) approach to tackle this issue. The original task is converted to a multi-task learning problem, in which distant supervision is used to retrieve hard-negative examples. The obtained hard-negative examples are then used as a regularizer. The original target objective of distinguishing positive examples from negative examples is jointly optimized with the auxiliary task objective of distinguishing softened positive (i.e., hard-negative examples plus positive examples) from easy-negative examples. In the neural context, this can be done by outputting the same representation from the last neural layer to different $softmax$ functions. Using this strategy, we can improve the performance of baseline models in a range of different NLP tasks, including text classification, sequence labeling and reading comprehension.

cs.CL↗

arXiv2013

Influence of the intensity gradient upon HHG from free electrons scattered by an intense laser beam

When an electron is scattered by a tightly-focused laser beam in vacuum, the intensity gradient is a critical factor to influence the electron dynamics, for example, the electron energy exchange with the laser fields as have been explored before [P.X.Wang et al.,J. Appl. Phys. 91, 856 (2002]. In this paper, we have further investigated its influence upon the electron high-harmonic generation (HHG) by treating the spacial gradient of the laser intensity as a ponderomotive potential. Based upon perturbative QED calculations, it has been found that the main effect of the intensity gradient is the broadening of the originally line HHG spectra. A one-to-one relationship can be built between the beam width and the corresponding line width. Hence this finding may provides us a promising way to measure the beam width of intense lasers in experiments. In addition, for a laser pulse, we have also studied the different influences from transverse and longitudinal intensity gradients upon HHG.

physics.optics↗

arXiv2013

The interference effect of laser-assisted bremsstrahlung emission in Coulumb fields of two nuclei

In this paper, the spontaneous bremsstrahlung emission from an electron scattered by two fixed nuclei in an intense laser field is investigated in details based upon the Volkov state and the Dirac-Volkov propagator. It has been found that the fundamental harmonic spectrum from the electron radiation exhibits distinctive fringes, which is dependent not only upon the internucleus distance and orientation, but also upon the initial energy of the electron and the laser intensity. By analyzing the differential cross section, we are able to explain these effects in terms of interference among the electron scattering by the nuclei. These results could have promising applications in probing the atomic or molecular dressed potentials in intense laser fields.

physics.optics↗

arXiv2007

Quark Mass Ratios in ChPT with the Difference of Quark Condensates Considered

The chiral effective Lagrangian for pseudoscalar nonet is constructed in consideration of isospin breaking. And the difference of quark condensates is taken accounted. The SU(3) singlet eta0 is not taken as Goldstone-boson. The mixing with and without isospin symmetry is considered. The quark mass ratios are obtained through solving the mass equations of mesons. We estimate the change of quark mass ratios according to the change of the masses of pion+ and kaon+ to see how the electromagnetic corrections affect our results. It turn out that massless up quark is possible. The upper limit for mu/md is around 0.39. 2ms/(mu+md)=24.23~25.12. The values for quark condensate ratios and other constants are limited in narrow ranges.

hep-ph↗