arXiv subjects

Jingjing Xu

Publications and source records attributed to Jingjing Xu.

84 records · Page 5Linked to original sources

arXiv2018

DP-GAN: Diversity-Promoting Generative Adversarial Network for Generating Informative and Diversified Text

Existing text generation methods tend to produce repeated and "boring" expressions. To tackle this problem, we propose a new text generation model, called Diversity-Promoting Generative Adversarial Network (DP-GAN). The proposed model assigns low reward for repeatedly generated text and high reward for "novel" and fluent text, encouraging the generator to produce diverse and informative text. Moreover, we propose a novel language-model based discriminator, which can better distinguish novel text from repeated text without the saturation problem compared with existing classifier-based discriminators. The experimental results on review generation and dialogue generation tasks demonstrate that our model can generate substantially more diverse and informative text than existing baselines. The code is available at https://github.com/lancopku/DPGAN

cs.CL↗

arXiv2018

Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach

The goal of sentiment-to-sentiment "translation" is to change the underlying sentiment of a sentence while keeping its content. The main challenge is the lack of parallel data. To solve this problem, we propose a cycled reinforcement learning method that enables training on unpaired data by collaboration between a neutralization module and an emotionalization module. We evaluate our approach on two review datasets, Yelp and Amazon. Experimental results show that our approach significantly outperforms the state-of-the-art systems. Especially, the proposed method substantially improves the content preservation performance. The BLEU score is improved from 1.64 to 22.46 and from 0.56 to 14.06 on the two datasets, respectively.

cs.CL↗

arXiv2018

Layered structure and leveled function of a human brain

The anatomically layered structure of a human brain results in leveled functions. In all these levels of different functions, comparison, feedback and imitation are the universal and crucial mechanisms. Languages, symbols and tools play key roles in the development of human brain and entire civilization.

q-bio.NC↗

arXiv2017

A memory mechanism based on two dimensional code of neurosome pattern

We have recognized that 2D codes, i.e., a group of strongly connected neurosomes that can be simultaneously excited, are the basic data carriers for memory in a brain. An echoing mechanism between two neighboring layers of neurosomes is assumed to establish temporary memory, and repeating processes enhance the formation of long-term memory. Creation and degradation of memory information are statistically. The maximum capacity of memory storage in a human brain is estimated to be one billion of 2D codes. By triggering one or more neurosomes in a neurosome-based 2D code, the whole strongly connected neurosome network is capable of exciting simultaneously and projecting its excitation onto an analysis layer of neurons in cortex, thus retrieving the stored memory data. The capability of comparing two 2D codes in the analysis layer is one of the major brain functions.

q-bio.NC↗

arXiv2017

Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning

In recent years, neural networks have proven to be effective in Chinese word segmentation. However, this promising performance relies on large-scale training data. Neural networks with conventional architectures cannot achieve the desired results in low-resource datasets due to the lack of labelled training data. In this paper, we propose a deep stacking framework to improve the performance on word segmentation tasks with insufficient data by integrating datasets from diverse domains. Our framework consists of two parts, domain-based models and deep stacking networks. The domain-based models are used to learn knowledge from different datasets. The deep stacking networks are designed to integrate domain-based models. To reduce model conflicts, we innovatively add communication paths among models and design various structures of deep stacking networks, including Gaussian-based Stacking Networks, Concatenate-based Stacking Networks, Sequence-based Stacking Networks and Tree-based Stacking Networks. We conduct experiments on six low-resource datasets from various domains. Our proposed framework shows significant performance improvements on all datasets compared with several strong baselines.

cs.CL↗

arXiv2017

Improving Social Media Text Summarization by Learning Sentence Weight Distribution

Recently, encoder-decoder models are widely used in social media text summarization. However, these models sometimes select noise words in irrelevant sentences as part of a summary by error, thus declining the performance. In order to inhibit irrelevant sentences and focus on key information, we propose an effective approach by learning sentence weight distribution. In our model, we build a multi-layer perceptron to predict sentence weights. During training, we use the ROUGE score as an alternative to the estimated sentence weight, and try to minimize the gap between estimated weights and predicted weights. In this way, we encourage our model to focus on the key sentences, which have high relevance with the summary. Experimental results show that our approach outperforms baselines on a large-scale social media corpus.

cs.CL↗

arXiv2017

Shallow Discourse Parsing with Maximum Entropy Model

In recent years, more research has been devoted to studying the subtask of the complete shallow discourse parsing, such as indentifying discourse connective and arguments of connective. There is a need to design a full discourse parser to pull these subtasks together. So we develop a discourse parser turning the free text into discourse relations. The parser includes connective identifier, arguments identifier, sense classifier and non-explicit identifier, which connects with each other in pipeline. Each component applies the maximum entropy model with abundant lexical and syntax features extracted from the Penn Discourse Tree-bank. The head-based representation of the PDTB is adopted in the arguments identifier, which turns the problem of indentifying the arguments of discourse connective into finding the head and end of the arguments. In the non-explicit identifier, the contextual type features like words which have high frequency and can reflect the discourse relation are introduced to improve the performance of non-explicit identifier. Compared with other methods, experimental results achieve the considerable performance.

cs.CL↗

arXiv2017

Minimal Effort Back Propagation for Convolutional Neural Networks

As traditional neural network consumes a significant amount of computing resources during back propagation, \citet{Sun2017mePropSB} propose a simple yet effective technique to alleviate this problem. In this technique, only a small subset of the full gradients are computed to update the model parameters. In this paper we extend this technique into the Convolutional Neural Network(CNN) to reduce calculation in back propagation, and the surprising results verify its validity in CNN: only 5\% of the gradients are passed back but the model still achieves the same effect as the traditional CNN, or even better. We also show that the top-$k$ selection of gradients leads to a sparse calculation in back propagation, which may bring significant computational benefits for high computational complexity of convolution operation in CNN.

cs.LG↗

arXiv2017

Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network

Recent studies have shown effectiveness in using neural networks for Chinese word segmentation. However, these models rely on large-scale data and are less effective for low-resource datasets because of insufficient training data. We propose a transfer learning method to improve low-resource word segmentation by leveraging high-resource corpora. First, we train a teacher model on high-resource corpora and then use the learned knowledge to initialize a student model. Second, a weighted data similarity method is proposed to train the student model on low-resource data. Experiment results show that our work significantly improves the performance on low-resource datasets: 2.3% and 1.5% F-score on PKU and CTB datasets. Furthermore, this paper achieves state-of-the-art results: 96.1%, and 96.2% F-score on PKU and CTB datasets.

cs.CL↗

arXiv2017

Wireless Communication in Biosystems

Electrical signals play the primary role in rapid communication among organs, tissues and cells in biosystems. We have analyzed and discussed three phenomena of electrical communication in biosystems, including direction-changing movement of paramecia, prey behavior of flytraps, and simultaneous discharge of electric ells. These discussions lead to a conclusion that in biosystems the electrical signals are mainly soliton-like electromagnetic pulses, which are generated by transient transmembrane ion currents through protein ion-channels. These electromagnetic signals mainly propagate along the softmaterial waveguide which is basically composed of dielectric membrane, sandwiched in two ionic liquid layers at both sides. In myelinated axons, the dielectric layer is as thick as several microns as it consists of tens of lipid bilayers. This transmission model implies that a wireless communication mechanism has been naturally developed in biosystem. This hypothesis may shed some light on the working mechanism of ten billion neurons in a human brain.

physics.bio-ph↗

arXiv2017

Improving Semantic Relevance for Sequence-to-Sequence Learning of Chinese Social Media Text Summarization

Current Chinese social media text summarization models are based on an encoder-decoder framework. Although its generated summaries are similar to source texts literally, they have low semantic relevance. In this work, our goal is to improve semantic relevance between source texts and summaries for Chinese social media summarization. We introduce a Semantic Relevance Based neural model to encourage high semantic similarity between texts and summaries. In our model, the source text is represented by a gated attention encoder, while the summary representation is produced by a decoder. Besides, the similarity score between the representations is maximized during training. Our experiments show that the proposed model outperforms baseline systems on a social media corpus.

cs.CL↗

arXiv2014

Quantum spatial-periodic harmonic model for daily price-limited stock markets

We investigate the behavior of stocks in daily price-limited stock markets by purposing a quantum spatial-periodic harmonic model. The stock price is presumed to oscillate and damp in a quantum spatial-periodic harmonic oscillator potential well. Complicated non-linear relations including inter-band positive correlation and intra-band negative correlation between the volatility and the trading volume of stocks are derived by considering the energy band structure of the model. The validity of price limitation is then examined and abnormal phenomena of a price-limited stock market (Shanghai Stock Exchange) of China are studied by applying our quantum model.

q-fin.GN↗