arXiv subjects

Jingzhou Liu

Publications and source records attributed to Jingzhou Liu.

23 records · Page 2Linked to original sources

arXiv2018

Stack-Pointer Networks for Dependency Parsing

We introduce a novel architecture for dependency parsing: \emph{stack-pointer networks} (\textbf{\textsc{StackPtr}}). Combining pointer networks~\citep{vinyals2015pointer} with an internal stack, the proposed model first reads and encodes the whole sentence, then builds the dependency tree top-down (from root-to-leaf) in a depth-first fashion. The stack tracks the status of the depth-first search and the pointer networks select one child for the word at the top of the stack at each step. The \textsc{StackPtr} parser benefits from the information of the whole sentence and all previously derived subtree structures, and removes the left-to-right restriction in classical transition-based parsers. Yet, the number of steps for building any (including non-projective) parse tree is linear in the length of the sentence just as other transition-based parsers, yielding an efficient decoding algorithm with $O(n^2)$ time complexity. We evaluate our model on 29 treebanks spanning 20 languages and different dependency annotation schemas, and achieve state-of-the-art performance on 21 of them.

cs.CL↗

arXiv2017

Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML

Reward augmented maximum likelihood (RAML), a simple and effective learning framework to directly optimize towards the reward function in structured prediction tasks, has led to a number of impressive empirical successes. RAML incorporates task-specific reward by performing maximum-likelihood updates on candidate outputs sampled according to an exponentiated payoff distribution, which gives higher probabilities to candidates that are close to the reference output. While RAML is notable for its simplicity, efficiency, and its impressive empirical successes, the theoretical properties of RAML, especially the behavior of the exponentiated payoff distribution, has not been examined thoroughly. In this work, we introduce softmax Q-distribution estimation, a novel theoretical interpretation of RAML, which reveals the relation between RAML and Bayesian decision theory. The softmax Q-distribution can be regarded as a smooth approximation of the Bayes decision boundary, and the Bayes decision rule is achieved by decoding with this Q-distribution. We further show that RAML is equivalent to approximately estimating the softmax Q-distribution, with the temperature $τ$ controlling approximation error. We perform two experiments, one on synthetic data of multi-class classification and one on real data of image captioning, to demonstrate the relationship between RAML and the proposed softmax Q-distribution estimation method, verifying our theoretical analysis. Additional experiments on three structured prediction tasks with rewards defined on sequential (named entity recognition), tree-based (dependency parsing) and irregular (machine translation) structures show notable improvements over maximum likelihood baselines.

cs.LG↗

arXiv2016

Visualizing Large-scale and High-dimensional Data

We study the problem of visualizing large-scale and high-dimensional data in a low-dimensional (typically 2D or 3D) space. Much success has been reported recently by techniques that first compute a similarity structure of the data points and then project them into a low-dimensional space with the structure preserved. These two steps suffer from considerable computational costs, preventing the state-of-the-art methods such as the t-SNE from scaling to large-scale and high-dimensional data (e.g., millions of data points and hundreds of dimensions). We propose the LargeVis, a technique that first constructs an accurately approximated K-nearest neighbor graph from the data and then layouts the graph in the low-dimensional space. Comparing to t-SNE, LargeVis significantly reduces the computational cost of the graph construction step and employs a principled probabilistic model for the visualization step, the objective of which can be effectively optimized through asynchronous stochastic gradient descent with a linear time complexity. The whole procedure thus easily scales to millions of high-dimensional data points. Experimental results on real-world data sets demonstrate that the LargeVis outperforms the state-of-the-art methods in both efficiency and effectiveness. The hyper-parameters of LargeVis are also much more stable over different data sets.

cs.LG↗

arXiv2004

The spread of disease with birth and death on networks

In this paper, we introduce a modified epidemic model on regular and scale-free networks respectively. We consider the birth rate $δ$, cure rate $γ$, infection rate $λ$, $α$ from the infectious disease, and death rate $β$ from other factors. Through mean-field analysis, we find that on regular network there is an epidemic threshold $λ_{c}$ dependent on the parameters $δ$, $γ$, $α$, and $β$; while for power law degree distribution network epidemic threshold is absent in the thermodynamic limit. The result is the same as that of the standard SIS model. This reminds us the structure of the networks plays a very important role in the spreading property of the infectious disease.

q-bio.PE↗

arXiv2004

The Spread of Infectious Disease with Household-Structure on the Complex Networks

In this paper we study the household-structure SIS epidemic spreading on general complex networks. The household structure gives us the way to distinguish inner and the outer infection rate. Unlike household-structure models on homogenous networks, such as regular and random networks, here we consider heterogeneous networks with arbitrary degree distribution p(k). First we introduce the epidemic model. Then rate equations under mean field appropriation and computer simulations are used here to analyze our model. Some unique phenomena only existing in divergent network with household structure is found, while we also get some similar conclusions that some simple geometrical quantities of networks have important impression on infection property of infectous disease. It seems that in our model even when local cure rate is greater than inner infection rate in every household, disease still can spread on scale-free network. It implies that no disease is spreading in every single household, but for the whole network, disease is spreading. Since our society network seems like this structure, maybe this conclusion remind us that during disease spreading we should pay more attention on network structure than local cure condition.

q-bio.PE↗