arXiv subjects

Hao Peng

Publications and source records attributed to Hao Peng.

406 records · Page 23Linked to original sources

arXiv2016

A Convolutional Attention Network for Extreme Summarization of Source Code

Attention mechanisms in neural networks have proved useful for problems in which the input and output do not have fixed dimension. Often there exist features that are locally translation invariant and would be valuable for directing the model's attention, but previous attentional architectures are not constructed to learn such features specifically. We introduce an attentional neural network that employs convolution on the input tokens to detect local time-invariant and long-range topical attention features in a context-dependent way. We apply this architecture to the problem of extreme summarization of source code snippets into short, descriptive function name-like summaries. Using those features, the model sequentially generates a summary by marginalizing over two attention mechanisms: one that predicts the next summary token based on the attention weights of the input tokens and another that is able to copy a code token as-is directly into the summary. We demonstrate our convolutional attention neural network's performance on 10 popular Java projects showing that it achieves better performance compared to previous attentional mechanisms.

cs.LG↗

arXiv2015

Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Path

Relation classification is an important research arena in the field of natural language processing (NLP). In this paper, we present SDP-LSTM, a novel neural network to classify the relation of two entities in a sentence. Our neural architecture leverages the shortest dependency path (SDP) between two entities; multichannel recurrent neural networks, with long short term memory (LSTM) units, pick up heterogeneous information along the SDP. Our proposed model has several distinct features: (1) The shortest dependency paths retain most relevant information (to relation classification), while eliminating irrelevant words in the sentence. (2) The multichannel LSTM networks allow effective information integration from heterogeneous sources over the dependency paths. (3) A customized dropout strategy regularizes the neural network to alleviate overfitting. We test our model on the SemEval 2010 relation classification task, and achieve an $F_1$-score of 83.7\%, higher than competing methods in the literature.

cs.CL↗

arXiv2015

A Comparative Study on Regularization Strategies for Embedding-based Neural Networks

This paper aims to compare different regularization strategies to address a common phenomenon, severe overfitting, in embedding-based neural networks for NLP. We chose two widely studied neural models and tasks as our testbed. We tried several frequently applied or newly proposed regularization strategies, including penalizing weights (embeddings excluded), penalizing embeddings, re-embedding words, and dropout. We also emphasized on incremental hyperparameter tuning, and combining different regularizations. The results provide a picture on tuning hyperparameters for neural NLP models.

cs.CL↗

arXiv2015

EigenGP: Gaussian Process Models with Adaptive Eigenfunctions

Gaussian processes (GPs) provide a nonparametric representation of functions. However, classical GP inference suffers from high computational cost for big data. In this paper, we propose a new Bayesian approach, EigenGP, that learns both basis dictionary elements--eigenfunctions of a GP prior--and prior precisions in a sparse finite model. It is well known that, among all orthogonal basis functions, eigenfunctions can provide the most compact representation. Unlike other sparse Bayesian finite models where the basis function has a fixed form, our eigenfunctions live in a reproducing kernel Hilbert space as a finite linear combination of kernel functions. We learn the dictionary elements--eigenfunctions--and the prior precisions over these elements as well as all the other hyperparameters from data by maximizing the model marginal likelihood. We explore computational linear algebra to simplify the gradient computation significantly. Our experimental results demonstrate improved predictive performance of EigenGP over alternative sparse GP methods as well as relevance vector machine.

cs.LG↗

arXiv2015

Discriminative Neural Sentence Modeling by Tree-Based Convolution

This paper proposes a tree-based convolutional neural network (TBCNN) for discriminative sentence modeling. Our models leverage either constituency trees or dependency trees of sentences. The tree-based convolution process extracts sentences' structural features, and these features are aggregated by max pooling. Such architecture allows short propagation paths between the output layer and underlying feature detectors, which enables effective structural feature learning and extraction. We evaluate our models on two tasks: sentiment analysis and question classification. In both experiments, TBCNN outperforms previous state-of-the-art results, including existing neural networks and dedicated feature/rule engineering. We also make efforts to visualize the tree-based convolution process, shedding light on how our models work.

cs.CL↗

arXiv2014

Building Program Vector Representations for Deep Learning

Deep learning has made significant breakthroughs in various fields of artificial intelligence. Advantages of deep learning include the ability to capture highly complicated features, weak involvement of human engineering, etc. However, it is still virtually impossible to use deep learning to analyze programs since deep architectures cannot be trained effectively with pure back propagation. In this pioneering paper, we propose the "coding criterion" to build program vector representations, which are the premise of deep learning for program analysis. Our representation learning approach directly makes deep learning a reality in this new field. We evaluate the learned vector representations both qualitatively and quantitatively. We conclude, based on the experiments, the coding criterion is successful in building program representations. To evaluate whether deep learning is beneficial for program analysis, we feed the representations to deep neural networks, and achieve higher accuracy in the program classification task than "shallow" methods, such as logistic regression and the support vector machine. This result confirms the feasibility of deep learning to analyze programs. It also gives primary evidence of its success in this new field. We believe deep learning will become an outstanding technique for program analysis in the near future.

cs.SE↗

arXiv2014

On Frankl and Furedi's conjecture for 3-uniform hypergraphs

The Lagrangian of a hypergraph has been a useful tool in hypergraph extremal problems. In most applications, we need an upper bound for the Lagrangian of a hypergraph. Frankl and Furedi in \cite{FF} conjectured that the $r$-graph with $m$ edges formed by taking the first $m$ sets in the colex ordering of ${\mathbb N}^{(r)}$ has the largest Lagrangian of all $r$-graphs with $m$ edges. In this paper, we give some partial results for this conjecture.

math.CO↗

arXiv2013

An extension of Motzkin-Straus Theorem to non-uniform hypergraphs and its applications

In 1965, Motzkin and Straus established a remarkable connection between the order of a maximum clique and the Lagrangian of a graph and provided a new proof of Turán's theorem using the connection. The connection of Lagrangians and Turán densities can be also used to prove the fundamental theorem of Erdös-Stone-Simonovits on Turán densities of graphs. Very recently, the study of Turán densities of non-uniform hypergraphs have been motivated by extremal poset problems. In this paper, we attempt to explore the applications of Lagrangian method in determining Turán densities of non-uniform hypergraphs. We first give a definition of the Lagrangian of a non-uniform hypergraph, then give an extension of Motzkin-Straus theorem to non-uniform hypergraphs whose edges contain 1 or 2 vertices. Applying it, we give an extension of Erdös-Stone-Simonovits theorem to non-uniform hypergraphs whose edges contain 1 or 2 vertices.

math.CO↗

arXiv2012

Nonperturbative tuning of an improved relativistic heavy-quark action with application to bottom spectroscopy

We calculate the masses of bottom mesons using an improved relativistic action for the b-quarks and the RBC/UKQCD Iwasaki gauge configurations with 2+1 flavors of dynamical domain-wall light quarks. We analyze configurations with two lattice spacings: a^{-1} = 1.729 GeV (a ~ 0.11 fm) and a^{-1} = 2.281 GeV (a ~ 0.086 fm). We use an anisotropic, clover-improved Wilson action for the b-quark, and tune the three parameters of the action nonperturbatively such that they reproduce the experimental values of the B_s and B_s* heavy-light meson states. The masses and mass-splittings of the low-lying bottomonium states (such as the eta_b and Upsilon) can then be computed with no additional inputs, and comparison between these predictions and experiment provides a test of the validity of our method. We obtain bottomonium masses with total uncertainties of ~0.5-0.6% and fine-structure splittings with uncertainties of ~35-45%; for all cases we find good agreement with experiment. The parameters of the relativistic heavy-quark action tuned for b-quarks presented in this work can be used for precise calculations of weak matrix elements such as B-meson decay constants and mixing parameters with lattice discretization errors that are of the same size as in light pseudoscalar meson quantities. This general method can also be used for charmed meson masses and matrix elements if the parameters of the heavy-quark action are appropriately tuned.

hep-lat↗

arXiv2011

An Anti-attack Model Based on Complex Network Theory in P2P networks

Complex network theory is a useful way to study many real systems. In this paper, an anti-attack model based on complex network theory is introduced. The mechanism of this model is based on dynamic compensation process and reverse percolation process in P2P networks. The main purpose of the paper is: (i) a dynamic compensation process can turn an attacked P2P network into a power-law (PL) network with exponential cutoff; (ii) a local healing process can restore the maximum degree of peers in an attacked P2P network to a normal level; (iii) a restoring process based on reverse percolation theory connects the fragmentary peers of an attacked P2P network together into a giant connected component. In this way, the model based on complex network theory can be effectively utilized for anti-attack and protection purposes in P2P networks.

cs.NI↗