arXiv subjects

Hui Jiang

Publications and source records attributed to Hui Jiang.

At least 109 records · Page 6Linked to original sources

arXiv2016

Higher Order Recurrent Neural Networks

In this paper, we study novel neural network structures to better model long term dependency in sequential data. We propose to use more memory units to keep track of more preceding states in recurrent neural networks (RNNs), which are all recurrently fed to the hidden layers as feedback through different weighted paths. By extending the popular recurrent structure in RNNs, we provide the models with better short-term memory mechanism to learn long term dependency in sequences. Analogous to digital filters in signal processing, we call these structures as higher order RNNs (HORNNs). Similar to RNNs, HORNNs can also be learned using the back-propagation through time method. HORNNs are generally applicable to a variety of sequence modelling tasks. In this work, we have examined HORNNs for the language modeling task using two popular data sets, namely the Penn Treebank (PTB) and English text8 data sets. Experimental results have shown that the proposed HORNNs yield the state-of-the-art performance on both data sets, significantly outperforming the regular RNNs as well as the popular LSTMs.

cs.NE↗

arXiv2016

More on total monochromatic connection of graphs

A graph is said to be {\it total-colored} if all the edges and the vertices of the graph are colored. A total-coloring of a graph is a {\it total monochromatically-connecting coloring} ({\it TMC-coloring}, for short) if any two vertices of the graph are connected by a path whose edges and internal vertices on the path have the same color. For a connected graph $G$, the {\it total monochromatic connection number}, denoted by $tmc(G)$, is defined as the maximum number of colors used in a TMC-coloring of $G$. Note that a TMC-coloring does not exist if $G$ is not connected, in which case we simply let $tmc(G)=0$. In this paper, we first characterize all graphs of order $n$ and size $m$ with $tmc(G)=3,4,5,6,m+n-2,m+n-3$ and $m+n-4$, respectively. Then we determine the threshold function for a random graph to have $tmc(G)\geq f(n)$, where $f(n)$ is a function satisfying $1\leq f(n)<\frac{1}{2}n(n-1)+n$. Finally, we show that for a given connected graph $G$, and a positive integer $L$ with $L\leq m+n$, it is NP-complete to decide whether $tmc(G)\geq L$.

math.CO↗

arXiv2016

Part-of-Speech Relevance Weights for Learning Word Embeddings

This paper proposes a model to learn word embeddings with weighted contexts based on part-of-speech (POS) relevance weights. POS is a fundamental element in natural language. However, state-of-the-art word embedding models fail to consider it. This paper proposes to use position-dependent POS relevance weighting matrices to model the inherent syntactic relationship among words within a context window. We utilize the POS relevance weights to model each word-context pairs during the word embedding training process. The model proposed in this paper paper jointly optimizes word vectors and the POS relevance matrices. Experiments conducted on popular word analogy and word similarity tasks all demonstrated the effectiveness of the proposed method.

cs.CL↗

arXiv2016

A Deep Learning Based Fast Image Saliency Detection Algorithm

In this paper, we propose a fast deep learning method for object saliency detection using convolutional neural networks. In our approach, we use a gradient descent method to iteratively modify the input images based on the pixel-wise gradients to reduce a pre-defined cost function, which is defined to measure the class-specific objectness and clamp the class-irrelevant outputs to maintain image background. The pixel-wise gradients can be efficiently computed using the back-propagation algorithm. We further apply SLIC superpixels and LAB color based low level saliency features to smooth and refine the gradients. Our methods are quite computationally efficient, much faster than other deep learning based saliency methods. Experimental results on two benchmark tasks, namely Pascal VOC 2012 and MSRA10k, have shown that our proposed methods can generate high-quality salience maps, at least comparable with many slow and complicated deep learning methods. Comparing with the pure low-level methods, our approach excels in handling many difficult images, which contain complex background, highly-variable salient objects, multiple objects, and/or very small salient objects.

cs.CV↗

arXiv2016

Total monochromatic connection of graphs

A graph is said to be {\it total-colored} if all the edges and the vertices of the graph are colored. A path in a total-colored graph is a {\it total monochromatic path} if all the edges and internal vertices on the path have the same color. A total-coloring of a graph is a {\it total monochromatically-connecting coloring} ({\it TMC-coloring}, for short) if any two vertices of the graph are connected by a total monochromatic path of the graph. For a connected graph $G$, the {\it total monochromatic connection number}, denoted by $tmc(G)$, is defined as the maximum number of colors used in a TMC-coloring of $G$. These concepts are inspired by the concepts of monochromatic connection number $mc(G)$, monochromatic vertex connection number $mvc(G)$ and total rainbow connection number $trc(G)$ of a connected graph $G$. Let $l(T)$ denote the number of leaves of a tree $T$, and let $l(G)=\max\{ l(T) | $ $T$ is a spanning tree of $G$ $\}$ for a connected graph $G$. In this paper, we show that there are many graphs $G$ such that $tmc(G)=m-n+2+l(G)$, and moreover, we prove that for almost all graphs $G$, $tmc(G)=m-n+2+l(G)$ holds. Furthermore, we compare $tmc(G)$ with $mvc(G)$ and $mc(G)$, respectively, and obtain that there exist graphs $G$ such that $tmc(G)$ is not less than $mvc(G)$ and vice versa, and that $tmc(G)=mc(G)+l(G)$ holds for almost all graphs. Finally, we prove that $tmc(G)\leq mc(G)+mvc(G)$, and the equality holds if and only if $G$ is a complete graph.

math.CO↗

arXiv2016

Feedforward Sequential Memory Networks: A New Structure to Learn Long-term Dependency

In this paper, we propose a novel neural network structure, namely \emph{feedforward sequential memory networks (FSMN)}, to model long-term dependency in time series without using recurrent feedback. The proposed FSMN is a standard fully-connected feedforward neural network equipped with some learnable memory blocks in its hidden layers. The memory blocks use a tapped-delay line structure to encode the long context information into a fixed-size representation as short-term memory mechanism. We have evaluated the proposed FSMNs in several standard benchmark tasks, including speech recognition and language modelling. Experimental results have shown FSMNs significantly outperform the conventional recurrent neural networks (RNN), including LSTMs, in modeling sequential signals like speech or language. Moreover, FSMNs can be learned much more reliably and faster than RNNs or LSTMs due to the inherent non-recurrent model structure.

cs.NE↗

arXiv2015

Total proper connection of graphs

A graph is said to be {\it total-colored} if all the edges and the vertices of the graph is colored. A path in a total-colored graph is a {\it total proper path} if $(i)$ any two adjacent edges on the path differ in color, $(ii)$ any two internal adjacent vertices on the path differ in color, and $(iii)$ any internal vertex of the path differs in color from its incident edges on the path. A total-colored graph is called {\it total-proper connected} if any two vertices of the graph are connected by a total proper path of the graph. For a connected graph $G$, the {\it total proper connection number} of $G$, denoted by $tpc(G)$, is defined as the smallest number of colors required to make $G$ total-proper connected. These concepts are inspired by the concepts of proper connection number $pc(G)$, proper vertex connection number $pvc(G)$ and total rainbow connection number $trc(G)$ of a connected graph $G$. In this paper, we first determine the value of the total proper connection number $tpc(G)$ for some special graphs $G$. Secondly, we obtain that $tpc(G)\leq 4$ for any $2$-connected graph $G$ and give examples to show that the upper bound $4$ is sharp. For general graphs, we also obtain an upper bound for $tpc(G)$. Furthermore, we prove that $tpc(G)\leq \frac{3n}{δ+1}+1$ for a connected graph $G$ with order $n$ and minimum degree $δ$. Finally, we compare $tpc(G)$ with $pvc(G)$ and $pc(G)$, respectively, and obtain that $tpc(G)>pvc(G)$ for any nontrivial connected graph $G$, and that $tpc(G)$ and $pc(G)$ can differ by $t$ for $0\leq t\leq 2$.

math.CO↗

arXiv2015

Feedforward Sequential Memory Neural Networks without Recurrent Feedback

We introduce a new structure for memory neural networks, called feedforward sequential memory networks (FSMN), which can learn long-term dependency without using recurrent feedback. The proposed FSMN is a standard feedforward neural networks equipped with learnable sequential memory blocks in the hidden layers. In this work, we have applied FSMN to several language modeling (LM) tasks. Experimental results have shown that the memory blocks in FSMN can learn effective representations of long history. Experiments have shown that FSMN based language models can significantly outperform not only feedforward neural network (FNN) based LMs but also the popular recurrent neural network (RNN) LMs.

cs.NE↗

arXiv2015

Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks

Financial news contains useful information on public companies and the market. In this paper we apply the popular word embedding methods and deep neural networks to leverage financial news to predict stock price movements in the market. Experimental results have shown that our proposed methods are simple but very effective, which can significantly improve the stock prediction accuracy on a standard financial database over the baseline system using only the historical price information.

cs.CE↗

arXiv2015

A Fixed-Size Encoding Method for Variable-Length Sequences with its Application to Neural Network Language Models

In this paper, we propose the new fixed-size ordinally-forgetting encoding (FOFE) method, which can almost uniquely encode any variable-length sequence of words into a fixed-size representation. FOFE can model the word order in a sequence using a simple ordinally-forgetting mechanism according to the positions of words. In this work, we have applied FOFE to feedforward neural network language models (FNN-LMs). Experimental results have shown that without using any recurrent feedbacks, FOFE based FNN-LMs can significantly outperform not only the standard fixed-input FNN-LMs but also the popular RNN-LMs.

cs.NE↗

arXiv2015

Hybrid Orthogonal Projection and Estimation (HOPE): A New Framework to Probe and Learn Neural Networks

In this paper, we propose a novel model for high-dimensional data, called the Hybrid Orthogonal Projection and Estimation (HOPE) model, which combines a linear orthogonal projection and a finite mixture model under a unified generative modeling framework. The HOPE model itself can be learned unsupervised from unlabelled data based on the maximum likelihood estimation as well as discriminatively from labelled data. More interestingly, we have shown the proposed HOPE models are closely related to neural networks (NNs) in a sense that each hidden layer can be reformulated as a HOPE model. As a result, the HOPE framework can be used as a novel tool to probe why and how NNs work, more importantly, to learn NNs in either supervised or unsupervised ways. In this work, we have investigated the HOPE framework to learn NNs for several standard tasks, including image recognition on MNIST and speech recognition on TIMIT. Experimental results have shown that the HOPE framework yields significant performance gains over the current state-of-the-art methods in various types of NN learning problems, including unsupervised feature learning, supervised or semi-supervised learning.

cs.LG↗

arXiv2015

On (strong) proper vertex-connection of graphs

A path in a vertex-colored graph is a {\it vertex-proper path} if any two internal adjacent vertices differ in color. A vertex-colored graph is {\it proper vertex $k$-connected} if any two vertices of the graph are connected by $k$ disjoint vertex-proper paths of the graph. For a $k$-connected graph $G$, the {\it proper vertex $k$-connection number} of $G$, denoted by $pvc_{k}(G)$, is defined as the smallest number of colors required to make $G$ proper vertex $k$-connected. A vertex-colored graph is {\it strong proper vertex-connected}, if for any two vertices $u,v$ of the graph, there exists a vertex-proper $u$-$v$ geodesic. For a connected graph $G$, the {\it strong proper vertex-connection number} of $G$, denoted by $spvc(G)$, is the smallest number of colors required to make $G$ strong proper vertex-connected. These concepts are inspired by the concepts of rainbow vertex $k$-connection number $rvc_k(G)$, strong rainbow vertex-connection number $srvc(G)$, and proper $k$-connection number $pc_k(G)$ of a $k$-connected graph $G$. Firstly, we determine the value of $pvc(G)$ for general graphs and $pvc_k(G)$ for some specific graphs. We also compare the values of $pvc_k(G)$ and $pc_k(G)$. Then, sharp bounds of $spvc(G)$ are given for a connected graph $G$ of order $n$, that is, $0\leq spvc(G)\leq n-2$. Moreover, we characterize the graphs of order $n$ such that $spvc(G)=n-2,n-3$, respectively. Finally, we study the relationship among the three vertex-coloring parameters, namely, $spvc(G), \ srvc(G)$ and the chromatic number $χ(G)$ of a connected graph $G$.

math.CO↗

arXiv2015

Deep Learning for Object Saliency Detection and Image Segmentation

In this paper, we propose several novel deep learning methods for object saliency detection based on the powerful convolutional neural networks. In our approach, we use a gradient descent method to iteratively modify an input image based on the pixel-wise gradients to reduce a cost function measuring the class-specific objectness of the image. The pixel-wise gradients can be efficiently computed using the back-propagation algorithm. The discrepancy between the modified image and the original one may be used as a saliency map for the image. Moreover, we have further proposed several new training methods to learn saliency-specific convolutional nets for object saliency detection, in order to leverage the available pixel-wise segmentation information. Our methods are extremely computationally efficient (processing 20-40 images per second in one GPU). In this work, we use the computed saliency maps for image segmentation. Experimental results on two benchmark tasks, namely Microsoft COCO and Pascal VOC 2012, have shown that our proposed methods can generate high-quality salience maps, clearly outperforming many existing methods. In particular, our approaches excel in handling many difficult images, which contain complex background, highly-variable salient objects, multiple objects, and/or very small salient objects.

cs.CV↗

arXiv2015

Large deviations of the Threshold estimator of integrated (co-)volatility vector in the presence of jumps

Recently a considerable interest has been paid on the estimation problem of the realized volatility and covolatility by using high-frequency data of financial price processes in financial econometrics. Threshold estimation is one of the useful techniques in the inference for jump-type stochastic processes from discrete observations. In this paper, we adopt the threshold estimator introduced by Mancini where only the variations under a given threshold function are taken into account. The purpose of this work is to investigate large and moderate deviations for the threshold estimator of the integrated variance-covariance vector. This paper is an extension of the previous work in Djellout et al. where the problem has been studied in absence of the jump component. We will use the approximation lemma to prove the LDP. As the reader can expect we obtain the same results as in the case without jump.

math.PR↗

arXiv2015

Good upper bounds for the total rainbow connection of graphs

A total-colored graph is a graph $G$ such that both all edges and all vertices of $G$ are colored. A path in a total-colored graph $G$ is a total rainbow path if its edges and internal vertices have distinct colors. A total-colored graph $G$ is total-rainbow connected if any two vertices of $G$ are connected by a total rainbow path of $G$. The total rainbow connection number of $G$, denoted by $trc(G)$, is defined as the smallest number of colors that are needed to make $G$ total-rainbow connected. These concepts were introduced by Liu et al. Notice that for a connected graph $G$, $2diam(G)-1\leq trc(G)\leq 2n-3$, where $diam(G)$ denotes the diameter of $G$ and $n$ is the order of $G$. In this paper we show, for a connected graph $G$ of order $n$ with minimum degree $δ$, that $trc(G)\leq6n/{(δ+1)}+28$ for $δ\geq\sqrt{n-2}-1$ and $n\geq 291$, while $trc(G)\leq7n/{(δ+1)}+32$ for $16\leqδ\leq\sqrt{n-2}-2$ and $trc(G)\leq7n/{(δ+1)}+4C(δ)+12$ for $6\leqδ\leq15$, where $C(δ)=e^{\frac{3\log(δ^3+2δ^2+3)-3(\log3-1)}{δ-3}}-2$. This implies that when $δ$ is in linear with $n$, then the total rainbow number $trc(G)$ is a constant. We also show that $trc(G)\leq 7n/4-3$ for $δ=3$, $trc(G)\leq8n/5-13/5$ for $δ=4$ and $trc(G)\leq3n/2-3$ for $δ=5$. Furthermore, an example shows that our bound can be seen tight up to additive factors when $δ\geq\sqrt{n-2}-1$.

math.CO↗

arXiv2014

Robust estimation of isoform expression with RNA-Seq data

Qualifying gene and isoform expression is one of the primary tasks for RNA-Seq experiments. Given a sequence of counts representing numbers of reads mapped to different positions (exons and junctions) of isoforms, methods based on Poisson generalized linear models (GLM) with the identity link function have been proposed to estimate isoform expression levels from these counts. These Poisson based models have very limited ability in handling the overdispersion in the counts brought by various sources, and some of them are not robust to outliers. We propose a negative binomial based GLM with identity link, and use a set of robustified quasi-likelihood equations to make it resistant to outliers. An efficient and reliable numeric algorithm has been identified to solve these equations. In simulations, we find that our approach seems to outperform existing approaches. We also find evidence supporting this conclusion in real RNA-Seq data.

stat.AP↗

arXiv2014

Asymptotic distributions related to mildly-explosive second order autoregressive models

In this paper, we consider the normalized least squares estimator of the parameter in a mildly-explosive first-order autoregressive model with dependent errors which are modeled as a mildly-explosive AR(1) process. We prove that the estimator has a Cauchy limit law which provides a bridge between moderate deviation asymptotics and the earlier results on the local to unity and explosive autoregressive models. In particular, the results can be applied to understand the near-integrated second order autoregressive processes. Simulation studies are also carried out to assess the performance of least squares estimation in finite samples.

math.PR↗

arXiv2013

A penalized likelihood approach for robust estimation of isoform expression

Ultra high-throughput sequencing of transcriptomes (RNA-Seq) has enabled the accurate estimation of gene expression at individual isoform level. However, systematic biases introduced during the sequencing and mapping processes as well as incompleteness of the transcript annotation databases may cause the estimates of isoform abundances to be unreliable, and in some cases, highly inaccurate. This paper introduces a penalized likelihood approach to detect and correct for such biases in a robust manner. Our model extends those previously proposed by introducing bias parameters for reads. An L1 penalty is used for the selection of non-zero bias parameters. We introduce an efficient algorithm for model fitting and analyze the statistical properties of the proposed model. Our experimental studies on both simulated and real datasets suggest that the model has the potential to improve isoform-specific gene expression estimates and identify incompletely annotated gene models.

stat.ME↗