arXiv subjects

Hui Jiang

Publications and source records attributed to Hui Jiang.

At least 73 records · Page 4Linked to original sources

arXiv2019

Dual-FOFE-net Neural Models for Entity Linking with PageRank

This paper presents a simple and computationally efficient approach for entity linking (EL), compared with recurrent neural networks (RNNs) or convolutional neural networks (CNNs), by making use of feedforward neural networks (FFNNs) and the recent dual fixed-size ordinally forgetting encoding (dual-FOFE) method to fully encode the sentence fragment and its left/right contexts into a fixed-size representation. Furthermore, in this work, we propose to incorporate PageRank based distillation in our candidate generation module. Our neural linking models consist of three parts: a PageRank based candidate generation module, a dual-FOFE-net neural ranking model and a simple NIL entity clustering system. Experimental results have shown that our proposed neural linking models achieved higher EL accuracy than state-of-the-art models on the TAC2016 task dataset over the baseline system, without requiring any in-house data or complicated handcrafted features. Moreover, it achieves a competitive accuracy on the TAC2017 task dataset.

cs.CL↗

arXiv2019

Bandlimiting Neural Networks Against Adversarial Attacks

In this paper, we study the adversarial attack and defence problem in deep learning from the perspective of Fourier analysis. We first explicitly compute the Fourier transform of deep ReLU neural networks and show that there exist decaying but non-zero high frequency components in the Fourier spectrum of neural networks. We demonstrate that the vulnerability of neural networks towards adversarial samples can be attributed to these insignificant but non-zero high frequency components. Based on this analysis, we propose to use a simple post-averaging technique to smooth out these high frequency components to improve the robustness of neural networks against adversarial attacks. Experimental results on the ImageNet dataset have shown that our proposed method is universally effective to defend many existing adversarial attacking methods proposed in the literature, including FGSM, PGD, DeepFool and C&W attacks. Our post-averaging method is simple since it does not require any re-training, and meanwhile it can successfully defend over 95% of the adversarial samples generated by these methods without introducing any significant performance degradation (less than 1%) on the original clean images.

cs.LG↗

arXiv2019

Content based News Recommendation via Shortest Entity Distance over Knowledge Graphs

Content-based news recommendation systems need to recommend news articles based on the topics and content of articles without using user specific information. Many news articles describe the occurrence of specific events and named entities including people, places or objects. In this paper, we propose a graph traversal algorithm as well as a novel weighting scheme for cold-start content based news recommendation utilizing these named entities. Seeking to create a higher degree of user-specific relevance, our algorithm computes the shortest distance between named entities, across news articles, over a large knowledge graph. Moreover, we have created a new human annotated data set for evaluating content based news recommendation systems. Experimental results show our method is suitable to tackle the hard cold-start problem and it produces stronger Pearson correlation to human similarity scores than other cold-start methods. Our method is also complementary and a combination with the conventional cold-start recommendation methods may yield significant performance gains. The dataset, CNRec, is available at: https://github.com/kevinj22/CNRec

cs.IR↗

arXiv2019

The Lower The Simpler: Simplifying Hierarchical Recurrent Models

To improve the training efficiency of hierarchical recurrent models without compromising their performance, we propose a strategy named as `the lower the simpler', which is to simplify the baseline models by making the lower layers simpler than the upper layers. We carry out this strategy to simplify two typical hierarchical recurrent models, namely Hierarchical Recurrent Encoder-Decoder (HRED) and R-NET, whose basic building block is GRU. Specifically, we propose Scalar Gated Unit (SGU), which is a simplified variant of GRU, and use it to replace the GRUs at the middle layers of HRED and R-NET. Besides, we also use Fixed-size Ordinally-Forgetting Encoding (FOFE), which is an efficient encoding method without any trainable parameter, to replace the GRUs at the bottom layers of HRED and R-NET. The experimental results show that the simplified HRED and the simplified R-NET contain significantly less trainable parameters, consume significantly less training time, and achieve slightly better performance than their baseline models.

cs.CL↗

arXiv2019

Explicit Utilization of General Knowledge in Machine Reading Comprehension

To bridge the gap between Machine Reading Comprehension (MRC) models and human beings, which is mainly reflected in the hunger for data and the robustness to noise, in this paper, we explore how to integrate the neural networks of MRC models with the general knowledge of human beings. On the one hand, we propose a data enrichment method, which uses WordNet to extract inter-word semantic connections as general knowledge from each given passage-question pair. On the other hand, we propose an end-to-end MRC model named as Knowledge Aided Reader (KAR), which explicitly uses the above extracted general knowledge to assist its attention mechanisms. Based on the data enrichment method, KAR is comparable in performance with the state-of-the-art MRC models, and significantly more robust to noise than them. When only a subset (20%-80%) of the training examples are available, KAR outperforms the state-of-the-art MRC models by a large margin, and is still reasonably robust to noise.

cs.AI↗

arXiv2019

A Multi-task Learning Approach for Named Entity Recognition using Local Detection

Named entity recognition (NER) systems that perform well require task-related and manually annotated datasets. However, they are expensive to develop, and are thus limited in size. As there already exists a large number of NER datasets that share a certain degree of relationship but differ in content, it is important to explore the question of whether such datasets can be combined as a simple method for improving NER performance. To investigate this, we developed a novel locally detecting multitask model using FFNNs. The model relies on encoding variable-length sequences of words into theoretically lossless and unique fixed-size representations. We applied this method to several well-known NER tasks and compared the results of our model to baseline models as well as other published results. As a result, we observed competitive performance in nearly all of the tasks.

cs.CL↗

arXiv2019

Effective Context and Fragment Feature Usage for Named Entity Recognition

In this paper, we explore a new approach to named entity recognition (NER) with the goal of learning from context and fragment features more effectively, contributing to the improvement of overall recognition performance. We use the recent fixed-size ordinally forgetting encoding (FOFE) method to fully encode each sentence fragment and its left-right contexts into a fixed-size representation. Next, we organize the context and fragment features into groups, and feed each feature group to dedicated fully-connected layers. Finally, we merge each group's final dedicated layers and add a shared layer leading to a single output. The outcome of our experiments show that, given only tokenized text and trained word embeddings, our system outperforms our baseline models, and is competitive to the state-of-the-arts of various well-known NER tasks.

cs.CL↗

arXiv2019

A General FOFE-net Framework for Simple and Effective Question Answering over Knowledge Bases

Question answering over knowledge base (KB-QA) has recently become a popular research topic in NLP. One popular way to solve the KB-QA problem is to make use of a pipeline of several NLP modules, including entity discovery and linking (EDL) and relation detection. Recent success on KB-QA task usually involves complex network structures with sophisticated heuristics. Inspired by a previous work that builds a strong KB-QA baseline, we propose a simple but general neural model composed of fixed-size ordinally forgetting encoding (FOFE) and deep neural networks, called FOFE-net to solve KB-QA problem at different stages. For evaluation, we use two popular KB-QA datasets, SimpleQuestions and WebQSP, and a newly created dataset, FreebaseQA. The experimental results show that FOFE-net performs well on KB-QA subtasks, entity discovery and linking (EDL) and relation detection, and in turn pushing overall KB-QA system to achieve strong results on all datasets.

cs.CL↗

arXiv2019

A New Perspective on Machine Learning: How to do Perfect Supervised Learning

In this work, we introduce the concept of bandlimiting into the theory of machine learning because all physical processes are bandlimited by nature, including real-world machine learning tasks. After the bandlimiting constraint is taken into account, our theoretical analysis has shown that all practical machine learning tasks are asymptotically solvable in a perfect sense. Furthermore, the key towards this solvability almost solely relies on two factors: i) a sufficiently large amount of training samples beyond a threshold determined by a difficulty measurement of the underlying task; ii) a sufficiently complex and bandlimited model. Moreover, for some special cases, we have derived new error bounds for perfect learning, which can quantify the difficulty of learning. These generalization bounds are not only asymptotically convergent but also irrelevant to model complexity. Our new results on generalization have provided a new perspective to explain the recent successes of large-scale supervised learning using complex models like neural networks.

cs.LG↗

arXiv2019

Fixed-Size Ordinally Forgetting Encoding Based Word Sense Disambiguation

In this paper, we present our method of using fixed-size ordinally forgetting encoding (FOFE) to solve the word sense disambiguation (WSD) problem. FOFE enables us to encode variable-length sequence of words into a theoretically unique fixed-size representation that can be fed into a feed forward neural network (FFNN), while keeping the positional information between words. In our method, a FOFE-based FFNN is used to train a pseudo language model over unlabelled corpus, then the pre-trained language model is capable of abstracting the surrounding context of polyseme instances in labelled corpus into context embeddings. Next, we take advantage of these context embeddings towards WSD classification. We conducted experiments on several WSD data sets, which demonstrates that our proposed method can achieve comparable performance to that of the state-of-the-art approach at the expense of much lower computational cost.

cs.CL↗

arXiv2019

Topological classification of non-Hermitian systems with reflection symmetry

We classify topological phases of non-Hermitian systems in the Altland-Zirnbauer classes with an additional reflection symmetry in all dimensions. By mapping the non-Hermitian system into an enlarged Hermitian Hamiltonian with an enforced chiral symmetry, our topological classification is thus equivalent to classifying Hermitian systems with both chiral and reflection symmetries, which effectively change the classifying space and shift the periodical table of topological phases. According to our classification tables, we provide concrete examples for all topologically nontrivial non-Hermitian classes in one dimension and also give explicitly the topological invariant for each nontrivial example. Our results show that there exist two kinds of topological invariants composed of either winding numbers or $\mathbb{Z}_2$ numbers. By studying the corresponding lattice models under the open boundary condition, we unveil the existence of bulk-edge correspondence for the one-dimensional topological non-Hermitian systems characterized by winding numbers, however we did not observe the bulk-edge correspondence for the $\mathbb{Z}_2$ topological number in our studied $\mathbb{Z}_2$-type model.

cond-mat.mes-hall↗

arXiv2019

Interplay of non-Hermitian skin effects and Anderson localization in non-reciprocal quasiperiodic lattices

Non-Hermiticity from non-reciprocal hoppings has been shown recently to demonstrate the non-Hermitian skin effect (NHSE) under open boundary conditions (OBCs). Here we study the interplay of this effect and the Anderson localization in a \textit{non-reciprocal} quasiperiodic lattice, dubbed non-reciprocal Aubry-André model, and a \textit{rescaled} transition point is exactly proved. The non-reciprocity can induce not only the NHSE, but also the asymmetry in localized states with two Lyapunov exponents for both sides. Meanwhile, this transition is also topological, characterized by a winding number associated with the complex eigenenergies under periodic boundary conditions (PBCs), establishing a \textit{bulk-bulk} correspondence. This interplay can be realized by an elaborately designed electronic circuit with only linear passive RLC devices instead of elusive non-reciprocal ones, where the transport of a continuous wave undergoes a transition between insulating and amplifying. This initiative scheme can be immediately applied in experiments to other non-reciprocal models, and will definitely inspires the study of interplay of NHSEs and more other quantum/topological phenomena.

cond-mat.mes-hall↗

arXiv2018

DropFilter: A Novel Regularization Method for Learning Convolutional Neural Networks

The past few years have witnessed the fast development of different regularization methods for deep learning models such as fully-connected deep neural networks (DNNs) and Convolutional Neural Networks (CNNs). Most of previous methods mainly consider to drop features from input data and hidden layers, such as Dropout, Cutout and DropBlocks. DropConnect select to drop connections between fully-connected layers. By randomly discard some features or connections, the above mentioned methods control the overfitting problem and improve the performance of neural networks. In this paper, we proposed two novel regularization methods, namely DropFilter and DropFilter-PLUS, for the learning of CNNs. Different from the previous methods, DropFilter and DropFilter-PLUS selects to modify the convolution filters. For DropFilter-PLUS, we find a suitable way to accelerate the learning process based on theoretical analysis. Experimental results on MNIST show that using DropFilter and DropFilter-PLUS may improve performance on image classification tasks.

cs.LG↗

arXiv2018

Topological invariants and phase diagrams for one-dimensional two-band non-Hermitian systems without chiral symmetry

We study topological properties of one-dimensional non-Hermitian systems without chiral symmetry and give phase diagrams characterized by topological invariants $ν_E$ and $ν_{total}$, associated with complex energy vorticity and summation of Berry phases of complex bands, respectively. In the absence of chiral symmetry, we find that the phase diagram determined by $ν_E$ is different from $ν_{tot}$. While the transition between phases with different $ν_{E}$ is closely related to the band-touching point, the transition between different $ν_{tot}$ is irrelevant to the band-touching condition. We give an interpretation for the discrepancy from the geometrical view by analyzing the relation of topological invariants with the winding numbers associated with exception points of the system. We then generalize the fidelity approach to study the phase transition in the non-Hermitian system and find that transition between phases with different $ν_{tot}$ can be well characterized by an abrupt change of fidelity and fidelity susceptibility around the transition point.

cond-mat.mes-hall↗

arXiv2018

P-splines with an l1 penalty for repeated measures

P-splines are penalized B-splines, in which finite order differences in coefficients are typically penalized with an $\ell_2$ norm. P-splines can be used for semiparametric regression and can include random effects to account for within-subject variability. In addition to $\ell_2$ penalties, $\ell_1$-type penalties have been used in nonparametric and semiparametric regression to achieve greater flexibility, such as in locally adaptive regression splines, $\ell_1$ trend filtering, and the fused lasso additive model. However, there has been less focus on using $\ell_1$ penalties in P-splines, particularly for estimating conditional means. In this paper, we demonstrate the potential benefits of using an $\ell_1$ penalty in P-splines with an emphasis on fitting non-smooth functions. We propose an estimation procedure using the alternating direction method of multipliers and cross validation, and provide degrees of freedom and approximate confidence bands based on a ridge approximation to the $\ell_1$ penalized fit. We also demonstrate potential uses through simulations and an application to electrodermal activity data collected as part of a stress study.

stat.ME↗

arXiv2018

Distributed Layered Grant-Free Non-Orthogonal Multiple Access for Massive MTC

Grant-free transmission is considered as a promising technology to support sporadic data transmission in massive machine-type communications (mMTC). Due to the distributed manner, high collision probability is an inherent drawback of grant-free access techniques. Non-orthogonal multiple access (NOMA) is expected to be used in uplink grant-free transmission, multiplying connection opportunities by exploiting power domain resources. However, it is usually applied for coordinated transmissions where the base station performs coordination with full channel state information, which is not suitable for grant-free techniques. In this paper, we propose a novel distributed layered grant-free NOMA framework. Under this framework, we divide the cell into different layers based on predetermined inter-layer received power difference. A distributed layered grant-free NOMA based hybrid transmission scheme is proposed to reduce collision probability. Moreover, we derive the closed-form expression of connection throughput. A joint access control and NOMA layer selection (JACNLS) algorithm is proposed to solve the connection throughput optimization problem. The numerical and simulation results reveal that, when the system is overloaded, our proposed scheme outperforms the grant-free-only scheme by three orders of magnitude in terms of expected connection throughput and outperforms coordinated OMA transmission schemes by 31.25\% with only 0.0189\% signaling overhead of the latter.

cs.IT↗

arXiv2018

Using Neural Network for Identifying Clickbaits in Online News Media

Online news media sometimes use misleading headlines to lure users to open the news article. These catchy headlines that attract users but disappointed them at the end, are called Clickbaits. Because of the importance of automatic clickbait detection in online medias, lots of machine learning methods were proposed and employed to find the clickbait headlines. In this research, a model using deep learning methods is proposed to find the clickbaits in Clickbait Challenge 2017's dataset. The proposed model gained the first rank in the Clickbait Challenge 2017 in terms of Mean Squared Error. Also, data analytics and visualization techniques are employed to explore and discover the provided dataset to get more insight from the data.

cs.CL↗

arXiv2018

Geometrical meaning of winding number and its characterization of topological phases in one-dimensional chiral non-Hermitian systems

We unveil the geometrical meaning of winding number and utilize it to characterize the topological phases in one-dimensional chiral non-Hermitian systems. While chiral symmetry ensures the winding number of Hermitian systems being integers, it can take half integers for non-Hermitian systems. We give a geometrical interpretation of the half integers by demonstrating that the winding number $ν$ of a non-Hermitian system is equal to half of the summation of two winding numbers $ν_1$ and $ν_2$ associated with two exceptional points respectively. The winding numbers $ν_1$ and $ν_2$ represent the times of real part of the Hamiltonian in momentum space encircling the exceptional points and can only take integers. We further find that the difference of $ν_1$ and $ν_2$ is related to the second winding number or energy vorticity. By applying our scheme to a non-Hermitian Su-Schrieffer-Heeger model and an extended version of it, we show that the topologically different phases can be well characterized by winding numbers. Furthermore, we demonstrate that the existence of left and right zero-mode edge states is closely related to the winding number $ν_1$ and $ν_2$.

cond-mat.mes-hall↗