arXiv subjects

Tao Lei

Publications and source records attributed to Tao Lei.

65 records · Page 4Linked to original sources

arXiv2017

Style Transfer from Non-Parallel Text by Cross-Alignment

This paper focuses on style transfer on the basis of non-parallel text. This is an instance of a broad family of problems including machine translation, decipherment, and sentiment modification. The key challenge is to separate the content from other aspects such as style. We assume a shared latent content distribution across different text corpora, and propose a method that leverages refined alignment of latent representations to perform style transfer. The transferred sentences from one style should match example sentences from the other style as a population. We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.

cs.CL↗

arXiv2017

Deriving Neural Architectures from Sequence and Graph Kernels

The design of neural architectures for structured objects is typically guided by experimental insights rather than a formal process. In this work, we appeal to kernels over combinatorial structures, such as sequences and graphs, to derive appropriate neural operations. We introduce a class of deep recurrent neural operations and formally characterize their associated kernel spaces. Our recurrent modules compare the input to virtual reference objects (cf. filters in CNN) via the kernels. Similar to traditional neural operations, these reference objects are parameterized and directly optimized in end-to-end training. We empirically evaluate the proposed class of neural architectures on standard applications such as language modeling and molecular graph regression, achieving state-of-the-art results across these applications.

cs.NE↗

arXiv2017

Small trees in supercritical random forests

We study the scaling limit of random forest with prescribed degree sequence in the regime that the largest tree consists of all but a vanishing fraction of nodes. We give a description of the limit of the forest consisting of the small trees, by relating plane forest to marked cyclic forest and its corresponding lattice path.

math.PR↗

arXiv2017

Scaling limit of random forests with prescribed degree sequences

In this paper, we consider the random plane forest uniformly drawn from all possible plane forests with a given degree sequence. Under suitable conditions on the degree sequences, we consider the limit of a sequence of such forests with the number of vertices tends to infinity in terms of Gromov-Hausdorff-Prokhorov topology. This work falls into the general framework of showing convergence of random combinatorial structures to certain Gromov-Hausdorff scaling limits, described in terms of the Brownian Continuum Random Tree, pioneered by the work of Aldous. In fact we identify the limiting random object as a sequence of random real trees encoded by excursions of some first passage bridges reflected at minimum. We establish such convergence by studying the associated Lukasiewicz walk of the degree sequences. In particular, our work is closely related to and uses the results from the recent work of Broutin and Marckert on scaling limit of random trees with prescribed degree sequences, and the work of Addario-Berry on tail bounds of the height of a random tree with prescribed degree sequence.

math.PR↗

arXiv2017

Gravitational Coupling from Active Gravity

We attempt to construct a gravitational coupling by pre-selecting an energy-momentum tensor as the source for gravitational field. The energy-momentum tensor we take is a recently derived new expression motivated by joint localization of energy and momentum in quantum measurement. This energy-momentum tensor differs from the traditional canonical and symmetric ones, and the theory we obtain is of an Einstein-Cartan type, but derived from a minimal coupling of a Lagrangian with second-derivative, and leads to additional interaction between torsion and matter, including the scalar field. For the scalar field, the theory can also be derived in the Riemann space-time by a non-minimal coupling. Our study gives hint on more general tests of general relativistic effects.

gr-qc↗

arXiv2017

New Energy-Momentum and Angular Momentum Tensors with Applications to Nucleon Structure

We present a new type of energy-momentum tensor and angular momentum tensor. They are motivated by a special consideration in quantum measurement: Given a wave in mutual eigen-state of more than one physical observables, the corresponding physical currents should be proportional to each other. Interestingly, this criterion denies the traditional canonical and symmetric expressions of energy-momentum tensor and their associated expressions of angular momentum tensor. The new tensors we propose can be derived as Noether currents from a Lagrangian with second derivative, and shed new light on the study of nucleon structures.

physics.gen-ph↗

arXiv2016

Rationalizing Neural Predictions

Prediction without justification has limited applicability. As a remedy, we learn to extract pieces of input text as justifications -- rationales -- that are tailored to be short and coherent, yet sufficient for making the same prediction. Our approach combines two modular components, generator and encoder, which are trained to operate well together. The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction. Rationales are never given during training. Instead, the model is regularized by desiderata for rationales. We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases. Our approach outperforms attention-based baseline by a significant margin. We also successfully illustrate the method on the question retrieval task.

cs.CL↗

arXiv2016

Semi-supervised Question Retrieval with Gated Convolutions

Question answering forums are rapidly growing in size with no effective automated ability to refer to and reuse answers already available for previous posted questions. In this paper, we develop a methodology for finding semantically related questions. The task is difficult since 1) key pieces of information are often buried in extraneous details in the question body and 2) available annotations on similar questions are scarce and fragmented. We design a recurrent and convolutional model (gated convolution) to effectively map questions to their semantic representations. The models are pre-trained within an encoder-decoder framework (from body to title) on the basis of the entire raw corpus, and fine-tuned discriminatively from limited annotations. Our evaluation demonstrates that our model yields substantial gains over a standard IR baseline and various neural network architectures (including CNNs, LSTMs and GRUs).

cs.CL↗

arXiv2015

Molding CNNs for text: non-linear, non-consecutive convolutions

The success of deep learning often derives from well-chosen operational building blocks. In this work, we revise the temporal convolution operation in CNNs to better adapt it to text processing. Instead of concatenating word representations, we appeal to tensor algebra and use low-rank n-gram tensors to directly exploit interactions between words already at the convolution stage. Moreover, we extend the n-gram convolution to non-consecutive words to recognize patterns with intervening words. Through a combination of low-rank tensors, and pattern weighting, we can efficiently evaluate the resulting convolution operation via dynamic programming. We test the resulting architecture on standard sentiment classification and news categorization tasks. Our model achieves state-of-the-art performance both in terms of accuracy and training speed. For instance, we obtain 51.2% accuracy on the fine-grained sentiment classification task.

cs.CL↗

arXiv2014

Effects of Oxygen Adsorption on the Surface State of Epitaxial Silicene on Ag(111)

Epitaxial silicene, which is one single layer of silicon atoms packed in a honeycomb structure, demonstrates a strong interaction with the substrate that dramatically affects its electronic structure. The role of electronic coupling in the chemical reactivity between the silicene and the substrate is still unclear so far, which is of great importance for functionalization of silicene layers. Here, we report the reconstructions and hybridized electronic structures of epitaxial 4x4 silicene on Ag(111), which are revealed by scanning tunneling microscopy and angle-resolved photoemission spectroscopy. The hybridization between Si and Ag results in a metallic surface state, which can gradually decay due to oxygen adsorption. X-ray photoemission spectroscopy confirms the decoupling of Si-Ag bonds after oxygen treatment as well as the relatively oxygen resistance of Ag(111) surface, in contrast to 4x4 silicene [with respect to Ag(111)]. First-principles calculations have confirmed the evolution of the electronic structure of silicene during oxidation. It has been verified experimentally and theoretically that the high chemical activity of 4x4 silicene is attributable to the Si pz state, while the Ag(111) substrate exhibits relatively inert chemical behavior.

cond-mat.mes-hall↗

arXiv2012

The mixing time of the Newman--Watts small world

"Small worlds" are large systems in which any given node has only a few connections to other points, but possessing the property that all pairs of points are connected by a short path, typically logarithmic in the number of nodes. The use of random walks for sampling a uniform element from a large state space is by now a classical technique; to prove that such a technique works for a given network, a bound on the mixing time is required. However, little detailed information is known about the behaviour of random walks on small-world networks, though many predictions can be found in the physics literature. The principal contribution of this paper is to show that for a famous small-world random graph model known as the Newman--Watts small world, the mixing time is of order (log n)^2. This confirms a prediction of Richard Durrett, who proved a lower bound of order (log n)^2 and an upper bound of order (log n)^3.

math.PR↗