arXiv subjects

Yao Ma

Publications and source records attributed to Yao Ma.

At least 181 records · Page 10Linked to original sources

arXiv2018

Automata Guided Reinforcement Learning With Demonstrations

Tasks with complex temporal structures and long horizons pose a challenge for reinforcement learning agents due to the difficulty in specifying the tasks in terms of reward functions as well as large variances in the learning signals. We propose to address these problems by combining temporal logic (TL) with reinforcement learning from demonstrations. Our method automatically generates intrinsic rewards that align with the overall task goal given a TL task specification. The policy resulting from our framework has an interpretable and hierarchical structure. We validate the proposed method experimentally on a set of robotic manipulation tasks.

cs.AI↗

arXiv2018

Signed Graph Convolutional Network

Due to the fact much of today's data can be represented as graphs, there has been a demand for generalizing neural network models for graph data. One recent direction that has shown fruitful results, and therefore growing interest, is the usage of graph convolutional neural networks (GCNs). They have been shown to provide a significant improvement on a wide range of tasks in network analysis, one of which being node representation learning. The task of learning low-dimensional node representations has shown to increase performance on a plethora of other tasks from link prediction and node classification, to community detection and visualization. Simultaneously, signed networks (or graphs having both positive and negative links) have become ubiquitous with the growing popularity of social media. However, since previous GCN models have primarily focused on unsigned networks (or graphs consisting of only positive links), it is unclear how they could be applied to signed networks due to the challenges presented by negative links. The primary challenges are based on negative links having not only a different semantic meaning as compared to positive links, but their principles are inherently different and they form complex relations with positive links. Therefore we propose a dedicated and principled effort that utilizes balance theory to correctly aggregate and propagate the information across layers of a signed GCN model. We perform empirical experiments comparing our proposed signed GCN against state-of-the-art baselines for learning node representations in signed networks. More specifically, our experiments are performed on four real-world datasets for the classical link sign prediction problem that is commonly used as the benchmark for signed network embeddings algorithms.

cs.SI↗

arXiv2018

Linked Recurrent Neural Networks

Recurrent Neural Networks (RNNs) have been proven to be effective in modeling sequential data and they have been applied to boost a variety of tasks such as document classification, speech recognition and machine translation. Most of existing RNN models have been designed for sequences assumed to be identically and independently distributed (i.i.d). However, in many real-world applications, sequences are naturally linked. For example, web documents are connected by hyperlinks; and genes interact with each other. On the one hand, linked sequences are inherently not i.i.d., which poses tremendous challenges to existing RNN models. On the other hand, linked sequences offer link information in addition to the sequential information, which enables unprecedented opportunities to build advanced RNN models. In this paper, we study the problem of RNN for linked sequences. In particular, we introduce a principled approach to capture link information and propose a linked Recurrent Neural Network (LinkedRNN), which models sequential and link information coherently. We conduct experiments on real-world datasets from multiple domains and the experimental results validate the effectiveness of the proposed framework.

cs.CL↗

arXiv2018

Multi-dimensional Graph Convolutional Networks

Convolutional neural networks (CNNs) leverage the great power in representation learning on regular grid data such as image and video. Recently, increasing attention has been paid on generalizing CNNs to graph or network data which is highly irregular. Some focus on graph-level representation learning while others aim to learn node-level representations. These methods have been shown to boost the performance of many graph-level tasks such as graph classification and node-level tasks such as node classification. Most of these methods have been designed for single-dimensional graphs where a pair of nodes can only be connected by one type of relation. However, many real-world graphs have multiple types of relations and they can be naturally modeled as multi-dimensional graphs with each type of relation as a dimension. Multi-dimensional graphs bring about richer interactions between dimensions, which poses tremendous challenges to the graph convolutional neural networks designed for single-dimensional graphs. In this paper, we study the problem of graph convolutional networks for multi-dimensional graphs and propose a multi-dimensional convolutional neural network model mGCN aiming to capture rich information in learning node-level representations for multi-dimensional graphs. Comprehensive experiments on real-world multi-dimensional graphs demonstrate the effectiveness of the proposed framework.

cs.SI↗

arXiv2018

Automata-Guided Hierarchical Reinforcement Learning for Skill Composition

Skills learned through (deep) reinforcement learning often generalizes poorly across domains and re-training is necessary when presented with a new task. We present a framework that combines techniques in \textit{formal methods} with \textit{reinforcement learning} (RL). The methods we provide allows for convenient specification of tasks with logical expressions, learns hierarchical policies (meta-controller and low-level controllers) with well-defined intrinsic rewards, and construct new skills from existing ones with little to no additional exploration. We evaluate the proposed methods in a simple grid world simulation as well as a more complicated kitchen environment in AI2Thor

cs.AI↗

arXiv2018

Emerging Market Corporate Bonds as First-to-Default Baskets

Emerging market hard-currency bonds are an asset class of growing importance, and contain exposure to an EM sovereign and the underlying industry. The authors investigate how to model this as a modification of the well-known first-to-default (FtD) basket, using the structural model, and find the approach feasible.

q-fin.PR↗

arXiv2017

Preserving Local and Global Information for Network Embedding

Networks such as social networks, airplane networks, and citation networks are ubiquitous. The adjacency matrix is often adopted to represent a network, which is usually high dimensional and sparse. However, to apply advanced machine learning algorithms to network data, low-dimensional and continuous representations are desired. To achieve this goal, many network embedding methods have been proposed recently. The majority of existing methods facilitate the local information i.e. local connections between nodes, to learn the representations, while completely neglecting global information (or node status), which has been proven to boost numerous network mining tasks such as link prediction and social recommendation. Hence, it also has potential to advance network embedding. In this paper, we study the problem of preserving local and global information for network embedding. In particular, we introduce an approach to capture global information and propose a network embedding framework LOG, which can coherently model {\bf LO}cal and {\bf G}lobal information. Experimental results demonstrate the ability to preserve global information of the proposed framework. Further experiments are conducted to demonstrate the effectiveness of learned representations of the proposed framework.

cs.SI↗

arXiv2017

A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks

Reward engineering is an important aspect of reinforcement learning. Whether or not the user's intentions can be correctly encapsulated in the reward function can significantly impact the learning outcome. Current methods rely on manually crafted reward functions that often require parameter tuning to obtain the desired behavior. This operation can be expensive when exploration requires systems to interact with the physical world. In this paper, we explore the use of temporal logic (TL) to specify tasks in reinforcement learning. TL formula can be translated to a real-valued function that measures its level of satisfaction against a trajectory. We take advantage of this function and propose temporal logic policy search (TLPS), a model-free learning technique that finds a policy that satisfies the TL specification. A set of simulated experiments are conducted to evaluate the proposed approach.

cs.AI↗

arXiv2017

Crowdsourcing with Sparsely Interacting Workers

We consider estimation of worker skills from worker-task interaction data (with unknown labels) for the single-coin crowd-sourcing binary classification model in symmetric noise. We define the (worker) interaction graph whose nodes are workers and an edge between two nodes indicates whether or not the two workers participated in a common task. We show that skills are asymptotically identifiable if and only if an appropriate limiting version of the interaction graph is irreducible and has odd-cycles. We then formulate a weighted rank-one optimization problem to estimate skills based on observations on an irreducible, aperiodic interaction graph. We propose a gradient descent scheme and show that for such interaction graphs estimates converge asymptotically to the global minimum. We characterize noise robustness of the gradient scheme in terms of spectral properties of signless Laplacians of the interaction graph. We then demonstrate that a plug-in estimator based on the estimated skills achieves state-of-art performance on a number of real-world datasets. Our results have implications for rank-one matrix completion problem in that gradient descent can provably recover $W \times W$ rank-one matrices based on $W+1$ off-diagonal observations of a connected graph with a single odd-cycle.

cs.LG↗

arXiv2016

Triple Derivations and Triple Homomorphisms of Perfect Lie Superalgebras

In this paper, we study triple derivations and triple homomorphisms of perfect Lie superalgebras over a commutative ring $R$. It is proved that, if the base ring contains $\frac{1}{2}$, $L$ is a perfect Lie superalgebra with zero center, then every triple derivation of $L$ is a derivation, and every triple derivation of the derivation algebra $ Der (L)$ is an inner derivation. Let $L,~L^{'}$ be Lie superalgebras over a commutative ring $R$, the notion of triple homomorphism from $L$ to $L^{'}$ is introduced. We proved that, under certain assumptions, homomorphisms, anti-homomorphisms, and sums of homomorphisms and anti-homomorphisms are all triple homomorphisms.

math.RA↗

arXiv2016

Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning

In PU learning, a binary classifier is trained from positive (P) and unlabeled (U) data without negative (N) data. Although N data is missing, it sometimes outperforms PN learning (i.e., ordinary supervised learning). Hitherto, neither theoretical nor experimental analysis has been given to explain this phenomenon. In this paper, we theoretically compare PU (and NU) learning against PN learning based on the upper bounds on estimation errors. We find simple conditions when PU and NU learning are likely to outperform PN learning, and we prove that, in terms of the upper bounds, either PU or NU learning (depending on the class-prior probability and the sizes of P and N data) given infinite U data will improve on PN learning. Our theoretical findings well agree with the experimental results on artificial and benchmark data even when the experimental setup does not match the theoretical assumptions exactly.

cs.LG↗

arXiv2015

Some structure theories of Leibniz triple systems

In this paper, we investigate the Leibniz triple system $T$ and its universal Leibniz envelope $U(T)$. The involutive automorphism of $U(T)$ determining $T$ is introduced, which gives a characterization of the $\Z_2$-grading of $U(T)$. We give the relationship between the solvable radical $R(T)$ of $T$ and $Rad(U(T))$, the solvable radical of $U(T)$. Further, Levi's theorem for Leibniz triple systems is obtained. Moreover, the relationship between the nilpotent radical of $T$ and that of $U(T)$ is studied. Finally, we introduce the notion of representations of a Leibniz triple system.

math.RA↗

arXiv2015

Online Markov decision processes with policy iteration

The online Markov decision process (MDP) is a generalization of the classical Markov decision process that incorporates changing reward functions. In this paper, we propose practical online MDP algorithms with policy iteration and theoretically establish a sublinear regret bound. A notable advantage of the proposed algorithm is that it can be easily combined with function approximation, and thus large and possibly continuous state spaces can be efficiently handled. Through experiments, we demonstrate the usefulness of the proposed algorithm.

cs.LG↗

arXiv2015

Bandit-Based Task Assignment for Heterogeneous Crowdsourcing

We consider a task assignment problem in crowdsourcing, which is aimed at collecting as many reliable labels as possible within a limited budget. A challenge in this scenario is how to cope with the diversity of tasks and the task-dependent reliability of workers, e.g., a worker may be good at recognizing the name of sports teams, but not be familiar with cosmetics brands. We refer to this practical setting as heterogeneous crowdsourcing. In this paper, we propose a contextual bandit formulation for task assignment in heterogeneous crowdsourcing, which is able to deal with the exploration-exploitation trade-off in worker selection. We also theoretically investigate the regret bounds for the proposed method, and demonstrate its practical usefulness experimentally.

cs.LG↗

arXiv2015

One-parameter formal deformations of Hom-Lie-Yamaguti algebras

This paper studies one-parameter formal deformations of Hom-Lie-Yamaguti algebras. The first, second and third cohomology groups on Hom-Lie-Yamaguti algebras extending ones on Lie-Yamaguti algebras are provided. It is proved that first and second cohomology groups are suitable to the deformation theory involving infinitesimals, equivalent deformations and rigidity. However, the third cohomology group is not suitable for the obstructions.

math.RA↗

arXiv2014

Generalized Derivations of Lie triple systems

In this paper, we present some basic properties concerning the derivation algebra ${\rm Der}(T)$, the quasiderivation algebra ${\rm QDer}(T)$ and the generalized derivation algebra ${\rm GDer}(T)$ of a Lie triple system $T$, with the relationship ${\rm Der}(T)\subseteq {\rm QDer}(T)\subseteq {\rm GDer}(T)\subseteq {\rm End}(T)$. Furthermore, we completely determine those Lie triple systems $T$ with condition ${\rm QDer}(T)={\rm End}(T)$. We also show that the quasiderivations of $T$ can be embedded as derivations in a larger Lie triple system.

math.RA↗

arXiv2014

Low-Temperature Pseudogap Phenomenon: Precursor of High-$T_c$ Superconductivity

In this paper, we try to understand the pseudogap phenomenon observed in the cuprate superconductor through a model study. Specifically, we explore the so-called low-temperature pseudogap state by turning off the superconducting off diagonal long range order in an ansatz state for the $t$-$J$ model [New Journal of Physics {\bf 13}, 103039 (2011)]. Besides strong non-Gaussian superconducting fluctuations, the resulting state also exhibits a systematic pseudogap behavior in both spin and charge degrees of freedom, manifested in the uniform spin susceptibility, specific heat, non-Drude resistivity, Nernst effect, as well as the quantum oscillation associated with small Fermi pockets emerging in strong magnetic fields, etc. These anomalous `normal state' properties are found in qualitative consistency with experimental measurements in the cuprates. Such a model study establishes an intrinsic connection between the peculiar pseudogap properties and the non-BCS nature of the superconducting ground state. Critical comparison with other approaches to the doped Mott insulator is also made.

cond-mat.supr-con↗

arXiv2014

Generalized Derivations of Hom-Lie Superalgebras

In this paper, we give some basic properties of the generalized derivation algebra ${\rm GDer}(L)$ of a Hom-Lie superalgebra $L$. In particular, we prove that ${\rm GDer}(L) = {\rm QDer}(L) + {\rm QC}(L)$, the sum of the quasiderivation algebra and the quasicentroid. We also prove that ${\rm QDer}(L)$ can be embedded as derivations in a larger Hom-Lie superalgebra.

math.RA↗