Searcharxiv⌕ Search

arXiv subjects

James Zhang

Publications and source records attributed to James Zhang.

At least 37 records · Page 2Linked to original sources
arXiv2022

Learning Large-scale Universal User Representation with Sparse Mixture of Experts

Learning user sequence behaviour embedding is very sophisticated and challenging due to the complicated feature interactions over time and high dimensions of user features. Recent emerging foundation models, e.g., BERT and its variants, encourage a large body of researchers to investigate in this field. However, unlike natural language processing (NLP) tasks, the parameters of user behaviour model come mostly from user embedding layer, which makes most existing works fail in training a universal user embedding of large scale. Furthermore, user representations are learned from multiple downstream tasks, and the past research work do not address the seesaw phenomenon. In this paper, we propose SUPERMOE, a generic framework to obtain high quality user representation from multiple tasks. Specifically, the user behaviour sequences are encoded by MoE transformer, and we can thus increase the model capacity to billions of parameters, or even to trillions of parameters. In order to deal with seesaw phenomenon when learning across multiple tasks, we design a new loss function with task indicators. We perform extensive offline experiments on public datasets and online experiments on private real-world business scenarios. Our approach achieves the best performance over state-of-the-art models, and the results demonstrate the effectiveness of our framework.

cs.LG↗
arXiv2022

A Meta Reinforcement Learning Approach for Predictive Autoscaling in the Cloud

Predictive autoscaling (autoscaling with workload forecasting) is an important mechanism that supports autonomous adjustment of computing resources in accordance with fluctuating workload demands in the Cloud. In recent works, Reinforcement Learning (RL) has been introduced as a promising approach to learn the resource management policies to guide the scaling actions under the dynamic and uncertain cloud environment. However, RL methods face the following challenges in steering predictive autoscaling, such as lack of accuracy in decision-making, inefficient sampling and significant variability in workload patterns that may cause policies to fail at test time. To this end, we propose an end-to-end predictive meta model-based RL algorithm, aiming to optimally allocate resource to maintain a stable CPU utilization level, which incorporates a specially-designed deep periodic workload prediction model as the input and embeds the Neural Process to guide the learning of the optimal scaling actions over numerous application services in the Cloud. Our algorithm not only ensures the predictability and accuracy of the scaling strategy, but also enables the scaling decisions to adapt to the changing workloads with high sample efficiency. Our method has achieved significant performance improvement compared to the existing algorithms and has been deployed online at Alipay, supporting the autoscaling of applications for the world-leading payment platform.

cs.LG↗
arXiv2022

Unit Ball Model for Embedding Hierarchical Structures in the Complex Hyperbolic Space

Learning the representation of data with hierarchical structures in the hyperbolic space attracts increasing attention in recent years. Due to the constant negative curvature, the hyperbolic space resembles tree metrics and captures the tree-like properties naturally, which enables the hyperbolic embeddings to improve over traditional Euclidean models. However, many real-world hierarchically structured data such as taxonomies and multitree networks have varying local structures and they are not trees, thus they do not ubiquitously match the constant curvature property of the hyperbolic space. To address this limitation of hyperbolic embeddings, we explore the complex hyperbolic space, which has the variable negative curvature, for representation learning. Specifically, we propose to learn the embeddings of hierarchically structured data in the unit ball model of the complex hyperbolic space. The unit ball model based embeddings have a more powerful representation capacity to capture a variety of hierarchical structures. Through experiments on synthetic and real-world data, we show that our approach improves over the hyperbolic embedding models significantly. We also explore the competence of complex hyperbolic geometry on the multitree structure and $1$-$N$ structure.

cs.LG↗
arXiv2022

Variational Policy Propagation for Multi-agent Reinforcement Learning

We propose a \emph{collaborative} multi-agent reinforcement learning algorithm named variational policy propagation (VPP) to learn a \emph{joint} policy through the interactions over agents. We prove that the joint policy is a Markov Random Field under some mild conditions, which in turn reduces the policy space effectively. We integrate the variational inference as special differentiable layers in policy such that the actions can be efficiently sampled from the Markov Random Field and the overall policy is differentiable. We evaluate our algorithm on several large scale challenging tasks and demonstrate that it outperforms previous state-of-the-arts.

cs.LG↗
arXiv2020

Model Embedding Model-Based Reinforcement Learning

Model-based reinforcement learning (MBRL) has shown its advantages in sample-efficiency over model-free reinforcement learning (MFRL). Despite the impressive results it achieves, it still faces a trade-off between the ease of data generation and model bias. In this paper, we propose a simple and elegant model-embedding model-based reinforcement learning (MEMB) algorithm in the framework of the probabilistic reinforcement learning. To balance the sample-efficiency and model bias, we exploit both real and imaginary data in the training. In particular, we embed the model in the policy update and learn $Q$ and $V$ functions from the real data set. We provide the theoretical analysis of MEMB with the Lipschitz continuity assumption on the model and policy. At last, we evaluate MEMB on several benchmarks and demonstrate our algorithm can achieve state-of-the-art performance.

cs.LG↗
arXiv2020

Neural Physicist: Learning Physical Dynamics from Image Sequences

We present a novel architecture named Neural Physicist (NeurPhy) to learn physical dynamics directly from image sequences using deep neural networks. For any physical system, given the global system parameters, the time evolution of states is governed by the underlying physical laws. How to learn meaningful system representations in an end-to-end way and estimate accurate state transition dynamics facilitating long-term prediction have been long-standing challenges. In this paper, by leveraging recent progresses in representation learning and state space models (SSMs), we propose NeurPhy, which uses variational auto-encoder (VAE) to extract underlying Markovian dynamic state at each time step, neural process (NP) to extract the global system parameters, and a non-linear non-recurrent stochastic state space model to learn the physical dynamic transition. We apply NeurPhy to two physical experimental environments, i.e., damped pendulum and planetary orbits motion, and achieve promising results. Our model can not only extract the physically meaningful state representations, but also learn the state transition dynamics enabling long-term predictions for unseen image sequences. Furthermore, from the manifold dimension of the latent state space, we can easily identify the degree of freedom (DoF) of the underlying physical systems.

cs.LG↗
arXiv2020

Riemannian Proximal Policy Optimization

In this paper, We propose a general Riemannian proximal optimization algorithm with guaranteed convergence to solve Markov decision process (MDP) problems. To model policy functions in MDP, we employ Gaussian mixture model (GMM) and formulate it as a nonconvex optimization problem in the Riemannian space of positive semidefinite matrices. For two given policy functions, we also provide its lower bound on policy improvement by using bounds derived from the Wasserstein distance of GMMs. Preliminary experiments show the efficacy of our proposed Riemannian proximal policy optimization algorithm.

cs.LG↗
arXiv2019

Noncommutative cyclic isolated singularities

The question of whether a noncommutative graded quotient singularity $A^G$ is isolated depends on a subtle invariant of the $G$-action on $A$, called the pertinency. We prove a partial dichotomy theorem for isolatedness, which applies to a family of noncommutative quotient singularities arising from a graded cyclic action on the $(-1)$-skew polynomial ring. Our results generalize and extend some results of Bao, He and the third-named author and results of Gaddis, Kirkman, Moore and Won.

math.RA↗
arXiv2018

McKay Correspondence for semisimple Hopf actions on regular graded algebras, I

In establishing a more general version of the McKay correspondence, we prove Auslander's theorem for actions of semisimple Hopf algebras H on noncommutative Artin-Schelter regular algebras A of global dimension two, where A is a graded H-module algebra, and the Hopf action on A is inner faithful with trivial homological determinant. We also show that each fixed ring A^H under such an action arises an analogue of a coordinate ring of a Kleinian singularity.

math.RA↗
arXiv2018

Anomaly detection in wide area network mesh using two machine learning anomaly detection algorithms

Anomaly detection is the practice of identifying items or events that do not conform to an expected behavior or do not correlate with other items in a dataset. It has previously been applied to areas such as intrusion detection, system health monitoring, and fraud detection in credit card transactions. In this paper, we describe a new method for detecting anomalous behavior over network performance data, gathered by perfSONAR, using two machine learning algorithms: Boosted Decision Trees (BDT) and Simple Feedforward Neural Network. The effectiveness of each algorithm was evaluated and compared. Both have shown sufficient performance and sensitivity.

cs.NI↗
arXiv2017

McKay Correspondence for semisimple Hopf actions on regular graded algebras, II

We continue our study of the McKay Correspondence for grading preserving actions of semisimple Hopf algebras H on (noncommutative) Artin-Schelter regular algebras A. Here, we establish correspondences between module categories over A^H, over A\#H, and over \End_{A^H} A. We also study homological properties of (endomorphism rings of) maximal Cohen-Macaulay modules over A^H.

math.RA↗
arXiv2014

Quantum binary polyhedral groups and their actions on quantum planes

We classify quantum analogues of actions of finite subgroups G of SL_2(k) on commutative polynomial rings k[u,v]. More precisely, we produce a classification of pairs (H,R), where H is a finite dimensional Hopf algebra that acts inner faithfully and preserves the grading of an Artin-Schelter regular algebra R of global dimension two. Remarkably, the corresponding invariant rings R^H share similar regularity and Gorenstein properties as the invariant rings k[u,v]^G in the classic setting. We also present several questions and directions for expanding this work in noncommutative invariant theory.

math.RA↗
arXiv2014

Hopf actions and Nakayama automorphisms

Let H be a Hopf algebra with antipode S, and let A be an N-Koszul Artin-Schelter regular algebra. We study connections between the Nakayama automorphism of A and S^2 of H when H coacts on A inner-faithfully. Several applications pertaining to Hopf actions on Artin-Schelter regular algebras are given.

math.RA↗