arXiv subjects

Ziyi Wang

Publications and source records attributed to Ziyi Wang.

At least 199 records · Page 11Linked to original sources

arXiv2021

Itinerary-aware Personalized Deep Matching at Fliggy

Matching items for a user from a travel item pool of large cardinality have been the most important technology for increasing the business at Fliggy, one of the most popular online travel platforms (OTPs) in China. There are three major challenges facing OTPs: sparsity, diversity, and implicitness. In this paper, we present a novel Fliggy ITinerary-aware deep matching NETwork (FitNET) to address these three challenges. FitNET is designed based on the popular deep matching network, which has been successfully employed in many industrial recommendation systems, due to its effectiveness. The concept itinerary is firstly proposed under the context of recommendation systems for OTPs, which is defined as the list of unconsumed orders of a user. All orders in a user itinerary are learned as a whole, based on which the implicit travel intention of each user can be more accurately inferred. To alleviate the sparsity problem, users' profiles are incorporated into FitNET. Meanwhile, a series of itinerary-aware attention mechanisms that capture the vital interactions between user's itinerary and other input categories are carefully designed. These mechanisms are very helpful in inferring a user's travel intention or preference, and handling the diversity in a user's need. Further, two training objectives, i.e., prediction accuracy of user's travel intention and prediction accuracy of user's click behavior, are utilized by FitNET, so that these two objectives can be optimized simultaneously. An offline experiment on Fliggy production dataset with over 0.27 million users and 1.55 million travel items, and an online A/B test both show that FitNET effectively learns users' travel intentions, preferences, and diverse needs, based on their itineraries and gains superior performance compared with state-of-the-art methods. FitNET now has been successfully deployed at Fliggy, serving major online traffic.

cs.IR↗

arXiv2021

LHRM: A LBS based Heterogeneous Relations Model for User Cold Start Recommendation in Online Travel Platform

Most current recommender systems used the historical behaviour data of user to predict user' preference. However, it is difficult to recommend items to new users accurately. To alleviate this problem, existing user cold start methods either apply deep learning to build a cross-domain recommender system or map user attributes into the space of user behaviour. These methods are more challenging when applied to online travel platform (e.g., Fliggy), because it is hard to find a cross-domain that user has similar behaviour with travel scenarios and the Location Based Services (LBS) information of users have not been paid sufficient attention. In this work, we propose a LBS-based Heterogeneous Relations Model (LHRM) for user cold start recommendation, which utilizes user's LBS information and behaviour information in related domains and user's behaviour information in travel platforms (e.g., Fliggy) to construct the heterogeneous relations between users and items. Moreover, an attention-based multi-layer perceptron is applied to extract latent factors of users and items. Through this way, LHRM has better generalization performance than existing methods. Experimental results on real data from Fliggy's offline log illustrate the effectiveness of LHRM.

cs.IR↗

arXiv2021

Learning in Nonzero-Sum Stochastic Games with Potentials

Multi-agent reinforcement learning (MARL) has become effective in tackling discrete cooperative game scenarios. However, MARL has yet to penetrate settings beyond those modelled by team and zero-sum games, confining it to a small subset of multi-agent systems. In this paper, we introduce a new generation of MARL learners that can handle nonzero-sum payoff structures and continuous settings. In particular, we study the MARL problem in a class of games known as stochastic potential games (SPGs) with continuous state-action spaces. Unlike cooperative games, in which all agents share a common reward, SPGs are capable of modelling real-world scenarios where agents seek to fulfil their individual goals. We prove theoretically our learning method, SPot-AC, enables independent agents to learn Nash equilibrium strategies in polynomial time. We demonstrate our framework tackles previously unsolvable tasks such as Coordination Navigation and large selfish routing games and that it outperforms the state of the art MARL baselines such as MADDPG and COMIX in such scenarios.

cs.MA↗

arXiv2021

Large-Scale Multi-Agent Deep FBSDEs

In this paper we present a scalable deep learning framework for finding Markovian Nash Equilibria in multi-agent stochastic games using fictitious play. The motivation is inspired by theoretical analysis of Forward Backward Stochastic Differential Equations (FBSDE) and their implementation in a deep learning setting, which is the source of our algorithm's sample efficiency improvement. By taking advantage of the permutation-invariant property of agents in symmetric games, the scalability and performance is further enhanced significantly. We showcase superior performance of our framework over the state-of-the-art deep fictitious play algorithm on an inter-bank lending/borrowing problem in terms of multiple metrics. More importantly, our approach scales up to 3000 agents in simulation, a scale which, to the best of our knowledge, represents a new state-of-the-art. We also demonstrate the applicability of our framework in robotics on a belief space autonomous racing problem.

cs.AI↗

arXiv2021

PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds

In this paper, we propose a Point-Voxel Recurrent All-Pairs Field Transforms (PV-RAFT) method to estimate scene flow from point clouds. Since point clouds are irregular and unordered, it is challenging to efficiently extract features from all-pairs fields in the 3D space, where all-pairs correlations play important roles in scene flow estimation. To tackle this problem, we present point-voxel correlation fields, which capture both local and long-range dependencies of point pairs. To capture point-based correlations, we adopt the K-Nearest Neighbors search that preserves fine-grained information in the local region. By voxelizing point clouds in a multi-scale manner, we construct pyramid correlation voxels to model long-range correspondences. Integrating these two types of correlations, our PV-RAFT makes use of all-pairs relations to handle both small and large displacements. We evaluate the proposed method on the FlyingThings3D and KITTI Scene Flow 2015 datasets. Experimental results show that PV-RAFT outperforms state-of-the-art methods by remarkable margins.

cs.CV↗

arXiv2021

NOVAS: Non-convex Optimization via Adaptive Stochastic Search for End-to-End Learning and Control

In this work we propose the use of adaptive stochastic search as a building block for general, non-convex optimization operations within deep neural network architectures. Specifically, for an objective function located at some layer in the network and parameterized by some network parameters, we employ adaptive stochastic search to perform optimization over its output. This operation is differentiable and does not obstruct the passing of gradients during backpropagation, thus enabling us to incorporate it as a component in end-to-end learning. We study the proposed optimization module's properties and benchmark it against two existing alternatives on a synthetic energy-based structured prediction task, and further showcase its use in stochastic optimal control applications.

cs.LG↗

arXiv2021

Variational Inference MPC using Tsallis Divergence

In this paper, we provide a generalized framework for Variational Inference-Stochastic Optimal Control by using thenon-extensive Tsallis divergence. By incorporating the deformed exponential function into the optimality likelihood function, a novel Tsallis Variational Inference-Model Predictive Control algorithm is derived, which includes prior works such as Variational Inference-Model Predictive Control, Model Predictive PathIntegral Control, Cross Entropy Method, and Stein VariationalInference Model Predictive Control as special cases. The proposed algorithm allows for effective control of the cost/reward transform and is characterized by superior performance in terms of mean and variance reduction of the associated cost. The aforementioned features are supported by a theoretical and numerical analysis on the level of risk sensitivity of the proposed algorithm as well as simulation experiments on 5 different robotic systems with 3 different policy parameterizations.

cs.LG↗

arXiv2021

Learning Deep Stochastic Optimal Control Policies using Forward-Backward SDEs

In this paper we propose a new methodology for decision-making under uncertainty using recent advancements in the areas of nonlinear stochastic optimal control theory, applied mathematics, and machine learning. Grounded on the fundamental relation between certain nonlinear partial differential equations and forward-backward stochastic differential equations, we develop a control framework that is scalable and applicable to general classes of stochastic systems and decision-making problem formulations in robotics and autonomy. The proposed deep neural network architectures for stochastic control consist of recurrent and fully connected layers. The performance and scalability of the aforementioned algorithm are investigated in three non-linear systems in simulation with and without control constraints. We conclude with a discussion on future directions and their implications to robotics.

cs.RO↗

arXiv2021

Adaptive Risk Sensitive Model Predictive Control with Stochastic Search

We present a general framework for optimizing the Conditional Value-at-Risk for dynamical systems using stochastic search. The framework is capable of handling the uncertainty from the initial condition, stochastic dynamics, and uncertain parameters in the model. The algorithm is compared against a risk-sensitive distributional reinforcement learning framework and demonstrates outperformance on a pendulum and cartpole with stochastic dynamics. We also showcase the applicability of the framework to robotics as an adaptive risk-sensitive controller by optimizing with respect to the fully nonlinear belief provided by a particle filter on a pendulum, cartpole, and quadcopter in simulation.

math.OC↗

arXiv2020

Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEs

This paper introduces a new formulation for stochastic optimal control and stochastic dynamic optimization that ensures safety with respect to state and control constraints. The proposed methodology brings together concepts such as Forward-Backward Stochastic Differential Equations, Stochastic Barrier Functions, Differentiable Convex Optimization and Deep Learning. Using the aforementioned concepts, a Neural Network architecture is designed for safe trajectory optimization in which learning can be performed in an end-to-end fashion. Simulations are performed on three systems to show the efficacy of the proposed methodology.

eess.SY↗

arXiv2020

Erase-hidden and Drivability-improved Magnetic Non-Volatile Flip-Flops with NAND-SPIN Devices

Non-volatile flip-flops (NVFFs) using power gating techniques promise to overcome the soaring leakage power consumption issue with the scaling of CMOS technology. Magnetic tunnel junction (MTJ) is a good candidate for constructing the NVFF thanks to its low power, high speed, good CMOS compatibility, etc. In this paper, we propose a novel magnetic NVFF based on an emerging memory device called NAND-SPIN. The data writing of NAND-SPIN is achieved by successively applying two unidirectional currents, which respectively generate the spin orbit torque (SOT) and spin transfer torque (STT) for erase and programming operations. This characteristic allows us to design an erase-hidden and drivability-improved magnetic NVFF. Furthermore, more design flexibility could be obtained since the backup operation of the proposed NVFF is not limited by the inherent slave latch. Simulation results show that our proposed NVFF achieves performance improvement in terms of power, delay and area, compared with conventional slave-latch-driven SOT-NVFF designs.

cs.ET↗

arXiv2020

Ensemble Bayesian Decision Making with Redundant Deep Perceptual Control Policies

This work presents a novel ensemble of Bayesian Neural Networks (BNNs) for control of safety-critical systems. Decision making for safety-critical systems is challenging due to performance requirements with significant consequences in the event of failure. In practice, failure of such systems can be avoided by introducing redundancies of control. Neural Networks (NNs) are generally not used for safety-critical systems as they can behave in unexpected ways in response to novel inputs. In addition, there may not be any indication as to when they will fail. BNNs have been recognized for their ability to produce not only viable outputs but also provide a measure of uncertainty in these outputs. This work combines the knowledge of prediction uncertainty obtained from BNNs and ensemble control for a redundant control methodology. Our technique is applied to an agile autonomous driving task. Multiple BNNs are trained to control a vehicle in an end-to-end fashion on different sensor inputs provided by the system. We show that an individual network is successful in maneuvering around the track but crashes in the presence of unforeseen input noise. Our proposed ensemble of BNNs shows successful task performance even in the event of multiple sensor failures.

cs.RO↗

arXiv2019

Deep 2FBSDEs For Systems With Control Multiplicative Noise

We present a deep recurrent neural network architecture to solve a class of stochastic optimal control problems described by fully nonlinear Hamilton Jacobi Bellmanpartial differential equations. Such PDEs arise when one considers stochastic dynamics characterized by uncertainties that are additive and control multiplicative. Stochastic models with the aforementioned characteristics have been used in computational neuroscience, biology, finance and aerospace systems and provide a more accurate representation of actuation than models with additive uncertainty. Previous literature has established the inadequacy of the linear HJB theory and instead rely on a non-linear Feynman-Kac lemma resulting in a second order forward-backward stochastic differential equations representation. However, the proposed solutions that use this representation suffer from compounding errors and computational complexity leading to lack of scalability. In this paper, we propose a deep learning based algorithm that leverages the second order Forward-Backward SDE representation and LSTM based recurrent neural networks to not only solve such Stochastic Optimal Control problems but also overcome the problems faced by previous approaches and scales well to high dimensional systems. The resulting control algorithm is tested on non-linear systems in robotics and biomechanics to demonstrate feasibility and out-performance against previous methods.

cs.LG↗

arXiv2019

Constrained Sampling-based Trajectory Optimization using Stochastic Approximation

We propose a sampling-based trajectory optimization methodology for constrained problems. We extend recent works on stochastic search to deal with box control constraints,as well as nonlinear state constraints for discrete dynamical systems. Regarding the former, our strategy is to optimize over truncated parameterized distributions on control inputs. Furthermore, we show how non-smooth penalty functions can be incorporated into our framework to handle state constraints. Simulations on cartpole and quadcopter show that our approach outperforms previous methods on constrained sampling-based optimization, in terms of quality of solutions and convergence speed.

math.OC↗

arXiv2019

An algorithm for Bragg coherent X-ray diffractive imaging of highly strained nanocrystals

By using phase retrieval, Bragg Coherent Diffractive Imaging (BCDI) allows tracking of three-dimensional displacement fields inside individual nanocrystals. Nevertheless, in the presence of significant (1% and higher) strains, such as in the process of a structural phase transformation, fails due to the Bragg peak distortions. Here we present an advanced BCDI algorithm enabling imaging three-dimensional strain fields in highly strained crystals. We test the algorithm on particles simulated to undergo a structural phase transformation. While the conventional algorithm fails in unambiguously reconstructing the phase morphology, our algorithm correctly retrieves the morphology of coexistent phases with a strain difference of 1%. The key novelty is the simultaneous reconstruction of multiple scans of the same nanoparticle at snapshots through the phase transformations. The algorithm enables visualizing phase transformations in nanoparticles of lithium-ion, sodium-ion nanoparticles, and other nanoparticulate materials in working conditions (operando).

cond-mat.mtrl-sci↗

arXiv2019

Information Theoretic Model Predictive Control on Jump Diffusion Processes

In this paper we present an information theoretic approach to stochastic optimal control problems for systems with compound Poisson noise. We generalize previous work on information theoretic path integral control to discontinuous dynamics with compound Poisson noise and develop an iterative model predictive control (MPC) algorithm using importance sampling. The proposed algorithm is parallelizable and when implemented on a Graphical Processing Unit (GPU) can run in real time. We test the performance of the proposed algorithm in simulation for two control tasks using a cartpole system and a quadrotor. Our simulations demonstrate improved performance of the new scheme and indicate the importance of incorporating the statistical characteristics of stochastic disturbances in the computation of the stochastic optimal control policies.

math.OC↗

arXiv2019

Deep Forward-Backward SDEs for Min-max Control

This paper presents a novel approach to numerically solve stochastic differential games for nonlinear systems. The proposed approach relies on the nonlinear Feynman-Kac theorem that establishes a connection between parabolic deterministic partial differential equations and forward-backward stochastic differential equations. Using this theorem the Hamilton-Jacobi-Isaacs partial differential equation associated with differential games is represented by a system of forward-backward stochastic differential equations. Numerical solution of the aforementioned system of stochastic differential equations is performed using importance sampling and a Long-Short Term Memory recurrent neural network, which is trained in an offline fashion. The resulting algorithm is tested on two example systems in simulation and compared against the standard risk neutral stochastic optimal control formulations.

math.OC↗

arXiv2018

A First Look at Mobile Intelligence: Architecture, Experimentation and Challenges

Artificial intelligence (AI) technology makes mobile devices become intelligent objects which can learn and act automatically. Although AI will bring great opportunities for mobile applications, little work has focused on the architecture and the interaction with the cloud. In this article, we present three existing architectures of mobile intelligence in detail and introduce its broad application prospects. Furthermore, we conduct a series of experiments to evaluate the performance of the prevalent commercial applications and intelligent frameworks. Our results show that there is a big gap between Quality of Experience (QoE) requirements and the status quo. So far, we have seen only the tip of the iceberg. We pose issues and challenges to advance the area of mobile intelligence and hope to pave the way for the forthcoming.

cs.CY↗