arXiv subjects

Jiekun Feng

Publications and source records attributed to Jiekun Feng.

3 recordsLinked to original sources

arXiv2020

Scalable Deep Reinforcement Learning for Ride-Hailing

Ride-hailing services, such as Didi Chuxing, Lyft, and Uber, arrange thousands of cars to meet ride requests throughout the day. We consider a Markov decision process (MDP) model of a ride-hailing service system, framing it as a reinforcement learning (RL) problem. The simultaneous control of many agents (cars) presents a challenge for the MDP optimization because the action space grows exponentially with the number of cars. We propose a special decomposition for the MDP actions by sequentially assigning tasks to the drivers. The new actions structure resolves the scalability problem and enables the use of deep RL algorithms for control policy optimization. We demonstrate the benefit of our proposed decomposition with a numerical experiment based on real data from Didi Chuxing.

math.OC↗

arXiv2018

Steady-state Diffusion Approximations for Discrete-time Queue in Hospital Inpatient Flow Management

In this paper, we analyze a discrete-time queue that is motivated from studying hospital inpatient flow management, where the customer count process captures the midnight inpatient census. The stationary distribution of the customer count has no explicit form and is difficult to compute in certain parameter regimes. Using the Stein's method framework, we identify a continuous random variable to approximate the steady-state customer count. The continuous random variable corresponds to the stationary distribution of a diffusion process with state-dependent diffusion coefficients. We characterize the error bounds of this approximation under a variety of system load conditions -- from lightly loaded to heavily loaded. We also identify the critical role that the service rate plays in the convergence rate of the error bounds. We perform extensive numerical experiments to support the theoretical findings and to demonstrate the approximation quality. In particular, we show that our approximation performs better than those based on constant diffusion coefficients when the number of servers is small, which is relevant to decision making in a single hospital ward.

math.PR↗

arXiv2017

Stein's method for steady-state diffusion approximations: an introduction through the Erlang-A and Erlang-C models

This paper provides an introduction to the Stein method framework in the context of steady-state diffusion approximations. The framework consists of three components: the Poisson equation and gradient bounds, generator coupling, and moment bounds. Working in the setting of the Erlang-A and Erlang-C models, we prove that both Wasserstein and Kolmogorov distances between the stationary distribution of a normalized customer count process, and that of an appropriately defined diffusion process decrease at a rate of $1/\sqrt{R}$, where $R$ is the offered load. Futhermore, these error bounds are \emph{universal}, valid in any load condition from lightly loaded to heavily loaded.

math.PR↗