arXiv subjects

Jim Dai

Publications and source records attributed to Jim Dai.

8 recordsLinked to original sources

arXiv2026

Deep Learning Method for Stationary Distribution of Reflected Brownian Motion

The stationary distribution of reflected Brownian motion (RBM) plays an important role in the analysis of high-dimensional stochastic systems, yet closed-form solutions are known only for a few special cases. Computing important performance metrics, such as tail probabilities, is even more intractable, despite their practical relevance. In this paper, we develop a deep learning approach that accurately and efficiently learns the Laplace transform of high-dimensional RBMs based on the basic adjoint relationship (BAR). Our framework combines a careful design of the loss function, training data sampling procedure, and neural network architecture. We evaluate the proposed method on RBM instances with known ground-truth tail probabilities and demonstrate near-perfect prediction in high-dimensional settings, highlighting its potential as a general tool for analyzing stochastic systems beyond analytically tractable regimes. Our code can be found at https://github.com/zhangz73/NN4MGF.

cs.LG↗

arXiv2025

Optimal Batched Scheduling of Stochastic Processing Networks Using Atomic Action Decomposition

Stochastic processing networks (SPNs) have broad applications in healthcare, transportation, and communication networks. The control of SPN is to dynamically assign servers in batches under uncertainty to optimize long-run performance. This problem is challenging as the policy dimension grows exponentially with the number of servers, making standard reinforcement learning and policy optimization methods intractable at scale. We propose an atomic action decomposition framework that addresses this scalability challenge by breaking joint assignments into sequential single-server assignments. This yields policies with constant dimension, independent of the number of servers. We study two classes of atomic policies, the step-dependent and step-independent atomic policies, and prove that both achieve the same optimal long-run average reward as the original joint policies. These results establish that computing the optimal SPN control can be made scalable without loss of optimality using the atomic framework. Our results offer theoretical justification for the strong empirical success of the atomic framework in large-scale applications reported in previous articles.

eess.SY↗

arXiv2025

Atomic Proximal Policy Optimization for Electric Robo-Taxi Dispatch and Charger Allocation

Pioneering companies such as Waymo have deployed robo-taxi services in several U.S. cities. These robo-taxis are electric vehicles, and their operations require the joint optimization of ride matching, vehicle repositioning, and charging scheduling in a stochastic environment. We model the operations of the ride-hailing system with robo-taxis as a discrete-time, average-reward Markov Decision Process with an infinite horizon. As the fleet size grows, dispatching becomes challenging, as both the system state space and the fleet dispatching action space grow exponentially with the number of vehicles. To address this, we introduce a scalable deep reinforcement learning algorithm, called Atomic Proximal Policy Optimization (Atomic-PPO), that reduces the action space using atomic action decomposition. We evaluate our algorithm using real-world NYC for-hire vehicle trip records and measure its performance by the long-run average reward achieved by the dispatching policy, relative to a fluid-based upper bound. Our experiments demonstrate the superior performance of Atomic-PPO compared to benchmark methods. Furthermore, we conduct extensive numerical experiments to analyze the efficient allocation of charging facilities and assess the impact of vehicle range and charger speed on system performance.

cs.AI↗

arXiv2024

Asymptotic Product-form Steady-state for Multiclass Queueing Networks: A Reentrant Line Case Study

This paper serves as a companion to "Asymptotic Product-form Steady-state for Multiclass Queueing Networks with SBP Service Policies in Multi-scale Heavy Traffic." In this short paper, we illustrate the main results of the main paper through a two-station, five-class reentrant line under a specific static buffer priority policy, while avoiding heavy notations. For this example, we prove the asymptotic steady-state limit and uniform moment bound under general inter-arrival and service time distributions.

math.PR↗

arXiv2024

Inpatient Overflow Management with Proximal Policy Optimization

Problem Definition: Managing inpatient flow in large hospital systems is challenging due to the complexity of assigning randomly arriving patients -- either waiting for primary units or being overflowed to alternative units. Current practices rely on ad-hoc rules, while prior analytical approaches struggle with the intractably large state and action spaces inherent in patient-unit matching. Scalable decision support is needed to optimize overflow management while accounting for time-periodic fluctuations in patient flow. Methodology/Results: We develop a scalable decision-making framework using Proximal Policy Optimization (PPO) to optimize overflow decisions in a time-periodic, long-run average cost setting. To address the combinatorial complexity, we introduce atomic actions, which decompose multi-patient routing into sequential assignments. We further enhance computational efficiency through a partially-shared policy network designed to balance parameter sharing with time-specific policy adaptations, and a queueing-informed value function approximation to improve policy evaluation. Our method significantly reduces the need for extensive simulation data, a common limitation in reinforcement learning applications. Case studies on hospital systems with up to twenty patient classes and twenty wards demonstrate that our approach matches or outperforms existing benchmarks, including approximate dynamic programming, which is computationally infeasible beyond five wards. Managerial Implications: Our framework offers a scalable, efficient, and explainable solution for managing patient flow in complex hospital systems. More broadly, our results highlight that domain-aware adaptation is more critical to improving algorithm performance than fine-tuning neural network parameters when applying general-purpose algorithms to specific applications.

math.OC↗

arXiv2014

Sprinklers: A Randomized Variable-Size Striping Approach to Reordering-Free Load-Balanced Switching

Internet traffic continues to grow exponentially, calling for switches that can scale well in both size and speed. While load-balanced switches can achieve such scalability, they suffer from a fundamental packet reordering problem. Existing proposals either suffer from poor worst-case packet delays or require sophisticated matching mechanisms. In this paper, we propose a new family of stable load-balanced switches called "Sprinklers" that has comparable implementation cost and performance as the baseline load-balanced switch, but yet can guarantee packet ordering. The main idea is to force all packets within the same virtual output queue (VOQ) to traverse the same "fat path" through the switch, so that packet reordering cannot occur. At the core of Sprinklers are two key innovations: a randomized way to determine the "fat path" for each VOQ, and a way to determine its "fatness" roughly in proportion to the rate of the VOQ. These innovations enable Sprinklers to achieve near-perfect load-balancing under arbitrary admissible traffic. Proving this property rigorously using novel worst-case large deviation techniques is another key contribution of this work.

cs.NI↗

arXiv2011

Optimal Control of Brownian Inventory Models with Convex Inventory Cost: Discounted Cost Case

math.OC↗

arXiv2011

Optimal Control of Brownian Inventory Models with Convex Holding Cost: Average Cost Case

We consider an inventory system in which inventory level fluctuates as a Brownian motion in the absence of control. The inventory continuously accumulates cost at a rate that is a general convex function of the inventory level, which can be negative when there is a backlog. At any time, the inventory level can be adjusted by a positive or negative amount, which incurs a fixed cost and a proportional cost. The challenge is to find an adjustment policy that balances the holding cost and adjustment cost to minimize the long-run average cost. When both upward and downward fixed costs are positive, our model is an impulse control problem. When both fixed costs are zero, our model is a singular or instantaneous control problem. For the impulse control problem, we prove that a four-parameter control band policy is optimal among all feasible policies. For the singular control problem, we prove that a two-parameter control band policy is optimal. We use a lower-bound approach, widely known as "the verification theorem", to prove the optimality of a control band policy for both the impulse and singular control problems. Our major contribution is to prove the existence of a "smooth" solution to the free boundary problem under some mild assumptions on the holding cost function. The existence proof leads naturally to numerical algorithms to compute the optimal control band parameters. We demonstrate that the lower-bound approach also works for Brownian inventory model in which no inventory backlog is allowed. In a companion paper, we will show how the lower-bound approach can be adapted to study a Brownian inventory model under a discounted cost criterion.

math.OC↗