arXiv subjects

Guodong Pang

Publications and source records attributed to Guodong Pang.

At least 19 recordsLinked to original sources

arXiv2026

Data-Driven Brownian Reflection Control

We study a data-driven reflection control problem for a Brownian model with unknown drift and volatility. We first propose a learn-then-optimize (LTO) algorithm: it estimates the policy-relevant parameter during exploration, plugs the estimate into the optimality equation, and exploits the resulting policy---achieving an $O(\sqrt{T})$ finite-time expected regret bound. We further propose two algorithms, adaptive-updating (AU) and full-history adaptive-updating (AU-FH), which continuously update the estimator and reflecting level, attaining an improved $O(\log T)$ regret bound. Notably, AU-FH algorithms leverages all historical data, yielding better performance in numerical simulations. Our analysis decomposes regret into exploration, transient, and learning components. Transient regret from nonstationarity is bounded by the time-integrated deviation of the transition semigroup from stationarity evaluated on the holding cost, which can be further bounded via a Foster-Lyapunov inequality for exponential convergence of the controlled reflected Brownian motion (RBM). For learning regret, we establish local regularity properties together with consistency and mean-squared error bounds for the estimator, which control the stationary cost gap between the learned and optimal reflection policies. In addition, we leverage the monotonicity of the moving boundary Skorokhod map to derive moment bounds for the AU algorithms' switching states, via pathwise comparison with fixed-boundary RBMs.

math.OC↗

arXiv2026

Jacobi-like relative value iteration algorithms for ergodic risk-sensitive control of Markov chains

We propose a Jacobi-like relative value iteration (RVI) algorithm and a Gauss-Seidel-like implementation for the ergodic risk-sensitive control (ERSC) problem of a controlled discrete time Markov chain (DTMC) on a finite state space. Under the assumption that the DTMC is irreducible and recurrent under every stationary Markov policy, we prove that the iterates of the proposed RVI algorithms converge at a geometric rate. The main challenge stems from the multiplicative structure of the ERSC cost criterion and the associated Bellman-like operators, which prevents us from adapting the analogous global contraction and bi-Lipschitz continuity properties that underlie the proof of convergence in the average cost setting. We overcome this by establishing local contraction properties for the risk-sensitive Bellman-like operators and a local bi-Lipschitz continuity property for their fixed points, and use these properties to show the iterates converge geometrically. We conclude by implementing our proposed RVI algorithms on two examples: service effort control for a single-server queue of finite capacity, and maximizing the exit rate from a finite domain (on a graph).

math.OC↗

arXiv2026

Exponential rate of convergence of relative value iteration algorithms for ergodic controls of diffusions

In this paper, we investigate the rate of convergence of the relative value iteration (RVI) algorithms for diffusions in $\mathbb{R}^d$ under both the conventional ergodic cost (CEC) and ergodic risk-sensitive cost (ERSC) criteria, and under the uniform exponential stability condition. The existing RVI algorithms for the CEC and ERSC problems solve the associated initial value Hamilton-Jacobi-Bellman type equations whose solutions are shown to converge asymptotically to the corresponding optimal values. However, the rates of convergence for such algorithms have remained open. This paper proposes discrete-time implementations for the RVI algorithms based on slight modifications of the associated PDEs, and proves that the rates of convergence of these RVI algorithms are exponential under a weighted sup-norm. These implementations have discrete-time iterates that can be explicitly expressed as recursive systems. The difference between these iterates and the desired value function in the CEC case can then be expressed in terms of the associated Markov kernels. Similarly, this can be done for the logarithms of the corresponding iterates and desired value function in the ERSC case in terms of the associated Markov kernels for the extended diffusion. As a result, we are able to prove the desirable contraction properties in order to establish the exponential rate of convergence by making use of a weighted semi-norm in which Markov kernel acts a contraction.

math.OC↗

arXiv2026

Growing open Markovian Jackson networks: Fluid limit and infinite-dimensional Skorokhod problem

We study growing open Jackson networks where each station is a single-server queue that follows the first-come first-served discipline with Poisson arrivals and exponentially distributed service times, characterized by node-specific rates. In applying a fluid scaling to the queue-length process, we show that under certain conditions the queueing system can be approximated by an infinite-dimensional fluid limit with a kernel function in place of the transition matrix. This limiting process can be characterized by an infinite-dimensional Skorokhod problem, for which we develop a new theory by considering a broader class of reflection operators and general infinite-dimensional processes. We establish existence and uniqueness of a solution along with Lipschitz continuity provided the reflecting operator has a spectral radius less than 1.By introducing an intermediate process in which the compensated Poisson components are removed, and then lifting this to an infinite-dimensional process, we exploit the new Lipschitz property of the infinite-dimensional Skorokhod mapping to prove convergence of the intermediate process. We then prove the necessary estimates for the difference between the original and intermediate processes by using martingale properties. Finally, we consider the empirical measure of the queueing processes, for which we show convergence to the measure associated with the path of the infinite-dimensional fluid limit, extending to the convergence of specific performance-related functionals.

math.PR↗

arXiv2025

Ergodic Risk Sensitive Control of Diffusions under a General Structural Hypothesis

We study the infinite-horizon average (ergodic) risk sensitive control problem for diffusion processes under a general structural hypothesis: there is a partition of state space into two subsets, where the controlled diffusion process satisfies a Foster-Lyapunov type drift condition in one subset, under any stationary Markov control, while the near-monotonicity condition is satisfied with the running cost function being inf-compact in its complement. Under these conditions, we completely characterize the optimal stationary Markov controls. To prove this, we consider an inf-compact perturbation to the running cost over the entire space such that the resulting ergodic risk sensitive control problem is well-defined and then use the corresponding existing results. The heart of the analysis lies in exploiting the variational formula of exponential functionals of Brownian motion and applying it to the objective exponential cost function of the controlled diffusion. This representation facilitates us to view the risk sensitive cost for any stationary Markov control as the optimal value of a control problem of an extended diffusion involving a new auxiliary control where the optimal criterion is to maximize the associated long-run average cost criterion that is a difference of the original running cost and an extra term that is quadratic in the auxiliary control. The main difficulty in using this approach lies in the fact that tightness of mean empirical measures of the extended diffusion is not a priori implied by the analogous tightness property of the original diffusion. We overcome this by establishing a priori estimates for the extended diffusion associated with the nearly optimal auxiliary controls.

math.OC↗

arXiv2025

Sample Path Moderate Deviation Principle for Queues with Waiting-time Dependent Interarrival and Service Times

We consider a single-server queue where interarrival and service times depend linearly and randomly on customer waiting times, and establish a sample-path moderate deviation principle (MDP) for the waiting time process. The waiting times for the queue can be written as a modified Lindley recursion with a random weight coefficient. Under a natural scaling of the random coefficients, we analyze the fluid behavior of the workload process and derive the stable equilibrium point, which can be zero or a positive value. The moderate-deviation-scaled process is centered around the stable equilibrium point and then represented as a linear stochastic differential equation driven by two random walks together with additional asymptotically negligible error terms and possibly a reflection at zero. The rate functions of MDPs in the two scenarios can be characterized explicitly, and they differ in that the case with zero centering term involves the linearly generalized Skorokhod reflection mapping while the case with positive centering term does not (similar to the corresponding diffusion limits). Our analysis involves the MDP for the associated linearly recursive Markov chains, invoking a perturbation of two independent random walks, and employing martingale techniques to prove the asymptotically exponentially vanishing error terms.

math.PR↗

arXiv2025

Queueing models with random resetting

We introduce and study some queueing models with random resetting, including Markovian and non--Markovian models under the first-come first-served (FCFS) discipline. The Markovian models include M/M/$r$ and M/M/1+M queues with random resetting, in which a continuous-time Markov chain is formulated, with transitions including a resetting to state zero in addition to arrivals and services. We explicitly characterize the stationary distributions of the queueing processes in these models by using parting balance equations. We derive expressions for standard performance measures such as the delay probability, expected queue length and waiting time, as well as probability of a customer completing service before resetting in the M/M/$r$ model, and probability of abandonment before service or resetting in the M/M/1+M model. The non--Markovian models include GI/GI/1, GI/GI/$r$ and GI/GI/$\infty$ queues with random resetting to state zero at arrival times. For GI/GI/1 and GI/GI/$r$ queues under the FCFS discipline, we introduce modified Lindley and Kiefer--Wolfowitz recursions, respectively. Using an operator representation for these recursions, we characterize the stationary distributions via convergent series, as solutions to the modified Wiener--Hopf equations. For GI/GI/$\infty$ queues with resettings, we utilize a version of the Kiefer--Wolfowitz recursion, and also characterize the corresponding stationary distribution.

math.PR↗

arXiv2025

Stochastic heterogeneous SIR model with infection-age dependent infectivity on large random graphs

We study an individual-based stochastic SIR epidemic model with infection-age dependent infectivity on a large random graph, capturing individual heterogeneity and non-homogeneous connectivity. Each individual is associated with particular characteristics (for example, spatial location and age structure), which may not be i.i.d., and is represented by a particular node. The connectivities among the individuals are given by a non-homogeneous random graph, whose connecting probabilities may depend on the individual characteristics of the edge. To each individual is associated a random infectivity function of its infection age, which is allowed to depend upon the individual characteristics. We use measure-valued processes to describe the epidemic evolution dynamics, tracking the infection age of all individuals, and their associated characteristics. We consider the epidemic dynamics as the population size grows to infinity under a specific scaling of the connectivity graph related to the convergence to a graphon. In the limit, we obtain a system of measure-valued equations, which can be also represented as a PDE model on graphon, and reflects the heterogeneities in individual characteristics and social connectivity.

math.PR↗

arXiv2024

SPDE for stochastic SIR epidemic models with infection-age dependent infectivity

We study the stochastic SIR epidemic model with infection-age dependent infectivity for which a measure-valued process is used to describe the ages of infection for each individual. We establish a functional law of large numbers (FLLN) and a functional central limit theorem (FCLT) for the properly scaled measure-valued processes together with the other epidemic processes to describe the evolution dynamics. In the FLLN, assuming that the hazard rate function of the infection periods is bounded and the ages at time 0 of the infections of the initially infected individuals are bounded, we obtain a PDE limit for the LLN-scaled measure-valued process, for which we characterize its solution explicitly. The PDE is linear with a boundary condition given by the unique solution to a set of Volterra-type nonlinear integral equations. In the FCLT, we obtain an SPDE for the CLT-scaled measure-valued process, driven by two independent white noises coming from the infection and recovery processes. The SPDE is also linear and coupled with the solution to a system of stochastic Volterra-type linear integral equations driven by three independent Gaussian noises, one from the random infection functions in addition to the two white noises mentioned above. The solution to the SPDE can be also explicitly characterized, given this auxiliary process. The uniqueness of the SPDE solution is established under stronger assumptions (density and its derivative being locally bounded) on the distribution function of an infectious duration.

math.PR↗

arXiv2024

Ergodic Risk Sensitive Control of Markovian Multiclass Many-Server Queues with Abandonment

We study the optimal scheduling problem for a Markovian multiclass queueing network with abandonment in the Halfin--Whitt regime, under the long run average (ergodic) risk sensitive cost criterion. The objective is to prove asymptotic optimality for the optimal control arising from the corresponding ergodic risk sensitive control (ERSC) problem for the limiting diffusion. In particular, we show that the optimal ERSC value associated with the diffusion-scaled queueing process converges to that of the limiting diffusion in the asymptotic regime. The challenge that ERSC poses is that one cannot express the ERSC cost as an expectation over the mean empirical measure associated with the queueing process, unlike in the usual case of a long run average (ergodic) cost. We develop a novel approach by exploiting the variational representations of the limiting diffusion and the Poisson-driven queueing dynamics, which both involve certain auxiliary controls. The ERSC costs for both the diffusion-scaled queueing process and the limiting diffusion can be represented as the integrals of an extended running cost over a mean empirical measure associated with the corresponding extended processes using these auxiliary controls. For the lower bound proof, we exploit the connections of the ERSC problem for the limiting diffusion with a two-person zero-sum stochastic differential game. We also make use of the mean empirical measures associated with the extended limiting diffusion and diffusion-scaled processes with the auxiliary controls. One major technical challenge in both lower and upper bound proofs, is to establish the tightness of the aforementioned mean empirical measures for the extended processes. We identify nearly optimal controls appropriately in both cases so that the existing ergodicity properties of the limiting diffusion and diffusion-scaled queueing processes can be used.

math.PR↗

arXiv2024

Unbiased approximation of the ergodic measure for piecewise $\alpha$-stable Ornstein-Uhlenbeck processes arising in queueing networks

Piecewise $\alpha$-stable Ornstein-Uhlenbeck (OU) processes arising in queue networks usually do not have an explicit dissipation, which makes the related numerical methods such as Euler-Maruyama (EM) scheme more difficult to analyze. We develop an EM scheme with decreasing step size $\Lambda=(\eta_n)_{n\in \mathbb{N}}$ to approximate their ergodic measures. This approximation does not have a bias and has a rate $\eta^{1/\alpha}_n$ in Wasserstein-1 distance. We show by the classical OU process that our convergence rate is optimal. We further prove the central limit theorem (CLT) and moderate derivation principle (MDP) for the empirical measure of these piecewise $\alpha$-stable Ornstein-Uhlenbeck processes. In addition, we use the Sinkhorn--Knopp algorithm to compute the Wasserstein-1 distance and conduct simulations for several concrete examples.

math.PR↗

arXiv2024

Stochastic dynamics of two-compartment cell proliferation models with regulatory mechanisms for hematopoiesis

We present an asymptotic analysis of a stochastic two-compartmental cell division system with regulatory mechanisms inspired by Getto et al. (2013). The hematopoietic system is modeled as a two-compartment system, where the first compartment consists of dividing cells in the bone marrow, referred to as type $0$ cells, and the second compartment consists of post-mitotic cells in the blood, referred to as type $1$ cells. Division and self-renewal of type $0$ cells are regulated by the population density of type $1$ cells. By scaling up the initial population, we demonstrate that the scaled dynamics converges in distribution to the solution of a system of ordinary differential equations (ODEs). This system of ODEs exhibits a unique non-trivial equilibrium that is globally stable. Furthermore, we establish that the scaled fluctuations of the density dynamics converge in law to a linear diffusion process with time-dependent coefficients. When the initial data is Gaussian, the limit process is a Gauss-Markov process. We analyze its asymptotic properties to elucidate the joint structure of both compartments over large times. This is achieved by proving exponential convergence in the 2-Wasserstein metric for the associated Gaussian measures on an $L^{2}$ Hilbert space. Finally, we apply our results to compare the effects of regulating division and self-renewal of type $0$ cells, providing insights into their respective roles in maintaining hematopoietic system stability.

q-bio.PE↗

arXiv2023

PDE model for multi-patch epidemic models with migration and infection-age dependent infectivity

We study a stochastic epidemic model with multiple patches (locations), where individuals in each patch are categorized into three compartments, Susceptible, Infected and Recovered/Removed, and may migrate from one patch to another in any of the compartments. Each individual is associated with a random infectivity function which dictates the force of infectivity while the interactive infection process depends on the age of infection (elapsed time since infection). We prove a functional law of large number for the epidemic evolution dynamics including the aggregate infectivity process, the numbers of susceptible and recovered individuals as well as the number of infected individuals at each time that have been infected for a certain amount of time. From the limits, we derive a PDE model for the density of the number of infected individuals with respect to the infection age, which is a systems of linear PDE equations with a boundary condition that is determined by a set of integral equations.

math.PR↗

arXiv2023

Spatially dense stochastic epidemic models with infection-age dependent infectivity

We study an individual-based stochastic spatial epidemic model where the number of locations and the number of individuals at each location both grow to infinity. Each individual is associated with a random infection-age dependent infectivity function. Individuals are infected through interactions across the locations with heterogeneous effects. The epidemic dynamics can be described using a time-space representation for the the total force of infection, the number of susceptible individuals, the number of infected individuals that are infected at each time and have been infected for a certain amount of time, as well as the number of recovered individuals. We prove a functional law of large numbers for these time-space processes, and in the limit, we obtain a set of time-space integral equations. We then derive the PDE models from the limiting time-space integral equations, in particular, the density (with respect to the infection age) of the time-age-space integral equation for the number of infected individuals tracking the age of infection satisfies a linear PDE in time and age with an integral boundary condition. These integral equation and PDE limits can be regarded as dynamics on graphon under certain conditions.

math.PR↗

arXiv2022

Functional law of large numbers and PDEs for epidemic models with infection-age dependent infectivity

We study epidemic models where the infectivity of each individual is a random function of the infection age (the elapsed time of infection). To describe the epidemic evolution dynamics, we use a stochastic process that tracks the number of individuals at each time that have been infected for less than or equal to a certain amount of time, together with the aggregate infectivity process. We establish the functional law of large numbers (FLLN) for the stochastic processes that describe the epidemic dynamics. The limits are described by a set of deterministic integral equations, which has a further characterization using PDEs under some regularity conditions. The solutions are characterized with boundary conditions that are given by a system of Volterra equations. We also characterize the equilibrium points for the PDEs in the SIS model with infection-age dependent infectivity. To establish the FLLNs, we employ a useful criterion for weak convergence for the two-parameter processes together with useful representations for the relevant processes via Poisson random measures.

math.PR↗

arXiv2022

Stochastic epidemic models with varying infectivity and waning immunity

We study an individual-based stochastic epidemic model in which infected individuals become susceptible again following each infection. In contrast to classical compartment models, after each infection, the infectivity is a random function of the time elapsed since one's infection. Similarly, recovered individuals become gradually susceptible after some time according to a random susceptibility function. We study the large population asymptotic behaviour of the model, by proving a functional law of large numbers (FLLN) and investigating the endemic equilibria properties of the limit. The limit depends on the law of the susceptibility random functions but only on the mean infectivity functions. The FLLN is proved by constructing a sequence of i.i.d. auxiliary processes and adapting the approach from the theory of propagation of chaos. The limit is a generalisation of a PDE model introduced by Kermack and McKendrick, and we show how this PDE model can be obtained as a special case of our FLLN limit.% for a particular set of infectivity and susceptibility random functions and initial conditions. For the endemic equilibria, if $ R_0 $ is lower than (or equal to) some threshold, the epidemic does not last forever and eventually disappears from the population, while if $ R_0 $ is larger than this threshold, the epidemic will not disappear and there exists an endemic equilibrium. The value of this threshold turns out to depend on the harmonic mean of the susceptibility a long time after an infection, a fact which was not previously known.

math.PR↗

arXiv2022

Recent Advances in Epidemic Modeling: Non-Markov Stochastic Models and their Scaling Limits

In this survey paper, we review the recent advances in individual based non--Markovian epidemic models. They include epidemic models with a constant infectivity rate, varying infectivity rate or infection-age dependent infectivity, infection-age recovery rate (or equivalently, general law of infectious period), as well as varying susceptibility/immunity. We focus on the scaling limits with a large population, functional law of large numbers (FLLN) and functional central limit theorems (FCLT), while the large and moderate deviations for some Markovian epidemic models are also reviewed. In the FLLN, the limits are a set of Volterra integral equations, and in the FCLT, the limits are stochastic Volterra integral equations driven by Gaussian processes. We relate our deterministic limits to the results in the seminal papers by Kermack and McKendrick published in 1927, 1932 and 1933, where the varying infectivity and susceptibility/immunity were already considered. We also discuss some extensions, including models with heterogeneous population, spatial models and control problems, as well as open problems.

math.PR↗

arXiv2022

An approximation to the invariant measure of the limiting diffusion of G/Ph/n+GI queues in the Halfin-Whitt regime and related asymptotics

In this paper, we develop a stochastic algorithm based on the Euler--Maruyama scheme to approximate the invariant measure of the limiting multidimensional diffusion of $G/Ph/n+GI$ queues in the Halfin-Whitt regime. Specifically, we prove a non-asymptotic error bound between the invariant measures of the approximate model from the algorithm and the limiting diffusion. To establish the error bound, we employ the recently developed Stein's method for multi-dimensional diffusions, in which the regularity of Stein's equation developed by Gurvich (2014, 2022) plays a crucial role. We further prove the central limit theorem (CLT) and the moderate deviation principle (MDP) for the occupation measures of the limiting diffusion of $G/Ph/n+GI$ queues and its Euler-Maruyama scheme. In particular, the variances in the CLT and MDP associated with the limiting diffusion are determined by Stein's equation and Malliavin calculus, in which properties of a mollified diffusion and an associated weighted occupation time play a crucial role.

math.PR↗