arXiv subjects
Publications and source records attributed to Yingdong Lu.
We study the accuracy of a scaled Poisson approximation to the weighted sum of independent Poisson random variables, focusing on in particular the relative error of the tail distribution. A bound on the relative approximation error is established using a modified Stein-Chen method.
In this short paper, we obtain non-asymptotic concentration results for magnetization of the Curie-Weiss model at subcritical temperatures, which leads to a diffusion limit theorem of the scaled and centered magnetization driven by a Metropolis-Hasting algorithm. These results are complementary to results at supercritical and critical temperatures in Bierkens et al (2017).
Motivated by few delay-optimal scheduling results, in comparison to results on throughput optimality, we investigate a canonical input-queued switch scheduling problem in which the objective is to minimize the discounted delay cost over an infinite time horizon. We derive an optimal scheduling policy and establish corresponding theoretical properties, which are expected to be of interest more broadly than input-queued switches. Computational experiments demonstrate and quantify the benefits of our optimal scheduling policy over alternative policies such as variants of MaxWeight scheduling, well-known to be throughput optimal and more recently shown to be delay optimal in the heavy-traffic regime limit.
The mean field variational inference (MFVI) formulation restricts the general Bayesian inference problem to the subspace of product measures. We present a framework to analyze MFVI algorithms, which is inspired by a similar development for general variational Bayesian formulations. Our approach enables the MFVI problem to be represented in three different manners: a gradient flow on Wasserstein space, a system of Fokker-Planck-like equations and a diffusion process. Rigorous guarantees are established to show that a time-discretized implementation of the coordinate ascent variational inference algorithm in the product Wasserstein space of measures yields a gradient flow in the limit. A similar result is obtained for their associated densities, with the limit being given by a quasi-linear partial differential equation. A popular class of practical algorithms falls in this framework, which provides tools to establish convergence. We hope this framework could be used to guarantee convergence of algorithms in a variety of approaches, old and new, to solve variational inference problems.
We study the convergence of a random iterative sequence of a family of operators on infinite dimensional Hilbert spaces, inspired by the Stochastic Gradient Descent (SGD) algorithm in the case of the noiseless regression, as studied in [1]. We identify conditions that are strictly broader than previously known for polynomial convergence rate in various norms, and characterize the roles the randomness plays in determining the best multiplicative constants. Additionally, we prove almost sure convergence of the sequence.
We establish $L_q$ convergence for Hamiltonian Monte Carlo algorithms. More specifically, under mild conditions for the associated Hamiltonian motion, we show that the outputs of the algorithms converge (strongly for $2\le q<\infty$ and weakly for $1<q<2$) to the desired target distribution.
We study an approximation method of stationary characters of a two-dimensional Markov chain via the Stein method. For this purpose, innovative methods are developed to estimate the moments of the Markov chain, as well as the solution to the Poisson equation with a partial differential operator.
We consider a new form of reinforcement learning (RL) that is based on opportunities to directly learn the optimal control policy and a general Markov decision process (MDP) framework devised to support these opportunities. Derivations of general classes of our control-based RL methods are presented, together with forms of exploration and exploitation in learning and applying the optimal control policy over time. Our general MDP framework extends the classical Bellman operator and optimality criteria by generalizing the definition and scope of a policy for any given state. We establish the convergence and optimality-both in general and within various control paradigms (e.g., piecewise linear control policies)-of our control-based methods through this general MDP framework, including convergence of $Q$-learning within the context of our MDP framework. Our empirical results demonstrate and quantify the significant benefits of our approach.
We study an initial value problem with fractional Laplacian and a singular drift term, and obtain local and global existence theorems similar to the results in Jourdain et al.(2005).
We compute the stationary performance metrics of a single server $M^X/G/1$ queue under a class of generalized processor-sharing scheduling policies that are proposed by Grishechkin. This class of processor-sharing policies allow service capacities to be allocated to jobs based on the amount of service they attained. In [11], Grishechkin derives an integral equation that is satisfied by the Laplace transform of the stationary performance metrics under these policies. Our main focus in this paper is to derive the solution to this integral equation. This is achieved through a series of transforms that convert the integral equation into a more tractable form, then solve it. Then we derive approximations to the density functions of the performance metrics through inverting the Laplace transforms. In the last part of the paper, we apply our results to some well-known scheduling policies that are either a special case of the Grishechkin processor-sharing policies or can be treated as the limits of a sequence Grishechkin processor-sharing policies. These examples include the egalitarian processor-sharing policy, the discriminatory processor-sharing policy, shortest residual processing time first rule, foreground-background policy and the time function scheduling policy.
In this paper, we quantify some known approximation to the Curie-Weiss model via applying the Stein method to the Markov chain whose stationary distribution coincides with Curie-Weiss model.
The main purpose of this paper is to facilitate the communication between the Analytic, Probabilistic and Algorithmic communities. We present a proof of convergence of the Hamiltonian (Hybrid) Monte Carlo algorithm from the point of view of the Dynamical Systems, where the evolving objects are densities of probability distributions and the tool are derived from the Functional Analysis.
Most of the early input-queued switch research focused on establishing throughput optimality of the max-weight scheduling policy, with some recent research showing that max-weight scheduling is optimal with respect to total expected delay asymptotically in the heavy-traffic regime. However, the question of delay-optimal scheduling in input-queued switches remains open in general, as does the question of delay-optimal scheduling under more general objective functions. To gain fundamental insights into these very difficult problems, we consider a fluid model of $n \times n$ input-queued switches with associated fluid-flow costs, and we derive an optimal scheduling control policy to an infinite horizon discounted control problem with a general linear objective function of fluid cost. Our optimal policy coincides with the $cμ$-rule in certain parameter domains. More generally, due to the input-queued switch constraints, the optimal policy takes the form of the solution to a flow maximization problem, after we identify the Lagrangian multipliers of some key constraints through carefully designed algorithms. Computational experiments demonstrate the benefits of our optimal scheduling policy over variants of max-weight scheduling within fluid models of input-queued switches.
We consider the optimal control of input-queued switches under a cost-weighted variant of MaxWeight scheduling, for which we establish theoretical properties that include showing the algorithm exhibits optimal heavy-traffic queue-length scaling. Our results are expected to be of theoretical interest more broadly than input-queued switches.
We study a single server queue under a processor-sharing type of scheduling policy, where the weights for determining the sharing are given by functions of each job's remaining service(processing) amount, and obtain a fluid limit for the scaled measure-valued system descriptors.
We study a mathematical model for revenue management under competition with multiple sellers. The model combines the stochastic knapsack problem, a classic revenue management model, with a non coorperative game model that characterizes the sellers' rational behavior. We are able to establish a dynamic recursive procedure that incorporate the value function with the utility function of the games. The formalization of the dynamic recursion allows us to establish some fundamental structural properties.
We consider a new family of operators for reinforcement learning with the goal of alleviating the negative effects and becoming more robust to approximation or estimation errors. Various theoretical results are established, which include showing on a sample path basis that our family of operators preserve optimality and increase the action gap. Our empirical results illustrate the strong benefits of our family of operators, significantly outperforming the classical Bellman operator and recently proposed operators.
In this paper, we consider the classic stochastic (dynamic) knapsack problem, a fundamental mathematical model in revenue management, with general time-varying random demand. Our main goal is to study the optimal policies, which can be obtained by solving the dynamic programming formulated for the problem, both qualitatively and quantitatively. It is well-known that when the demand size is fixed and the demand distribution is stationary over time, the value function of the dynamic programming exhibits extremely useful first and second order monotonicity properties, which lead to monotonicity properties of the optimal policies. In this paper, we are able to verify that these results still hold even in the case that the price distributions are time-dependent. When we further relax the demand size distribution assumptions and allow them to be arbitrary, for example in random batches, we develop a scheme for using value function of alternative unit demand systems to provide bounds to the value function. These results confirm some of the basic understandings of the stochastic knapsack problem. As a natural consequence, we propose a new class of heuristic policies, two-dimensional switch-over policies, and discuss numerical procedures for optimizing the parameters of the policies. Extensive numerical experiments demonstrate that these switch-over policies can provide performances that are in a close neighborhood of the optimum statistically.