Searcharxiv⌕ Search

arXiv subjects

Ari Arapostathis

Publications and source records attributed to Ari Arapostathis.

At least 37 records · Page 2Linked to original sources
arXiv2019

Ergodic control of diffusions with compound Poisson jumps under a general structural hypothesis

We study the ergodic control problem for a class of controlled jump diffusions driven by a compound Poisson process. This extends the results of [SIAM J. Control Optim. 57 (2019), no. 2, 1516-1540] to running costs that are not near-monotone. This generality is needed in applications such as optimal scheduling of large-scale parallel server networks. We provide a full characterization of optimality via the Hamilton-Jacobi-Bellman (HJB) equation, for which we additionally exhibit regularity of solutions under mild hypotheses. In addition, we show that optimal stationary Markov controls are a.s. pathwise optimal. Lastly, we show that one can fix a stable control outside a compact set and obtain near-optimal solutions by solving the HJB on a sufficiently large bounded domain. This is useful for constructing asymptotically optimal scheduling policies for multiclass parallel server networks.

math.OC↗
arXiv2019

On uniqueness of solutions to viscous HJB equations with a subquadratic nonlinearity in the gradient

Uniqueness of positive solutions to viscous Hamilton-Jacobi-Bellman (HJB) equations of the form $-Δu(x) + \frac{1}γ |D{u}(x)|^γ= f(x) - λ$, with $f$ a coercive function and $λ$ a constant, in the subquadratic case, that is, $γ\in(1,2)$, appears to be an open problem. Barles and Meireles [Comm. Partial Differential Equations 41 (2016)] show uniqueness in the case that $f(x) \approx |x|^β$ and $|D f(x)|\lessapprox |x|^{(β-1)_+}$ for some $β>0$, essentially matching earlier results of Ichihara, who considered more general Hamiltonians but with better regularity for $f$. Without enforcing this assumption, to our knowledge, there are no results on uniqueness in the literature. In this short article, we show that the equation has a unique positive solution for any locally Lipschitz continuous, coercive $f$ which satisfies $|D f(x)| \le κ\bigl(1 +|f(x)|^{2-\frac{1}γ}\bigr)$ for some positive constant $κ$. Since $2-\frac{1}γ>1$, this assumption imposes very mild restrictions on the growth of the potential $f$. We also show that this solution fully characterizes optimality for the associated ergodic problem. Our method involves the study of an infinite dimensional linear program for elliptic equations for measures, and is very different from earlier approaches. It also applies to the larger class of Hamiltonians studied by Ichihara, and we show that it is well suited to provide verification of optimality results for the associated ergodic control problems, even in a pathwise sense, and without resorting to the parabolic problem.

math.AP↗
arXiv2019

A variational formula for risk-sensitive control of diffusions in $\mathbb{R}^d$

We address the variational problem for the generalized principal eigenvalue on $\mathbb{R}^d$ of linear and semilinear elliptic operators associated with nondegenerate diffusions controlled through the drift. We establish the Collatz-Wielandt formula for potentials that vanish at infinity under minimal hypotheses, and also for general potentials under blanket geometric ergodicity assumptions. We also present associated results having the flavor of a refined maximum principle.

math.OC↗
arXiv2019

`Controlled' versions of the Collatz-Wielandt and Donsker-Varadhan formulae

This is an overview of the work of the authors and their collaborators on the characterization of risk sensitive costs and rewards in terms of an abstract Collatz-Wielandt formula and in case of rewards, also a controlled version of the Donsker-Varadhan formula. For the finite state and action case, this leads to useful linear and dynamic programming formulations in the reducible case.

math.OC↗
arXiv2019

Uniform polynomial rates of convergence for a class of Lévy-driven controlled SDEs arising in multiclass many-server queues

We study the ergodic properties of a class of controlled stochastic differential equations (SDEs) driven by $α$-stable processes which arise as the limiting equations of multiclass queueing models in the Halfin-Whitt regime that have heavy-tailed arrival processes. When the safety staffing parameter is positive, we show that the SDEs are uniformly ergodic and enjoy a polynomial rate of convergence to the invariant probability measure in total variation, which is uniform over all stationary Markov controls resulting in a locally Lipschitz continuous drift. We also derive a matching lower bound on the rate of convergence (under no abandonment). On the other hand, when all abandonment rates are positive, we show that the SDEs are exponentially ergodic uniformly over the above-mentioned class of controls. Analogous results are obtained for Lévy-driven SDEs arising from multiclass many-server queues under asymptotically negligible service interruptions. For these equations, we show that the aforementioned ergodic properties are uniform over all stationary Markov controls. We also extend a key functional central limit theorem concerning diffusion approximations so as to make it applicable to the models studied here.

math.PR↗
arXiv2018

Ergodicity of Lévy-driven SDEs arising from multiclass many-server queues

We study the ergodic properties of a class of multidimensional piecewise Ornstein-Uhlenbeck processes with jumps, which contains the limit of the queueing processes arising in multiclass many-server queues with heavy-tailed arrivals and/or asymptotically negligible service interruptions in the Halfin-Whitt regime as special cases. In these queueing models, the Itô equations have a piecewise linear drift, and are driven by either (1) a Brownian motion and a pure-jump Lévy process, or (2) an anisotropic Lévy process with independent one-dimensional symmetric $α$-stable components, or (3) an anisotropic Lévy process as in (2) and a pure-jump Lévy process. We also study the class of models driven by a subordinate Brownian motion, which contains an isotropic (or rotationally invariant) $α$-stable Lévy process as a special case. We identify conditions on the parameters in the drift, the Lévy measure and/or covariance function which result in subexponential and/or exponential ergodicity. We show that these assumptions are sharp, and we identify some key necessary conditions for the process to be ergodic. In addition, we show that for the queueing models described above with no abandonment, the rate of convergence is polynomial, and we provide a sharp quantitative characterization of the rate via matching upper and lower bounds.

math.PR↗
arXiv2018

Certain Liouville properties of eigenfunctions of elliptic operators

We present certain Liouville properties of eigenfunctions of second-order elliptic operators with real coefficients, via an approach that is based on stochastic representations of positive solutions, and criticality theory of second-order elliptic operators. These extend results of Y. Pinchover to the case of nonsymmetric operators of Schrödinger type. In particular, we provide an answer to an open problem posed by Pinchover in [Comm. Math. Phys. 272 (2007), no. 1, 75-84, Problem 5]. In addition, we prove a lower bound on the decay of positive supersolutions of general second-order elliptic operators in any dimension, and discuss its implications to the Landis conjecture.

math.FA↗
arXiv2018

Optimal control of Markov-modulated multiclass many-server queues

We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means that it is critically loaded in the average sense, although not necessarily in each state of the Markov process. We show that under any static priority policy, the Markov-modulated diffusion-scaled queueing process is geometrically ergodic. This is accomplished by employing a solution to an associated Poisson equation in order to construct a suitable Lyapunov function. We establish a functional central limit theorem for the diffusion-scaled queueing process and show that the limiting process is a controlled diffusion with piecewise linear drift and constant covariance matrix. We address the infinite-horizon discounted and long-run average (ergodic) optimal control problems and establish asymptotic optimality.

math.PR↗
arXiv2018

Augmenting Max-Weight with Explicit Learning for Wireless Scheduling with Switching Costs

In small-cell wireless networks where users are connected to multiple base stations (BSs), it is often advantageous to switch off dynamically a subset of BSs to minimize energy costs. We consider two types of energy cost: (i) the cost of maintaining a BS in the active state, and (ii) the cost of switching a BS from the active state to inactive state. The problem is to operate the network at the lowest possible energy cost (sum of activation and switching costs) subject to queue stability. In this setting, the traditional approach -- a Max-Weight algorithm along with a Lyapunov-based stability argument -- does not suffice to show queue stability, essentially due to the temporal co-evolution between channel scheduling and the BS activation decisions induced by the switching cost. Instead, we develop a learning and BS activation algorithm with slow temporal dynamics, and a Max-Weight based channel scheduler that has fast temporal dynamics. We show using convergence of time-inhomogeneous Markov chains, that the co-evolving dynamics of learning, BS activation and queue lengths lead to near optimal average energy costs along with queue stability.

eess.SY↗
arXiv2018

On Learning the $cμ$ Rule in Single and Parallel Server Networks

We consider learning-based variants of the $c μ$ rule for scheduling in single and parallel server settings of multi-class queueing systems. In the single server setting, the $c μ$ rule is known to minimize the expected holding-cost (weighted queue-lengths summed over classes and a fixed time horizon). We focus on the problem where the service rates $μ$ are unknown with the holding-cost regret (regret against the $c μ$ rule with known $μ$) as our objective. We show that the greedy algorithm that uses empirically learned service rates results in a constant holding-cost regret (the regret is independent of the time horizon). This free exploration can be explained in the single server setting by the fact that any work-conserving policy obtains the same number of samples in a busy cycle. In the parallel server setting, we show that the $c μ$ rule may result in unstable queues, even for arrival rates within the capacity region. We then present sufficient conditions for geometric ergodicity under the $c μ$ rule. Using these results, we propose an almost greedy algorithm that explores only when the number of samples falls below a threshold. We show that this algorithm delivers constant holding-cost regret because a free exploration condition is eventually satisfied.

cs.PF↗
arXiv2018

Strict monotonicity of principal eigenvalues of elliptic operators in $\mathbb{R}^d$ and risk-sensitive control

This paper studies the eigenvalue problem on $\mathbb{R}^d$ for a class of second order, elliptic operators of the form $\mathscr{L} = a^{ij}\partial_{x_i}\partial_{x_j} + b^{i}\partial_{x_i} + f$, associated with non-degenerate diffusions. We show that strict monotonicity of the principal eigenvalue of the operator with respect to the potential function $f$ fully characterizes the ergodic properties of the associated ground state diffusion, and the unicity of the ground state, and we present a comprehensive study of the eigenvalue problem from this point of view. This allows us to extend or strengthen various results in the literature for a class of viscous Hamilton-Jacobi equations of ergodic type with smooth coefficients to equations with measurable drift and potential. In addition, we establish the strong duality for the equivalent infinite dimensional linear programming formulation of these ergodic control problems. We also apply these results to the study of the infinite horizon risk-sensitive control problem for diffusions, and establish existence of optimal Markov controls, verification of optimality results, and the continuity of the controlled principal eigenvalue with respect to stationary Markov controls.

math.AP↗
arXiv2018

Ergodic control of a class of jump diffusions with finite Lévy measures and rough kernels

We study the ergodic control problem for a class of jump diffusions in $\mathbb{R}^d$, which are controlled through the drift with bounded controls. The Levy measure is finite, but has no particular structure; it can be anisotropic and singular. Moreover, there is no blanket ergodicity assumption for the controlled process. Unstable behavior is `discouraged' by the running cost which satisfies a mild coercive hypothesis (i.e., is near-monotone). We first study the problem in its weak formulation as an optimization problem on the space of infinitesimal ergodic occupation measures, and derive the Hamilton-Jacobi-Bellman equation under minimal assumptions on the parameters, including verification of optimality results, using only analytical arguments. We also examine the regularity of invariant measures. Then, we address the jump diffusion model, and obtain a complete characterization of optimality.

math.OC↗
arXiv2018

Infinite horizon asymptotic average optimality for large-scale parallel server networks

We study infinite-horizon asymptotic average optimality for parallel server network with multiple classes of jobs and multiple server pools in the Halfin-Whitt regime. Three control formulations are considered: 1) minimizing the queueing and idleness cost, 2) minimizing the queueing cost under a constraints on idleness at each server pool, and 3) fairly allocating the idle servers among different server pools. For the third problem, we consider a class of bounded-queue, bounded-state (BQBS) stable networks, in which any moment of the state is bounded by that of the queue only (for both the limiting diffusion and diffusion-scaled state processes). We show that the optimal values for the diffusion-scaled state processes converge to the corresponding values of the ergodic control problems for the limiting diffusion. We present a family of state-dependent Markov balanced saturation policies (BSPs) that stabilize the controlled diffusion-scaled state processes. It is shown that under these policies, the diffusion-scaled state process is exponentially ergodic, provided that at least one class of jobs has a positive abandonment rate. We also establish useful moment bounds, and study the ergodic properties of the diffusion-scaled state processes, which play a crucial role in proving the asymptotic optimality.

math.OC↗
arXiv2017

Infinite Horizon Average Optimality of the N-network Queueing Model in the Halfin-Whitt Regime

We study the infinite horizon optimal control problem for N-network queueing systems, which consist of two customer classes and two server pools, under average (ergodic) criteria in the Halfin-Whitt regime. We consider three control objectives: 1) minimizing the queueing (and idleness) cost, 2) minimizing the queueing cost while imposing a constraint on idleness at each server pool, and 3) minimizing the queueing cost while requiring fairness on idleness. The running costs can be any nonnegative convex functions having at most polynomial growth. For all three problems we establish asymptotic optimality, namely, the convergence of the value functions of the diffusion-scaled state process to the corresponding values of the controlled diffusion limit. We also present a simple state-dependent priority scheduling policy under which the diffusion-scaled state process is geometrically ergodic in the Halfin-Whitt regime, and some results on convergence of mean empirical measures which facilitate the proofs.

math.OC↗
arXiv2017

Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions

We consider the infinite horizon risk-sensitive problem for nondegenerate diffusions with a compact action space, and controlled through the drift. We only impose a structural assumption on the running cost function, namely near-monotonicity, and show that there always exists a solution to the risk-sensitive Hamilton-Jacobi-Bellman (HJB) equation, and that any minimizer in the Hamiltonian is optimal in the class of stationary Markov controls. Under the additional hypothesis that the coefficients of the diffusion are bounded, and satisfy a condition that limits (even though it still allows) transient behavior, we show that any minimizer in the Hamiltonian is optimal in the class of all admissible controls. In addition, we present a sufficient condition, under which the solution of the HJB is unique (up to a multiplicative constant), and establish the usual verification result. We also present some new results concerning the multiplicative Poisson equation for elliptic operators in $\mathbb{R}^d$.

math.OC↗
arXiv2017

Controlled equilibrium selection in stochastically perturbed dynamics

We consider a dynamical system with finitely many equilibria and perturbed by small noise, in addition to being controlled by an `expensive' control. The controlled process is optimal for an ergodic criterion with a running cost that consists the sum of the control effort and a penalty function on the state space. We study the optimal stationary distribution of the controlled process as the variance of the noise becomes vanishingly small. It is shown that depending on the relative magnitudes of the noise variance and the `running cost' for control, one can identify three regimes, in each of which the optimal control forces the invariant distribution of the process to concentrate near equilibria that can be characterized according to the regime. We also obtain moment bounds for the optimal stationary distribution. Moreover, we show that in the vicinity of the points of concentration the density of optimal stationary distribution approximates the density of a Gaussian, and we explicitly solve for its covariance matrix.

math.PR↗
arXiv2017

Some new results on sample path optimality in ergodic control of diffusions

We present some new results on sample path optimality for the ergodic control problem of a class of non-degenerate diffusions controlled through the drift. The hypothesis most often used in the literature to ensure the existence of an a.s. sample path optimal stationary Markov control requires finite second moments of the first hitting times $τ$ of bounded domains over all admissible controls. We show that this can be considerably weakened: ${\mathrm E}[τ^2]$ may be replaced with ${\mathrm E}[τ\ln^+(τ)]$, thus reducing the required rate of convergence of averages from polynomial to logarithmic. A Foster-Lyapunov condition which guarantees this is also exhibited. Moreover, we study a large class of models that are neither uniformly stable, nor have a near-monotone running cost, and we exhibit sufficient conditions for the existence of a sample path optimal stationary Markov control.

math.OC↗