arXiv subjects

Subhamay Saha

Publications and source records attributed to Subhamay Saha.

17 recordsLinked to original sources

arXiv2024

Nonzero-sum Discrete-time Stochastic Games with Risk-sensitive Ergodic Cost Criterion

In this paper we study infinite horizon nonzero-sum stochastic games for controlled discrete-time Markov chains on a Polish state space with risk-sensitive ergodic cost criterion. Under suitable assumptions we show that the associated ergodic optimality equations admit unique solutions. Finally, the existence of Nash-equilibrium in randomized stationary strategies is established by showing that an appropriate set-valued map has a fixed point.

math.OC↗

arXiv2022

Semimartingle Representation of a class of Semi-Markov Dynamics

We consider a class of semi-Markov processes (SMP) such that the embedded discrete time Markov chain may be non-homogeneous. The corresponding augmented processes are represented as semi-martingales using stochastic integral equation involving a Poisson random measure. The existence and uniqueness of the equation are established. Subsequently, we show that the solution is indeed a SMP with desired transition rate. Finally, we derive the law of the bivariate process obtained from two solutions of the equation having two different initial conditions.

math.PR↗

arXiv2021

Zero and Non-zero Sum Risk-sensitive Semi-Markov Games

In this article we consider zero and non-zero sum risk-sensitive average criterion games for semi-Markov processes with a finite state space. For the zero-sum case, under suitable assumptions we show that the game has a value. We also establish the existence of a stationary saddle point equilibrium. For the non-zero sum case, under suitable assumptions we establish the existence of a stationary Nash equilibrium.

math.OC↗

arXiv2021

Risk-sensitive Semi-Markov Decision Problems with Discounted Cost and General Utilities

In this article we consider risk-sensitive control of semi-Markov processes with a discrete state space. We consider general utility functions and discounted cost in the optimization criteria. We consider random finite horizon and infinite horizon problems. Using a state augmentation technique we characterise the value functions and also prescribe optimal controls.

math.OC↗

arXiv2020

Continuous-time Zero-Sum Stochastic Game with Stopping and Control

We consider a zero-sum stochastic game for continuous-time Markov chain with countable state space and unbounded transition and pay-off rates. The additional feature of the game is that the controllers together with taking actions are also allowed to stop the process. Under suitable hypothesis we show that the game has a value and it is the unique solution of certain dynamic programming inequalities with bilateral constraints. In the process we also prescribe a saddle point equilibrium.

math.OC↗

arXiv2020

Asymptotic optimality of the generalized $c\mu$ rule under model uncertainty

We consider a critically-loaded multiclass queueing control problem with model uncertainty. The model consists of $I$ types of customers and a single server. At any time instant, a decision-maker (DM) allocates the server's effort to the customers. The DM's goal is to minimize a convex holding cost that accounts for the ambiguity with respect to the model, i.e., the arrival and service rates. For this, we consider an adversary player whose role is to choose the worst-case scenario. Specifically, we assume that the DM has a reference probability model in mind and that the cost function is formulated by the supremum over equivalent admissible probability measures to the reference measure with two components, the first is the expected holding cost, and the second one is a penalty for the adversary player for deviating from the reference model. The penalty term is formulated by a general divergence measure. We show that although that under the equivalent admissible measures the critically-load condition might be violated, the generalized $c\mu$ rule is asymptotically optimal for this problem.

math.PR↗

arXiv2017

Zero-sum stochastic differential game with risk-sensitive cost

Zero sum games with risk-sensitive cost criterion are considered with underlying dynamics being given by controlled stochastic differential equations. Under the assumption of geometric stability on the dynamics , we completely characterize all possible saddle point strategies in the class of stationary Markov controls. In addition, we also establish existence-uniqueness result for the value function of the Hamilton-Jacobi-Isaacs equation.

math.OC↗

arXiv2017

Strict monotonicity of principal eigenvalues of elliptic operators in $\mathbb{R}^d$ and risk-sensitive control

This paper studies the eigenvalue problem on $\mathbb{R}^d$ for a class of second order, elliptic operators of the form $\mathscr{L} = a^{ij}\partial_{x_i}\partial_{x_j} + b^{i}\partial_{x_i} + f$, associated with non-degenerate diffusions. We show that strict monotonicity of the principal eigenvalue of the operator with respect to the potential function $f$ fully characterizes the ergodic properties of the associated ground state diffusion, and the unicity of the ground state, and we present a comprehensive study of the eigenvalue problem from this point of view. This allows us to extend or strengthen various results in the literature for a class of viscous Hamilton-Jacobi equations of ergodic type with smooth coefficients to equations with measurable drift and potential. In addition, we establish the strong duality for the equivalent infinite dimensional linear programming formulation of these ergodic control problems. We also apply these results to the study of the infinite horizon risk-sensitive control problem for diffusions, and establish existence of optimal Markov controls, verification of optimality results, and the continuity of the controlled principal eigenvalue with respect to stationary Markov controls.

math.AP↗

arXiv2017

Optimality of the generalized cμ- rule in the moderate deviation regime

This paper studies a multiclass queueing system with an associated risk- sensitive cost observed in heavy traffic at the moderate deviation scale, accounting for convex queue length penalties. The main result is the asymptotic optimality of a dynamic index policy known from the diffusion scale heavy traffic literature as the generalized cμ- rule.

math.OC↗

arXiv2016

On viscosity solution of HJB equations with state constraints and reflection control

Motivated by a control problem of a certain queueing network we consider a control problem where the dynamics is constrained in the nonnegative orthant $\mathbb{R}_+$ of the $d$-dimensional Euclidean space and controlled by the reflections at the faces/boundaries. We define a discounted value function associated to this problem and show that the value function is a viscosity solution to a certain HJB equation in $\mathbb{R}_+$ with nonlinear Neumann type boundary condition. Under certain conditions, we also characterize this value function as the unique solution to this HJB equation.

math.OC↗

arXiv2016

A note on non-existence of diffusion limits for serve-the-longest-queue when the buffers are equal in size

We consider the serve-the-longest-queue discipline for a multiclass queue with buffers of equal size, operating under (i) the conventional and (ii) the Halfin-Whitt heavy traffic regimes, and show that while the queue length process' scaling limits are fully determined by the first and second order data in case (i), they depend on finer properties in case (ii). The proof of the latter relies on the construction of a {\it deterministic} arrival pattern.

math.PR↗

arXiv2015

Central Limit Results for Jump-Diffusions with Mean Field Interaction and a Common Factor

A system of $N$ weakly interacting particles whose dynamics is given in terms of jump-diffusions with a common factor is considered. The common factor is described through another jump-diffusion and the coefficients of the evolution equation for each particle depend, in addition to its own state value, on the empirical measure of the states of the $N$ particles and the common factor. A Central Limit Theorem, as $N \to \infty$, is established. The limit law is described in terms of a certain Gaussian mixture. An application to models in Mathematical Finance of self-excited correlated defaults is described.

math.PR↗

arXiv2015

An $ε$-Nash equilibrium with high probability for strategic customers in heavy traffic

A multiclass queue with many servers is considered, where customers make a join-or-leave decision upon arrival based on queue length information, without knowing the scheduling policy or the state of other queues. A game theoretic formulation is proposed and analyzed, that takes advantage of a phenomenon unique to heavy traffic regimes, namely Reiman's snaphshot principle, by which waiting times are predicted with high precision by the information available upon arrival. The payoff considered is given as a random variable, which depends on the customer's decision, accounting for waiting time in the queue and penalty for leaving. The notion of an equilibrium is only meaningful in an asymptotic framework, which is taken here to be the Halfin-Whitt heavy traffic regime. The main result is the identification of an $ε$-Nash equilibrium with probability approaching 1. On way to proving this result, new diffusion limit results for systems with finite buffers are obtained.

math.OC↗

arXiv2015

Construction of Asymptotically Optimal Control for a Stochastic Network from a Free Boundary Problem

An asymptotic framework for optimal control of multiclass stochastic processing networks, using formal diffusion approximations under suitable temporal and spatial scaling, by Brownian control problems (BCP) and their equivalent workload formulations (EWF), has been developed by Harrison (1988). This framework has been implemented in many works for constructing asymptotically optimal control policies for a broad range of stochastic network models. To date all asymptotic optimality results for such networks correspond to settings where the solution of the EWF is a reflected Brownian motion in the positive orthant with normal reflections. In this work we consider a well studied stochastic network which is perhaps the simplest example of a model with more than one dimensional workload process. In the regime considered here, the singular control problem corresponding to the EWF does not have a simple form explicit solution, however by considering an associated free boundary problem one can give a representation for an optimal controlled process as a two dimensional reflected Brownian motion in a Lipschitz domain whose boundary is determined by the solution of the free boundary problem. Using the form of the optimal solution we propose a sequence of control policies, given in terms of suitable thresholds, for the scaled stochastic network control problems and prove that this sequence of policies is asymptotically optimal. As suggested by the solution of the EWF, the policy we propose requires a server to idle under certain conditions which are specified in terms of the thresholds determined from the free boundary.

math.PR↗

arXiv2014

Optimal Control of Markov Processes with Age-Dependent Transition Rates

We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite horizon average cost problems. Our approach is via the construction of an equivalent semi-Markov decision process. We characterize the value function and optimal controls for both discounted and average cost cases.

math.OC↗

arXiv2014

Zero-Sum Stochastic Games with Partial Information and Average Payoff

We consider discrete time partially observable zero-sum stochastic game with average payoff criterion. We study the game using an equivalent completely observable game. We show that the game has a value and also we come up with a pair of optimal strategies for both the players.

math.OC↗

arXiv2014

Risk-sensitive control of continuous time Markov chains

We study risk-sensitive control of continuous time Markov chains taking values in discrete state space. We study both finite and infinite horizon problems. In the finite horizon problem we characterise the value function via HJB equation and obtain an optimal Markov control. We do the same for infinite horizon discounted cost case. In the infinite horizon average cost case we establish the existence of an optimal stationary control under certain Lyapunov condition. We also develop a policy iteration algorithm for finding an optimal control.

math.OC↗