arXiv subjects

Mark E. Lewis

Publications and source records attributed to Mark E. Lewis.

5 recordsLinked to original sources

arXiv2026

Telehealth Control Policies: Bridging the Gap Between Patients and Doctors

This paper studies a sequential decision-making problem in a two-stage queueing system modeled after operations in CVS MinuteClinics, where nurse practitioners (NPs) oversee patient care throughout the entire visit. All services are non-preemptive, and NPs cannot begin treating a new patient until the current patient has completed both stages of care. Following an initial diagnosis in the upstream phase, NPs must decide for low-acuity patients whether to proceed with treatment independently through immediate service, or to collaborate with a dedicated general physician (GP) via telemedicine. While collaboration typically improves service quality and is preferred by individual patients, it may introduce delays as the NP-patient pair waits for a GP to become available. This work explores the structural properties of optimal policies under different system parameters, with a focus on large initial upstream queues, revealing unconventional and complex policy behaviors. Leveraging these structural insights and supporting theoretical results, we design simple and effective heuristics that are computable in linear time and suitable for practical implementation. These heuristics are robust across the entire parameter space of interest, and offer clear, actionable guidance for NPs as system parameters vary. They also achieve near-optimal performance, averaging within 0.1% of the optimal, while commonly used benchmark policies are highly sensitive to parameter shifts and can incur costs more than 100% higher than optimal. The work provides applicable insights for decision-makers on improving policy robustness and effectiveness, as well as recommendations for stakeholders on the value of investing in telemedicine infrastructure. For instance, we identify scenarios where such investments may be either unnecessary or essential based on specific system parameters.

math.OC↗

arXiv2026

Control policies for a two-stage queueing system with parallel and single server options

We study a two-stage tandem service queue attended by two servers. Each job-server pair must complete both service phases together, with the server unable to begin a new job until the current one is fully processed after two stages. Immediately after the first phase of service, the server decides whether to send the job/customer to a downstream station that allows parallel processing or to a single-service facility that offers faster or higher-quality service but handles only one job at a time. This choice determines whether the second phase commences immediately or (potentially) after waiting in a queue for the single-service facility to become available. The decision-making scenario is modeled via a Markov decision process formulation, of a clearing system with holding costs at each station. We fully characterize the structural properties of an optimal control policy based on the relationship between the service rates at the downstream stations. A numerical study highlights the significance of optimal control by comparing its performance against several natural heuristic policies.

math.OC↗

arXiv2026

Balancing Independent and Collaborative Service

We study a two-type server queueing system where flexible Type-I servers, upon their initial interaction with jobs, decide in real time whether to process them independently or in collaboration with dedicated Type-II servers. Independent processing begins immediately, as does collaborative service if a Type-II server is available. Otherwise, the job and its paired Type-I server wait in queue for collaboration. Type-I servers are non-preemptive and cannot engage with new jobs until their current job is completed. We provide a complete characterization of the structural properties of the optimal policy for the clearing system. In particular, an optimal control is shown to follow a threshold structure based on the number of jobs in the queue before a Type-I first interaction and on the number of jobs in either independent or collaborative service. We propose simple threshold heuristics, based on linear approximations, for real-time decision-making. In much of the parameter and state spaces, we establish theoretical bounds that compare the thresholds proposed by our heuristics to those of optimal policies and identify parameter configurations where these bounds are attained. Outside of these regions, the optimal thresholds are infinite. Numerical experiments further demonstrate the accuracy and robustness of our heuristics, particularly when the initial queue length is high. Our proposed heuristics achieve costs within 0.5% of the optimal policy on average and significantly outperform benchmark policies that exhibit extreme sensitivity to system parameters, sometimes incurring costs exceeding 100% of the optimal.

math.OC↗

arXiv2017

On the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of $(s,S)$ Inventory Policies

This paper studies convergence properties of optimal values and actions for discounted and average-cost Markov Decision Processes (MDPs) with weakly continuous transition probabilities and applies these properties to the stochastic periodic-review inventory control problem with backorders, positive setup costs, and convex holding/backordering costs. The following results are established for MDPs with possibly noncompact action sets and unbounded cost functions: (i) convergence of value iterations to optimal values for discounted problems with possibly non-zero terminal costs, (ii) convergence of optimal finite-horizon actions to optimal infinite-horizon actions for total discounted costs, as the time horizon tends to infinity, and (iii) convergence of optimal discount-cost actions to optimal average-cost actions for infinite-horizon problems, as the discount factor tends to 1. Being applied to the setup-cost inventory control problem, the general results on MDPs imply the optimality of $(s,S)$ policies and convergence properties of optimal thresholds. In particular this paper analyzes the setup-cost inventory control problem without two assumptions often used in the literature: (a) the demand is either discrete or continuous or (b) the backordering cost is higher than the cost of backordered inventory if the amount of backordered inventory is large.

math.OC↗

arXiv2013

Dynamic Service Rate Control for a Single Server Queue with Markov Modulated Arrivals

We consider the problem of service rate control of a single server queueing system with a finite-state Markov-modulated Poisson arrival process. We show that the optimal service rate is non-decreasing in the number of customers in the system; higher congestion rates warrant higher service rates. On the contrary, however, we show that the optimal service rate is not necessarily monotone in the current arrival rate. If the modulating process satisfies a stochastic monotonicity property the monotonicity is recovered. We examine several heuristics and show where heuristics are reasonable substitutes for the optimal control. None of the heuristics perform well in all the regimes. Secondly, we discuss when the Markov-modulated Poisson process with service rate control can act as a heuristic itself to approximate the control of a system with a periodic non-homogeneous Poisson arrival process. Not only is the current model of interest in the control of Internet or mobile networks with bursty traffic, but it is also useful in providing a tractable alternative for the control of service centers with non-stationary arrival rates.

math.OC↗