arXiv subjects

Rob Shone

Publications and source records attributed to Rob Shone.

6 recordsLinked to original sources

arXiv2026

Robust Patrol in a Dispersed Environment

We consider a patrol problem in which a patroller moves among several geographically dispersed locations to detect attackers that arrive over time. Once the patroller arrives at a location, they can spend any amount of time searching for attackers at that location---and detect an attacker there with some location-dependent instantaneous detection rate---before moving to a different location. The objective of the patroller is to minimize the expected time an attacker stays undetected at a location, regardless of where and when the attack occurs. In the special case where travel times are negligible, we elucidate an optimal cyclic policy in which the patroller allocates a fixed fraction of their effort to each location continuously. The patrol problem becomes significantly more challenging when travel times cannot be ignored. We introduce two types of cyclic patrol patterns based on common patrol practice for perimeter patrol and border patrol, respectively, and derive formulae for the expected time to detect an attack in both cases. We also provide an algorithm for finding the best search time parameters in both of these cases. We give several examples where these cycle types perform well and numerically demonstrate that the optimal patrol policy depends highly on the structure and parameters of each patrol problem.

math.OC↗

arXiv2026

Dynamic Repair and Maintenance of Heterogeneous Machines Dispersed on a Network: A Rollout Method for Online Reinforcement Learning

We consider a problem in which a single repairer is responsible for the maintenance and repair of a collection of machines, positioned at different locations on a network of nodes and edges. Machines deteriorate according to stochastic processes and incur increasing costs as they approach complete failure. The times needed for repairs to be performed, and the amounts of time needed for the repairer to switch between different machines, are random and machine-dependent. The problem is formulated as a Markov decision process (MDP) in which the objective is to minimize long-run average costs. We prove the equivalence of an alternative formulation based on rewards and use this to develop an index heuristic policy, which is shown to be optimal in certain special cases. We then use rollout-based reinforcement learning techniques to develop a novel online policy improvement (OPI) approach, which uses the index heuristic as a base policy and also as an insurance option at decision epochs where the best action cannot be selected with sufficient confidence. Results from extensive numerical experiments, involving randomly-generated network layouts and parameter values, show that the OPI heuristic is able to achieve close-to-optimal performance in fast-changing systems with state transitions occurring 100 times per second, suggesting that it is suitable for online implementation.

math.OC↗

arXiv2025

Stochastic dynamic job scheduling with interruptible setup and processing times: An approach based on queueing control

We consider a stochastic, dynamic job scheduling problem, formulated as a queueing control problem, in which a single server processes jobs of different types that arrive according to independent Poisson processes. The problem is defined on a network, with jobs arriving at designated demand points and waiting in queues to be processed by the server, which travels around the network dynamically and is able to change its course at any time. In the context of machine scheduling, this enables us to consider sequence-dependent, interruptible setup and processing times, with the network structure encoding the amounts of effort needed to switch between different tasks. We formulate the problem as a Markov decision process in which the objective is to minimize long-run average holding costs and prove the existence of a stationary policy under which the system is stable, subject to a condition on the workload of the system. We then propose a class of index-based heuristic policies, show that these possess intuitively appealing structural properties and suggest how to modify these heuristics to ensure scalability to larger problem sizes. Results from extensive numerical experiments are presented in order to show that our heuristic policies perform well against suitable benchmarks.

math.OC↗

arXiv2025

A Bi-Objective Markov Decision Process Design approach to redundancy allocation with dynamic maintenance for a parallel system

The reliability of a system can be improved by the addition of redundant elements, giving rise to the well-known redundancy allocation problem (RAP). We propose a novel extension to the RAP called the Bi-Objective Integrated Design and Dynamic Maintenance Problem (BO-IDDMP) which allows for future dynamic maintenance decisions to be incorporated. This leads to a problem with first-stage redundancy design decisions and second-stage sequential maintenance decisions under uncertainty. To the best of our knowledge, this is the first use of a continuous-time Markov Decision Process Design framework to formulate a problem with non-trivial dynamics, as well as its first use alongside bi-objective optimization. A general heuristic optimization methodology for bi-objective MDP Design problems is developed, and then applied to the BO-IDDMP. The efficiency and accuracy of our methodology are demonstrated against an exact mixed-integer linear programming solver. The heuristic is shown to be orders of magnitude faster in the majority of cases, and in only 2 out of 84 cases produces a solution that is dominated by the exact method. The inclusion of dynamic maintenance policies is shown to yield stronger and better-populated Pareto fronts, allowing more flexibility for the decision-maker. The impacts of varying parameters unique to our problem are also investigated.

math.OC↗

arXiv2022

A New Simheuristic Approach for Stochastic Runway Scheduling

We consider a stochastic, dynamic runway scheduling problem involving aircraft landings on a single runway. Sequencing decisions are made with knowledge of the estimated arrival times (ETAs) of all aircraft due to arrive at the airport, and these ETAs vary according to continuous-time stochastic processes. Time separations between consecutive runway landings are modeled via sequence-dependent Erlang distributions and are affected by weather conditions, which also evolve continuously over time. The resulting multi-stage optimization problem is intractable using exact methods and we propose a novel simheuristic approach, based on the application of methods analogous to variable neighborhood search (VNS) in a high-dimensional stochastic environment. Our model is calibrated using flight tracking data for over 98,000 arrivals at Heathrow Airport. Results from numerical experiments indicate that our proposed simheuristic algorithm outperforms an alternative based on deterministic forecasts under a wide range of parameter values, with the largest benefits being seen when the underlying stochastic processes become more volatile and also when the on-time requirements of individual flights are given greater weight in the objective function.

math.OC↗

arXiv2022

A stochastic game framework for patrolling a border

In this paper we consider a stochastic game for modelling the interactions between smugglers and a patroller along a border. The problem we examine involves a group of cooperating smugglers making regular attempts to bring small amounts of illicit goods across a border. A single patroller has the goal of preventing the smugglers from doing so, but must pay a cost to travel from one location to another. We model the problem as a two-player stochastic game and look to find the Nash equilibrium to gain insight to real world problems. Our framework extends the literature by assuming that the smugglers choose a continuous quantity of contraband, complicating the analysis of the game. We discuss a number of properties of Nash equilibria, including the aggregation of smugglers, the discount factors of the players, and the equivalence to a zero-sum game. Additionally, we present algorithms to find Nash equilibria that are more computationally efficient than existing methods. We also consider certain assumptions on the parameters of the model that give interesting equilibrium strategies for the players.

cs.GT↗