Improved Disease Outbreak Detection from Out-of-sequence measurements Using Markov-switching Fixed-lag Particle Filters

Abstract

Particle filters (PFs) have become an essential tool for disease surveillance, as they can estimate hidden epidemic states in nonlinear and non-Gaussian models. In epidemic modelling, population dynamics may be governed by distinct regimes such as endemic or outbreak phases which can be represented using Markov-switching state-space models. In many real-world surveillance systems, data often arrives with delays or in the wrong temporal order, producing out-of-sequence (OOS) measurements that pertain to past time points rather than the current one. While existing PF methods can incorporate OOS measurements through particle reweighting, these approaches are limited in their ability to fully adjust past latent trajectories. To address this, we introduce a Markov-switching fixed-lag particle filter (FL-PF) that resimulates particle trajectories within a user-specified lag window, allowing OOS measurements to retroactively update both state and model estimates. By explicitly reevaluating historical samples, the FL-PF improves the accuracy and timeliness of outbreak detection and reduces false alarms. We also show how to compute the log-likelihood within the FL-PF framework, enabling parameter estimation using Sequential Monte Carlo squared (SMC$^2$). Together, these contributions extend the applicability of PFs to surveillance systems where retrospective data are common, offering a more robust framework for monitoring disease outbreaks and parameter inference.