When Should Forecasting Models Be Re-Specified? A Cost-Sensitive Trigger for Adaptive Model-Form Updating

Abstract

Routine refresh bundles two operations that need not travel together: estimating parameters and selecting the model form. The second is often unnecessary. Under a reduced-update policy the maintenance question is a stopping problem: when has enough evidence accumulated against the deployed form to justify re-specifying it? We define specification debt as that evidence, derive a cost-sensitive rule that re-specifies when expected avoidable loss exceeds re-specification cost, and recover fixed update frequencies as the constant-accumulation case. The rule then raises its own question: how much machinery the monitor needs. We take this to all 47,982 monthly M4 series over an exponential smoothing grid and field three rules ordered by machinery: a fixed cadence with no monitoring, a one-parameter tracking signal after Trigg and Brown on the deployed model's one-step errors, and an evidence-gated trigger that fits a twelve-candidate grid for a validation score gap. Model form matters little on average here, so a cheap cadence matches full updating. The ordering within that band is informative. The tracking signal matches full updating at the three-step benchmark, but that edge dies under rate matching: against a cadence making the same nine searches, the gap is null at every horizon. What survives is the detector, once separated from the selector. The score-gap signal loses to its cap-matched cadence when a fire deploys the validation winner, but beats it at all five horizons when a fire re-selects by AICc, never worse at any lead; the validation-winner selector trades the first lead for the largest long-horizon gains, beating the cadence at horizon 18 by 0.4 percent (t of -9.0). Whether a gate repays its compute reduces to a break-even weight the evaluation reports. Everything rests on realized out-of-sample loss; the in-sample analogue does not predict degradation.