Audit Before You Update: Recording Artifacts and the Limits of Bounded Model Updates in MIMIC-IV ICU Prediction
Clinical prediction models in intensive care degrade as data shift, and regulators such as the FDA ask developers to specify bounded update protocols in advance. Evaluations of such protocols assume that apparent drift reflects clinical practice rather than changes in how the database records care. We test whether update strategies can be evaluated on unaudited ICU data, and whether a parameter-bounded update gives up accuracy or attribution stability relative to full fine-tuning once the data are audited. We audit MIMIC-IV 3.1 (2008-2022) for recording artifacts and define a cleaning protocol. On the cleaned cohort, a label-free classifier two-sample test gates updates across three eras. We compare a bounded update, which retrains only treatment-context and fusion parameters, with full fine-tuning for two model families and two targets under a pre-specified non-inferiority margin of 0.015 AUROC, and measure attribution stability against the source model. In 2020-2022 about half of ICU stays have no input or procedure-event records, and a vasopressor ItemID migration makes the 2017-2019 label rate appear to fall when it actually rises. On the cleaned cohort, the gate detects drift in every era, including in physiology-only features, with no false alarms on null data. Bounded updating is non-inferior or superior in 10 of 12 primary cells, but this depends on the target, the model family and learning-rate tuning, and adaptation gains over never updating are small. At a shared learning rate, bounded updates keep physiological attributions more stable in neural models, clearly for vasopressor and less clearly for intubation; with separately tuned rates, full updates are more stable. Audits should precede update evaluation, and the update boundary should be chosen per target.