What does the model actually see? Evaluation protocols and input availability in data-driven prediction of room acoustic parameters

Abstract

Machine-learnt models are increasingly used to predict ISO 3382-1 room acoustic parameters at unmeasured seats from sparse measurements, with reported coefficients of determination frequently above 0.85. This paper shows that such figures are often determined by the evaluation protocol rather than by the model. Using a multi-condition measurement campaign in a 264-seat conference hall and a 180-seat concert hall, three model families were evaluated under a factorial protocol ablation: validation splits either row-based or grouped by receiver position, and inputs either including measured-at-test quantities (the target position's impulse response, co-located parameter measurements, position identifiers) or restricted to source-receiver geometry and environmental state. Row-based splits with measured-at-test inputs reproduce the high reported accuracies (mean R^2 of 0.81 for the core parameters); grouped splits with deployment-consistent inputs reduce these to 0.09-0.60 and reorder the apparent difficulty of parameter classes. Access to the target's own impulse response at test time reproduces the high accuracy of the row-based folds but provides no benefit under grouped folds. The row-based figure reflects condition interpolation at the measured positions rather than transferable acoustic information. Under the deployment-consistent protocol the spread between Random Forest, the hybrid network, and inverse-distance weighting is several times smaller than the spread between protocols for a fixed model; the learnt models retain a measurable advantage for sound strength and reverberation time, and the high accuracy of the original pipelines re-emerges as condition interpolation at measured positions, a distinct and operationally useful task. A reporting checklist operationalises the findings.