Null-Corrected Conditional Coverage in Materials Property Prediction: A Five-Dataset Replication
Uncertainty quantification for materials property prediction increasingly reports conditional coverage, coverage within strata of the predicted value, alongside marginal coverage. The worst-bin statistics used for this are downward-biased in finite samples, a known defect that applied materials work does not generally correct for. We replicate a single-dataset conditional coverage study across five regression datasets (n = 437 to 21,263), seven interval methods and 50 random splits each, correcting the worst-bin statistic against its expected value under exact conditional validity. The null is simulated from realized per-bin test counts and referenced to each method's own marginal coverage, separating marginal miscalibration from uneven coverage. On the smallest dataset the correction runs from 0.061 to 0.096, between a third and two thirds of the shortfalls actually reported, against 0.009 to 0.011 on the two largest. Split conformal prediction's failure survives the correction on four of five datasets; two conclusions of the antecedent study do not. Conformalized quantile regression, characterized there as a negative case, fails on one dataset only. Cross-fitted normalized conformal prediction, grouped there with the successful methods, fails on three of five. Group-conditional (Mondrian) conformal prediction stays below our significance criterion on every dataset, yet exceeds the null on three of five under an octile refinement of the variable it calibrates on. A level-matched ablation locates that uniformity in group-conditional calibration rather than in the elevated quantile level, and removes Mondrian's apparent width cost entirely. Conditional coverage evaluations should report the null distribution of the worst-bin statistic used, the method's realized marginal coverage, and the resolution of the evaluation partition relative to any partition the method calibrates on.