Generating is not discovering: a pre-registered physics judge for AI-proposed superconductors, calibrated on six known superconductors and one negative control

Abstract

Generative models now propose millions of "stable" crystal structures, and their own referees have shown that stability is not discovery; no public benchmark asks whether a candidate has the physics to do a job. For superconductors the job is twofold: to pair, with a definite gap symmetry, and to become wire. We present a physics judge that answers both from a structure alone (DFT+U, a gated Wannier model, the full rank-4 RPA susceptibility on a $24^3$ mesh, the linearized gap equation, an irreducible-representation classifier, a manufacturability funnel) and measure what it says about what generators propose. With answer keys pre-registered before computing, the judge was calibrated on six known superconductors (Nb, Nb$_3$Sn, MgB$_2$, BaFe$_2$As$_2$, La$_{2-x}$Sr$_x$CuO$_4$, YBa$_2$Cu$_3$O$_7$), recovering the expected class in all six after two dated corrections. On the ruthenium analogue of the iron pnictide, with two runs and their criterion pre-registered the day before, it returns a null verdict at the same physical interaction where iron gives s$\pm$ ($\lambda_1 \le 0.0007$ versus 0.096; critical U 4.17 versus 0.97 eV); the pre-registered equal-alpha run fails the letter of the criterion, and we declare the resolution in favour of absolute U as post-hoc. The funnel reproduces the industrial map with thresholds fixed before running. Three generation audits follow: a descriptor sweep over 150 Materials Project metals re-finds the canon; 1,248 MatterGen structures yield 0 candidates that are new, stable and carry a pairing motif; mechanism filters over 47,893 compounds, validated by a blind hold-out, re-derive the community's analogies: geometry plus d-count is necessary but not sufficient. Generation re-finds what is known; judgment is the bottleneck. We propose a public benchmark over the eleven lists of AI-proposed superconductor candidates, none of which classifies gap symmetry.