No Value Is Better Than a Wrong Value: Finding and Handling Erroneous Data in Literature-Mined Datasets for AI
Machine learning across materials science, chemistry, energy, and medicine increasingly relies on literature-extracted datasets due to a lack of curated databases. However, automated extraction introduces errors: text recognition misreads digits, units and notations are confused, values are misattributed, and LLMs hallucinate numbers. Drawing on our work building an electro-optic property database, we present a taxonomy of errors organized by their origin and signatures, alongside a layered framework for error detection. The framework combines schema and provenance checks, physical admissibility (bounds, symmetry, and governing equations), cross-source statistical consensus, model auditing, and suspicion-ranked review. In a production run, our verification gate halted 402 of 816 candidate values with typed reasons. Yet, an audit against curated references showed that 30 of 112 accepted values were defective; standard symmetry rules alone caught 21 of these. On measured BaTiO3 and KNbO3 tensors, constitutive identities caught 8 of 10 single-value errors and isolated the corrupted entries. In a controlled benchmark, the combined framework caught 81% of errors at 85% precision, including errors in single-report materials, bringing model accuracy on held-out data to within 3% of a model trained on clean data. Uncleaned data severely impairs discovery: 89% of the twenty largest values were errors, causing AI searches to prioritize false positives. Once bad numbers are controlled, missing measurement conditions emerge as the main limit, as distinct records become indistinguishable without full context. We outline directions for the field, including shared physics rule libraries and self-reporting databases.