An Evidence-First Multi-LLM Framework for Auditable Critical-Infrastructure Dependency Modeling

Abstract

Critical-infrastructure knowledge is distributed across heterogeneous, incomplete, and weakly structured evidence, making dependency models difficult to construct automatically and difficult to trust. Large language models (LLMs) can extract structured knowledge from such evidence, but direct LLM-to-graph generation risks unsupported relationships, inconsistent terminology, incorrect entity identities, and erroneous dependency endpoints. We present an evidence-first multi-LLM framework for constructing Infrastructure Knowledge Bases (IKBs) and Infrastructure Dependency Graphs (IDGs) from heterogeneous infrastructure documentation. Multiple open-weight LLMs independently extract candidate entities and dependencies from normalized evidence, after which the framework separates evidence verification, ontology grounding, entity resolution, dependency alignment, validation, fusion, and human review. Evidence support, ontology reconciliation, endpoint resolution, model agreement, and human validation remain distinct states, while provenance and unresolved cases are preserved throughout. The validated IKB is then projected deterministically into the IDG without introducing new LLM-generated knowledge. Evaluation in nine infrastructure projects shows that entity recovery achieves substantially higher recall than complete directed dependency recovery and that canonical endpoint resolution is a major constraint in dependency construction. Cross-model overlap is also much lower for dependencies than for entities, indicating that the models often produce non-overlapping candidate assertions rather than a stable majority consensus. These findings support an auditable evidence-to-IKB-to-IDG process in which uncertainty is preserved and resolved progressively rather than collapsed into a single confidence or voting decision.