Constructing Reliable Social Networks from Conversational Data: An Ensemble Prompt Engineering Approach with Uncertainty Quantification

Abstract

Conversational data are central to the study of interaction dynamics and social structures across psychological research. However, constructing structured social networks from unstructured conversational data remains a major methodological challenge. This study presents a pipeline for reliable network construction using prompt engineering. We employ an ensemble of multiple Large Language Models (LLMs) with majority voting to automate utterance classification, overcoming the scalability limitations of manual coding and the generalizability constraints of supervised deep learning. Classification reliability is assessed through an uncertainty quantification framework based on Shannon entropy, which supports systematic human-in-the-loop review of ambiguous cases. The classified utterances are used to construct directed interaction networks for subsequent analysis. We demonstrate the utility of this approach through two illustrative applications to classroom interaction data: network centrality analysis to characterize participant roles, and network mediation analysis using the additive and multiplicative effects network (AMEN) model to examine how interaction structures mediate the relationship between gender and mathematics performance. This pipeline provides a scalable foundation for automated network construction from conversational data across diverse research contexts.