arXiv subjects

Klint Kanopka

Publications and source records attributed to Klint Kanopka.

2 recordsLinked to original sources

arXiv2026

Measurement Induced Confounding

A critical assumption of observational studies is that all confounding variables must be known and sufficiently adjusted for to estimate causal effects. An implicit, and often overlooked, aspect of this assumption is that all confounding variables have been measured without error. In the social and medical sciences, latent traits such as motivation, self-efficacy, and ability measures are likely confounding variables. Because latent traits are not directly observable, conventional approaches to adjust for them in observational studies rely on collecting responses to individual items on a test or survey instrument and then adjust for sum scores, measurement model-derived ability estimates, or item responses directly. Through a process we describe as measurement induced confounding, we show that measurement error propagates through the estimation process and that current conventional approaches to adjusting for latent traits in observational studies produce biased estimates of the average treatment effect with incorrectly calibrated coverage properties. A critical implication of this finding is that current observational studies that attempt to adjust for latent confounding variables likely put forth biased causal estimates with incorrect uncertainty intervals. We show that measurement induced confounding can be resolved through a Bayesian Joint Estimation approach that simultaneously estimates the measurement model, the treatment assignment model, and the response model.

stat.ME↗

arXiv2025

Algorithmic Tradeoffs, Applied NLP, and the State-of-the-Art Fallacy

Computational sociology is growing in popularity, yet the analytic tools employed differ widely in power, transparency, and interpretability. In computer science, methods gain popularity after surpassing benchmarks of predictive accuracy, becoming the "state of the art." Computer scientists favor novelty and innovation for different reasons, but prioritizing technical prestige over methodological fit could unintentionally limit the scope of sociological inquiry. To illustrate, we focus on computational text analysis and revisit a prior study of college admissions essays, comparing analyses with both older and newer methods. These methods vary in flexibility and opacity, allowing us to compare performance across distinct methodological regimes. We find that newer techniques did not outperform prior results in meaningful ways. We also find that using the current state of the art, generative AI and large language models, could introduce bias and confounding that is difficult to extricate. We therefore argue that sociological inquiry benefits from methodological pluralism that aligns analytic choices with theoretical and empirical questions. While we frame this sociologically, scholars in other disciplines may confront what we call the "state-of-the-art fallacy", the belief that the tool computer scientists deem to be the best will work across topics, domains, and questions.

cs.CY↗