arXiv subjects

Kristen M. Altenburger

Publications and source records attributed to Kristen M. Altenburger.

4 recordsLinked to original sources

arXiv2024

Examining the Role of Relationship Alignment in Large Language Models

The rapid development and deployment of Generative AI in social settings raise important questions about how to optimally personalize them for users while maintaining accuracy and realism. Based on a Facebook public post-comment dataset, this study evaluates the ability of Llama 3.0 (70B) to predict the semantic tones across different combinations of a commenter's and poster's gender, age, and friendship closeness and to replicate these differences in LLM-generated comments. The study consists of two parts: Part I assesses differences in semantic tones across social relationship categories, and Part II examines the similarity between comments generated by Llama 3.0 (70B) and human comments from Part I given public Facebook posts as input. Part I results show that including social relationship information improves the ability of a model to predict the semantic tone of human comments. However, Part II results show that even without including social context information in the prompt, LLM-generated comments and human comments are equally sensitive to social context, suggesting that LLMs can comprehend semantics from the original post alone. When we include all social relationship information in the prompt, the similarity between human comments and LLM-generated comments decreases. This inconsistency may occur because LLMs did not include social context information as part of their training data. Together these results demonstrate the ability of LLMs to comprehend semantics from the original post and respond similarly to human comments, but also highlights their limitations in generalizing personalized comments through prompting alone.

cs.CL↗

arXiv2024

A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing

The reliability of controlled experiments, commonly referred to as "A/B tests," is often compromised by network interference, where the outcomes of individual units are influenced by interactions with others. Significant challenges in this domain include the lack of accounting for complex social network structures and the difficulty in suitably characterizing network interference. To address these challenges, we propose a machine learning-based method. We introduce "causal network motifs" and utilize transparent machine learning models to characterize network interference patterns underlying an A/B test on networks. Our method's performance has been demonstrated through simulations on both a synthetic experiment and a large-scale test on Instagram. Our experiments show that our approach outperforms conventional methods such as design-based cluster randomization and conventional analysis-based neighborhood exposure mapping. Our approach provides a comprehensive and automated solution to address network interference for A/B testing practitioners. This aids in informing strategic business decisions in areas such as marketing effectiveness and product customization.

stat.ML↗

arXiv2021

Causal Network Motifs: Identifying Heterogeneous Spillover Effects in A/B Tests

Randomized experiments, or "A/B" tests, remain the gold standard for evaluating the causal effect of a policy intervention or product change. However, experimental settings, such as social networks, where users are interacting and influencing one another, may violate conventional assumptions of no interference for credible causal inference. Existing solutions to the network setting include accounting for the fraction or count of treated neighbors in a user's network, yet most current methods do not account for the local network structure beyond simply counting the number of neighbors. Our study provides an approach that accounts for both the local structure in a user's social network via motifs as well as the treatment assignment conditions of neighbors. We propose a two-part approach. We first introduce and employ "causal network motifs", which are network motifs that characterize the assignment conditions in local ego networks; and then we propose a tree-based algorithm for identifying different network interference conditions and estimating their average potential outcomes. Our approach can account for social network theories, such as structural diversity and echo chambers, and also can help specify network interference conditions that are suitable to each experiment. We test our method on a synthetic network setting and on a real-world experiment on a large-scale network, which highlight how accounting for local structures can better account for different interference patterns in networks.

cs.SI↗

arXiv2017

Bias and variance in the social structure of gender

The observation that individuals tend to be friends with people who are similar to themselves, commonly known as homophily, is a prominent and well-studied feature of social networks. Many machine learning methods exploit homophily to predict attributes of individuals based on the attributes of their friends. Meanwhile, recent work has shown that gender homophily can be weak or nonexistent in practice, making gender prediction particularly challenging. In this work, we identify another useful structural feature for predicting gender, an overdispersion of gender preferences introduced by individuals who have extreme preferences for a particular gender, regardless of their own gender. We call this property monophily for "love of one," and jointly characterize the statistical structure of homophily and monophily in social networks in terms of preference bias and preference variance. For prediction, we find that this pattern of extreme gender preferences introduces friend-of-friend correlations, where individuals are similar to their friends-of-friends without necessarily being similar to their friends. We analyze a population of online friendship networks in U.S. colleges and offline friendship networks in U.S. high schools and observe a fundamental difference between the success of prediction methods based on friends, "the company you keep," compared to methods based on friends-of-friends, "the company you're kept in." These findings offer an alternative perspective on attribute prediction in general and gender in particular, complicating the already difficult task of protecting attribute privacy.

cs.SI↗