arXiv subjects

Daniel de Vassimon Manela

Publications and source records attributed to Daniel de Vassimon Manela.

4 recordsLinked to original sources

arXiv2025

Testing Generalizability in Causal Inference

Ensuring robust model performance in diverse real-world scenarios requires addressing generalizability across domains with covariate shifts. However, no formal procedure exists for statistically evaluating generalizability in machine learning algorithms. Existing predictive metrics like mean squared error (MSE) help to quantify the relative performance between models, but do not directly answer whether a model can or cannot generalize. To address this gap in the domain of causal inference, we propose a systematic framework for statistically evaluating the generalizability of high-dimensional causal inference models. Our approach uses the frugal parameterization to flexibly simulate from fully and semi-synthetic causal benchmarks, offering a comprehensive evaluation for both mean and distributional regression methods. Grounded in real-world data, our method ensures more realistic evaluations, which is often missing in current work relying on simplified datasets. Furthermore, using simulations and statistical testing, our framework is robust and avoids over-reliance on conventional metrics, providing statistical safeguards for decision making.

cs.LG↗

arXiv2025

Exact Simulation of Longitudinal Data from Marginal Structural Models

Simulating longitudinal data from specified marginal structural models is a crucial but challenging task for evaluating causal inference methods and informing study design. While data generation typically proceeds in a fully conditional manner using structural equations according to a temporal ordering, it is difficult to ensure alignment between conditional distributions and the target marginal causal effects, which presents a fundamental challenge. To address this, we propose a flexible and efficient algorithm for simulating longitudinal data that adheres exactly to a specified marginal structural model. Our approach accommodates time-to-event outcomes and extends naturally to survival settings, which are prevalent in applied research. Compared to existing approaches, it offers several advantages: it enables exact simulation from a known causal model rather than relying on approximations; avoids restrictive assumptions about the data-generating process; and remains computationally efficient by requiring only the evaluation of analytical expressions, rather than Monte Carlo methods or numerical integration. Through simulation studies replicating realistic scenarios, we validate the method's accuracy and utility. Our method will facilitate researchers in effectively simulating data with target causal structures for their specific scenarios.

stat.ME↗

arXiv2024

Marginal Causal Flows for Validation and Inference

Investigating the marginal causal effect of an intervention on an outcome from complex data remains challenging due to the inflexibility of employed models and the lack of complexity in causal benchmark datasets, which often fail to reproduce intricate real-world data patterns. In this paper we introduce Frugal Flows, a novel likelihood-based machine learning model that uses normalising flows to flexibly learn the data-generating process, while also directly inferring the marginal causal quantities from observational data. We propose that these models are exceptionally well suited for generating synthetic data to validate causal methods. They can create synthetic datasets that closely resemble the empirical dataset, while automatically and exactly satisfying a user-defined average treatment effect. To our knowledge, Frugal Flows are the first generative model to both learn flexible data representations and also exactly parameterise quantities such as the average treatment effect and the degree of unobserved confounding. We demonstrate the above with experiments on both simulated and real-world datasets.

cs.LG↗

arXiv2021

Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Models

This paper proposes two intuitive metrics, skew and stereotype, that quantify and analyse the gender bias present in contextual language models when tackling the WinoBias pronoun resolution task. We find evidence that gender stereotype correlates approximately negatively with gender skew in out-of-the-box models, suggesting that there is a trade-off between these two forms of bias. We investigate two methods to mitigate bias. The first approach is an online method which is effective at removing skew at the expense of stereotype. The second, inspired by previous work on ELMo, involves the fine-tuning of BERT using an augmented gender-balanced dataset. We show that this reduces both skew and stereotype relative to its unaugmented fine-tuned counterpart. However, we find that existing gender bias benchmarks do not fully probe professional bias as pronoun resolution may be obfuscated by cross-correlations from other manifestations of gender prejudice. Our code is available online, at https://github.com/12kleingordon34/NLP_masters_project.

cs.CL↗