arXiv subjects

François Grolleau

Publications and source records attributed to François Grolleau.

4 recordsLinked to original sources

arXiv2025

MedFactEval and MedAgentBrief: A Framework and Workflow for Generating and Evaluating Factual Clinical Summaries

Evaluating factual accuracy in Large Language Model (LLM)-generated clinical text is a critical barrier to adoption, as expert review is unscalable for the continuous quality assurance these systems require. We address this challenge with two complementary contributions. First, we introduce MedFactEval, a framework for scalable, fact-grounded evaluation where clinicians define high-salience key facts and an "LLM Jury"--a multi-LLM majority vote--assesses their inclusion in generated summaries. Second, we present MedAgentBrief, a model-agnostic, multi-step workflow designed to generate high-quality, factual discharge summaries. To validate our evaluation framework, we established a gold-standard reference using a seven-physician majority vote on clinician-defined key facts from inpatient cases. The MedFactEval LLM Jury achieved almost perfect agreement with this panel (Cohen's kappa=81%), a performance statistically non-inferior to that of a single human expert (kappa=67%, P < 0.001). Our work provides both a robust evaluation framework (MedFactEval) and a high-performing generation workflow (MedAgentBrief), offering a comprehensive approach to advance the responsible deployment of generative AI in clinical workflows.

cs.CL↗

arXiv2025

Estimating Complier Average Causal Effects with Mixtures of Experts

Treatment non-compliance, where individuals deviate from their assigned experimental conditions, frequently complicates the estimation of causal effects. To address this, we introduce a novel learning framework based on a mixture of experts architecture to estimate the Complier Average Causal Effect (CACE). Our framework provides a flexible alternative to classical instrumental variable methods by relaxing their strict monotonicity and exclusion restriction assumptions. We develop a principled, two-step procedure where each step is optimized with a dedicated Expectation-Maximization (EM) algorithm. Crucially, we provide formal proofs that the model's components are identifiable, ensuring the learning procedure is well-posed. The resulting CACE estimators are proven to be consistent and asymptotically normal. Extensive simulations demonstrate that our method achieves a substantially lower root mean squared error than traditional instrumental variable approaches when their assumptions fail, an advantage that persists even when our own mixture of experts are misspecified. We illustrate the framework's practical utility on data from a large-scale randomized trial.

stat.ME↗

arXiv2025

powerROC: An Interactive Web Tool for Sample Size Calculation in Assessing Models' Discriminative Abilities

Rigorous external validation is crucial for assessing the generalizability of prediction models, particularly by evaluating their discrimination (AUROC) on new data. This often involves comparing a new model's AUROC to that of an established reference model. However, many studies rely on arbitrary rules of thumb for sample size calculations, often resulting in underpowered analyses and unreliable conclusions. This paper reviews crucial concepts for accurate sample size determination in AUROC-based external validation studies, making the theory and practice more accessible to researchers and clinicians. We introduce powerROC, an open-source web tool designed to simplify these calculations, enabling both the evaluation of a single model and the comparison of two models. The tool offers guidance on selecting target precision levels and employs flexible approaches, leveraging either pilot data or user-defined probability distributions. We illustrate powerROC's utility through a case study on hospital mortality prediction using the MIMIC database.

stat.ME↗

arXiv2023

A Comprehensive Framework for the Evaluation of Individual Treatment Rules From Observational Data

Individualized treatment rules (ITRs) are deterministic decision rules that recommend treatments to individuals based on their characteristics. Though ubiquitous in medicine, ITRs are hardly ever evaluated in randomized controlled trials. To evaluate ITRs from observational data, we introduce a new probabilistic model and distinguish two situations: i) the situation of a newly developed ITR, where data are from a population where no patient implements the ITR, and ii) the situation of a partially implemented ITR, where data are from a population where the ITR is implemented in some unidentified patients. In the former situation, we propose a procedure to explore the impact of an ITR under various implementation schemes. In the latter situation, on top of the fundamental problem of causal inference, we need to handle an additional latent variable denoting implementation. To evaluate ITRs in this situation, we propose an estimation procedure that relies on an expectation-maximization algorithm. In Monte Carlo simulations our estimators appear unbiased with confidence intervals achieving nominal coverage. We illustrate our approach on the MIMIC-III database, focusing on ITRs for dialysis initiation in patients with acute kidney injury.

stat.ME↗