arXiv subjects

Qunhua Li

Publications and source records attributed to Qunhua Li.

7 recordsLinked to original sources

arXiv2025

AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science

Large language models (LLMs) are increasingly used to automate data analysis through executable code generation. Yet, data science tasks often admit multiple statistically valid solutions, e.g. different modeling strategies, making it critical to understand the reasoning behind analyses, not just their outcomes. While manual review of LLM-generated code can help ensure statistical soundness, it is labor-intensive and requires expertise. A more scalable approach is to evaluate the underlying workflows-the logical plans guiding code generation. However, it remains unclear how to assess whether an LLM-generated workflow supports reproducible implementations. To address this, we present AIRepr, an Analyst-Inspector framework for automatically evaluating and improving the reproducibility of LLM-generated data analysis workflows. Our framework is grounded in statistical principles and supports scalable, automated assessment. We introduce two novel reproducibility-enhancing prompting strategies and benchmark them against standard prompting across 15 analyst-inspector LLM pairs and 1,032 tasks from three public benchmarks. Our findings show that workflows with higher reproducibility also yield more accurate analyses, and that reproducibility-enhancing prompts substantially improve both metrics. This work provides a foundation for transparent, reliable, and efficient human-AI collaboration in data science. Our code is publicly available.

cs.LG↗

arXiv2018

Segmented correspondence curve regression model for quantifying reproducibility of high-throughput experiments

The reliability of a high-throughput biological experiment relies highly on the settings of the operational factors in its experimental and data-analytic procedures. Understanding how operational factors influence the reproducibility of the experimental outcome is critical for constructing robust workflows and obtaining reliable results. One challenge in this area is that candidates at different levels of significance may respond to the operational factors differently. To model this heterogeneity, we develop a novel segmented regression model, based on the rank concordance between candidates from different replicate samples, to characterize the varying effects of operational factors for candidates at different levels of significance. A grid search method is developed to identify the change point in response to the operational factors and estimate the covariate effects accounting for the change. A sup-likelihood-ratio-type test is proposed to test the existence of a change point. Simulation studies show that our method yields a well-calibrated type I error, is powerful in detecting the difference in reproducibility, and achieves a better model fitting than the existing method. An application on a ChIP-seq dataset reveals interesting insights on how sequencing depth affects the reproducibility of experimental results, demonstrating the usefulness of our method in designing cost-effective and reliable high-throughput workflows.

stat.ME↗

arXiv2016

A Continuous Threshold Expectile Model

Expectile regression is a useful tool for exploring the relation between the response and the explanatory variables beyond the conditional mean. This article develops a continuous threshold expectile regression for modeling data in which the effect of a covariate on the response variable is linear but varies below and above an unknown threshold in a continuous way. Based on a grid search approach, we obtain estimators for the threshold and the regression coefficients via an asymmetric least squares regression method. We derive the asymptotic properties for all the estimators and show that the estimator for the threshold achieves root-n consistency. We also develop a weighted CUSUM type test statistic for the existence of a threshold in a given expectile, and derive its asymptotic properties under both the null and the local alternative models. This test only requires fitting the model under the null hypothesis in the absence of a threshold, thus it is computationally more efficient than the likelihood-ratio type tests. Simulation studies show desirable finite sample performance in both homoscedastic and heteroscedastic cases. The application of our methods on a Dutch growth data and a baseball pitcher salary data reveals interesting insights.

stat.ME↗

arXiv2016

Robust bent line regression

We introduce a rank-based bent linear regression with an unknown change point. Using a linear reparameterization technique, we propose a rank-based estimate that can make simultaneous inference on all model parameters, including the location of the change point, in a computationally efficient manner. We also develop a score-like test for the existence of a change point, based on a weighted CUSUM process. This test only requires fitting the model under the null hypothesis in absence of a change point, thus it is computationally more efficient than likelihood-ratio type tests. The asymptotic properties of the test are derived under both the null and the local alternative models. Simulation studies and two real data examples show that the proposed methods are robust against outliers and heavy-tailed errors in both parameter estimation and hypothesis testing.

stat.ME↗

arXiv2013

A likelihood-based scoring method for peptide identification using mass spectrometry

Mass spectrometry provides a high-throughput approach to identify proteins in biological samples. A key step in the analysis of mass spectrometry data is to identify the peptide sequence that, most probably, gave rise to each observed spectrum. This is often tackled using a database search: each observed spectrum is compared against a large number of theoretical "expected" spectra predicted from candidate peptide sequences in a database, and the best match is identified using some heuristic scoring criterion. Here we provide a more principled, likelihood-based, scoring criterion for this problem. Specifically, we introduce a probabilistic model that allows one to assess, for each theoretical spectrum, the probability that it would produce the observed spectrum. This probabilistic model takes account of peak locations and intensities, in both observed and theoretical spectra, which enables incorporation of detailed knowledge of chemical plausibility in peptide identification. Besides placing peptide scoring on a sounder theoretical footing, the likelihood-based score also has important practical benefits: it provides natural measures for assessing the uncertainty of each identification, and in comparisons on benchmark data it produced more accurate peptide identifications than other methods, including SEQUEST. Although we focus here on peptide identification, our scoring rule could easily be integrated into any downstream analyses that require peptide-spectrum match scores.

stat.AP↗

arXiv2011

Measuring reproducibility of high-throughput experiments

Reproducibility is essential to reliable scientific discovery in high-throughput experiments. In this work we propose a unified approach to measure the reproducibility of findings identified from replicate experiments and identify putative discoveries using reproducibility. Unlike the usual scalar measures of reproducibility, our approach creates a curve, which quantitatively assesses when the findings are no longer consistent across replicates. Our curve is fitted by a copula mixture model, from which we derive a quantitative reproducibility score, which we call the "irreproducible discovery rate" (IDR) analogous to the FDR. This score can be computed at each set of paired replicate ranks and permits the principled setting of thresholds both for assessing reproducibility and combining replicates. Since our approach permits an arbitrary scale for each replicate, it provides useful descriptive measures in a wide variety of situations to be explored. We study the performance of the algorithm using simulations and give a heuristic analysis of its theoretical properties. We demonstrate the effectiveness of our method in a ChIP-seq experiment.

stat.AP↗

arXiv2010

A nested mixture model for protein identification using mass spectrometry

Mass spectrometry provides a high-throughput way to identify proteins in biological samples. In a typical experiment, proteins in a sample are first broken into their constituent peptides. The resulting mixture of peptides is then subjected to mass spectrometry, which generates thousands of spectra, each characteristic of its generating peptide. Here we consider the problem of inferring, from these spectra, which proteins and peptides are present in the sample. We develop a statistical approach to the problem, based on a nested mixture model. In contrast to commonly used two-stage approaches, this model provides a one-stage solution that simultaneously identifies which proteins are present, and which peptides are correctly identified. In this way our model incorporates the evidence feedback between proteins and their constituent peptides. Using simulated data and a yeast data set, we compare and contrast our method with existing widely used approaches (PeptideProphet/ProteinProphet) and with a recently published new approach, HSM. For peptide identification, our single-stage approach yields consistently more accurate results. For protein identification the methods have similar accuracy in most settings, although we exhibit some scenarios in which the existing methods perform poorly.

stat.AP↗