T. J. Sego | Searcharxiv

arXiv2025

EFECT: A Method to Quantify the Reproducibility of Stochastic Simulations

Reproducibility is a fundamental requirement for validating scientific claims in computational research. Stochastic computational models are widely used in fields such as systems biology, financial modeling and environmental sciences. However, achieving reproducibility in stochastic simulations remains challenging, as each run can produce different outcomes. Existing infrastructure and software tools do not address independent reproduction of simulation results. Without independent reproducibility, results and conclusions lack credibility, as it remains unclear whether observed findings reflect model behavior or are artifacts of stochastic variation or an underpowered study. To bridge this gap, we introduce the Empirical Characteristic Function Equality Convergence Test (EFECT), a data-driven method to quantify the reproducibility of stochastic simulation results. EFECT employs empirical characteristic functions to compare reported results with those independently generated by assessing distributional inequality, termed EFECT error. Additionally, we establish the EFECT convergence point, a quantitative metric for determining the required number of simulation runs to achieve an EFECT error value of a priori significance. EFECT is applicable to all bounded, real-valued outputs, regardless of the model type or simulation method that produced them. We tested EFECT with over 40 use cases to demonstrate its broad applicability and effectiveness. EFECT standardizes stochastic simulation reproducibility, establishing a workflow that guarantees reliable results, supporting a wide range of stakeholders, and thereby enhancing validation of stochastic simulation studies, across a model's lifecycle. To promote standardization, we are developing the open-source software library libSSR in multiple programming languages for easy integration of EFECT.

stat.ME↗

arXiv2025

From FAIR to CURE: Guidelines for Computational Models of Biological Systems

Guidelines for managing scientific data have been established under the FAIR principles requiring that data be Findable, Accessible, Interoperable, and Reusable. In many scientific disciplines, especially computational biology, both data and models are key to progress. For this reason, and recognizing that such models are a very special type of 'data', we argue that computational models, especially mechanistic models prevalent in medicine, physiology and systems biology, deserve a complementary set of guidelines. We propose the CURE principles, emphasizing that models should be Credible, Understandable, Reproducible, and Extensible. We delve into each principle, discussing verification, validation, and uncertainty quantification for model credibility; the clarity of model descriptions and annotations for understandability; adherence to standards and open science practices for reproducibility; and the use of open standards and modular code for extensibility and reuse. We outline recommended and baseline requirements for each aspect of CURE, aiming to enhance the impact and trustworthiness of computational models, particularly in biomedical applications where credibility is paramount. Our perspective underscores the need for a more disciplined approach to modeling, aligning with emerging trends such as Digital Twins and emphasizing the importance of data and modeling standards for interoperability and reuse. Finally, we emphasize that given the non-trivial effort required to implement the guidelines, the community moves to automate as many of the guidelines as possible.

q-bio.OT↗