Ali Sayedsalehi | Searcharxiv

arXiv2026

DRS-OSS: A Diff-Risk Scoring Tool for Continuous Integration Workflows

Software teams need change-risk scores that can guide continuous integration decisions such as review prioritization, test scheduling, and downstream validation before risky changes are merged or released. However, open-source teams often lack deployable tools for surfacing these risk signals in everyday CI workflows. We present DRS-OSS, an open-source diff-risk scoring tool for continuous integration workflows. DRS-OSS is designed as a deployable and customizable pipeline rather than as a standalone prediction model. It combines a REST API gateway, containerized model services, a developer dashboard, GitHub integration, and a replication package that lets users retrain or replace the backend with other transformer models. The bundled workflow combines commit messages, commit diffs, and change metrics in a single risk-prediction pipeline. The default packaged backend uses a Llama 3.1 8B sequence classifier configured for long diffs. Its training recipe uses parameter-efficient tuning, quantization, CPU offloading, and customization helper scripts so that it can be adapted on modest hardware. We compare DRS-OSS with similar tools and evaluate the bundled classifier on ApacheJIT, where it reaches an ROC-AUC of 0.895 and outperforms prior baselines. From a user-feedback perspective, DRS-OSS has received interest from Uber, Duolingo, and Microsoft in adapting the workflow to their own continuous integration settings. The full tool is released with source code, customization scripts, deployment artifacts, a public repository, a live demo at worldofcode.org/drs, and a demonstration video at youtube.com/watch?v=2FzeRRdNaco.

cs.SE↗

arXiv2026

Risk-Aware Batch Testing for Performance Regression Detection

Performance regression testing is essential in large-scale continuous-integration (CI) systems, yet executing full performance suites for every commit is prohibitively expensive. Prior work on performance regression prediction and batch testing has shown independent benefits, but each faces practical limitations: predictive models are rarely integrated into CI decision-making, and conventional batching strategies ignore commit-level heterogeneity. We unify these strands by introducing a risk-aware framework that integrates machine-learned commit risk with adaptive batching. Using Mozilla Firefox as a case study, we construct a production-derived dataset of human-confirmed regressions aligned chronologically with Autoland, and fine-tune ModernBERT, CodeBERT, and LLaMA-3.1 variants to estimate commit-level performance regression risk, achieving up to 0.694 ROC-AUC with CodeBERT. The risk scores drive a family of risk-aware batching strategies, including Risk-Aged Priority Batching and Risk-Adaptive Stream Batching, evaluated through realistic CI simulations. Across thousands of historical Firefox commits, our best overall configuration, Risk-Aged Priority Batching with linear aggregation (RAPB-la), yields a Pareto improvement over Mozilla's production-inspired baseline. RAPB-la reduces total test executions by 32.4%, decreases mean feedback time by 3.8%, maintains mean time-to-culprit at approximately the baseline level, reduces maximum time-to-culprit by 26.2%, and corresponds to an estimated annual infrastructure cost savings of approximately $491K under our cost model. These results demonstrate that risk-aware batch testing can reduce CI resource consumption while improving diagnostic timeliness. To support reproducibility and future research, we release a complete replication package containing all datasets, fine-tuning pipelines, and implementations of our batching algorithms.

cs.SE↗