arXiv subjects

Qurban Ali

Publications and source records attributed to Qurban Ali.

4 recordsLinked to original sources

arXiv2026

Beyond Ensemble Disagreement: Predictive STL Monitoring of RL Locomotion under Distribution Shift

RL locomotion policies trained under nominal simulation conditions may encounter distribution shifts that compromise safety. Predictive monitors could anticipate violations, but their world-model forecasts may become unreliable under shift, and ensemble disagreement is not calibrated. We evaluate predictive Signal Temporal Logic (STL) monitoring for frozen Ant-v5 and HalfCheetah-v5 policies using closed-loop world-model rollouts and a three-model ensemble. Across external-force, observation-noise, and mass shifts, predicted robustness ranks HalfCheetah's future STL violations strongly, whereas disagreement does not reliably track observation prediction error or violations. Under severe noise, violations remain common ($64.2\pm24.4\%$ at $K{=}20$ across three PPO seeds), yet current-state robustness without prediction comes within $0.03$ AUROC of predictive robustness. Penalizing predicted robustness by disagreement does not consistently improve ranking. Ant provides complementary per-specification evidence, but its composite target has no positive violations and its any-violation target is single-class in most evaluated seed-condition slices. Overall, predictive robustness provides useful STL-specific ranking but little demonstrated additional ranking value over the current state; these findings caution against treating ensemble disagreement as a reliable indicator of safety uncertainty.

cs.RO↗

arXiv2026

Coverage-Guided Road Selection and Prioritization for Efficient Testing in Autonomous Driving Systems

Autonomous Driving Assistance Systems (ADAS) rely on extensive testing to ensure safety and reliability, yet road scenario datasets often contain redundant cases that slow down the testing process without improving fault detection. To address this issue, we present a novel test prioritization framework that reduces redundancy while preserving geometric and behavioral diversity. Road scenarios are clustered based on geometric and dynamic features of the ADAS driving behavior, from which representative cases are selected to guarantee coverage. Roads are finally prioritized based on geometric complexity, driving difficulty, and historical failures, ensuring that the most critical and challenging tests are executed first. We evaluate our framework on the OPENCAT dataset and the Udacity self-driving car simulator using two ADAS models. On average, our approach achieves an 89% reduction in test suite size while retaining an average of 79% of failed road scenarios. The prioritization strategy improves early failure detection by up to 95x compared to random baselines.

cs.SE↗

arXiv2025

OpenCat: Improving Interoperability of ADS Testing

Testing Advanced Driving Assistance Systems (ADAS), such as lane-keeping functions, requires creating road topologies or using predefined benchmarks. However, the test cases in existing ADAS benchmarks are often designed in specific formats (e.g., OpenDRIVE) and tailored to specific ADAS models. This limits their reusability and interoperability with other simulators and models, making it challenging to assess ADAS functionalities independently of the platform-specific details used to create the test cases. This paper evaluates the interoperability of SensoDat, a benchmark developed for ADAS regression testing. We introduce OpenCat, a converter that transforms OpenDRIVE test cases into the Catmull-Rom spline format, which is widely supported by many current test generators. By applying OpenCat to the SensoDat dataset, we achieved high accuracy in converting test cases into reusable road scenarios. To validate the converted scenarios, we used them to evaluate a lane-keeping ADAS model using the Udacity simulator. Both the simulator and the ADAS model operate independently of the technologies underlying SensoDat, ensuring an unbiased evaluation of the original test cases. Our findings reveal that benchmarks built with specific ADAS models hinder their effective usage for regression testing. We conclude by offering insights and recommendations to enhance the reusability and transferability of ADAS benchmarks for more extensive applications.

cs.SE↗

arXiv2024

Testing in the Evolving World of DL Systems:Insights from Python GitHub Projects

In the ever-evolving field of Deep Learning (DL), ensuring project quality and reliability remains a crucial challenge. This research investigates testing practices within DL projects in GitHub. It quantifies the adoption of testing methodologies, focusing on aspects like test automation, the types of tests (e.g., unit, integration, and system), test suite growth rate, and evolution of testing practices across different project versions. We analyze a subset of 300 carefully selected repositories based on quantitative and qualitative criteria. This study reports insights on the prevalence of testing practices in DL projects within the open-source community.

cs.SE↗