arXiv subjects

Hamza Bin Mazhar

Publications and source records attributed to Hamza Bin Mazhar.

3 recordsLinked to original sources

arXiv2026

Fault Injection in OpenAPI Specifications for Evaluating Black-Box Testing Effectiveness

OpenAPI specifications are the primary input for black-box testing tools in microservice systems (MSS), yet prior work shows these specifications are often incomplete, inconsistent, or incorrect. Despite this, most studies on OpenAPI-based black-box testing assume correct specifications and evaluate tool performance. We address this gap by introducing a literature-grounded taxonomy of six OpenAPI specification fault classes. We inject faults at five severity levels, and evaluate the resulting mutated specifications on two microservice benchmarks, TrainTicket and SocialNetwork, using three testing tools: EvoMaster, RESTler, and Schemathesis. We measure the impact of these faults using code coverage, specification coverage, request/response quality, and behavioral diversity. Our results show that specification faults cause strong and heterogeneous degradation patterns across testing tools and systems. Faults in method semantics cause broad degradation across all metrics, while others, such as modifications to response codes, remain weak. Relaxations of schema constraints cause hidden degradation, with no impact on code and specification coverage but a large impact on request/response quality. These findings demonstrate that specification quality directly shapes black-box API testing effectiveness. Also, code and specification coverage-only evaluations can understate the impact of specification faults on black-box testing in MSS and should be complemented by request/response quality and behavioral diversity.

cs.SE↗

arXiv2026

Towards LLM Accelerated Rapid Reviews for Software Tool Discovery -- Case for Log Anomaly Detection

In software engineering research, the primary outcome is frequently a tool. However, for practitioners and academics alike, it is hard to tell which tools are maintained and do they work out of the box. In this paper, we propose a pipeline to identify relevant studies with LLM screening, extract the tools presented in them, and run them with LLM-based coding agent. To evaluate the feasibility of our approach we focus on software log anomaly detection tools. We begin the study by designing a broad search string that yields 3233 hits from Scopus. We request two LLMs to provide an inclusion probability for each title-abstract pair according to the inclusion and exclusion criteria. From the 3233 exported abstracts, this screening reduced the number of included papers to 569, out of which we could download 470. These papers included 206 unique links and after manual evaluation we determined 83 to be tools. Finally, we ran the LLM-based coding agent on these 83 links, and got 24 successfully running tools. As replicating our approach would require roughly only 4 hours of human effort, of which 3 hours were manual PDF downloading, and 12 hours of LLM running time, this demonstrates promising efficiency when utilizing LLMs in rapid reviews. Because practitioner-built tools often lack academic papers, in the future we aim to expand our analysis to tool-hosting platforms such as GitHub and PyPI. In the future, we plan to formalize our workflow as LLM Agent Skills to make our approach easier to adopt.

cs.SE↗

arXiv2026

AnoMod: A Dataset for Anomaly Detection and Root Cause Analysis in Microservice Systems

Microservice systems (MSS) have become a predominant architectural style for cloud services. Yet the community still lacks high-quality, publicly available datasets for anomaly detection (AD) and root cause analysis (RCA) in MSS. Most benchmarks emphasize performance-related faults and provide only one or two monitoring modalities, limiting research on broader failure modes and cross-modal methods. To address these gaps, we introduce a new multimodal anomaly dataset built on two open-source microservice systems: SocialNetwork and TrainTicket. We design and inject four categories of anomalies (Ano): performance-level, service-level, database-level, and code-level, to emulate realistic anomaly modes. For each scenario, we collect five modalities (Mod): logs, metrics, distributed traces, API responses, and code coverage reports, offering a richer, end-to-end view of system state and inter-service interactions. We name our dataset, reflecting its unique properties, as AnoMod. This dataset enables (1) evaluation of cross-modal anomaly detection and fusion/ablation strategies, and (2) fine-grained RCA studies across service and code regions, supporting end-to-end troubleshooting pipelines that jointly consider detection and localization.

cs.SE↗