arXiv subjects

Olsi Rrjolli

Publications and source records attributed to Olsi Rrjolli.

3 recordsLinked to original sources

arXiv2026

Using OAI Overlay to Enhance REST API Fuzzing

REST APIs are widely used in industry. Therefore, a lot of research has been focused on how to automatically generate test cases for REST APIs, with few different open-source fuzzers existing in the literature. For a thorough testing, especially in black-box scenarios, just relying on the information provided in the OpenAPI schemas is not enough. Testers typically need to provide extra input data to help steer the fuzzers in the right direction. Dedicated formats specific to each different fuzzer would work, but they would create a vendor lock-in, as well as increasing cognitive load. The OpenAPI Initiative (OAI) standard Overlay might be a solution to this problem. Such standard enables to define transformations on the OpenAPI schemas, where testers can provide input data in Overlay files where such data is provided as ``examples'' entries. In this paper, we have extended the state-of-the-art fuzzer EvoMaster to support Overlay files natively. Experiments are carried out in industry on five APIs from five enterprises from around the world (e.g., Belgium, China, Germany and Türkiye), including two Fortune500 enterprises as well as a 3-man startup. Our industrial results show that Overlay is a viable solution to better enable black-box fuzzing of REST APIs in industry.

cs.SE↗

arXiv2026

Fuzzing REST APIs in Industry: Necessary Features and Open Problems

REST APIs are widely used in industry, in all different kinds of domains. An example is Volkswagen AG, a German automobile manufacturer. Established testing approaches for REST APIs are time consuming, and require expertise from professional test engineers. Due to its cost and importance, in the scientific literature several approaches have been proposed to automatically test REST APIs. The open-source, search-based fuzzer EvoMaster is one of such tools proposed in the academic literature. However, how academic prototypes can be integrated in industry and have real impact to software engineering practice requires more investigation. In this paper, we report on our experience in using EvoMaster at Volkswagen AG, as an EvoMaster user from 2023 to 2026. We share our learnt lessons, and discuss several features needed to be implemented in EvoMaster to make its use in an industrial context successful. Feedback about value in industrial setups of EvoMaster was given from Volkswagen AG about 4 APIs. Additionally, a user study was conducted involving 11 testing specialists from 4 different companies. We further identify several real-world research challenges that still need to be solved.

cs.SE↗

arXiv2025

Generating REST API Tests With Descriptive Names

Automated test generation has become a key technique for ensuring software quality, particularly in modern API-based architectures. However, automatically generated test cases are typically assigned non-descriptive names (e.g., test0, test1), which reduces their readability and hinders their usefulness during comprehension and maintenance. In this work, we present three novel deterministic techniques to generate REST API test names. We then compare eight techniques in total for generating descriptive names for REST API tests automatically produced by the fuzzer EvoMaster, using 10 test cases generated for 9 different open-source APIs. The eight techniques include rule-based heuristics and large language model (LLM)-based approaches. Their effectiveness was empirically evaluated through two surveys (involving up to 39 people recruited via LinkedIn). Our results show that a rule-based approach achieves the highest clarity ratings among deterministic methods, performs on par with state-of-the-art LLM-based models such as Gemini and GPT-4o, and significantly outperforms GPT-3.5. To further evaluate the practical impact of our results, an industrial case study was carried out with practitioners who actively use EvoMaster at Volkswagen AG. A developer questionnaire was then carried out based on the use of EvoMaster on four different APIs by four different users, for a total of 74 evaluated test cases. Feedback from practitioners further confirms that descriptive names produced by this approach improve test suite readability. These findings highlight that lightweight, deterministic techniques can serve as effective alternatives to computationally expensive and security-sensitive LLM-based approaches for automated system-level test naming, providing a practical step toward more developer-friendly API test generation.

cs.SE↗