arXiv subjects

Alper Buğra Keleş

Publications and source records attributed to Alper Buğra Keleş.

6 recordsLinked to original sources

arXiv2026

ISTQB Certifications Under the Lens: Their Contributions to the Software-Testing Profession; and AI-assisted Synthesis of Practitioners' Endorsements and Criticisms

Context: The International Software Testing Qualifications Board (ISTQB) certification dominates global software testing practice, with 1.2+ million certifications issued across 130+ countries. Yet it remains contested: practitioners value it for career advancement and shared terminology, while others criticize it as overly theoretical and somewhat disconnected from real-world testing practices. Objective: This study investigates the perceived value and critique of ISTQB certifications, the most widely recognized testing qualifications worldwide. Method: We conducted an AI-assisted Multivocal Literature Review (MLR), combining academic and grey literature to synthesize practitioner endorsements (RQ1) and criticisms (RQ2). ChatGPT's deep research capability was employed under continuous human oversight, with QA strategies ensuring transparency and reliability. As another analysis, we asked a panel of four independent experts to evaluate the precision of endorsements and fairness of criticisms. Results: Practitioner endorsements emphasized career benefits, improved communication, and a shared vocabulary as the main values of ISTQB certifications. Criticisms focused on excessive theoretical content, limited relevance in agile and automation-intensive contexts, and weak support for real testing skills. Expert review confirmed that while many endorsements were precise, several criticisms reflected broader tensions in the discipline, including contrasting schools of thought in testing practice. Conclusions: ISTQB certifications provide recognizable career and communication value but remain contested in terms of practical utility. By triangulating practitioner voices with expert validation, this study delivers an evidence-based reflection on the strengths and weaknesses of ISTQB in shaping the software testing body of knowledge.

cs.SE↗

arXiv2025

MBTModelGenerator: Automated Reverse Engineering of Test Models from Clickstream Data for Model-Based Testing of Web Applications

Context: Model-Based Testing (MBT) was first introduced in 1970's, and has the potential to improve efficiency and effectiveness of testing. However, its adoption-especially for web applications-has been hindered by the effort required to manually design MBT models, and keep them updated. Objective: Based on the above challenge in a real industrial context, this study introduces an automated approach to reduce that effort by reverse engineering MBT models from clickstream data captured during users' interaction with web applications. Method: We have developed and present in this paper an open-source tool, named MBTModelGenerator, which logs user interactions via a lightweight JavaScript module in the front-end, and transmits them to a REST API backend. These interactions are then transformed into directly executable MBT models in the input format of an open-source MBT tool named GraphWalker. Results: The tool was evaluated on two representative open-source web applications, Spring PetClinic and a Task Manager web app, and is under evaluation in several large-scale industrial testing projects. The generated MBT models accurately reflected user navigation flows and could be executed in the GraphWalker MBT tool without any manual changes. Using the tool has significantly reduced the effort of MBT model design by more than 90%, while still allowing test engineers to inspect and refine the generated models for completeness. Conclusion: Our approach facilitates lightweight adoption of MBT by automating model generation, which is the most effort intensive phase of MBT. To ensure correctness and completeness, the generated models should still be reviewed by test engineers -- but that effort remains substantially lower than designing MBT models from scratch. The tool is in active industrial use and available as open-source for reuse and further development.

cs.SE↗

arXiv2025

AI-powered software testing tools: A systematic review and empirical assessment of their features and limitations

Context: The rise of Artificial Intelligence (AI) in software engineering has led to the development of AI-powered test automation tools, promising improved efficiency, reduced maintenance effort, and enhanced defect-detection. However, a systematic evaluation of these tools is needed to understand their capabilities, benefits, and limitations. Objective: This study has two objectives: (1) A systematic review of AI-assisted test automation tools, categorizing their key AI features; (2) an empirical study of two selected AI-powered tools on two software under test, to investigate the effectiveness and limitations of the tools. Method: A systematic review of 55 AI-based test automation tools was conducted, classifying them based on their AI-assisted capabilities such as self-healing tests, visual testing, and AI-powered test generation. In the second phase, two representative tools were selected for the empirical study, in which we applied them to test two open-source software systems. Their performance was compared with traditional test automation approaches to evaluate efficiency and adaptability. Results: The review provides a comprehensive taxonomy of AI-driven testing tools, highlighting common features and trends. The empirical evaluation demonstrates that AI-powered automation enhances test execution efficiency and reduces maintenance effort but also exposes limitations such as handling complex UI changes and contextual understanding. Conclusion: AI-driven test automation tools show strong potential in improving software quality and reducing manual testing effort. However, their current limitations-such as false positives, lack of domain knowledge, and dependency on predefined models-indicate the need for further refinement. Future research should focus on advancing AI models to improve adaptability, reliability, and robustness in software testing.

cs.SE↗

arXiv2024

A pragmatic look at education and training of software test engineers: Further cooperation of academia and industry is needed

Alongside software testing education in universities, a great extent of effort and resources are spent on software-testing training activities in industry. For example, there are several international certification schemes in testing, such as those provided by the International Software Testing Qualifications Board (ISTQB), which have been issued to more than 914K testers so far. To train the highly qualified test engineers of tomorrow, it is important for both university educators and trainers in industry to be aware of the status of software testing education in academia versus its training in industry, to analyze the relationships of these two approaches, and to assess ways on how to improve the education / training landscape. For that purpose, this paper provides a pragmatic overview of the issue, presents several recommendations, and hopes to trigger further discussions in the community, between industry and academia, on how to further improve the status-quo, and to find further best practices for more effective education and training of software testers. The paper is based on combined ~40 years of the two authors' technical experience in test engineering, and their ~30 years of experience in providing testing education and training in more than six countries.

cs.SE↗

arXiv2024

Coverage measurement in model-based testing of web applications: Tool support and an industrial experience report

There are many widely used tools for measuring test-coverage and code-coverage. Test coverage is the ratio of requirements or other non-code artifacts covered by a test suite, while code-coverage is the ratio of source code covered by tests. Almost all coverage tools show a few certain subset of coverage values, and almost always either test-coverage or code-coverage measures. In a large-scale industrial web-application-testing setting, we were faced with the need to "integrate" several types of coverage data (including front-end and back-end code coverage with requirements coverage), and to see all of them "live" as large model-based test suites were running. By being unable to find any off-the-shelf toolset to address the above need, we have developed an open-source test coverage tool, specific for MBT, named MBTCover. In addition to code coverage, the tool measures and reports requirements and model coverage, "live" as a given MBT test suite is executing. In this paper, we present the features of the MBTCover tool and our experience from using it in multiple large test-automation projects in practice. Other software test engineers, who conduct web application testing and MBT, may find the tool useful in their projects.

cs.SE↗

arXiv2021

Model-based testing in practice: An experience report from the web applications domain

In the context of a large software testing company, we have deployed the model-based testing (MBT) approach to take the company's test automation practices to higher levels of maturity /and capability. We have chosen, from a set of open-source/commercial MBT tools, an open-source tool named GraphWalker, and have pragmatically used MBT for end-to-end test automation of several large web and mobile applications under test. The MBT approach has provided, so far in our project, various tangible and intangible benefits in terms of improved test coverage (number of paths tested), improved test-design practices, and also improved real-fault detection effectiveness. The goal of this experience report (applied research report), done based on "action research", is to share our experience of applying and evaluating MBT as a software technology (technique and tool) in a real industrial setting. We aim at contributing to the body of empirical evidence in industrial application of MBT by sharing our industry-academia project on applying MBT in practice, the insights that we have gained, and the challenges and questions that we have faced and tackled so far. We discuss an overview of the industrial setting, provide motivation, explain the events leading to the outcomes, discuss the challenges faced, summarize the outcomes, and conclude with lessons learned, take-away messages, and practical advices based on the described experience. By learning from the best practices in this paper, other test engineers could conduct more mature MBT in their test projects.

cs.SE↗