arXiv subjects

Taher A. Ghaleb

Publications and source records attributed to Taher A. Ghaleb.

28 records · Page 2Linked to original sources

arXiv2026

Code Change Characteristics and Description Alignment: A Comparative Study of Agentic versus Human Pull Requests

AI coding agents can autonomously generate pull requests (PRs), yet little is known about how their contributions compare to those of humans. We analyze 33,596 agent-generated PRs (APRs) and 6,618 human PRs (HPRs) to compare code-change characteristics and message quality. We observe that APR-introduced symbols (functions and classes) are removed much sooner than those in HPRs (median time to removal 3 vs. 34 days) and are also removed more often (symbol churn 7.33% vs. 4.10%), reflecting a focus on other tasks like documentation and test updates. Agents generate stronger commit-level messages (semantic similarity 0.72 vs. 0.68) but lag humans at PR-level summarization (PR-commit similarity 0.86 vs. 0.88). Commit message length is the best predictor of description quality, indicating reliance on individual commits over full-PR reasoning. These findings highlight a gap between agents' micro-level precision and macro-level communication, suggesting opportunities to improve agent-driven development workflows.

cs.SE↗

arXiv2026

Self-Admitted Technical Debt in LLM Software: An Empirical Comparison with ML and Non-ML Software

Self-admitted technical debt (SATD), referring to comments flagged by developers that explicitly acknowledge suboptimal code or incomplete functionality, has received extensive attention in machine learning (ML) and traditional (Non-ML) software. However, little is known about how SATD manifests and evolves in contemporary Large Language Model (LLM)-based systems, whose architectures, workflows, and dependencies differ fundamentally from both traditional and pre-LLM ML software. In this paper, we conduct the first empirical study of SATD in the LLM era, replicating and extending prior work on ML technical debt to modern LLM-based systems. We compare SATD prevalence across LLM, ML, and non-ML repositories across a total of 477 repositories (159 per category). We perform survival analysis of SATD introduction and removal to understand the dynamics of technical debt across different development paradigms. Surprisingly, despite their architectural complexity, our results reveal that LLM repositories accumulate SATD at similar rates to ML systems (3.95% vs. 4.10%). However, we observe that LLM repositories remain debt-free 2.4x longer than ML repositories (a median of 492 days vs. 204 days), and then start to accumulate technical debt rapidly. Moreover, our qualitative analysis of 377 SATD instances reveals three new forms of technical debt unique to LLM-based development that have not been reported in prior research: Model-Stack Workaround Debt, Model Dependency Debt, and Performance Optimization Debt. Finally, by mapping SATD to stages of the LLM development pipeline, we observe that debt concentrates

cs.SE↗

arXiv2025

Task-Aware Reduction for Scalable LLM-Database Systems

Large Language Models (LLMs) are increasingly applied to data-intensive workflows, from database querying to developer observability. Yet the effectiveness of these systems is constrained by the volume, verbosity, and noise of real-world text-rich data such as logs, telemetry, and monitoring streams. Feeding such data directly into LLMs is costly, environmentally unsustainable, and often misaligned with task objectives. Parallel efforts in LLM efficiency have focused on model- or architecture-level optimizations, but the challenge of reducing upstream input verbosity remains underexplored. In this paper, we argue for treating the token budget of an LLM as an attention budget and elevating task-aware text reduction as a first-class design principle for language -- data systems. We position input-side reduction not as compression, but as attention allocation: prioritizing information most relevant to downstream tasks. We outline open research challenges for building benchmarks, designing adaptive reduction pipelines, and integrating token-budget--aware preprocessing into database and retrieval systems. Our vision is to channel scarce attention resources toward meaningful signals in noisy, data-intensive workflows, enabling scalable, accurate, and sustainable LLM--data integration.

cs.SE↗

arXiv2025

From First Use to Final Commit: Studying the Evolution of Multi-CI Service Adoption

Continuous Integration (CI) services, such as GitHub Actions and Travis CI, are widely adopted in open-source development to automate testing and deployment. Though existing research often examines individual services in isolation, it remains unclear how projects adopt and transition between multiple services over time. To understand how CI adoption is evolving across services, we present a preliminary study analyzing the historical CI adoption of 18,924 Java projects hosted on GitHub between January 2008 and December 2024, adopting at least one of eight CI services, namely Travis CI, AppVeyor, CircleCI, Azure Pipelines, GitHub Actions, Bitbucket, GitLab CI, and Cirrus CI. Specifically, we investigate: (1) how frequently CI services are co-adopted or replaced, and (2) how maintenance activity varies across different services. Our analysis shows that the use of multiple CI services within the same project is a recurring pattern observed in nearly one in five projects, often reflecting migration across CI services. Our study is among the first to examine multi-CI adoption in practice, offering new insights for future research and highlighting the need for strategies and tools to support service selection, coordination, and migration in evolving CI environments.

cs.SE↗

arXiv2025

Can LLMs Write CI? A Study on Automatic Generation of GitHub Actions Configurations

Continuous Integration (CI) services, such as GitHub Actions, require developers to write YAML-based configurations, which can be tedious and error-prone. Despite the increasing use of Large Language Models (LLMs) to automate software engineering tasks, their ability to generate CI configurations remains underexplored. This paper presents a preliminary study evaluating six LLMs for generating GitHub Actions configurations from natural language descriptions. We assess three general-purpose foundation models (GPT-4o, Llama, and Gemma) and three code-pretrained models (GPT-4.1, Code Llama, and CodeGemma). We also introduce the first labeled dataset of its kind, constructed from GitHub Actions documentation, pairing descriptions with corresponding best-practice YAML configurations. Zero-shot prompting achieves up to 69% similarity with the ground truth, with only 3% perfect matches. Code-pretrained models slightly underperform compared to general-purpose ones in YAML-based CI tasks, revealing LLM limitations for CI configuration generation. Analyzing GPT-4o outputs reveals issues like missing or renamed steps, misinterpreted descriptions, and unnecessary additions that may affect structural and contextual correctness, indicating a gap between generation quality and the precision required for executable CI configurations. Our research offers insights for improving LLM alignment with configuration languages and guiding future efforts on CI automation and tooling support.

cs.SE↗

arXiv2025

CI/CD Configuration Practices in Open-Source Android Apps: An Empirical Study

Continuous Integration and Continuous Delivery (CI/CD) is a well-established practice that automatically builds, tests, packages, and deploys software systems. To adopt CI/CD, software developers need to configure their projects using dedicated YML configuration files. Mobile apps have distinct characteristics with respect to CI/CD practices, such as testing on various emulators and deploying to app stores. However, little is known about the challenges and added value of adopting CI/CD in mobile apps and how developers maintain such a practice. In this paper, we conduct an empirical study on CI/CD practices in 2,557 Android apps adopting four popular CI/CD services, namely GitHub Actions, Travis CI, CircleCI, and GitLab CI/CD. We also compare our findings with those reported in prior research on general CI/CD practices to situate them within broader trends. We observe a lack of commonality and standardization across CI/CD services and Android apps, leading to complex YML configurations and associated maintenance efforts. We also observe that CI/CD configurations focus primarily on the build setup, with around half of the projects performing standard testing and only 9% incorporating deployment. In addition, we find that CI/CD configurations are changed bi-monthly on average, with frequent maintenance correlating with active issue tracking, project size/age, and community engagement. Our qualitative analysis of commits uncovered 11 themes in CI/CD maintenance activities, with over a third of the changes focusing on improving workflows and fixing build issues, whereas another third involves updating the build environment, tools, and dependencies. Our study emphasizes the necessity for automation and AI-powered tools to improve CI/CD processes for mobile apps and advocates creating adaptable open-source tools to efficiently manage resources, especially in testing and deployment.

cs.SE↗

arXiv2024

LTM: Scalable and Black-box Similarity-based Test Suite Minimization based on Language Models

Test suites tend to grow when software evolves, making it often infeasible to execute all test cases with the allocated testing budgets, especially for large software systems. Test suite minimization (TSM) is employed to improve the efficiency of software testing by removing redundant test cases, thus reducing testing time and resources, while maintaining the fault detection capability of the test suite. Most existing TSM approaches rely on code coverage (white-box) or model-based features, which are not always available to test engineers. Recent TSM approaches that rely only on test code (black-box) have been proposed, such as ATM and FAST-R. To address the scalability, we propose LTM (Language model-based Test suite Minimization), a novel, scalable, and black-box similarity-based TSM approach based on large language models (LLMs), which is the first application of LLMs in the context of TSM. To support similarity measurement for test code embeddings, we investigate five pre-trained language models: CodeBERT, GraphCodeBERT, UniXcoder, StarEncoder, and CodeLlama, on which we compute two similarity measures: Cosine Similarity and Euclidean Distance. Our goal is to find similarity measures that are not only computationally more efficient but can also better guide a Genetic Algorithm (GA) to search for optimal minimized test suites, thus reducing the overall search time. Experimental results show that the best configuration of LTM (UniXcoder/Cosine) outperforms ATM in three aspects: (a) achieving a slightly greater saving rate of testing time (41.72% versus 41.02%, on average); (b) attaining a significantly higher fault detection rate (0.84 versus 0.81, on average); and, most importantly, (c) minimizing test suites nearly five times faster on average, with higher gains for larger test suites and systems, thus achieving much higher scalability.

cs.SE↗

arXiv2022

ATM: Black-box Test Case Minimization based on Test Code Similarity and Evolutionary Search

Executing large test suites is time and resource consuming, sometimes impossible, and such test suites typically contain many redundant test cases. Hence, test case minimization is used to remove redundant test cases that are unlikely to detect new faults. However, most test case (suite) minimization techniques rely on code coverage (white-box), model-based features, or requirements specifications, which are not always accessible by test engineers. Recently, a set of novel techniques was proposed, called FAST-R, relying solely on test case code for test case minimization, which appeared to be much more efficient than white-box techniques. However, it achieved a comparable low fault detection capability for Java projects, making its application challenging in practice. This paper proposes ATM (AST-based Test case Minimizer), a similarity-based, search-based test case minimization technique, taking a specific budget as input, that also relies exclusively on the source code of test cases but attempts to achieve higher fault detection through finer-grained similarity analysis and a dedicated search algorithm. ATM transforms test case code into Abstract Syntax Trees (AST) and relies on four tree-based similarity measures to apply evolutionary search, specifically genetic algorithms, to minimize test cases. We evaluated the effectiveness and efficiency of ATM on a large dataset of 16 Java projects with 661 faulty versions using three budgets ranging from 25% to 75% of test suites. ATM achieved significantly higher fault detection rates (0.82 on average), compared to FAST-R (0.61 on average) and random minimization (0.52 on average), when running only 50% of the test cases, within practically acceptable time (1.1-4.3 hours, on average), given that minimization is only occasionally applied when many new test cases are created (major releases). Results achieved for other budgets were consistent.

cs.SE↗

arXiv2022

Flakify: A Black-Box, Language Model-based Predictor for Flaky Tests

Software testing assures that code changes do not adversely affect existing functionality. However, a test case can be flaky, i.e., passing and failing across executions, even for the same version of the source code. Flaky test cases introduce overhead to software development as they can lead to unnecessary attempts to debug production or testing code. The state-of-the-art ML-based flaky test case predictors rely on pre-defined sets of features that are either project-specific, require access to production code, which is not always available to software test engineers. Therefore, in this paper, we propose Flakify, a black-box, language model-based predictor for flaky test cases. Flakify relies exclusively on the source code of test cases, thus not requiring to (a) access to production code (black-box), (b) rerun test cases, (c) pre-define features. To this end, we employed CodeBERT, a pre-trained language model, and fine-tuned it to predict flaky test cases using the source code of test cases. We evaluated Flakify on two publicly available datasets (FlakeFlagger and IDoFT) for flaky test cases and compared our technique with the FlakeFlagger approach using two different evaluation procedures: cross-validation and per-project validation. Flakify achieved high F1-scores on both datasets using cross-validation and per-project validation, and surpassed FlakeFlagger by 10 and 18 percentage points in terms of precision and recall, respectively, when evaluated on the FlakeFlagger dataset, thus reducing the cost bound to be wasted on unnecessarily debugging test cases and production code by the same percentages. Flakify also achieved significantly higher prediction results when used to predict test cases on new projects, suggesting better generalizability over FlakeFlagger. Our results further show that a black-box version of FlakeFlagger is not a viable option for predicting flaky test cases.

cs.SE↗

arXiv2021

Test Case Selection and Prioritization Using Machine Learning: A Systematic Literature Review

Regression testing is an essential activity to assure that software code changes do not adversely affect existing functionalities. With the wide adoption of Continuous Integration (CI) in software projects, which increases the frequency of running software builds, running all tests can be time-consuming and resource-intensive. To alleviate that problem, Test case Selection and Prioritization (TSP) techniques have been proposed to improve regression testing by selecting and prioritizing test cases in order to provide early feedback to developers. In recent years, researchers have relied on Machine Learning (ML) techniques to achieve effective TSP (ML-based TSP). Such techniques help combine information about test cases, from partial and imperfect sources, into accurate prediction models. This work conducts a systematic literature review focused on ML-based TSP techniques, aiming to perform an in-depth analysis of the state of the art, thus gaining insights regarding future avenues of research. To that end, we analyze 29 primary studies published from 2006 to 2020, which have been identified through a systematic and documented process. This paper addresses five research questions addressing variations in ML-based TSP techniques and feature sets for training and testing ML models, alternative metrics used for evaluating the techniques, the performance of techniques, and the reproducibility of the published studies. We summarize the results related to our research questions in a high-level summary that can be used as a taxonomy for classifying future TSP studies.

cs.SE↗