arXiv subjects

Raula Gaikovina Kula

Publications and source records attributed to Raula Gaikovina Kula.

At least 37 records · Page 2Linked to original sources

arXiv2025

How Natural Language Proficiency Shapes GenAI Code for Software Engineering Tasks

With the widespread adoption of Foundation Model (FM)-powered tools in software engineering, the natural language prompt has become a critical interface between developers and Large Language Models (LLMs). While much research has focused on prompt structure, the natural language proficiency is an underexplored factor that can influence the quality of generated code. This paper investigates whether the English language proficiency itself independent of the prompting technique affects the proficiency and correctness of code generated by LLMs. Using the HumanEval dataset, we systematically varied the English proficiency of prompts from basic to advanced for 164 programming tasks and measured the resulting code proficiency and correctness. Our findings show that LLMs default to an intermediate (B2) natural language level. While the effect on the resulting code proficiency was model-dependent, we found that higher-proficiency prompts consistently yielded more correct code across all models. These results demonstrate that natural language proficiency is a key lever for controlling code generation, helping developers tailor AI output and improve the reliability of solutions.

cs.SE↗

arXiv2025

Human to Document, AI to Code: Comparing GenAI for Notebook Competitions

Computational notebooks have become the preferred tool of choice for data scientists and practitioners to perform analyses and share results. Notebooks uniquely combine scripts with documentation. With the emergence of generative AI (GenAI) technologies, it is increasingly important, especially in competitive settings, to distinguish the characteristics of human-written versus GenAI. In this study, we present three case studies to explore potential strengths of both humans and GenAI through the coding and documenting activities in notebooks. We first characterize differences between 25 code and documentation features in human-written, medal-winning Kaggle notebooks. We find that gold medalists are primarily distinguished by longer and more detailed documentation. Second, we analyze the distinctions between human-written and GenAI notebooks. Our results show that while GenAI notebooks tend to achieve higher code quality (as measured by metrics like code smells and technical debt), human-written notebooks display greater structural diversity, complexity, and innovative approaches to problem-solving. Based on these results, we envision the work as groundwork that highlight four agendas to further investigate how GenAI could be utilized in notebooks that maximizes the potential collaboration between human and AI.

cs.SE↗

arXiv2025

An Empirical Study of Security-Policy Related Issues in Open Source Projects

GitHub recommends that projects adopt a security file that outlines vulnerability reporting procedures. However, the effectiveness and operational challenges of such files are not yet fully understood. This study aims to clarify the challenges that security files face in the vulnerability reporting process within open-source communities. Specifically, we classified and analyzed the content of 711 randomly sampled issues related to security files. We also conducted a quantitative comparative analysis of the close time and number of responses for issues concerning six community health files, including security files. Our analysis revealed that 79.5% of security file-related issues were requests to add the file, and reports that included links were closed, with a median time that was 2 days shorter. These findings offer practical insights for improving security reporting policies and community management, ultimately contributing to a more secure open-source ecosystem.

cs.SE↗

arXiv2025

Towards Sustainable and Secure Reuse in Dependency Supply Chains: Initial Analysis of NPM packages at the End of the Chain

Much of the success of modern software development can be attributed to code reuse. The ability to reuse existing functionality via third-party dependencies has enabled massive gains in productivity, but for a long time the dominant philosophy has been to 'reuse as much as possible, without thought for what is being depended upon', creating fragile dependency chains. Heavy reliance has raised resiliency and maintenance concerns. In this vision paper, we investigate packages that challenge the typical concepts of reuse - that is, packages with no dependencies themselves that bear the responsibility of being at the end of the dependency supply chain. By avoiding dependencies, these packages at the end of the chain may also avoid the associated risks. Our initial analysis of the most depended upon NPM packages shows that such end-of-chain packages make up a significant portion of these critical dependency chain (over 50%). We find that these end-of-chain packages vary in characteristics and are not just packages that can be easily replaced, and present five cases. We then ask ourselves: Should maintainers minimize external dependencies? We argue that these packages reveal important lessons for strategic reuse-balancing the undeniable benefits of dependency ecosystems with sustainable, secure practices.

cs.SE↗

arXiv2025

Uncovering Intention through LLM-Driven Code Snippet Description Generation

Documenting code snippets is essential to pinpoint key areas where both developers and users should pay attention. Examples include usage examples and other Application Programming Interfaces (APIs), which are especially important for third-party libraries. With the rise of Large Language Models (LLMs), the key goal is to investigate the kinds of description developers commonly use and evaluate how well an LLM, in this case Llama, can support description generation. We use NPM Code Snippets, consisting of 185,412 packages with 1,024,579 code snippets. From there, we use 400 code snippets (and their descriptions) as samples. First, our manual classification found that the majority of original descriptions (55.5%) highlight example-based usage. This finding emphasizes the importance of clear documentation, as some descriptions lacked sufficient detail to convey intent. Second, the LLM correctly identified the majority of original descriptions as "Example" (79.75%), which is identical to our manual finding, showing a propensity for generalization. Third, compared to the originals, the produced description had an average similarity score of 0.7173, suggesting relevance but room for improvement. Scores below 0.9 indicate some irrelevance. Our results show that depending on the task of the code snippet, the intention of the document may differ from being instructions for usage, installations, or descriptive learning examples for any user of a library.

cs.SE↗

arXiv2025

Using LLMs for Security Advisory Investigations: How Far Are We?

Large Language Models (LLMs) are increasingly used in software security, but their trustworthiness in generating accurate vulnerability advisories remains uncertain. This study investigates the ability of ChatGPT to (1) generate plausible security advisories from CVE-IDs, (2) differentiate real from fake CVE-IDs, and (3) extract CVE-IDs from advisory descriptions. Using a curated dataset of 100 real and 100 fake CVE-IDs, we manually analyzed the credibility and consistency of the model's outputs. The results show that ChatGPT generated plausible security advisories for 96% of given input real CVE-IDs and 97% of given input fake CVE-IDs, demonstrating a limitation in differentiating between real and fake IDs. Furthermore, when these generated advisories were reintroduced to ChatGPT to identify their original CVE-ID, the model produced a fake CVE-ID in 6% of cases from real advisories. These findings highlight both the strengths and limitations of ChatGPT in cybersecurity applications. While the model demonstrates potential for automating advisory generation, its inability to reliably authenticate CVE-IDs or maintain consistency upon re-evaluation underscores the risks associated with its deployment in critical security tasks. Our study emphasizes the importance of using LLMs with caution in cybersecurity workflows and suggests the need for further improvements in their design to improve reliability and applicability in security advisory generation.

cs.CR↗

arXiv2025

Open Source at a Crossroads: The Future of Licensing Driven by Monetization

The widespread adoption of open source libraries and frameworks can be attributed to their licensing. Open Source Software Licenses (OSS licenses) ensure that software can be sold or distributed as part of aggregate programs from various sources without requiring a royalty or fee. The quality of such code rivals that of commercial software, with open source libraries forming large parts of the supply chain for critical commercial systems in industry. Despite this, most open source projects rely on volunteer contributions, and unpaid library maintainers face significant pressure to sustain their projects. One potential solution for these projects is to change their licensing to ensure that maintainers are compensated accordingly for their work. In this paper, we explore the potential of licensing to help alleviate funding issues, with a review of three different cases where OSS licenses were modified to allow for monetization. In addition, we explore licensing concerns related to the emergence of the use of artificial intelligence (AI) in software development. We argue that open source is at a crossroads, with a growing need to redefine its licensing models and support communities and critical software. We identify specific research opportunities and conclude with a research agenda comprising a series of research questions to guide future studies in this area.

cs.SE↗

arXiv2025

Mining the Characteristics of Jupyter Notebooks in Data Science Projects

Nowadays, numerous industries have exceptional demand for skills in data science, such as data analysis, data mining, and machine learning. The computational notebook (e.g., Jupyter Notebook) is a well-known data science tool adopted in practice. Kaggle and GitHub are two platforms where data science communities are used for knowledge-sharing, skill-practicing, and collaboration. While tutorials and guidelines for novice data science are available on both platforms, there is a low number of Jupyter Notebooks that received high numbers of votes from the community. The high-voted notebook is considered well-documented, easy to understand, and applies the best data science and software engineering practices. In this research, we aim to understand the characteristics of high-voted Jupyter Notebooks on Kaggle and the popular Jupyter Notebooks for data science projects on GitHub. We plan to mine and analyse the Jupyter Notebooks on both platforms. We will perform exploratory analytics, data visualization, and feature importances to understand the overall structure of these notebooks and to identify common patterns and best-practice features separating the low-voted and high-voted notebooks. Upon the completion of this research, the discovered insights can be applied as training guidelines for aspiring data scientists and machine learning practitioners looking to improve their performance from novice ranking Jupyter Notebook on Kaggle to a deployable project on GitHub.

cs.SE↗

arXiv2025

Mining for Lags in Updating Critical Security Threats: A Case Study of Log4j Library

The Log4j-Core vulnerability, known as Log4Shell, exposed significant challenges to dependency management in software ecosystems. When a critical vulnerability is disclosed, it is imperative that dependent packages quickly adopt patched versions to mitigate risks. However, delays in applying these updates can leave client systems exposed to exploitation. Previous research has primarily focused on NPM, but there is a need for similar analysis in other ecosystems, such as Maven. Leveraging the 2025 mining challenge dataset of Java dependencies, we identify factors influencing update lags and categorize them based on version classification (major, minor, patch release cycles). Results show that lags exist, but projects with higher release cycle rates tend to address severe security issues more swiftly. In addition, over half of vulnerability fixes are implemented through patch updates, highlighting the critical role of incremental changes in maintaining software security. Our findings confirm that these lags also appear in the Maven ecosystem, even when migrating away from severe threats.

cs.SE↗

arXiv2025

Do Developers Depend on Deprecated Library Versions? A Mining Study of Log4j

Log4j has become a widely adopted logging library for Java programs due to its long history and high reliability. Its widespread use is notable not only because of its maturity but also due to the complexity and depth of its features, which have made it an essential tool for many developers. However, Log4j 1.x, which reached its end of support (deprecated), poses significant security risks and has numerous deprecated features that can be exploited by attackers. Despite this, some clients may still rely on this library. We aim to understand whether clients are still using Log4j 1.x despite its official support ending. We utilized the Mining Software Repositories 2025 challenge dataset, which provides a large and representative sample of open-source software projects. We analyzed over 10,000 log entries from the Mining Software Repositories 2025 challenge dataset using the Goblin framework to identify trends in usage rates for both Log4j 1.x and Log4j-core 2.x. Specifically, our study addressed two key issues: (1) We examined the usage rates and trends for these two libraries, highlighting any notable differences or patterns in their adoption. (2) We demonstrate that projects initiated after a deprecated library has reached the end of its support lifecycle can still maintain significant popularity. These findings highlight how deprecated are still popular, with the next step being to understand the reasoning behind these adoptions.

cs.SE↗

arXiv2025

The Shift from Writing to Pruning Software: A Bonsai-Inspired IDE for Reshaping AI Generated Code

The rise of AI-driven coding assistants signals a fundamental shift in how software is built. While AI coding assistants have been integrated into existing Integrated Development Environments (IDEs), their full potential remains largely untapped. A key challenge is that these AI assistants can suffer from hallucinations, leading developers down decision paths that the AI should not dictate, sometimes even without the users awareness or consent. Moreover, current static-file IDEs lack the mechanisms to address critical issues such as tracking the provenance of AI-generated code and integrating version control in a way that aligns with the dynamic nature of AI-assisted development. As a result, developers are left without the necessary tools to manage, refine, and validate AI generated code systematically, making it difficult to ensure correctness, maintainability, and trust in the development process. Existing IDEs treat AI-generated code as static text, offering limited support for managing its evolution, refinement, or multiple alternative paths. Drawing inspiration from the ancient art of Japanese Bonsai gardening focused on balance, structure, and deliberate pruning: we propose a new approach to IDEs, where AI is allowed to generate in its true, unconstrained form, free from traditional file structures. This approach fosters a more fluid and interactive method for code evolution. We introduce the concept of a Bonsai-inspired IDE, structured as a graph of generated code snippets and multiple code paths, enabling developers to reshape AI generated code to suit their needs. Our vision calls for a shift away from a static file based model toward a dynamic, evolving system that allows for continuous refinement of generated code, with the IDE evolving alongside AI powered modifications rather than merely serving as a place to write and edit code.

cs.SE↗

arXiv2025

Interacting with AI Reasoning Models: Harnessing "Thoughts" for AI-Driven Software Engineering

Recent advances in AI reasoning models provide unprecedented transparency into their decision-making processes, transforming them from traditional black-box systems into models that articulate step-by-step chains of thought rather than producing opaque outputs. This shift has the potential to improve software quality, explainability, and trust in AI-augmented development. However, software engineers rarely have the time or cognitive bandwidth to analyze, verify, and interpret every AI-generated thought in detail. Without an effective interface, this transparency could become a burden rather than a benefit. In this paper, we propose a vision for structuring the interaction between AI reasoning models and software engineers to maximize trust, efficiency, and decision-making power. We argue that simply exposing AI's reasoning is not enough -- software engineers need tools and frameworks that selectively highlight critical insights, filter out noise, and facilitate rapid validation of key assumptions. To illustrate this challenge, we present motivating examples in which AI reasoning models state their assumptions when deciding which external library to use and produce divergent reasoning paths and recommendations about security vulnerabilities, highlighting the need for an interface that prioritizes actionable insights while managing uncertainty and resolving conflicts. We then outline a research roadmap for integrating automated summarization, assumption validation, and multi-model conflict resolution into software engineering workflows. Achieving this vision will unlock the full potential of AI reasoning models to enable software engineers to make faster, more informed decisions without being overwhelmed by unnecessary detail.

cs.SE↗

arXiv2025

On Categorizing Open Source Software Security Vulnerability Reporting Mechanisms on GitHub

Open-source projects are essential to software development, but publicly disclosing vulnerabilities without fixes increases the risk of exploitation. The Open Source Security Foundation (OpenSSF) addresses this issue by promoting robust security policies to enhance project security. Current research reveals that many projects perform poorly on OpenSSF criteria, indicating a need for stronger security practices and underscoring the value of SECURITY$.$md files for structured vulnerability reporting. This study aims to provide recommendations for improving security policies. By examining 679 open-source projects, we find that email is still the main source of reporting. Furthermore, we find that projects without SECURITY$.$md files tend to be less secure (lower OpenSSF scores). Our analysis also indicates that, although many maintainers encourage private reporting methods, some contributors continue to disclose vulnerabilities publicly, bypassing established protocols. The results from this preliminary study pave the way for understanding how developers react and communicate a potential security threat. Future challenges include understanding the impact and effectiveness of these mechanisms and what factors may influence how the security threat is addressed.

cs.SE↗

arXiv2025

Reducing Alert Fatigue via AI-Assisted Negotiation: A Case for Dependabot

The increasing complexity of software dependencies has led to the emergence of automated dependency management tools, such as Dependabot. However, these tools often overwhelm developers with a high volume of alerts and notifications, leading to alert fatigue. This paper presents a position on using Artificial Intelligence (AI) agents as dependency negotiators to reduce alert fatigue. We then examine specific use cases where AI agents can facilitate dependency negotiations, such as when working with external dependencies or managing complex, multi-component systems. Our findings highlight the need for more research on the design and evaluation of AI-driven dependency mediation mechanisms. With a focus on ensuring transparency, explainability, and human trustworthiness in these GitHub software projects, our goal is to reduce alert fatigue to an extent that maintainers no longer feel overwhelmed and welcome pull requests just like any other contribution into their projects.

cs.SE↗

arXiv2025

Building Bridges across Papua New Guinea's Digital Divide in Growing the ICT Industry

Papua New Guinea (PNG) is an emerging tech society with an opportunity to overcome geographic and social boundaries, in order to engage with the global market. However, the current tech landscape, dominated by Big Tech in Silicon Valley and other multinational companies in the Global North, tends to overlook the requirements of emerging economies such as PNG. This is becoming more obvious as issues such as algorithmic bias (in tech product deployments) and the digital divide (as in the case of non-affordable commercial software) are affecting PNG users. The Open Source Software (OSS) movement, based on extant research, is seen as a way to level the playing field in the digitalization and adoption of Information and Communications Technologies (ICTs) in PNG. This perspectives paper documents the outcome of the second International Workshop on BRIdging the Divides with Globally Engineered Software} (BRIDGES2023) in the hopes of proposing ideas for future research into ICT education, uplifting software engineering (SE) capability, and OSS adoption in promoting a more equitable digital future for PNG.

cs.CY↗

arXiv2025

Nigerian Software Engineer or American Data Scientist? GitHub Profile Recruitment Bias in Large Language Models

Large Language Models (LLMs) have taken the world by storm, demonstrating their ability not only to automate tedious tasks, but also to show some degree of proficiency in completing software engineering tasks. A key concern with LLMs is their "black-box" nature, which obscures their internal workings and could lead to societal biases in their outputs. In the software engineering context, in this early results paper, we empirically explore how well LLMs can automate recruitment tasks for a geographically diverse software team. We use OpenAI's ChatGPT to conduct an initial set of experiments using GitHub User Profiles from four regions to recruit a six-person software development team, analyzing a total of 3,657 profiles over a five-year period (2019-2023). Results indicate that ChatGPT shows preference for some regions over others, even when swapping the location strings of two profiles (counterfactuals). Furthermore, ChatGPT was more likely to assign certain developer roles to users from a specific country, revealing an implicit bias. Overall, this study reveals insights into the inner workings of LLMs and has implications for mitigating such societal biases in these models.

cs.SE↗

arXiv2024

Developer Reactions to Protestware in Open Source Software: The cases of color.js and es5.ext

There is growing concern about maintainers self-sabotaging their work in order to take political or economic stances, a practice referred to as "protestware". Our objective is to understand the discourse around discussions on such an attack, how it is received by the community, and whether developers respond to the attack in a timely manner. We study two notable protestware cases i.e., colors.js and es5-ext. Results indicate that protestware discussions are spread more quickly on the GitHub platform, while security vulnerabilities are faster on social media. By establishing a taxonomy of protestware discussions, we identify posts that express stances and provide technical mitigation instructions. We applied a thematic analysis to 684 protestware related posts to identify five major themes during the discussions: i. disseminate and response, ii. stance, iii. reputation, iv. communicative styles, v. rights and ethics. This work sheds light on the nuanced landscape of protestware discussions, offering insights for both researchers and developers into maintaining a healthy balance between the political or social actions of developers and the collective well-being of the open-source community.

cs.SE↗

arXiv2024

How Maintainable is Proficient Code? A Case Study of Three PyPI Libraries

Python is very popular because it can be used for a wider audience of developers, data scientists, machine learning experts and so on. Like other programming languages, there are beginner to advanced levels of writing Python code. However, like all software, code constantly needs to be maintained as bugs and the need for new features emerge. Although the Zen of Python states that "Simple is better than complex," we hypothesize that more elegant and proficient code might be harder for the developer to maintain. To study this relationship between the understanding of code maintainability and code proficiency, we present an exploratory study into the complexity of Python code on three Python libraries. Specifically, we investigate the risk level of proficient code inside a file. As a starting point, we mined and collected the proficiency of code from three PyPI libraries totaling 3,003 files. We identified several instances of high proficient code that was also high risk, with examples being simple list comprehensions, 'enumerate' calls, generator expressions, simple dictionary comprehensions, and the 'super' function. Our early examples revealed that most code-proficient development presented a low maintainability risk, yet there are some cases where proficient code is also risky to maintenance. We envision that the study should help developers identify scenarios where and when using proficient code might be detrimental to future code maintenance activities.

cs.SE↗