arXiv subjects

Sebastian Baltes

Publications and source records attributed to Sebastian Baltes.

At least 37 records · Page 2Linked to original sources

arXiv2026

Forecasting Developer Environments with GenAI: A Research Perspective

Generative Artificial Intelligence (GenAI) models are achieving remarkable performance in various tasks, including code generation, testing, code review, and program repair. The ability to increase the level of abstraction away from writing code has the potential to change the Human-AI interaction within the integrated development environment (IDE). To explore the impact of GenAI on IDEs, 33 experts from the Software Engineering, Artificial Intelligence, and Human-Computer Interaction domains gathered to discuss challenges and opportunities at Shonan Meeting 222, a four-day intensive research meeting. Four themes emerged as areas of interest for researchers and practitioners.

cs.SE↗

arXiv2026

Context Engineering for AI Agents in Open-Source Software

GenAI-based coding assistants have disrupted software development. The next generation of these tools is agent-based, operating with more autonomy and potentially without human oversight. Like human developers, AI agents require contextual information to develop solutions that are in line with the standards, policies, and workflows of the software projects they operate in. Vendors of popular agentic tools (e.g., Claude Code) recommend maintaining version-controlled Markdown files that describe aspects such as the project structure, code style, or building and testing. The content of these files is then automatically added to each prompt. Recently, AGENTS$.$md has emerged as a potential standard that consolidates existing tool-specific formats. However, little is known about whether and how developers adopt this format. Therefore, in this paper, we present the results of a preliminary study investigating the adoption of AI context files in 466 open-source software projects. We analyze the information that developers provide in AGENTS$.$md files, how they present that information, and how the files evolve over time. Our findings indicate that there is no established content structure yet and that there is a lot of variation in terms of how context is provided (descriptive, prescriptive, prohibitive, explanatory, conditional). Our commit-level analysis provides first insights into the evolution of the provided context. AI context files provide a unique opportunity to study real-world context engineering. In particular, we see great potential in studying which structural or presentational modifications can positively affect the quality of the generated content.

cs.SE↗

arXiv2026

Can We Classify Flaky Tests Using Only Test Code? An LLM-Based Empirical Study

Flaky tests yield inconsistent results when they are repeatedly executed on the same code revision. They interfere with automated quality assurance of code changes and hinder efficient software testing. Previous work evaluated approaches to train machine learning models to classify flaky tests based on identifiers in the test code. However, the resulting classifiers have been shown to lack generalizability, hindering their applicability in practical environments. Recently, pre-trained Large Language Models (LLMs) have shown the capability to generalize across various tasks. Thus, they represent a promising approach to address the generalizability problem of previous approaches. In this study, we evaluated three LLMs (two general-purpose models, one code-specific model) using three prompting techniques on two benchmark datasets from prior studies on flaky test classification. Furthermore, we manually investigated 50 samples from the given datasets to determine whether classifying flaky tests based only on test code is feasible for humans. Our findings indicate that LLMs struggle to classify flaky tests given only the test code. The results of our best prompt-model combination were only marginally better than random guessing. In our manual analysis, we found that the test code does not necessarily contain sufficient information for a flakiness classification. Our findings motivate future work to evaluate LLMs for flakiness classification with additional context, for example, using retrieval-augmented generation or agentic AI.

cs.SE↗

arXiv2026

Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA

Flaky tests yield different results when executed multiple times for the same version of the source code. Thus, they provide an ambiguous signal about the quality of the code and interfere with the automated assessment of code changes. While a variety of factors can cause test flakiness, approaches to fix flaky tests are typically tailored to address specific causes. However, the prevalent root causes of flaky tests can vary depending on the programming language, application domain, or size of the software project. Since manually labeling flaky tests is time-consuming and tedious, this work proposes an LLMs-as-annotators approach that leverages intra- and inter-model consistency to label issue reports related to fixed flakiness issues with the relevant root cause category. This allows us to gain an overview of prevalent flakiness categories in the issue reports. We evaluated our labeling approach in the context of SAP HANA, a large industrial database management system. Our results suggest that SAP HANA's tests most commonly suffer from issues related to concurrency (23%, 130 of 559 analyzed issue reports). Moreover, our results suggest that different test types face different flakiness challenges. Therefore, we encourage future research on flakiness mitigation to consider evaluating the generalizability of proposed approaches across different test types.

cs.SE↗

arXiv2026

How Does Cognitive Capability and Personality Influence Problem Solving in Coding Interview Puzzles?

Software engineering involves cognitively demanding activities impacted by individual differences. We investigate how cognitive capability and personality traits are associated with software problem solving accuracy. We assessed cognitive capability using Baddeley's three-minute grammatical reasoning test. Personality was measured using the IPIP-NEO-50 test. Eighty participants (40 software practitioners and 40 software engineering students) completed these two tests with nine interview-style problem solving tasks, comprising six coding-related and three logical-reasoning questions. Our practitioners achieved slightly higher grammatical reasoning accuracy than students, although this difference was not statistically significant. Students achieved higher accuracy on the coding and logical-reasoning tasks. For all, grammatical reasoning accuracy was positively correlated with problem solving accuracy, indicating that individuals with higher reasoning accuracy tended to perform better on applied problem solving tasks. Conscientiousness was the strongest personality-related association, positively correlated with both grammatical reasoning and problem solving accuracy. Openness to experience was also positively correlated with grammatical reasoning and problem solving accuracy. Neuroticism showed a small negative correlation with problem solving accuracy and weak negative correlation with grammatical reasoning accuracy. Practical implications for education and industry include integrating structured reasoning tasks in curricula, and considering the interplay of personality and cognition in recruitment and role allocation.

cs.SE↗

arXiv2026

On the Need to Rethink Trust in AI Assistants for Software Development: A Critical Review

Trust is a fundamental concept in human decision-making and collaboration that has long been studied in philosophy and psychology. However, software engineering (SE) articles often use the term trust informally; providing an explicit definition or embedding results in established trust models is rare. In SE research on AI assistants, this practice culminates in equating trust with the likelihood of accepting generated content, which, in isolation, does not capture the full conceptual complexity of trust. Without a common definition, true secondary research on trust is impossible. The objectives of our research were: (1) to present the psychological and philosophical foundations of human trust, (2) to systematically study how trust is conceptualized in SE and the related disciplines human-computer interaction and information systems, and (3) to discuss limitations of equating trust with content acceptance, outlining how SE research can adopt existing trust models to overcome the widespread informal use of the term trust. We conducted a literature review across disciplines and a critical review of recent SE articles with a focus on trust conceptualizations. We found that trust is rarely defined or conceptualized in SE articles. Related disciplines commonly embed their methodology and results in established trust models, clearly distinguishing, for example, between initial trust and trust formation and between appropriate and inappropriate trust. On a meta-scientific level, other disciplines even discuss whether and when trust can be applied to AI assistants at all. Our study reveals a significant maturity gap of trust research in SE compared to other disciplines. We provide concrete recommendations on how SE researchers can adopt established trust models and instruments to study trust in AI assistants beyond the acceptance of generated software artifacts.

cs.SE↗

arXiv2026

Political and Ideological Pressure in Software Engineering Research: The Case of DEI Backlash

Political and ideological pressures shape global research. Recently, these pressures have become particularly visible in research related to diversity, equity, and inclusion (DEI). Drastic changes in national funding and governmental guidance, especially in the US, have affected the global software engineering research ecosystem. The impacts of these pressures on research are not always direct, as they operate at multiple levels. However, what is clear is that these pressures affect every field, including software engineering (SE), despite the belief that our field is politically and ideologically neutral. In this position paper, we examine cases of political and ideological pressures on the SE research ecosystem. We investigate the community's perceptions of political and ideological pressures by analyzing community survey responses and outlining case examples of DEI backlash in SE research across three levels: macro, meso, and micro. Our research shows how recent political and ideological pressures have affected SE research across these levels, and, as a result, we propose actionable steps for the community to address these issues at different levels.

cs.SE↗

arXiv2026

Ethics of Care for Software Engineering

Software engineering researchers repeatedly argue that the impact of their research on industrial practice, while desired and intended, is rarely achieved. We believe that a possible explanation of this phenomenon is the opposition of "caring about" and "caring for", based on the ethics of care. Indeed, while software engineering is collaborative and hence builds on interpersonal relations, researchers tend to care about "industrial impact" and "practitioners" in abstract terms, but rarely care for specific individuals working in specific contexts facing specific challenges. In this position paper, we advocate for the adoption of ethics of care in software engineering and discuss the implications of this adoption for researchers and conference organizers.

cs.SE↗

arXiv2026

On the Flakiness of LLM-Generated Tests for Industrial and Open-Source Database Management Systems

Flaky tests are a common problem in software testing. They produce inconsistent results when executed multiple times on the same code, invalidating the assumption that a test failure indicates a software defect. Recent work on LLM-based test generation has identified flakiness as a potential problem with generated tests. However, its prevalence and underlying causes are unclear. We examined the flakiness of LLM-generated tests in the context of four relational database management systems: SAP HANA, DuckDB, MySQL, and SQLite. We amplified test suites with two LLMs, GPT-4o and Mistral-Large-Instruct-2407, to assess the flakiness of the generated test cases. Our results suggest that generated tests have a slightly higher proportion of flaky tests compared to existing tests. Based on a manual inspection, we found that the most common root cause of flakiness was the reliance of a test on a certain order that is not guaranteed ("unordered collection"), which was present in 72 of 115 flaky tests (63%). Furthermore, both LLMs transferred the flakiness from the existing tests to the newly generated tests via the provided prompt context. Our experiments suggest that flakiness transfer is more prevalent in closed-source systems such as SAP HANA than in open-source systems. Our study informs developers on what types of flakiness to expect from LLM-generated tests. It also highlights the importance of providing LLMs with tailored context when employing LLMs for test generation.

cs.SE↗

arXiv2025

Staying or Leaving? How Job Satisfaction, Embeddedness and Antecedents Predict Turnover Intentions of Software Professionals

Context: Voluntary turnover is common in the software industry, increasing recruitment and onboarding costs and the risk of losing organizational and tacit knowledge. Objective: This study investigates how job satisfaction, work-life balance, job embeddedness, and their antecedents, including job quality, personality traits, attitudes toward technical and sociotechnical infrastructure, and perceptions of organizational justice, relate to software professionals' turnover intentions. Method: We conducted a geographically diverse cross-sectional survey of software professionals (N = 224) and analyzed the data using partial least squares structural equation modeling (PLS-SEM). Our model includes both reflective and formative constructs and tests 15 hypotheses grounded in occupational psychology and software engineering literature. Results: Job satisfaction and embeddedness were significantly negatively associated with software professionals' turnover intentions, while work-life balance showed no direct effect. The strongest antecedents for job satisfaction were work-life balance and job quality, while organizational justice was the strongest predictor of job embeddedness. Discussion: The resulting PLS-SEM model has considerably higher explanatory power for key outcome variables than previous work conducted in the software development context, highlighting the importance of both psychological (e.g., job satisfaction, job embeddedness) and organizational (e.g., organizational justice, job quality) factors in understanding turnover intentions of software professionals. Our results imply that improving job satisfaction and job embeddedness is the key to retaining software professionals. In turn, enhancing job quality, supporting work-life balance, and ensuring high organizational justice can improve job satisfaction and embeddedness, indirectly reducing turnover intentions.

cs.SE↗

arXiv2025

The Software Infrastructure Attitude Scale (SIAS): A Questionnaire Instrument for Measuring Professionals' Attitudes Toward Technical and Sociotechnical Infrastructure

Context: Recent software engineering (SE) research has highlighted the need for sociotechnical research, implying a demand for customized psychometric scales. Objective: We define the concepts of technical and sociotechnical infrastructure in software engineering, and develop and validate a psychometric scale that measures attitudes toward them. Method: Grounded in theories of infrastructure, attitudes, and prior work on psychometric measurement, we defined the target constructs and generated scale items. The scale was administered to 225 software professionals and evaluated using a split sample. We conducted an exploratory factor analysis (EFA) on one half of the sample to uncover the underlying factor structure and performed a confirmatory factor analysis (CFA) on the other half to validate the structure. Further analyses with the whole sample assessed face, criterion-related, and discriminant validity. Results: EFA supported a two-factor structure (technical and sociotechnical infrastructure), accounting for 65% of the total variance with strong loadings. CFA confirmed excellent model fit. Face and content validity were supported by the item content reflecting cognitive, affective, and behavioral components. Both subscales were correlated with job satisfaction, perceived autonomy, and feedback from the job itself, supporting convergent validity. Regression analysis supported criterion-related validity, while the Heterotrait-Monotrait ratio of correlations (HTMT), the Fornell-Larcker criterion, and model comparison all supported discriminant validity. Discussion: The resulting scale is a valid instrument for measuring attitudes toward technical and sociotechnical infrastructure in software engineering research. Our work contributes to ongoing efforts to integrate psychological measurement rigor into empirical and behavioral software engineering research.

cs.SE↗

arXiv2025

A Multifaceted View on Discrimination in Software Development Careers

Conversations around diversity and inclusion in software engineering often focus on gender and racial disparities. However, the State of Devs 2025 survey with 8,717 participants revealed that other forms of discrimination are similarly prevalent but receive considerably less attention. This includes discrimination based on age, political perspective, disabilities, or cognitive differences such as neurodivergence. We conducted a secondary analysis of 800 open-ended survey responses to examine patterns of perceived discrimination, as well as related challenges and negative impacts. Our study covers multiple identity facets, including age, gender, race, and disability. We found that age- and gender-related discrimination was the most frequently reported workplace issue, but discrimination based on political and religious views emerged as further notable concerns. Most of the participants who identified as female cited gender as the primary source of discrimination, often accompanied by intersectional factors such as race, political views, age, or sexual orientation. Discrimination related to caregiving responsibilities was reported by all gender identities. Regarding the negative impacts of workplace issues, many participants described modifying their appearance or behavior in response to gender biases. Gender also appeared to influence broader career challenges, as women and non-binary respondents reported experiencing almost all workplace issues at higher rates, particularly discrimination (35%) and mental health challenges (62%). Our goal is to raise awareness in the research community that discrimination in software development is multifaceted, and to encourage researchers to select and assess relevant facets beyond age and gender when designing software engineering studies.

cs.SE↗

arXiv2025

Lost in Transition: The Struggle of Women Returning to Software Engineering Research after Career Breaks

The IT industry provides supportive pathways such as returnship programs, coding boot camps, and buddy systems for women re-entering their job after a career break. Academia, however, offers limited opportunities to motivate women to return. We propose a diverse multicultural research project investigating the challenges faced by women with software engineering (SE) backgrounds re-entering academia or related research roles after a career break. Career disruptions due to pregnancy, immigration status, or lack of flexible work options can significantly impact women's career progress, creating barriers for returning as lecturers, professors, or senior researchers. Although many companies promote gender diversity policies, such measures are less prominent and often under-recognized within academic institutions. Our goal is to explore the specific challenges women encounter when re-entering academic roles compared to industry roles; to understand the institutional perspective, including a comparative analysis of existing policies and opportunities in different countries for women to return to the field; and finally, to provide recommendations that support transparent hiring practices. The research project will be carried out in multiple universities and in multiple countries to capture the diverse challenges and policies that vary by location.

cs.SE↗

arXiv2025

Information-Theoretic Detection of Unusual Source Code Changes

The code base of software projects evolves essentially through inserting and removing information to and from the source code. We can measure this evolution via the elements of information - tokens, words, nodes - of the respective representation of the code. In this work, we approach the measurement of the information content of the source code of open-source projects from an information-theoretic standpoint. Our focus is on the entropy of two fundamental representations of code: tokens and abstract syntax tree nodes, from which we derive definitions of textual and structural entropy. We proceed with an empirical assessment where we evaluate the evolution patterns of the entropy of 95 actively maintained open source projects. We calculate the statistical relationships between our derived entropy metrics and classic methods of measuring code complexity and learn that entropy may capture different dimensions of complexity than classic metrics. Finally, we conduct entropy-based anomaly detection of unusual changes to demonstrate that our approach may effectively recognise unusual source code change events with over 60% precision, and lay the groundwork for improvements to information-theoretic measurement of source code evolution, thus paving the way for a new approach to statically gauging program complexity throughout its development.

cs.SE↗

arXiv2025

The Silent Scientist: When Software Research Fails to Reach Its Audience

If software research were a performance, it would be a thoughtful theater play -- full of rich content but confined to the traditional stage of academic publishing. Meanwhile, its potential audience is immersed in engaging on-demand experiences, leaving the theater half-empty, and the research findings lost in the wings. As long as this remains the case, discussions about research relevance and impact lack meaningful context.

cs.SE↗

arXiv2025

Making Software Development More Diverse and Inclusive: Key Themes, Challenges, and Future Directions

Introduction: Digital products increasingly reshape industries, influencing human behavior and decision-making. However, the software development teams developing these systems often lack diversity, which may lead to designs that overlook the needs, equal treatment or safety of diverse user groups. These risks highlight the need for fostering diversity and inclusion in software development to create safer, more equitable technology. Method: This research is based on insights from an academic meeting in June 2023 involving 23 software engineering researchers and practitioners. We used the collaborative discussion method 1-2-4-ALL as a systematic research approach and identified six themes around the theme challenges and opportunities to improve Software Developer Diversity and Inclusion (SDDI). We identified benefits, harms, and future research directions for the four main themes. Then, we discuss the remaining two themes, Artificial Intelligence & SDDI and AI & Computer Science education, which have a cross-cutting effect on the other themes. Results: This research explores the key challenges and research opportunities for promoting SDDI, providing a roadmap to guide both researchers and practitioners. We underline that research around SDDI requires a constant focus on maximizing benefits while minimizing harms, especially to vulnerable groups. As a research community, we must strike this balance in a responsible way.

cs.SE↗

arXiv2025

Towards Evaluation Guidelines for Empirical Studies involving LLMs

In the short period since the release of ChatGPT, large language models (LLMs) have changed the software engineering research landscape. While there are numerous opportunities to use LLMs for supporting research or software engineering tasks, solid science needs rigorous empirical evaluations. However, so far, there are no specific guidelines for conducting and assessing studies involving LLMs in software engineering research. Our focus is on empirical studies that either use LLMs as part of the research process or studies that evaluate existing or new tools that are based on LLMs. This paper contributes the first set of holistic guidelines for such studies. Our goal is to start a discussion in the software engineering research community to reach a common understanding of our standards for high-quality empirical studies involving LLMs.

cs.SE↗

arXiv2025

Not real or too soft? On the challenges of publishing interdisciplinary software engineering research

The discipline of software engineering (SE) combines social and technological dimensions. It is an interdisciplinary research field. However, interdisciplinary research submitted to software engineering venues may not receive the same level of recognition as more traditional or technical topics such as software testing. For this paper, we conducted an online survey of 73 SE researchers and used a mixed-method data analysis approach to investigate their challenges and recommendations when publishing interdisciplinary research in SE. We found that the challenges of publishing interdisciplinary research in SE can be divided into topic-related and reviewing-related challenges. Furthermore, while our initial focus was on publishing interdisciplinary research, the impact of current reviewing practices on marginalized groups emerged from our data, as we found that marginalized groups are more likely to receive negative feedback. In addition, we found that experienced researchers are less likely to change their research direction due to feedback they receive. To address the identified challenges, our participants emphasize the importance of highlighting the impact and value of interdisciplinary work for SE, collaborating with experienced researchers, and establishing clearer submission guidelines and new interdisciplinary SE publication venues. Our findings contribute to the understanding of the current state of the SE research community and how we could better support interdisciplinary research in our field.

cs.SE↗