arXiv subjects

Nasir Eisty

Publications and source records attributed to Nasir Eisty.

7 recordsLinked to original sources

arXiv2026

How Researchers Use and Verify AI Coding Assistants: Tasks and Validation Practices in Scientific Programming

Generative AI has entered research programming, yet there is little evidence about which tasks researchers hand to it or how they decide whether its code is correct. We draw on 527 free-text responses to a 2025 survey of researchers who write code, most of them at U.S. universities. In each response, a researcher recounts a single task from their own work, the way they used an AI tool for it, and what they did to assess the result. We coded the task and the evaluation strategies reported, and related both to programming experience, research area, and confidence ratings. Use was concentrated in five tasks: data handling, visualization, debugging, mathematical/scientific computing, and statistical analysis. Evaluation was informal and individual. Over half of accounts described running the generated code, while automated tests and review by another person were rare. Use cases and evaluation strategies varied little with programming experience, but confidence did: Less experienced programmers trusted the AI more than themselves, and experienced programmers the reverse. Evaluation confidence was not associated with the strategies reported. Its strongest correlates were confidence in the tool and in oneself. Validating AI contributions to scientific code rested largely on individual judgment, outside shared infrastructure for testing or review. Interfaces could support task-appropriate evaluation rather than leave it to the user.

cs.SE↗

arXiv2026

A survey of generative AI adoption and perceived productivity among scientists who program

Programming is essential to modern scientific research, yet most scientists report inadequate training for the software development their work demands. Generative AI tools capable of code generation may support scientific programmers, but user studies indicate risks of over-reliance, particularly among inexperienced users. We surveyed 868 scientists who program, examining adoption patterns, tool preferences, and factors associated with perceived productivity. Adoption is highest among students and less experienced programmers, with variation across fields. Scientific programmers overwhelmingly prefer general-purpose conversational interfaces like ChatGPT over developer-specific tools. Both inexperience and limited use of development practices (like testing, code review, and version control) are associated with greater perceived productivity -- but these factors interact, suggesting formal practices may partially compensate for inexperience. The strongest predictor of perceived productivity is the number of lines of generated code typically accepted at once. These findings suggest scientific programmers using generative AI may gauge productivity by code generation rather than validation.

cs.SE↗

arXiv2025

Beyond Pass/Fail: The Story of Learning-Based Testing

Learning-Based Testing (LBT) merges learning and testing processes to achieve both testing and behavioral adequacy. LBT utilizes active learning to infer the model of the System Under Test (SUT), enabling scalability for large and complex programs by requiring only a minimal set of initial test cases. The core principle of LBT is that the SUT's behavior can be thoroughly inferred by progressively generating test cases and subjecting the SUT to testing, thereby ensuring comprehensive testing. Despite being in its early stages, LBT has a solid foundation of theoretical research demonstrating its efficacy in testing both procedural and reactive programs. This paper provides a systematic literature review of various LBT implementations across different program types and evaluates the current state of research in this field. We explore diverse theoretical frameworks, existing tools, and libraries within the LBT domain to illustrate the concept's evolution and current research status. Additionally, we examine case studies involving the application of LBT tools in industrial settings, highlighting their potential and effectiveness in commercial software testing. This systematic literature review aims to offer researchers a comprehensive perspective on the inception and development of LBT, presenting it as a promising technique in software testing. By unveiling LBT's underutilized potential, this paper seeks to significantly benefit the practitioners and research community.

cs.SE↗

arXiv2025

Deciphering WONTFIX: A Mixed-Method Study on Why GitHub Issues Get Rejected

Context: The ``wontfix'' label is a widely used yet narrowly understood tool in GitHub repositories, indicating that an issue will not be pursued further. Despite its prevalence, the impact of this label on project management and community dynamics within open-source software development is not clearly defined. Objective: This study examines the prevalence and reasons behind issues being labeled as wontfix across various open-source repositories on GitHub. Method: Employing a mixed-method approach, we analyze both quantitative data to assess the prevalence of the wontfix label and qualitative data to explore the reasoning that it was used. Data were collected from 3,132 of GitHub's most-popular repositories. Later, we employ open coding and thematic analysis to categorize the reasons behind wontfix labels, providing a structured understanding of the issue management landscape. Results: Our findings show that about 30% of projects on GitHub apply the wontfix label to some issues. These issues most often occur on user-submitted issues for bug reports and feature requests. The study identified eight common themes behind labeling issues as wontfix, ranging from user-specific control factors to maintainer-specific decisions. Conclusions: The wontfix label is a critical tool for managing resources and guiding contributor efforts in GitHub projects. However, it can also discourage community involvement and obscure the transparency of project management. Understanding these reasons aids project managers in making informed decisions and fostering efficient collaboration within open-source communities.

cs.SE↗

arXiv2025

Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritization

Context: Deep Neural Networks (DNNs) are increasingly deployed in critical applications, where resilience against adversarial inputs is paramount. However, whether coverage-based or confidence-based, existing test prioritization methods often fail to efficiently identify the most fault-revealing inputs, limiting their practical effectiveness. Aims: This project aims to enhance fault detection and model robustness in DNNs by integrating Learning-Based Testing (LBT) with hypothesis and mutation testing to efficiently prioritize adversarial test cases. Methods: Our method selects a subset of adversarial inputs with a high likelihood of exposing model faults, without relying on architecture-specific characteristics or formal verification, making it adaptable across diverse DNNs. Results: Our results demonstrate that the proposed LBT method consistently surpasses baseline approaches in prioritizing fault-revealing inputs and accelerating fault detection. By efficiently organizing test permutations, it uncovers all potential faults significantly faster across various datasets, model architectures, and adversarial attack techniques. Conclusion: Beyond improving fault detection, our method preserves input diversity and provides effective guidance for model retraining, further enhancing robustness. These advantages establish our approach as a powerful and practical solution for adversarial test prioritization in real-world DNN applications.

cs.SE↗

arXiv2023

Evaluating Code Metrics in GitHub Repositories Related to Fake News and Misinformation

The surge of research on fake news and misinformation in the aftermath of the 2016 election has led to a significant increase in publicly available source code repositories. Our study aims to systematically analyze and evaluate the most relevant repositories and their Python source code in this area to improve awareness, quality, and understanding of these resources within the research community. Additionally, our work aims to measure the quality and complexity metrics of these repositories and identify their fundamental features to aid researchers in advancing the fields knowledge in understanding and preventing the spread of misinformation on social media. As a result, we found that more popular fake news repositories and associated papers with higher citation counts tend to have more maintainable code measures, more complex code paths, a larger number of lines of code, a higher Halstead effort, and fewer comments.

cs.SE↗

arXiv2023

Applications of Causality and Causal Inference in Software Engineering

Causal inference is a study of causal relationships between events and the statistical study of inferring these relationships through interventions and other statistical techniques. Causal reasoning is any line of work toward determining causal relationships, including causal inference. This paper explores the relationship between causal reasoning and various fields of software engineering. This paper aims to uncover which software engineering fields are currently benefiting from the study of causal inference and causal reasoning, as well as which aspects of various problems are best addressed using this methodology. With this information, this paper also aims to find future subjects and fields that would benefit from this form of reasoning and to provide that information to future researchers. This paper follows a systematic literature review, including; the formulation of a search query, inclusion and exclusion criteria of the search results, clarifying questions answered by the found literature, and synthesizing the results from the literature review. Through close examination of the 45 found papers relevant to the research questions, it was revealed that the majority of causal reasoning as related to software engineering is related to testing through root cause localization. Furthermore, most causal reasoning is done informally through an exploratory process of forming a Causality Graph as opposed to strict statistical analysis or introduction of interventions. Finally, causal reasoning is also used as a justification for many tools intended to make the software more human-readable by providing additional causal information to logging processes or modeling languages.

cs.SE↗