arXiv subjects

Nasir U. Eisty

Publications and source records attributed to Nasir U. Eisty.

At least 19 recordsLinked to original sources

arXiv2026

Uncovering Scientific Software Sustainability through Community Engagement and Software Quality Metrics

Scientific open-source software (Sci-OSS) projects are critical for advancing research, yet sustaining these projects long-term remains a major challenge. This paper explores the sustainability of Sci-OSS hosted on GitHub, focusing on two factors drawn from stewardship organizations: community engagement and software quality. We map sustainability to repository metrics from the literature and mined data from ten prominent Sci-OSS projects. A multimodal analysis of these projects led us to a novel visualization technique, providing a robust way to display both current and evolving software metrics over time, replacing multiple traditional visualizations with one. Additionally, our statistical analysis shows that even similar-domain projects sustain themselves differently. Natural language analysis supports claims from the literature, highlighting that project-specific feedback plays a key role in maintaining software quality. Our visualization and analysis methods offer researchers, funders, and developers key insights into long-term software sustainability.

cs.SE↗

arXiv2026

From Business Requirements to Test Assertions: Evaluating LLM-Generated Oracles on Real Bugs

The oracle problem (determining the correct expected outcome for a test) remains a major bottleneck in automated testing, and is increasingly relevant as non-experts rely on AI-generated code they cannot reliably validate. We study whether large language models (LLMs) can generate generalizable test oracles directly from natural-language business requirements, without access to source code or example input-output pairs. We propose a reproducible, requirement-driven pipeline grounded in Defects4J. For each of 10 real bugs from Defects4J Lang (Bugs 1 and 3-11), we (i) extract behavioral changes via buggy/fixed diffs, (ii) manually translate the change into a business requirement, (iii) construct a requirement-derived oracle (REQ) as a gold standard, and (iv) prompt five LLMs (DeepSeek-V3, Gemma-3n, Llama-3, Mistral-7B, and Qwen-3) to generate Java oracle code. We evaluate oracle correctness and generalization under two targets: agreement with REQ and agreement with the system under test (SUT), reporting macro-averaged accuracy, precision, recall, and F1. LLMs achieve non-trivial generalization but with substantial bug- and model-level variance. Generated oracles align more closely with REQ than with SUT, and correlations between requirement technicality/ambiguity ratings and oracle accuracy are weak with wide confidence intervals. No detectable linear relationship exists between requirement properties and oracle accuracy in this dataset, suggesting that pretraining coverage and the semantic specificity of the required behavior dominate oracle correctness. As a pilot proof of concept, these findings are preliminary and are intended to establish feasibility and motivate larger-scale empirical investigation.

cs.SE↗

arXiv2026

Self-Admitted Technical Debt in Scientific Software: Prioritization, Sentiment, and Propagation Across Artifacts

Self-admitted technical debt (SATD) impairs scientific software (SSW), yet its prioritization, sentiment, persistence, and propagation remains underexplored. Understanding how SSW developers express, and address SATD is crucial for improving SSW maintenance, and tooling. This study investigates how SATD types and artifacts in SSW are prioritized, how sentiment relates to urgency, SATD removal and resolution rates, and the extent to which SATD propagates across artifacts. We analyzed nine SSW repositories using a SATD classification model and a semantic embedding-based prioritization heuristic. SATD was examined across multiple artifacts, with sentiment assessed via a fine-tuned transformer. Propagation was traced, priority scores compared to static analysis, and removal and resolution rates quantified. SATD in comments, commits, and pull requests receive higher priority than SATD in issues, with negative sentiment amplifying urgency. Resolution and removal rates lag behind open-source software (OSS) averages. Most SATD remains confined to the originating artifact, but longer propagation chains are rare and correlate with higher priority, highlighting persistent and high impact debt. Prioritization is influenced by artifact type and sentiment, while low removal and resolution rates signal persistent debt. Cross-artifact propagation marks high priority, unresolved SATD, providing empirical guidance for targeted monitoring, review prioritization, and tool supported maintenance in SSW.

cs.SE↗

arXiv2026

Thinking Out Loud: Real-Time Deception Monitoring in Asymmetric LLM Negotiations

As LLM-based agents are increasingly deployed to negotiate, delegate, or transact on a user's behalf, software pipelines need runtime mechanisms to verify that an agent's stated intentions match its actual behavior. We study whether a lightweight, real-time chain-of-thought (CoT) monitor can detect strategic deception during asymmetric negotiations, using a used-car sales scenario where a seller agent has private knowledge of an undisclosed defect and a buyer agent has only public market data. The monitor, implemented as a third agent, audits the seller's internal reasoning against its messages and alerts the buyer whenever concealment is detected, across multiple buyer-seller model pairings. Our experiments show that this monitor increases the buyer's walk-away rate, but reveal a persistent intelligence gap: lower-capability buyers often cannot translate an alert into an equitable counter-offer and still accept exploitative deals after being warned. Sellers also change their behavior when told they are monitored, though concealment is not eliminated. These results highlight both the promise and limits of lightweight real-time oversight, offering practical guidance for engineers building and validating monitoring infrastructure for agentic systems with conflicting stakeholder incentives.

cs.CY↗

arXiv2026

SentTrack: Sentiment-Driven Bottleneck Detection in GitHub Issue Repositories

Software engineering teams increasingly depend on GitHub issue threads to coordinate work, report bugs, and negotiate technical decisions, yet most repository health tools focus on code metrics and ignore the conversational dynamics that drive or stall development. This paper presents SentTrack, a dual-lens framework for detecting socio-technical bottlenecks from GitHub issue discussions. Applied to the AvaloniaUI open-source repository across approximately 9,000 issue threads, the framework addresses three questions: how to automate workflow-inefficiency detection from real-time conversational data, whether sentiment signals can surface risk earlier than traditional label-based methods, and how to isolate human narrative from machine-generated noise in mixed-media issue text. SentTrack combines two complementary pipelines. A horizontal pipeline translates raw issue reports into clean summaries using a large language model, extracts mid-level concern phrases, and clusters them through UMAP and HDBSCAN, producing 613 semantic clusters from the first 3,608 issues processed. A vertical pipeline applies the ABCDE collaborative interaction framework to classify each comment and infer thread-level outcomes. Across the full corpus, 49\% of threads ended in stagnation and only 13\% reached resolution, with the resolution gap identified as the dominant bottleneck signal. A weighted scoring engine that combines negativity, stagnation, resolution gap, and thread length gives maintainers an interpretable prioritization tool for high-friction discussions before they stall development.

cs.SE↗

arXiv2026

LLM vs. Human Unit Tests: Fault Detection on Real Python Bugs

Large language models (LLMs) have shown considerable promise for automated unit test generation, yet their practical effectiveness relative to human-written tests remains poorly understood. Existing evaluations commonly rely on coverage-oriented benchmarks that do not assess fault-detection capability directly. We present an empirical comparison of LLM-generated and human-written unit tests across three complementary Python benchmarks: 29 real historical bugs from BugsInPy, a function-level benchmark drawn from python-slugify and packaging, and a controlled paired benchmark. Our generation pipeline couples Gemini 2.5 Flash with a lightweight lexical retrieval mechanism that supplies bug-relevant context at generation time. Across eight quality dimensions, LLM-generated tests with retrieval-augmented context detect faults in 69% of cases compared to 17.2% for general-purpose human-written tests (Fisher's exact, $p < 0.001$, Cohen's $h = 1.10$). Critically, line and branch coverage are nearly identical between the two approaches (84.8% vs. 88.5% and 75.2% vs. 82.1%), confirming that coverage is an insufficient proxy for fault-detection capability. We discuss the conditions under which each approach excels, characterize their complementary strengths, and identify the critical role of retrieval context and reproducible benchmark construction in meaningful test-quality evaluation.

cs.SE↗

arXiv2026

Exploring Sustainability in Scientific Software through Code Quality & Test Coverage Metrics

Context: Scientific open-source software (SciOSS) plays a foundational role in research and engineering, yet its long-term sustainability has often been overlooked and remains a significant concern. Objective: This study investigates the long-term sustainability of SciOSS through code and test quality metrics. Method: We analyze CASS Software Portfolio projects, classifying them by sustainability and comparing their code structure, test coverage, and links between code quality and testing across the dataset. Results: Sustainable projects show higher, more consistent test coverage and clearer code-test correlations, while unsustainable ones show weaker patterns. Overall, test coverage is low in scientific software, and high complexity and coupling reduce testability. Conclusion: In this study, we present a practical, data-driven approach for assessing sustainability in scientific software, offering a foundation for evaluating long-term software health and supporting future efforts in quality assurance and sustainability monitoring.

cs.SE↗

arXiv2026

Characterizing the Usefulness of Code Review Comments in Scientific Software for Software Quality and Scientific Rigor

Context: Innovation thrives on scientific software, with useful code review feedback enhancing its correctness and impact. However, unlike general-purpose commercial and open-source software, the usefulness of code review feedback (CR comment) in scientific software remains largely unstudied. Objective: This paper aims to characterize the usefulness of CR comment in scientific opens ource software (Sci-OSS), leveraging existing research on useful CR comment. Method: To achieve this objective, we mine successful Sci-OSS from GitHub, analyze their CR comments with usefulness related features, and compare the findings from prior research on general-purpose commercial and open-source CR comments. Results: The investigation on the usefulness of CR comments in SciOSS confirms many characteristics that prior research identified in general-purpose software. For example, subjective or negative CR comments remain not useful for the Sci-OSS. We also find CR comments which receive negative emoji reactions have a very small correlation with not useful comments, whereas the positive emojis show mixed correlations. Importantly, 6-33% CR comments in Sci-OSS are not useful in our mined repositories. Conclusions: Our investigation into Sci-OSS extends findings from CR comments' usefulness research on general-purpose software, benefiting developers, scientists, and researchers in the Sci-OSS community.

cs.SE↗

arXiv2026

Applications of Causality in Software Testing: A Rapid Review

Causal inference offers a principled framework for understanding how interventions influence software behavior, yet its adoption in software testing remains fragmented across different tasks and research communities. In this rapid review, we systematically analyze 27 studies that apply causal reasoning to software testing activities such as debugging, fairness assessment, and performance evaluation. We organize the literature using a layered causal inference pipeline, spanning causal representation, structure discovery, identification, and effect estimation, to reveal how existing work maps onto fundamental causal reasoning stages. Our analysis shows a concentration of research on identification and estimation, while representation and discovery techniques are underexplored in testing contexts. We also identify cross-layer challenges, including model misspecification, untested assumptions, and limited empirical evaluation, which hinder practical application. Based on these insights, we propose a research agenda that highlights underrepresented opportunities for advancing causal methods in software testing. This structured perspective aims to unify disparate contributions and guide future empirical and methodological work in the intersection of causality and software testing.

cs.SE↗

arXiv2026

Precision or Peril: A PoC of Python Code Quality from Quantized Large Language Models

Context: Large Language Models (LLMs) like GPT-5 and LLaMA-405b exhibit advanced code generation abilities, but their deployment demands substantial computation resources and energy. Quantization can reduce memory footprint and hardware requirements, yet may degrade code quality. Objective: This study investigates code generation performance of smaller LLMs, examines the effect of quantization, and identifies common code quality issues as a proof of concepts (PoC). Method: Four open-source LLMs are evaluated on Python benchmarks using code similarity metrics, with an analysis on 8-bit and 4-bit quantization, alongside static code quality assessment. Results: While smaller LLMs can generate functional code, benchmark performance is limited. Quantization impacts are variable, and generated code exhibits quality and maintainability concerns. Conclusions: LLM-generated code should be carefully validated before integration into software projects.

cs.SE↗

arXiv2026

MALTA: Maintenance-Aware Technical Lag, Estimation to Address Software Abandonment

Context: Open-source ecosystems rely on sustained package maintenance. When maintenance slows or stops, Technical Lag (TL), the gap between installed and latest dependency versions accumulates, creating security and sustainability risks. However, some existing TL metrics, such as Version Lag, struggle to distinguish between actively maintained and abandoned packages, leading to a systematic underestimation of risk. Objective: We investigate the relationship between Version Lag and software abandonment by (i) identifying which repository-level signals reliably distinguish sustained maintenance from long-term decline, (ii) quantifying how Version Lag magnitude and persistence differ across maintenance states, and (iii) evaluating how maintenance-aware metrics change the identification of high-risk dependencies. Method: We introduce Maintenance-Aware Lag and Technical Abandonment (MALTA), a scoring framework comprising three metrics: Development Activity Score (DAS), Maintainer Responsiveness Score (MRS), and Repository Metadata Viability Score (RMVS). We evaluate MALTA on a dataset of 11,047 Debian packages linked to upstream GitHub repositories, encompassing 1.7 million commits and 4.2 million pull requests. Results: MALTA achieves AUC = 0.783 for classifying active versus declining maintenance. Most significantly, 62.2% of packages classified as "Low Risk" by Version Lag alone are reclassified as "High Risk" when MALTA signals are incorporated. These discordant packages average 2019 days since their last commit, with 9.8% having archived repositories. Conclusions: Version Lag metrics systematically miss abandoned packages, a blind spot affecting the majority of dependencies in distribution ecosystems. MALTA identifies a substantial discordant population invisible to Version Lag by distinguishing resolvable lag from terminal lag caused by upstream abandonment.

cs.SE↗

arXiv2026

Technical Lag as Latent Technical Debt: A Rapid Review

Context: Technical lag accumulates when software systems fail to keep pace with technological advancements, leading to a deterioration in software quality. Objective: This paper aims to consolidate existing research on technical lag, clarify definitions, explore its detection and quantification methods, examine underlying causes and consequences, review current management practices, and lay out a vision as an indicator of passively accumulated technical debt. Method: We conducted a Rapid Review with snowballing to select the appropriate peer-reviewed studies. We leveraged the ACM Digital Library, IEEE Xplore, Scopus, and Springer as our primary source databases. Results: Technical lag accumulates passively, often unnoticed due to inadequate detection metrics and tools. It negatively impacts software quality through outdated dependencies, obsolete APIs, unsupported platforms, and aging infrastructure. Strategies to manage technical lag primarily involve automated dependency updates, continuous integration processes, and regular auditing. Conclusions: Enhancing and extending the current standardized metrics, detection methods, and empirical studies to use technical lag as an indication of accumulated latent debt can greatly improve the process of maintaining large codebases that are heavily dependent on external packages. We have identified the research gaps and outlined a future vision for researchers and practitioners to explore.

cs.SE↗

arXiv2026

Multi-Artifact Analysis of Self-Admitted Technical Debt in Scientific Software

Context: Self-admitted technical debt (SATD) occurs when developers acknowledge shortcuts in code. In scientific software (SSW), such debt poses unique risks to the validity and reproducibility of results. Objective: This study aims to identify, categorize, and evaluate scientific debt, a specialized form of SATD in SSW, and assess the extent to which traditional SATD categories capture these domain-specific issues. Method: We conduct a multi-artifact analysis across code comments, commit messages, pull requests, and issue trackers from 23 open-source SSW projects. We construct and validate a curated dataset of scientific debt, develop a multi-source SATD classifier, and conduct a practitioner validation to assess the practical relevance of scientific debt. Results: Our classifier performs strongly across 900,358 artifacts from 23 SSW projects. SATD is most prevalent in pull requests and issue trackers, underscoring the value of multi-artifact analysis. Models trained on traditional SATD often miss scientific debt, emphasizing the need for its explicit detection in SSW. Practitioner validation confirmed that scientific debt is both recognizable and useful in practice. Conclusions: Scientific debt represents a unique form of SATD in SSW that that is not adequately captured by traditional categories and requires specialized identification and management. Our dataset, classification analysis, and practitioner validation results provide the first formal multi-artifact perspective on scientific debt, highlighting the need for tailored SATD detection approaches in SSW.

cs.SE↗

arXiv2026

LLMs in Code Vulnerability Analysis: A Proof of Concept

Context: Traditional software security analysis methods struggle to keep pace with the scale and complexity of modern codebases, requiring intelligent automation to detect, assess, and remediate vulnerabilities more efficiently and accurately. Objective: This paper explores the incorporation of code-specific and general-purpose Large Language Models (LLMs) to automate critical software security tasks, such as identifying vulnerabilities, predicting severity and access complexity, and generating fixes as a proof of concept. Method: We evaluate five pairs of recent LLMs, including both code-based and general-purpose open-source models, on two recognized C/C++ vulnerability datasets, namely Big-Vul and Vul-Repair. Additionally, we compare fine-tuning and prompt-based approaches. Results: The results show that fine-tuning uniformly outperforms both zero-shot and few-shot approaches across all tasks and models. Notably, code-specialized models excel in zero-shot and few-shot settings on complex tasks, while general-purpose models remain nearly as effective. Discrepancies among CodeBLEU, CodeBERTScore, BLEU, and ChrF highlight the inadequacy of current metrics for measuring repair quality. Conclusions: This study contributes to the software security community by investigating the potential of advanced LLMs to improve vulnerability analysis and remediation.

cs.SE↗

arXiv2026

Integrating APK Image and Text Data for Enhanced Threat Detection: A Multimodal Deep Learning Approach to Android Malware

As zero-day Android malware attacks grow more sophisticated, recent research highlights the effectiveness of using image-based representations of malware bytecode to detect previously unseen threats. However, existing studies often overlook how image type and resolution affect detection and ignore valuable textual data in Android Application Packages (APKs), such as permissions and metadata, limiting their ability to fully capture malicious behavior. The integration of multimodality, which combines image and text data, has gained momentum as a promising approach to address these limitations. This paper proposes a multimodal deep learning framework integrating APK images and textual features to enhance Android malware detection. We systematically evaluate various image types and resolutions across different Convolutional Neural Networks (CNN) architectures, including VGG, ResNet-152, MobileNet, DenseNet, EfficientNet-B4, and use LLaMA-2, a large language model, to extract and annotate textual features for improved analysis. The findings demonstrate that RGB images at higher resolutions (e.g., 256x256, 512x512) achieve superior classification performance, while the multimodal integration of image and text using the CLIP model reveals limited potential. Overall, this research highlights the importance of systematically evaluating image attributes and integrating multimodal data to develop effective malware detection for Android systems.

cs.CR↗

arXiv2025

The Role of AI in Modern Penetration Testing

Penetration testing is a cornerstone of cybersecurity, traditionally driven by manual, time-intensive processes. As systems grow in complexity, there is a pressing need for more scalable and efficient testing methodologies. This systematic literature review examines how Artificial Intelligence (AI) is reshaping penetration testing, analyzing 58 peer-reviewed studies from major academic databases. Our findings reveal that while AI-assisted pentesting is still in its early stages, notable progress is underway, particularly through Reinforcement Learning (RL), which was the focus of 77% of the reviewed works. Most research centers on the discovery and exploitation phases of pentesting, where AI shows the greatest promise in automating repetitive tasks, optimizing attack strategies, and improving vulnerability identification. Real-world applications remain limited but encouraging, including the European Space Agency's PenBox and various open-source tools. These demonstrate AI's potential to streamline attack path analysis, analyze complex network topology, and reduce manual workload. However, challenges persist: current models often lack flexibility and are underdeveloped for the reconnaissance and post-exploitation phases of pentesting. Applications involving Large Language Models (LLMs) remain relatively under-researched, pointing to a promising direction for future exploration. This paper offers a critical overview of AI's current and potential role in penetration testing, providing valuable insights for researchers, practitioners, and organizations aiming to enhance security assessments through advanced automation or looking for gaps in existing research.

cs.SE↗

arXiv2025

Exploring Scientific Debt: Harnessing AI for SATD Identification in Scientific Software

Developers often leave behind clues in their code, admitting where it falls short, known as Self-Admitted Technical Debt (SATD). In the world of Scientific Software (SSW), where innovation moves fast and collaboration is key, such debt is not just common but deeply impactful. As research relies on accurate and reproducible results, accumulating SATD can threaten the very foundations of scientific discovery. Yet, despite its significance, the relationship between SATD and SSW remains largely unexplored, leaving a crucial gap in understanding how to manage SATD in this critical domain. This study explores SATD in SSW repositories, comparing SATD in scientific versus general-purpose open-source software and evaluating transformer-based models for SATD identification. We analyzed SATD in 27 scientific and general-purpose repositories across multiple domains and languages. We fine-tuned and compared 10 transformer-based models (100M-7B parameters) on 67,066 labeled code comments. SSW contains 9.25x more Scientific Debt and 4.93x more SATD than general-purpose software due to complex computations, domain constraints, and evolving research needs. Furthermore, our best model outperforms existing ones. This study uncovers how SATD in SSW differs from general software, revealing its impact on quality and scientific validity. By recognizing these challenges, developers and researchers can adopt smarter strategies to manage debt and safeguard the integrity of scientific discovery.

cs.SE↗

arXiv2025

Peer Code Review in Research Software Development: The Research Software Engineer Perspective

Background: Research software is crucial for enabling research discoveries and supporting data analysis, simulation, and interpretation across domains. However, evolving requirements, complex inputs, and legacy dependencies hinder the software quality and maintainability. While peer code review can improve software quality, its adoption by research software engineers (RSEs) remains unexplored. Aims: This study explores RSE perspectives on peer code review, focusing on their practices, challenges, and potential improvements. Building on prior work, it aims to uncover how RSEs insights differ from those of other research software developers and identify factors that can enhance code review adoption in this domain. Method: We surveyed RSEs to gather insights into their perspectives on peer code review. The survey design aligned with previous research to enable comparative analysis while including additional questions tailored to RSEs. Results: We received 61 valid responses from the survey. The findings align with prior research while uncovering unique insights about the challenges and practices of RSEs compared to broader developer groups. Conclusions: Peer code review is vital in improving research software's quality, maintainability, and reliability. Despite the unique challenges RSEs face, addressing these through structured processes, improved tools, and targeted training can enhance peer review adoption and effectiveness in research software development.

cs.SE↗