subject

cs.SE

cs.SE: explore 370 source-linked works published from 2025 to 2026, with original documents and citations.

This collection is a preview while coverage and quality are evaluated.

Coverage and selection

Includes records with this source-supplied label or an explicit phrase match in their metadata. Matches indicate a mention, not proof that a paper uses a method or tests a material. Source versions are consolidated by DOI.

Sources: arxiv. Collection updated 2026-09-16. Counts describe this index, not the complete source archives.

arXiv2026

ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluation

Evaluating Artificial Intelligence (AI) tutor feedback before deployment requires anticipating student engagement, typically assessed through real interaction data. We introduce ParaStudent, a fine-tuning framework for simulating novice programming revisions to support AI tutor evaluation. Compared with prompted baselines, ParaStudent's revisions more closely match real student code distributions across functional, stylistic, and semantic metrics. Our best variant achieves AUCs of 0.80 for both feedback relevance and successful uptake when distinguishing streams with real engagement above versus at or below the median, while prompted baselines remain near chance on successful uptake. These findings demonstrate the promise of simulated engagement for pre-deployment feedback triage.

cs.CY↗

arXiv2026

Essence Coach: A Bot for Software Practice Adoption

Although Essence has been proposed as a unifying framework for understanding and evaluating software engineering practices, its adoption remains challenging, calling for tools that can act as coaches for learners and practitioners. We present Essence Coach, a chatbot that integrates LLMs with retrieval-augmented generation (RAG) from a curated Essence knowledge base. Experiments with multiple LLMs show that RAG consistently improves response quality for Essence-related queries. Some preliminary tests suggest that such systems can help both technical and non-technical students understand and apply Essence concepts. While validation through broader user studies is needed, this work-in-progress highlights how LLMs can bridge abstract SE frameworks and practice adoption.

cs.SE↗

arXiv2026

Oops!... I did it again. Analysing and Handling Conclusion (In-)Stability in Socio-Technical Software Engineering

Context: Mining software repositories is a popular means to gain insights into a software project's evolution, monitor project health, support decisions and derive best practices. Tools supporting the mining process are commonly applied by researchers and practitioners, but their limitations and agreement are often not well understood. Objective: This study investigates some threats to validity in complex tool pipelines for evolutionary socio-technical software analyses. We evaluate the tools' agreement in terms of data, study outcomes and conclusions for the same research questions to derive actionable advice for researchers and practitioners. Method: We conduct a lightweight literature review to select \emph{three} studies on collaboration and coordination, software maintenance and software quality from high-ranked venues, which we formally replicate with \emph{four} independent, systematically selected mining tools to quantitatively and qualitatively compare the extracted data, analysis results and conclusions. Results: We find and summarise numerous technical details in tool design and implementation which accumulate along the complex mining pipelines and can cause substantial differences in the extracted baseline data, its derivatives, subsequent results of statistical analyses and, under specific circumstances, conclusions. Conclusions: Users should evaluate mining tools and their limitations carefully to scope the validity of their conclusions, for instance with the checklist we provide. Researchers and tool authors can further reduce uncertainty through reproduction packages and comparative studies in other MSR fields following our methodology.

cs.SE↗

arXiv2026

Measuring Computer Science Enthusiasm: A Questionnaire-Based Analysis of Age and Gender Effects on Students' Interest

This study examines how age and gender independently shape adolescents' interest in computer science (CS) education. Building on the Person-Object Theory of Interest (POI), we define enthusiasm as a short-term, activating response that combines positive affect, perceived relevance, and intention to re-engage. Because such enthusiasm can shift CS attitudes and engagement intentions even briefly, it offers a useful measure for short outreach activities. We developed a 28-item pre-post questionnaire to assess whether CS interventions raise enthusiasm, then applied it to more than 400 students (244 female, 187 male, aged 10-18) in CS courses. Contrary to the common assumption that early exposure secures lasting interest, we found a marked decline during early adolescence, especially among girls, along with wide variation in interest trajectories across ages. Exploratory factor analysis and ANOVA show that age predicts interest development more strongly than gender, and reveal specific developmental breakpoints. Although older students began with lower baseline attitudes, they showed the largest gains after the intervention, indicating that well-designed short activities can re-engage interest even later in adolescence. These results point to the need for CS education strategies that adapt to developmental stage rather than assuming a single early window matters most. Our validated questionnaire offers a way to measure immediate affective and motivational responses, giving researchers and practitioners a tool to evaluate whether specific interventions succeed in raising enthusiasm.

cs.SE↗

arXiv2026

Can Coding Agents Reproduce Findings in Computational Materials Science?

Large language models are increasingly deployed as autonomous coding agents and have achieved remarkably strong performance on software engineering benchmarks. However, it is unclear whether such success transfers to computational scientific workflows, where tasks require not only strong coding ability, but also the ability to navigate complex, domain-specific procedures and to interpret results in the context of scientific claims. To address this question, we present AutoMat, a benchmark for evaluating LLM-based agents' ability to reproduce claims from computational materials science. AutoMat poses three interrelated challenges: recovering underspecified computational procedures, navigating specialized toolchains, and determining whether the resulting evidence supports a claim. By working closely with subject matter experts, we curate a set of claims from real materials science papers to test whether coding agents can recover and execute the end-to-end workflow needed to support (or undermine) such claims. We then evaluate multiple representative coding agent settings across several foundation models. Our results show that current LLM-based agents obtain low overall success rates on AutoMat, with the best-performing setting achieving a success rate of only 53%. Error analysis further reveals that agents perform worst when workflows must be reconstructed from paper text alone and that they fail primarily due to incomplete procedures, methodological deviations, and execution fragility. Taken together, these findings position AutoMat as both a benchmark for computational scientific reproducibility and a tool for diagnosing the current limitations of agentic systems in AI-for-science settings.

cs.CL↗

arXiv2026

Steer, Don't Solve: Training Small Critic Models for Large Code Agents

Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While coding agents are optimized for the joint capabilities, individual capabilities such as high-level planning may have different optima and remain a major bottleneck. To address this challenge, we train a separate critic model that is specialized in high-level planning to steer the coding agent in inference. We construct SFT and DPO data to train the critic model to identify errors made by the coding agent and provide correct and clear high-level guidance without generating concrete actions. Experiments show that our fine-tuned 4B and 8B critic models significantly improve the performance of 6 larger coding agents (e.g., improving the resolved rates of GLM-4.7-Flash-30B-A3B and GPT-OSS-120B by 16.0% and 14.4% on SWE-Bench Verified). The critic model also reduces the total inference costs for some coding agents by solving tasks in fewer steps (e.g., reducing the per-example inference cost for GPT-OSS-20B from \$0.07 to \$0.03). Code: https://github.com/shubhamrgandhi/critic-training

cs.SE↗

arXiv2026

ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance

Language model (LM) agents are increasingly deployed to maintain codebases over extended periods, fixing streams of related defects while carrying context from one fix to the next. Yet existing software engineering (SWE) benchmarks evaluate models one bug at a time: the repository is reset, the codebase is re-read, and a single self-contained issue is graded in isolation. This setting collapses a continuous maintenance workflow into a series of independent sessions, ignoring the cumulative dependencies that make real-world bug fixing challenging. To bridge this gap, we introduce ChainSWE, the first benchmark for evaluating agents on sequential, dependent bug fixes within a shared codebase. We collect chronological chains of 304 issues across 54 Python projects, mined from six SWE-bench-family datasets. Our evaluation across a range of agents and models reveals a consistent performance drop by up to 70% as the chain length increases.

cs.SE↗

arXiv2026

SEDCoT: Enhancing LLM-Based COBOL Code Translation via Symbolic Execution and Delta Debugging

COBOL remains critical across banking, insurance, and government infrastructure. However, maintenance is increasingly challenging due to outdated technologies, sparse documentation, and developer retirement, necessitating code translation into modern languages like C. Traditional rule-based transcompilers yield outputs that are difficult to read and maintain, while general-purpose large language models (LLMs) achieve suboptimal correctness because COBOL is a low-resource language with distinct logic patterns. To bridge this gap, we propose SEDCoT, a novel COBOL-to-C translation framework. SEDCoT first leverages LLMs for initial translation, then combines symbolic execution with LLM guidance to generate test suites and iteratively repair semantic discrepancies. Finally, it integrates delta debugging to minimize failing tests into succinct counterexamples, accelerating automated code repair. Evaluating SEDCoT on a public COBOL-to-C dataset demonstrates that it outperforms state-of-the-art baselines by at least 12% while producing translations with substantially higher readability than rule-based alternatives.

cs.SE↗

arXiv2026

Automatic Model-Hardware Co-Adaptation for Heterogeneous AI Accelerators

Large language models now evolve faster than production inference systems can be ported and optimized. New releases change attention, MoE routing, quantization formats, KV-cache layout, and parallel execution patterns, while deployed accelerator fleets remain heterogeneous across hardware generations, framework forks, operator libraries, compiler backends, and communication runtimes. Serving a new model on existing hardware is therefore a model-framework-kernel-hardware co-adaptation problem. We present MetaInfer, an LLM-agent system that formulates inference adaptation as route search over a costed execution-adaptation graph. The graph connects model semantics, framework dispatch, kernel choices, hardware capabilities, runtime evidence, and serving objectives. MetaInfer constructs and updates this graph during execution, restores missing or blocked routes through patches, and reduces route cost through staged validation and end-to-end profiling. Three real episodes -- DeepSeek V4 Flash on NVIDIA A800, GLM 5.3 Flash on NVIDIA A800, and DeepSeek V4 Flash on Hygon K100AI DCU -- demonstrate deployment repair, cross-model knowledge transfer, and portability across heterogeneous accelerator software stacks.

cs.MA↗

arXiv2026

SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?

Software increasingly functions as part of the scientific instrument itself, making failures in scientific code capable of compromising not only program behavior but also the evidence underlying scientific conclusions. Yet existing evaluations of coding agents largely emphasize aggregate task success, providing limited insight into why agents fail when repairing scientific software. We introduce \textbf{SWE-bench Science}, a repository-level benchmark for scientific software engineering comprising 119 tasks from 98 GitHub repositories across 20 scientific domains. Each task is organized into one of three paradigms: Issue-driven, Expert-exploratory, and Engineering-integration. Even the best-performing agent, \textbf{Claude Code with Opus-5 (max), achieves a pass@1 below 50\%}, highlighting the substantial challenges posed by scientific software engineering. We identify four recurring failure mechanisms: deficits in scientific knowledge or abstraction, misguided exploration or surface-level repair, incomplete repair coverage or system integration, and failures to generalize scientific knowledge beyond observed cases in our analysis. We further conduct a paired ablation that removes explicit scientific guidance while preserving the repository and executable engineering context. The results show that scientific knowledge is not uniformly beneficial: well-grounded information can constrain repair and improve average performance and token efficiency, whereas poorly aligned guidance can induce anchoring and does not necessarily improve exact repair success. Together, SWE-bench Science provides a broad testbed for studying both the capabilities and failure mechanisms of coding agents in scientific software engineering.

cs.CL↗

arXiv2026

Spec-Driven Development as Prescribed and as Practiced

AI coding agents implement software from naturallanguage instructions, but the intent behind that work is lost when stated only in chat sessions. Spec-driven development (SDD) addresses this by having developers and agents write specifications that guide implementation, yet evidence on its use comes largely from tool vendors. We study SDD in practice through a thematic analysis of Reddit discussions, interviews with 21 practitioners, including eight tool makers, and 266 active and popular repositories in SpecMine, a corpus of 470,795 spec files from 73,030 public GitHub repositories that we release. Practitioners agreed that specs should capture intent, but many also included implementation details, used SDD selectively and mainly for new features, and reported more reliable agent behavior alongside greater planning and review costs. Our findings show that implementation detail may help agents while making specs prone to decay, that tool-specific record structures shape what people and agents see, and that participants described spec-code checks mainly when changes are completed rather than over time. We also find that individual benefits may come from clarifying intent before delegation, while team benefits may depend on SDD becoming a shared process. We derive implications for tool builders, practitioners, and researchers.

cs.SE↗

arXiv2026

Software Aging in LLM-Generated Applications: Runtime Evidence, Static Analysis, and Human-Written Comparisons

Large Language Models (LLMs) are increasingly used to generate executable software systems from natural language specifications, accelerating development and reducing manual implementation effort. Although recent studies have investigated the functional correctness, security, maintainability, and robustness of LLM-generated code, little is known about the long-term reliability of such systems under sustained execution. In this paper, we experimentally investigate software aging symptoms in LLM-generated service-based applications across generation-and-execution environments. Using backend scenarios derived from BaxBench, we generated applications targeting JavaScript, Python, and Rust through LLM-based generation platforms, validated them with BaxBench-derived tests, and subjected them to 48-hour workload executions. We monitored memory usage, response time, and throughput and analyzed them using the Mann--Kendall test and Sen's slope estimator. We further complemented the runtime evaluation with static analysis of the generated source code and an exploratory comparison with human-written implementations of related backend scenarios. The results show that memory usage is the most consistent indicator of potential software aging, with statistically significant upward trends in most application-environment combinations, while response time and throughput exhibit more heterogeneous behavior. Static analysis identified plausible code-level aging mechanisms, and the comparison with human-written systems showed that the aging symptoms observed in LLM-generated applications align with degradation patterns also found in manually developed implementations. These findings indicate that functional correctness alone is insufficient to assess the operational reliability of LLM-generated software before deployment in continuously running environments.

cs.SE↗

arXiv2026

Runtime-Independent Persistent Agents: Preserving Identity, Memory, and Code Across Models, Harnesses, and Servers

Long-lived AI agents may replace their models, orchestration harnesses, interaction surfaces and hosts while retaining identity, memory and executable body lineage. We present a runtime-independent architecture that separates a continuity-bearing substrate $P_t=(I_t,M_t,B_t)$ from replaceable execution and interaction bindings. Six continuity invariants and a quiesce--checkpoint--validate--bind--rehydrate--resume protocol govern state preservation, capability changes and continuation authority. Enoch implements the design through a reusable software body, private installed state, provider contracts and fenced migration. We report complementary evidence from frozen implementation snapshots and three kinds of live study. An operational case retains an established instance's identity, nonempty memory, body revision and historical task IDs across host, joint Codex-to-Claude model/harness, and chat-surface substitutions. A separate cross-host task resumes from a verified artifact checkpoint. A supervised Codex--Muse--Codex study completes five ordinary round trips and five matched same-runtime controls on their first attempts after a separate qualification run. A planned worker interruption recovers through the native task API with the checkpoint preserved, a stale reply rejected and one task completion recorded. These results support mechanical continuity for the tested deployments and bounded workflows. They do not establish behavioral equivalence, arbitrary-task portability, general exactly-once external effects or unattended reliability.

cs.SE↗

arXiv2026

WiseSpec: Requirements-Driven Agents for Code Generation

Code generation aims to automatically generate source code from task requirements and has attracted significant attention with the rapid advancement of large language models (LLMs). Despite remarkable progress, LLMs often struggle to generate correct code for complex software engineering tasks because task descriptions are frequently incomplete, ambiguous, or lack critical contextual information. Existing approaches primarily improve the capabilities of coding agents through more sophisticated tools, skills, and workflows, while largely overlooking the quality of the task requirements themselves. To address this limitation, we draw inspiration from software requirements engineering and propose WiseSpec, a novel requirements-driven agent framework for repository-level code generation. WiseSpec automatically constructs structured and information-rich requirements, assesses their quality through execution-based evaluation, and iteratively refines them to better guide code generation. Experimental results show that WiseSpec consistently outperforms all baselines, achieving an average improvement of 13.17% in %Resolved.

cs.SE↗

arXiv2026

Predicting Program Exit Code with LLMs and Programming Language Semantics

Large language models (LLMs) have shown proficiency in various software engineering tasks, such as code generation and translation. However, a key limitation in their performance may be their (lack of) understanding of programming-language semantics. Even when explicit semantics are given, it remains unclear whether LLMs apply those rules or lean on priors learned during pre-training instead. We study if LLMs lean on priors or given semantics with a novel task--Program Executability Prediction (PrEx)--that asks models to predict whether a program is semantically valid or invalid (and, if invalid, which formal rule it violates) given the program's syntax and operational semantics. Because PrEx requires both valid and invalid programs, we build a dataset with systematically generated invalid transformations derived from valid programs. We evaluate open-source coding LLMs under two semantic formalisms and two semantic shifts across Human-Written, LLM-Translated, and Fuzzer-Generated program splits. Our findings show that LLMs lean on pre-training priors rather than systematically applying the given rules, performing especially poorly on modified semantics and degrading further as program complexity increases. PrEx is available at https://github.com/EngineeringSoftware/prex.

cs.PL↗

arXiv2026

Towards a Reliable and Practical Eval Pipeline

LLM-based software systems increasingly require effective "evals" as quality gates in the development lifecycle. However, existing work typically addresses individual aspects of eval reliability rather than the full set of practical requirements. We present an end-to-end eval pipeline that combines eval checklist creation, with learned aggregation for checklist responses, to improve agreement across LLM judges and accuracy against human judgments. The framework additionally pro- vides self-consistency, explanations, and prediction uncertainty, and we empirically demonstrate its effectiveness.

cs.AI↗

arXiv2026

Replacing Training with Memory: Listwise Selection for Text-to-SQL

Modern Text-to-SQL systems often follow generate-execute-select pipelines, generating multiple candidate queries then selecting the best one. Listwise selection, by jointly comparing multiple candidates, has been widely adopted, but fine-tuning listwise selectors is costly. We thus propose a fine-tuning-free listwise selector. We replace two major fine-tuning objectives with inference-time strategies: (1) learning selection criteria as ordering and (2) mitigating positional bias. First, we build reusable structured memories instead of learning selection behavior as model parameters. Given a question, MaP-SQL retrieves memories distilled from training data that encode how natural language maps to schema elements, SQL operations, and expected outputs. These memories serve as explicit decision criteria for evaluating candidates in a listwise manner. Second, to mitigate ordering bias of listwise selectors, we aggregate rankings across multiple input permutations, with inference cost optimized by execution results and pointwise scoring. Our approach improves selection accuracy while maintaining efficiency and compatibility with existing large language models. Across Text-to-SQL benchmarks, it produces more stable selection without fine-tuning and fewer unnecessary comparisons than existing methods. On BIRD-dev, it outperforms the previous state-of-the-art selector-based method R^3-SQL by 2.02 execution accuracy points on average using the same candidate sets, with 2.92x fewer tokens.

cs.SE↗

arXiv2026

Probabilistic Model Checking of Autoregressive Neural Sequence Models

Test-set accuracy is silent on two issues that matter when deploying autoregressive neural sequence models: how much probability mass the system under test (SUT) places on constraint-violating alternatives that are reachable under sampling and what fraction of the input population satisfies a domain requirement. We answer both with probabilistic model checking. The pipeline extracts a discrete-time Markov chain (DTMC) from the SUT's token-by-token generation, verifies formal PCTL specifications with the PRISM model checker, and aggregates the per-input verdicts into a coverage curve over the input space. A soundness theorem establishes the DTMC as an under-approximation, so every verdict yields a certified interval on the SUT's true reachability probability. The coverage built from those verdicts is, therefore, conservative by construction. A counterexample-guided abstraction refinement (CEGAR) loop adaptively tightens the interval, and a maximum-likelihood algorithm extracts the most probable falsifying trace. Two case studies exercise the pipeline. On a GPT-2 computer-aided process-planning (CAPP) model with 100% test accuracy, the pipeline quantifies the probability mass greedy decoding hides, but that is reachable with sampling; and identifies the smallest training fraction at which an ordering requirement holds population-wide, neither of which test accuracy can report. We then verify the SMILES molecular generator with a 50x larger vocabulary. The only change is an external chemical-validity oracle, and the pipeline identifies the gap between structural completeness and chemical validity.

cs.SE↗

Compare source metadata on this page

Work	Published	Source identifier	Source
ParaStudent: Closing the Sim2Real Gap in User Simulators for AI Tutor Evaluation	2026-09-01	2507.12674	arxiv
Essence Coach: A Bot for Software Practice Adoption	2026-09-01	2508.16445	arxiv
Oops!... I did it again. Analysing and Handling Conclusion (In-)Stability in Socio-Technical Software Engineering	2026-09-01	2510.06844	arxiv
Measuring Computer Science Enthusiasm: A Questionnaire-Based Analysis of Age and Gender Effects on Students' Interest	2026-09-01	2512.08472	arxiv
Can Coding Agents Reproduce Findings in Computational Materials Science?	2026-09-01	2605.00803	arxiv
Steer, Don't Solve: Training Small Critic Models for Large Code Agents	2026-09-01	2606.21811	arxiv
ChainSWE: Benchmarking Coding Agents on Multi-Bug Software Maintenance	2026-09-01	2607.02606	arxiv
SEDCoT: Enhancing LLM-Based COBOL Code Translation via Symbolic Execution and Delta Debugging	2026-09-01	2607.04092	arxiv
Automatic Model-Hardware Co-Adaptation for Heterogeneous AI Accelerators	2026-09-01	2607.12875	arxiv
SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?	2026-09-01	2608.19799	arxiv
Spec-Driven Development as Prescribed and as Practiced	2026-10-15	2608.25202	arxiv
Software Aging in LLM-Generated Applications: Runtime Evidence, Static Analysis, and Human-Written Comparisons	2026-09-01	2608.26391	arxiv
Runtime-Independent Persistent Agents: Preserving Identity, Memory, and Code Across Models, Harnesses, and Servers	2026-09-19	2609.00546	arxiv
WiseSpec: Requirements-Driven Agents for Code Generation	2026-09-01	2609.00568	arxiv
Predicting Program Exit Code with LLMs and Programming Language Semantics	2026-09-01	2609.00579	arxiv
Towards a Reliable and Practical Eval Pipeline	2026-09-01	2609.00805	arxiv
Replacing Training with Memory: Listwise Selection for Text-to-SQL	2026-09-01	2609.00834	arxiv
Probabilistic Model Checking of Autoregressive Neural Sequence Models	2026-09-01	2609.00838	arxiv

These are bibliographic comparisons, not experimental rankings. Follow the original document for methods and conditions.