SearcharxivSearch

arXiv subjects

Yuhong Fu

Publications and source records attributed to Yuhong Fu.

5 recordsLinked to original sources

Requirements After the First Edit: Mining Late Requirement Emergence and Rework in Real-World Coding-Agent Sessions

Coding agents often implement changes before users have fully articulated their requirements, echoing a pattern from requirements engineering: stakeholders cannot express a constraint until part of the system exists to react to. This volatility is associated with schedule and budget overruns in traditional projects, but only at release-cycle granularity. Existing work on coding agents narrows this gap only partway: curated benchmarks fix requirements before implementation by design, and observational studies report pushback frequency without linking arrivals to the code invalidation they cause. We address this using 3,553 eligible SWE-chat sessions, coding post-implementation requirement arrivals along three dimensions and, where repository state can be replayed, linking each arrival to a proxy: deletion or replacement of prior agent-authored lines. A requirement's arrival is followed by roughly twice as much invalidation as matched non-requirement edits, robust to user-turn and net-deletion checks, though not demonstrated as causal. This burden shows no detectable decline over a session and no detected association with operation type once multiplicity is accounted for; several intervals remain wide. A controlled experiment shows delayed disclosure relocates implementation post-reveal, while advance warning produces no detected effect on overwriting. These results establish late requirement emergence as a measurable source of code invalidation.

cs.SE

Can LLMs Learn and Apply Multi-Level Modelling Semantics? A First Empirical Study

Industry 5.0 emphasises human-centric industrial system design, placing additional demands on modelling tools. Multi-level modelling (MLM) can directly represent three or more abstraction levels, but this comes at the cost of more complex semantic constraints that model correctness depends on. Large Language Models (LLMs) have been increasingly studied in model-driven engineering, but this evidence rests entirely on two-level modelling tasks, and whether it generalises to MLM, whose semantics differ in kind, remains untested. This paper presents the first empirical study of this question. We have three commercial LLMs (GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro) generate multi-level models for the MULTI Warehouse Challenge in the SLICER language under six prompting strategies, yielding 90 generated models compared against a manually validated reference using fourteen metrics. Syntactic correctness is within reach, but semantic correctness is only partially achieved, with Instantiation/Specialisation Correctness ranging from 52% to 79%. Models reproduce content stated explicitly in the task text, but rarely complete structure and constraints the text implies without stating. Prompting strategies trade off precision against completeness, and self-checking functions mainly as a rule checker rather than reliably improving alignment with the reference design. Among the three LLMs, Claude shows the most balanced profile. These results clarify the boundaries of current LLM capability for MLM and inform the design of human-centred, AI-assisted modelling workflows for Industry 5.0.

cs.SE

Two-Level vs. Multi-Level Modelling: An Empirical Study of Cascading Maintenance Burden

When a core definition changes, every dependent artefact must be updated, a cascading problem central to software maintenance. In Model-Driven Engineering (MDE), the dominant two-level modelling (2LM) paradigm fragments domain knowledge across metamodel and model artefacts that must be kept mutually consistent, making co-evolution a persistent source of inconsistencies and effort. Multi-level modelling (MLM) unifies these artefacts and is claimed to reduce co-evolution burden, but this has not been tested in a controlled, paired comparison against 2LM. We hypothesise that MLM's structural unification yields fewer post-change inconsistencies and a smaller modification footprint than 2LM for semantically equivalent evolution scenarios. To test this, we present a pre-registered, mutation-based empirical comparison of co-evolution behaviour in both paradigms. From a curated corpus of published 2LM co-evolution scenarios, we construct semantically equivalent MLM counterparts, apply identical evolution mutations to both, and measure outcomes through automated consistency checking and pre-registered hypothesis tests. Positive controls and a blinded mapping protocol guard against bias. This design provides the first empirical framework for assessing whether paradigm-level structural choices affect cascading maintenance burden, operationalising co-evolution burden as two automatically measurable outcome variables and delivering a reusable benchmarking protocol for replication and extension.

cs.SE

Domain-Driven Design in Practice: A Mining Study of Maintenance and Evolution in Open-Source Repositories

Domain-Driven Design (DDD) is an influential software development methodology that structures software around business domain complexity through tactical building blocks such as Entities, Value Objects, Aggregates, and Repositories. Despite its prominence in software engineering, large-scale empirical evidence on how DDD is practiced, how it evolves, and how it relates to software maintenance quality in open-source projects remains scarce. This study presents a pre-registered empirical investigation of the distribution, evolution, and maintenance implications of DDD tactical building blocks in open-source GitHub repositories, addressing the call for large-scale empirical evaluations identified in a recent systematic literature review. We will collect DDD-related repositories from GitHub using the Search API, apply automated keyword filtering and manual relevance assessment, and analyze the resulting dataset through four research questions covering: (RQ1) the static distribution and co-usage of DDD building blocks across repository types, (RQ2) their longitudinal evolution over commit history, (RQ3) the extent and maintenance implications of Bounded Context boundary violations (the primary technical challenge in DDD adoption, unquantified at scale), and (RQ4) the temporal association between maintenance activities and building block churn or Bounded Context violations in DDD repositories. Together, these RQs trace the full maintenance and evolution lifecycle of open-source DDD projects and establish an empirical foundation for future DDD tool support and methodology refinement.

cs.SE

Leveraging LLMs to support co-evolution between definitions and instances of textual DSLs: A Systematic Evaluation

Software languages evolve over time for reasons such as feature additions. When grammars evolve, textual instances that originally conformed to them may become outdated. While model-driven engineering provides many techniques for co-evolving models with metamodel changes, these approaches are not designed for textual DSLs and may lose human-relevant information such as layout and comments. This study systematically evaluates the potential of large language models (LLMs) for co-evolving grammars and instances of textual DSLs. Using Claude Sonnet 4.5 and GPT-5.2 across ten case languages with ten runs each, we assess both correctness and preservation of human-oriented information. Results show strong performance on small-scale cases ($\geq$94% precision and recall for instances requiring fewer than 20 modified lines), but performance degraded with scale: Claude maintains 85% recall at 40 lines, while GPT fails on the largest instances. Response time increases substantially with instance size, and grammar evolution complexity and deletion granularity affect performance more than change type. These findings clarify when LLM-based co-evolution is effective and where current limitations remain.

cs.SE