arXiv subjects

Minxiao Li

Publications and source records attributed to Minxiao Li.

3 recordsLinked to original sources

arXiv2026

Benchmarking Requirement-to-Architecture Generation with Hybrid Evaluation

Software architecture serves as the blueprint of a software system, capturing high-level structural decisions that shape downstream implementation and system quality. Despite this central role, generating architecture designs from requirement documents remains underexplored, with limited task-specific benchmarks for generation or rigorous evaluation. To bridge this gap, we introduce R2ABench, a benchmark for requirements-to-architecture (R2A) reasoning. R2ABench contains 68 projects with human-validated references collected from both educational-style settings and GitHub repositories. Each project is packaged as a unified instance that provides a structured software requirements specification (SRS), a reference architecture view in PlantUML, and the corresponding source requirement artifacts. To evaluate generated architecture views, we design a layered hybrid evaluation framework spanning syntax validation, structural graph diagnostics, and semantic and evidence-based architecture scoring. Using this framework, we conduct a comprehensive empirical study of state-of-the-art LLMs and agents on R2ABench. We find that current systems can often generate syntactically valid and readable architecture views, but still struggle with relation-level architecture modeling across both subsets: component identification is substantially stronger than edge recovery, and edge hallucination is the dominant structural failure mode. Semantically, structural fidelity does not guarantee requirement coverage or traceability, which emerge as the primary quality gaps. The associated data files are available at https://figshare.com/s/01f0a5fb6243a6a60f23.

cs.SE↗

arXiv2025

CodeIF-Bench: Evaluating Instruction-Following Capabilities of Large Language Models in Interactive Code Generation

Large Language Models (LLMs) have demonstrated exceptional performance in code generation tasks and have become indispensable programming assistants for developers. However, existing code generation benchmarks primarily assess the functional correctness of code generated by LLMs in single-turn interactions. They offer limited insight into LLMs' abilities to generate code that strictly follows users' instructions in multi-turn interaction scenarios. In this paper, we introduce CodeIF-Bench, a benchmark for evaluating the instruction-following capabilities of LLMs in interactive code generation. Specifically, CodeIF-Bench incorporates nine types of verifiable instructions aligned with the real-world software development requirements, which can be independently and objectively validated through specified test cases, facilitating the evaluation of instruction-following capability in multi-turn interactions. In both Static Conversation and Dynamic Conversation settings, we evaluate the performance of 6 state-of-the-art LLMs and summarize the important factors, additional repository context and gradually increasing interaction history influencing the instruction-following ability of LLMs in multi-turn interactions. Furthermore, we identify the potential direction for improvement: context management. The code and data are available at \href{https://github.com/zhu-zhu-ding/CodeIF-Bench}{https://github.com/zhu-zhu-ding/CodeIF-Bench}.

cs.SE↗

arXiv2025

EfficientEdit: Accelerating Code Editing via Edit-Oriented Speculative Decoding

Large Language Models (LLMs) have demonstrated remarkable capabilities in code editing, substantially enhancing software development productivity. However, the inherent complexity of code editing tasks forces existing approaches to rely on LLMs' autoregressive end-to-end generation, where decoding speed plays a critical role in efficiency. While inference acceleration techniques like speculative decoding are applied to improve the decoding efficiency, these methods fail to account for the unique characteristics of code editing tasks where changes are typically localized and existing code segments are reused. To address this limitation, we propose EfficientEdit, a novel method that improves LLM-based code editing efficiency through two key mechanisms based on speculative decoding: (1) effective reuse of original code segments while identifying potential edit locations, and (2) efficient generate edit content via high-quality drafts from edit-oriented draft models and a dynamic verification mechanism that balances quality and acceleration. Experimental results show that EfficientEdit can achieve up to 10.38$\times$ and 13.09$\times$ speedup compared to standard autoregressive decoding in CanItEdit and CodeIF-Bench, respectively, outperforming state-of-the-art inference acceleration approaches by up to 90.6%. The code and data are available at https://github.com/zhu-zhu-ding/EfficientEdit.

cs.SE↗