SearcharxivSearch

arXiv subjects

Mehedi Sun

Publications and source records attributed to Mehedi Sun.

2 recordsLinked to original sources

Recovering Fine-Grained Code Change Rationale from Multiple Software Artifacts

Understanding the reasons behind past code changes is critical for refactoring, code review, and debugging. However, code change rationale is often fragmented, inconsistently documented, and scattered across heterogeneous artifacts. We address this challenge with two contributions. First, we conduct an empirical study of nine rationale components from an established taxonomy and trace where they are documented across artifacts associated with 63 commits from five widely used open-source Java projects. Seven components appear in practice, and rationale is highly fragmented: commit messages and pull requests primarily capture GOAL, while NEED and ALTERNATIVE are more often found in issues and pull requests. No single artifact type consistently captures all components, which presents the need for cross-document reasoning. Second, we introduce ARGUS, an LLM-based approach that identifies sentences expressing GOAL, NEED, and ALTERNATIVE across a commit's artifacts and synthesizes them into concise rationale summaries. ARGUS achieved 51.4% overall precision and 93.2% recall for rationale identification and generated summaries rated as accurate relative to reference summaries. Experiments across different LLMs showed varying identification performance but consistently accurate summaries. A user study with 12 Java programmers found these summaries useful for understanding unfamiliar code changes and supporting code review, documentation, debugging, and maintenance.

cs.SE

Evaluating Language Model Applications for Identifying Solution-Related Content in Issue Report Discussions

During issue resolution, software developers rely on issue reports to discuss solutions for defects, feature requests, and other changes. These discussions contain proposed solutions--from design changes to code implementations--as well as their evaluations. Locating solution-related content is essential for investigating reopened issues, addressing regressions, reusing solutions, and understanding code change rationale. Manually understanding long discussions to identify such content can be difficult and time-consuming. This paper automates solution identification using language models as supervised classifiers. We investigate three applications--embeddings, prompting, and fine-tuning--across three classifier types: traditional ML models (MLMs), pre-trained language models (PLMs), and large language models (LLMs). Using 356 Mozilla Firefox issues, we created a dataset to train and evaluate six MLMs, four PLMs, and two LLMs across 68 configurations. Results show that MLMs with LLM embeddings outperform TF-IDF features, prompting underperforms, and fine-tuned LLMs achieve the highest performance, with LLAMAft reaching 0.716 F1 score. Ensembles of the best models further improve results (0.737 F1). Misclassifications often arise from misleading clues or missing context, highlighting the need for context-aware classifiers. Models trained on Mozilla transfer to other projects, with a small amount of project-specific data, further enhancing results. This work supports software maintenance, issue understanding, and solution reuse.

cs.SE