Searcharxiv⌕ Search

arXiv subjects

Mehedi Hasan Shanto

Publications and source records attributed to Mehedi Hasan Shanto.

4 recordsLinked to original sources

Understanding Suggested Edits in Stack Overflow: A Large-Scale Empirical Study and Review Prioritization Framework

Suggested edits allow low-reputation users to improve Stack Overflow posts and contribute to the platform's knowledge base. However, we still know little about why these edits are accepted or rejected and how reviewers can efficiently identify problematic edits. In this study, we analyze 3,888,805 suggested edits submitted between 2011 and May 2023. First, we compare accepted and rejected edits using 35 features. We find significant differences between them, with the largest differences related to users' editing history. Second, we manually analyze rejected edits and identify 17 fine-grained rejection reasons. These reasons provide a more detailed understanding of why edits are rejected than Stack Overflow's predefined rejection reasons. Third, we evaluate three models for prioritizing edits based on their likelihood of rejection. The Feature Model, built using the 35 studied features, provides the best trade-off between review rate and rejected-edit recall. At a decision threshold of $τ= 0.40$, it prioritizes 52.3% of edits for human review while capturing 80.8% of rejected edits. In comparison, the Embedding Model captures 78.8% of rejected edits while prioritizing 60.9% of edits, and the EditEx Model captures 72.5% while prioritizing 59.0%. We also conduct a complementary zero-shot evaluation on a separate set of 500 edits. The Feature Model achieves 78.0% recall at $τ= 0.40$, compared with 23.6%-31.2% for GPT-4o, Qwen2.5-32B, and Llama-3.3-70B. Overall, our findings provide a detailed understanding of suggested-edit rejection and show that rejection-risk prediction can help prioritize potentially problematic edits for human review.

cs.SE↗

Toward an Evidence-Based Template for Agent Instruction Files: An Empirical Study of Adoption, Consistency, and Maintenance

AI coding agents rely on repository-level instruction files to understand project conventions, development workflows, and constraints. However, different agents support different instruction formats, and limited empirical evidence exists about how these files are adopted, what information they contain, whether coexisting formats provide consistent guidance, and how they are maintained. We conduct an empirical study of 3,387 instruction files from 1,000 GitHub repositories across four formats: AGENTS.md, CLAUDE.md, GitHub Copilot instructions, and Cursor rules. We find that 42.2% of instruction-bearing repositories in the higher-popularity cohort maintain multiple formats. Among 386 manually examined pairs of corresponding sections, 16.1% provide divergent guidance. Our maintenance analysis further shows that 40.4% of the analyzed files have only one recorded modification event and that maintenance patterns differ across formats. Based on category prevalence and category addition ratios (CAR), we derive an evidence-based template containing ten Required, four Gap, and two Recommended categories. In an exploratory generation experiment, explicit template guidance increases structural category coverage from 18.7% to 82.8%. We further introduce AGIT-Instruct, a benchmark for evaluating instruction-file generation through structural coverage and content alignment. Across four generation configurations, retrieval-augmented generation achieves the highest full-file ROUGE-1 F1 of 0.459. A manual assessment of 278 generated category instances finds that 19.4% fully preserve and 61.2% partially preserve developer-written intent. Our findings provide empirical guidance for organizing and maintaining agent instruction

cs.SE↗

RAG-Reflect: Agentic Retrieval-Augmented Generation with Reflections for Comment-Driven Code Maintenance on Stack Overflow

User comments on online programming platforms such as Stack Overflow play a vital role in maintaining the correctness and relevance of shared code examples. However, the majority of comments express gratitude or clarification, while only a small fraction highlight actionable issues that drive meaningful edits. This paper demonstrates how agentic AI principles can revolutionize software maintenance tasks by presenting RAG-Reflect, a modular framework that achieves fine-tuned-level performance for valid comment-edit prediction without task-specific training. Valid Comment-Edit Prediction (VCP) is the task of determining whether a user comment directly triggered a subsequent code edit. The framework integrates large language models (LLMs) with retrieval-augmented reasoning and self-reflection mechanisms. RAG-Reflect operates through a three-stage runtime workflow built on a one-time pattern analysis phase. During initialization, an Interpretation module analyzes the knowledge base to generate validation rules. At inference time, the system (1) retrieves contextual examples, (2) reasons about comment-edit causality, and (3) reflects on decisions using the pre-established rules. We evaluate RAG-Reflect on the publicly available SOUP benchmark, achieving Precision = 0.81, Recall = 0.74, and F1 = 0.78, outperforming traditional baselines (e.g., Logistic Regression, XGBoost, different prompting techniques) and closely approaching the performance of fine-tuned models (F1 = 0.773) without retraining. Our ablation and stage-level analyses show that both retrieval and reflection modules substantially enhance performance.

cs.SE↗

Dependency Dilemmas: A Comparative Study of Independent and Dependent Artifacts in Maven Central Ecosystem

The Maven Central ecosystem forms the backbone of Java dependency management, hosting artifacts that vary significantly in their adoption, security, and ecosystem roles. Artifact reuse is fundamental in software development, with ecosystems like Maven Central facilitating this process. However, prior studies predominantly analyzed popular artifacts with numerous dependencies, leaving those without incoming dependencies (independent artifacts) unexplored. In this study, we analyzed 658,078 artifacts, of which 635,003 had at least one release. Among these, 93,101 artifacts (15.4%) were identified as independent (in-degree = 0), while the rest were classified as dependent. We looked at the impact of separate artifacts using PageRank and out-degree centrality and discovered that they were very important to the ecosystem. Further analysis across 18 different metrics revealed several advantages and comparability of independent artifacts with dependent artifacts: comparable popularity (25.58 vs. 7.30), fewer vulnerabilities (60 CVEs vs. 179 CVEs), and zero propagated vulnerabilities. Based on these results, it seems that independent artifacts make a big difference in the ecosystem and give developers a safe, self-contained alternative to traditional dependencies. These findings suggest that independent artifacts might be a beneficial choice for dependencies but have some maintainability issues. Therefore, developers should carefully incorporate independent artifacts into their projects, and artifact maintainers should prioritize this group of artifacts to mitigate the risk of transitive vulnerability propagation and improve software sustainability.

cs.SE↗