arXiv subjects

Kazuki Kusama

Publications and source records attributed to Kazuki Kusama.

4 recordsLinked to original sources

arXiv2026

ThinkLog: Leveraging Reasoning for Log Statement Generation

Runtime logs are an important source of information that supports software maintenance. To obtain useful logs, developers spend significant effort identifying appropriate log locations, assigning correct severity levels, and writing concise yet informative messages. Therefore, end-to-end automated log statement generation can help reduce this burden, and prior work has proposed many methods for this task. However, existing methods still exhibit limited accuracy. To address this problem, we propose ThinkLog, an LLM-based end-to-end log statement generation method. The core idea of ThinkLog is to incorporate reasoning that helps LLMs make decisions about log insertion, severity level assignment, and message generation, thereby improving log statement generation accuracy. ThinkLog injects reasoning into prompts as few-shot examples and guides LLMs to generate appropriate log statements. Evaluated on 9,619 Java methods extracted from public GitHub repositories, ThinkLog achieves 20.55% log statement generation accuracy, representing a 15.4% improvement over the best existing method. Moreover, these improvements were achieved at approximately 50% of the inference cost (USD) compared to the best existing method. These results show that leveraging reasoning is an effective and cost-efficient way to improve the accuracy of end-to-end log statement generation.

cs.SE↗

arXiv2026

Leveraging Language Models for Log Statement Generation in Multilingual Scenarios: How Far Are We?

Log statements capture critical information for software maintenance activities such as testing, debugging, and failure analysis. Because of this importance, developers must carefully design log statements, which requires significant effort. To support developers, various end-to-end automated log statement generation approaches have been proposed, whereas these approaches have mainly been evaluated within a single programming language environment and their effectiveness in multilingual environments remains underexplored. In this paper, we therefore comparatively evaluate three state-of-the-art log statement generation approaches and five large language models (LLMs) across multiple programming languages. For this purpose, we constructed a multilingual benchmark comprising 150,000 instances across five programming languages. Our empirical results demonstrate that UniLog, a state-of-the-art approach, achieves the best overall performance, maintaining high effectiveness even in multilingual environments. We also observe substantial variance in the difficulty of log generation across languages: Python presents a greater challenge, whereas JavaScript yields comparatively better performance. Detailed analysis reveals that these disparities stem from variations in log insertion distributions and language-specific logging idioms. Our findings indicate that simply scaling model size or the volume of training data is insufficient for multilingual log generation; rather, designing approaches tailored to the specific characteristics of target languages is crucial. These findings suggest that future automated logging techniques should explicitly account for language-specific logging characteristics to achieve robust performance in multilingual software development environments.

cs.SE↗

arXiv2026

Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study

AI coding agents increasingly submit pull requests (Agentic-PRs) to open-source repositories, yet their performance is commonly assessed using merge and rejection outcomes alone. We hypothesized that these outcome labels do not reliably reflect agent capability without considering review interactions. To test this, we conducted a decision-oriented analysis of 11,048 closed Agentic Pull Requests, refined to 9,799 human-reviewed PRs, and manually inspected 717 representative cases to recover decision rationale from interaction artifacts. We found that rejection outcomes substantially overstate agent error: only 35.7% of rejected PRs reflected clear agentic failures, while 31.2% were driven by workflow constraints and 33.1% lacked observable decision rationale. Among merged PRs, 15.4% required explicit reviewer involvement through feedback or direct commits, and 5.5% showed no visible interaction trace. We further observed systematic differences across agents, with Copilot and Devin more often embedded in reviewer-mediated workflows, while Codex and Cursor PRs were typically merged with minimal interaction. These results reject the assumption that PR outcomes alone capture agent performance and demonstrate the need for interaction-aware evaluation grounded in review behavior.

cs.SE↗

arXiv2025

How Small is Enough? Empirical Evidence of Quantized Small Language Models for Automated Program Repair

Background: Large language models (LLMs) have greatly improved the accuracy of automated program repair (APR) methods. However, LLMs are constrained by high computational resource requirements. Aims: We focus on small language models (SLMs), which perform well even with limited computational resources compared to LLMs. We aim to evaluate whether SLMs can achieve competitive performance in APR tasks. Method: We conducted experiments on the QuixBugs benchmark to compare the bug-fixing accuracy of SLMs and LLMs. We also analyzed the impact of int8 quantization on APR performance. Results: The latest SLMs can fix bugs as accurately as--or even more accurately than--LLMs. Also, int8 quantization had minimal effect on APR accuracy while significantly reducing memory requirements. Conclusions: SLMs present a viable alternative to LLMs for APR, offering competitive accuracy with lower computational costs, and quantization can further enhance their efficiency without compromising effectiveness.

cs.SE↗