arXiv · 2611.12083
SWE-Time: Benchmarking Temporal Repository Understanding
Abstract
Repository-level code-understanding benchmarks have progressively expanded the spatial scope of reasoning, from local code to entire repositories, while largely treating each repository as a static snapshot. However, how models reason about repository evolution over time remains largely unexplored. We introduce SWE-Time, a benchmark for temporal repository understanding that makes repository evolution itself the reasoning target. SWE-Time evaluates two complementary temporal directions: retrospective understanding of repository evolution through historical regression analysis and prospective understanding of intended changes from pre-change repository states. To enable realistic yet reliable evaluation in these settings, we introduce the RealWorld-to-Verifiable Construction Framework, which transforms real-world developer queries and historical evidence into verifiable evaluation tasks. We evaluate 15 coding agents on 1,661 cases from 65 GitHub repositories. Within Claude Code, Claude Opus 4.8 achieves the highest SWE-Time Avg. of 61.9, leaving substantial headroom. Controlled analyses highlight the importance of historical code evidence and developer intent, while consistent reasoning across repository states remains challenging. These results highlight repository evolution as a distinct and challenging dimension of code understanding.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Shijun Xiao, Xinrong Chen, Hao Liu, Wensheng Zhang, Yongqiang Tang. 2026-11-09. SWE-Time: Benchmarking Temporal Repository Understanding. https://searcharxiv.com/abs/2611.12083
Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Searcharxiv by Indiciae: BibTeX · RIS.