SWE-PolyVision: Benchmarking Cross-Image Abductive Reasoning for Repository-Level Software Engineering

Jiajun Wu; Leixin Sun; Zihan Tan; Yitao Liu; Zhenjun Zhao; Danyang Zhang; Shuo Li; Jiaru Qian; Yuxin Wu; Shanghaoran Quan; Chuangxin Zhao; Yangxu Liao; Yang Liu; Bin Chong; Guancheng Wan

arXiv · 2609.29754

SWE-PolyVision: Benchmarking Cross-Image Abductive Reasoning for Repository-Level Software Engineering

Jiajun Wu·Leixin Sun·Zihan Tan·Yitao Liu·Zhenjun Zhao·Danyang Zhang·Shuo Li·Jiaru Qian·Yuxin Wu·Shanghaoran Quan·Chuangxin Zhao·Yangxu Liao·Yang Liu·Bin Chong·Guancheng Wan

View original source ↗Read source document ↗Export metadata ↓

Abstract

Current multimodal software-engineering benchmarks expose images as additional context, but do not test whether an agent can integrate evidence distributed across images into a verified repository-level repair. We present SWE-PolyVision, an executable benchmark of 92 real tasks from 36 open-source organizations, with 48 public tasks and 44 private holdouts. The release contains 402 static images and 6 videos, with at least two visual inputs per task. Each task pairs a fixed pre-fix repository with an isolated verifier and is evaluated under the supported conditions among three access modes: Text-only, Native Vision, and Tool-mediated Vision. Across eleven coding models, visual access changes which tasks are solved, but effects depend on both model and task. Two trace-linked Native Vision cases illustrate how complementary visual and textual clues can lead to source-localized, verified repairs; controlled interventions show that this conversion is not yet stable across inputs. SWE-PolyVision thus separates the availability of multi-image evidence from its successful use in repository-level repair, without treating patch success alone as proof of explicit reasoning.

Explore related subjects

cs.SE

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Jiajun Wu, Leixin Sun, Zihan Tan, Yitao Liu, Zhenjun Zhao, Danyang Zhang, Shuo Li, Jiaru Qian, Yuxin Wu, Shanghaoran Quan, Chuangxin Zhao, Yangxu Liao, Yang Liu, Bin Chong, Guancheng Wan. 2026-10-15. SWE-PolyVision: Benchmarking Cross-Image Abductive Reasoning for Repository-Level Software Engineering. https://arxiv.org/abs/2609.29754

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Psychological Safety Framework in Pull-based Open Source Projects

Psychological safety (PS) refers to the belief that team members can speak up, ask questions, and make mistakes without fear of negative consequences. Although PS has been studied in traditional software teams, less is known about how it may appear in pull-based open-source software development, where contributors are self-directed and often collaborate voluntarily. This paper introduces a theory-informed framework for understanding how PS may be reflected in pull request (PR) interactions. We investigate the framework through a mixed-methods approach combining repository-based analysis and a survey of OSS contributors. First, we constructed a PS index using nine observable PR interaction indicators extracted from 60,685 PRs across 26 GitHub repositories. Second, we surveyed 57 OSS contributors about their perceived PS and experiences of corresponding PR interactions. All nine surveyed interaction indicators were significantly and positively associated with perceived PS. However, the repository-based index was not significantly associated with survey-reported PS. Contributors' open-ended responses further highlighted the importance of feedback quality, opportunities to participate, and how PR outcomes are interpreted. Together, our framework and mixed-methods findings provide initial insights into how PS may be reflected in PR interactions, highlighting the importance of both PR interaction patterns and contributors' experiences.

cs.SE↗

arXiv2026

Structuring license permissiveness from pairwise comparisons

Licenses are legal instruments that inventors rely upon to protect the technologies they build and regulate how they are used---however, the nature of their authorship and selection implies that how they are interpreted, chosen, and enforced is largely unstructured. In practice, this makes it difficult to compare licenses at scale---when is one license considered more permissive than the other, and when are their terms incomparable to each other? Currently, there is a growing list of licenses that are introduced and used, yet no systematic way to study their relationships. This matters for platforms such as Hugging Face, GitHub, and the Python Package Index, where developers publish or build upon technologies that each have their own licenses. Using large language models (LLMs), we introduce methods for comparing licenses at scale: first, in a pairwise fashion to construct and validate a partial ordering based on permissiveness; and by drawing on existing taxonomies of software licenses. Then, we try to recover the structure with the Bradley-Terry model to see if permissiveness can be judged more cheaply and observe a loss of $\sim$20\%---and classify this loss to feature coverage. The former coupled with model rationale allows us to trace restrictiveness, and the latter allows us to understand license selection as a combination of shared provisions.

cs.SE↗

arXiv2026

Repository and Issue Characteristics of Early GenAI Adopters on GitHub

Generative artificial intelligence (GenAI) has introduced new forms of AI assistance into software development, but less is known about the repositories and issues of early adopters of GenAI tools. We study the repository and issue characteristics of early adopters who state interest in GenAI on their public profiles on GitHub by analyzing 531 early adopters, 100 of their repositories with AI-tool configuration files (ACFs), 100 matched control repositories without ACFs, and 370 issues from the two repository groups. Repositories with ACFs show modest distributional differences in selected README structural characteristics. They contain longer README files with more code blocks and include a license more often, while repositories without ACFs contain more external URLs. The category distributions of the two repository groups show weak evidence of a difference. Once GenAI relatedness is held fixed, we detect no difference in issue Type or Outcome between the two repository groups. Our data, collected from May to August 2025, record how early adopters developed software in Software Engineering 2.0, the stage in which developers use both AI assistants and autonomous AI agents. When these agents become the norm, later studies can compare their data with ours.

cs.SE↗