OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use

Hanzhang Zhou; Long Li; Quyu Kong; Lei Wang; Panrong Tong; Long Chen; Tianyu Xia; Gongjie Zhang; Xu Zhang; Chen Liu; Jianan Zhang; Chenglin Cai; Zhengwen Zeng; Steven Hoi; Yue Wang

arXiv · 2610.22200

OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use

Hanzhang Zhou·Long Li·Quyu Kong·Lei Wang·Panrong Tong·Long Chen·Tianyu Xia·Gongjie Zhang·Xu Zhang·Chen Liu·Jianan Zhang·Chenglin Cai·Zhengwen Zeng·Steven Hoi·Yue Wang

View original source ↗Read source document ↗Export metadata ↓

Abstract

Computer use has reached a turning point: the focus is expanding from GUI-only interaction to systems that complete tasks on users' computers through visual and programmatic interfaces. We define a generalist computer-use agent (GCUA) as a system in which the model chooses how to observe, act, and organize its work across visual and programmatic interfaces. Yet frontier implementations, such as GPT-6 Astra's computer-use system, remain largely closed, limiting systematic study of their design. We introduce Open Generalist CUA (OpenGCUA), an ongoing project to develop and open the full stack for GCUAs with the community. This report presents the first release of OpenGCUA. Code execution is a key feature of our agent design: it lets intermediate results determine subsequent operations and enables models to apply their existing programming capabilities directly to computer interaction. We also propose harness-to-model decision transfer, moving selected observations and task-state decisions from fixed harness rules to the model. On OSWorld-v2, our full configuration improves task scores across all eight evaluated models, including GPT-6 Astra and Claude Opus 5, raising mean partial score by +19.6 points and reducing reported steps per task by an average of 53.0% relative to batched GUI-only execution. We release our implementation and prompts, evaluation trajectories from all eight models, and preliminary versions of a real-device harness and a cross-device SDK. The harness addresses real-device challenges not fully captured by benchmarks, while the SDK provides a common programming interface across computers, mobiles and browsers. Across four real macOS tasks under our real-device harness, selected Qwen3.8-Max runs use 19.4 times as many reported reasoning tokens as GPT-6 Astra, highlighting substantial differences between models in real-device execution efficiency.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Hanzhang Zhou, Long Li, Quyu Kong, Lei Wang, Panrong Tong, Long Chen, Tianyu Xia, Gongjie Zhang, Xu Zhang, Chen Liu, Jianan Zhang, Chenglin Cai, Zhengwen Zeng, Steven Hoi, Yue Wang. 2026-10-15. OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use. https://arxiv.org/abs/2610.22200

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Text-to-Image Synthesis: A Decade Survey

Text-to-image synthesis (T2I) focuses on generating high-quality images from textual descriptions In this survey, we review 490 papers on T2I spanning the past decade, with representative advances covered through 2026. We start by briefly introducing how GANs, autoregressive models, and diffusion models have been used for image generation, and we additionally examine the shared components whose evolution has driven recent progress. Building on this foundation, we discuss the development of these models for T2I and organize cutting-edge research by problem rather than by chronology alone, covering controllability, personalized and identity-preserving generation, instruction-based image editing, preference alignment and post-training with human and AI feedback, efficiency, safety, and consistency in content and spatial relationships. We further review unified multimodal understanding-and-generation models, summarize the datasets and evaluation metrics commonly used in T2I research, and provide a comparative discussion of the three paradigms together with their respective trade-offs. Finally, we discuss the potential applications of T2I within AIGC, along with the challenges and forward-looking research opportunities in this field.

cs.CV↗

arXiv2026

Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Tracking

Robust 3D state estimation in multi-object tracking is crucial for object localization in self-driving vehicles. Misalignment between the Kalman filter (KF) motion model and an object's motion can cause state drift, particularly in distant and occlusion cases. The static motion model assumption in the conventional KF is widely used in 3D MOT due to its low latency; however, its estimation accuracy degrades as the object's motion disaligns with the chosen model due to traffic. Although recent work introduces a multimodel KF that runs multiple motion models simultaneously to improve state robustness, it incurs significant computational overhead, so many studies still favor the constant-motion KF to maintain efficiency. This work addresses this trade-off by introducing a motion dynamics Kalman filter (MD-KF) that accounts for motion variation through temporal modeling of object motion and adaptively scales the contribution of the motion components in the motion model to correct state-prediction drift caused by this variation, while maintaining computational efficiency by retaining a single-model structure. Evaluations on KITTI and the Waymo Open Dataset, complemented by vehicle experiments, show that MD-KF achieves tracking performance comparable to multimodel KF with an average 1.72x speedup, while outperforming it in handling highly distant and prolonged occluded objects.

cs.CV↗

arXiv2026

RUSplatting: Robust 3D Gaussian Splatting for Sparse-View Underwater Scene Reconstruction

Reconstructing high-fidelity underwater scenes remains a challenging task due to light absorption, scattering, and limited visibility inherent in aquatic environments. This paper presents an enhanced Gaussian Splatting-based framework that improves both the visual quality and geometric accuracy of deep underwater rendering. We propose decoupled learning for RGB channels, guided by the physics of underwater attenuation, to enable more accurate colour restoration. To address sparse-view limitations and improve view consistency, we introduce a frame interpolation strategy with a novel adaptive weighting scheme. Additionally, we introduce a new loss function aimed at reducing noise while preserving edges, which is essential for deep-sea content. We also release a newly collected dataset, Submerged3D, captured specifically in deep-sea environments. Experimental results demonstrate that our framework consistently outperforms state-of-the-art methods with PSNR gains up to 1.90dB, delivering superior perceptual quality and robustness, and offering promising directions for marine robotics and underwater visual analytics. The code of RUSplatting is available at https://github.com/theflash987/RUSplatting and the dataset Submerged3D can be downloaded at https://zenodo.org/records/15482420.

cs.CV↗