arXiv subjects

Ruihang Zhang

Publications and source records attributed to Ruihang Zhang.

8 recordsLinked to original sources

arXiv2026

ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation

Tracking the six-degree-of-freedom (6-DoF) pose of objects and surfaces from monocular video is a long-standing problem in computer vision. To tackle this problem, existing methods require inputs beyond the video itself-such as 3D models, depth maps, object masks, or task-specific learned features-and they struggle with textureless, transparent, reflective, or deformable surfaces. Here, we introduce ProxyPose, which recasts 6-DoF pose tracking as video-to-video translation. Given only a video and a single marked pixel in the first frame, a fine-tuned video diffusion model translates the input into a proxy video-a synthetic video depicting a colored polyhedron undergoing the same local rigid-body motion as the surface region at the marked pixel. Because the proxy's geometry and appearance are known by construction, recovering its full 6-DoF trajectory reduces to classical pose estimation with off-the-shelf solvers. This formulation leverages large-scale video pre-training to absorb the hardest aspects of pose tracking-handling challenging materials, occlusions, and deformations-into the translation step, while operating at the pixel level with no assumptions about object identity, boundaries, or global rigidity. ProxyPose achieves state-of-the-art 6-DoF pose tracking accuracy without the additional inputs required by competing methods and after fine-tuning the video model only on synthetic data. We further demonstrate that ProxyPose extends to face tracking, camera pose estimation, and challenging in-the-wild scenes that are beyond the reach of existing approaches. Project page: https://ruihangzhang97.github.io/proxypose/.

cs.CV↗

arXiv2025

Optimizing Sensor Placement for Flow Reconstruction in Urban Drainage Networks: A Digital Twin-Based Sparse Sensing Approach

Urban flooding triggered by intense rainfall is becoming increasingly frequent and widespread. While flood prediction and monitoring in high spatio-temporal resolution are desired, practical constraints in time, budget, and technology hinder its full implementation. How to monitor urban drainage networks and predict flow conditions under constrained resources is a major challenge. To address this, we introduced a data-driven sparse sensing (DSS) approach, demonstrated via a digital-twin of the Woodland catchment in Duluth, Minnesota. Specifically, we coupled EPA-SWMM with singular value decomposition and QR factorization-based sensor selection to optimize monitoring locations for system-level flow reconstruction. An ensemble of SWMM simulations, driven by diverse scenarios, provided the necessary hydraulic data to extract the reduced basis and identify informative sensor locations. Cross-event validation showed that three strategically placed sensors among 77 candidate nodes achieved a mean system-level Nash-Sutcliffe efficiency (NSE) of 0.949 across observed storm events. The QR-selected sensor sets were benchmarked against reference sensor configurations obtained from exhaustive searches and Monte Carlo random-placements. This comparison further showed that flow reconstruction based on QR-selected sensors closely tracked the exhaustive optimum while substantially outperforming random placements. We further evaluated the framework's robustness by introducing multiplicative Gaussian noise and simulating individual sensor failures. While the model is relatively resilient to noise, the impact of sensor dropouts depends heavily on the number of sensors allocated and their specific locations.

cs.AI↗

arXiv2025

CanvasComposer: Personalized Group Photo Generation via a Multi-Reference Canvas

Existing personalized image generators still struggle to preserve multiple reference identities in natural and coherent multi-human generations. To address these limitations, we present CanvasComposer, an interactive framework for personalized group photo generation. Inspired by professional image-editing software, CanvasComposer allows users to place reference subjects on a shared canvas, where each subject keeps its own RGBA cutout of the input. This multi-reference canvas preserves reference content under overlap while providing an intuitive interface for organizing multiple identities; the subjects remain separate elements on the input canvas, and the model outputs a single personalized and harmonized image. To keep this representation efficient, transparent latent pruning retains only tokens from each subject's non-transparent region, and cross-reference training mitigates copy-paste artifacts by learning to harmonize references sampled from different images. Extensive experiments demonstrate that CanvasComposer achieves coherent generation and strong identity preservation compared to state-of-the-art methods in multi-human personalized image generation. Project page: https://snap-research.github.io/canvascomposer

cs.CV↗

arXiv2025

Can LLMs Correct Themselves? A Benchmark of Self-Correction in LLMs

Self-correction of large language models (LLMs) emerges as a critical component for enhancing their reasoning performance. Although various self-correction methods have been proposed, a comprehensive evaluation of these methods remains largely unexplored, and the question of whether LLMs can truly correct themselves is a matter of significant interest and concern. In this study, we introduce CorrectBench, a benchmark developed to evaluate the effectiveness of self-correction strategies, including intrinsic, external, and fine-tuned approaches, across three tasks: commonsense reasoning, mathematical reasoning, and code generation. Our findings reveal that: 1) Self-correction methods can improve accuracy, especially for complex reasoning tasks; 2) Mixing different self-correction strategies yields further improvements, though it reduces efficiency; 3) Reasoning LLMs (e.g., DeepSeek-R1) have limited optimization under additional self-correction methods and have high time costs. Interestingly, a comparatively simple chain-of-thought (CoT) baseline demonstrates competitive accuracy and efficiency. These results underscore the potential of self-correction to enhance LLM's reasoning performance while highlighting the ongoing challenge of improving their efficiency. Consequently, we advocate for further research focused on optimizing the balance between reasoning capabilities and operational efficiency. Project Page: https://correctbench.github.io/

cs.CL↗

arXiv2025

MVP4D: Multi-View Portrait Video Diffusion for Animatable 4D Avatars

Digital human avatars aim to simulate the dynamic appearance of humans in virtual environments, enabling immersive experiences across gaming, film, virtual reality, and more. However, the conventional process for creating and animating photorealistic human avatars is expensive and time-consuming, requiring large camera capture rigs and significant manual effort from professional 3D artists. With the advent of capable image and video generation models, recent methods enable automatic rendering of realistic animated avatars from a single casually captured reference image of a target subject. While these techniques significantly lower barriers to avatar creation and offer compelling realism, they lack constraints provided by multi-view information or an explicit 3D representation. So, image quality and realism degrade when rendered from viewpoints that deviate strongly from the reference image. Here, we build a video model that generates animatable multi-view videos of digital humans based on a single reference image and target expressions. Our model, MVP4D, is based on a state-of-the-art pre-trained video diffusion model and generates hundreds of frames simultaneously from viewpoints varying by up to 360 degrees around a target subject. We show how to distill the outputs of this model into a 4D avatar that can be rendered in real-time. Our approach significantly improves the realism, temporal consistency, and 3D consistency of generated avatars compared to previous methods.

cs.CV↗

arXiv2025

MMLU-Reason: Benchmarking Multi-Task Multi-modal Language Understanding and Reasoning

Recent advances in Multi-Modal Large Language Models (MLLMs) have enabled unified processing of language, vision, and structured inputs, opening the door to complex tasks such as logical deduction, spatial reasoning, and scientific analysis. Despite their promise, the reasoning capabilities of MLLMs, particularly those augmented with intermediate thinking traces (MLLMs-T), remain poorly understood and lack standardized evaluation benchmarks. Existing work focuses primarily on perception or final answer correctness, offering limited insight into how models reason or fail across modalities. To address this gap, we introduce the MMLU-Reason, a new benchmark designed to rigorously evaluate multi-modal reasoning with explicit thinking. The MMLU-Reason comprises 1) a high-difficulty dataset of 1,083 questions spanning six diverse reasoning types with symbolic depth and multi-hop demands and 2) a modular Reasoning Trace Evaluation Pipeline (RTEP) for assessing reasoning quality beyond accuracy through metrics like relevance, consistency, and structured error annotations. Empirical results show that MLLMs-T overall outperform non-thinking counterparts, but even top models like Claude-3.7-Sonnet and Gemini-2.5 Pro suffer from reasoning pathologies such as inconsistency and overthinking. This benchmark reveals persistent gaps between accuracy and reasoning quality and provides an actionable evaluation pipeline for future model development. Overall, the MMLU-Reason offers a scalable foundation for evaluating, comparing, and improving the next generation of multi-modal reasoning systems.

cs.AI↗

arXiv2024

CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models

Reconstructing photorealistic and dynamic portrait avatars from images is essential to many applications including advertising, visual effects, and virtual reality. Depending on the application, avatar reconstruction involves different capture setups and constraints $-$ for example, visual effects studios use camera arrays to capture hundreds of reference images, while content creators may seek to animate a single portrait image downloaded from the internet. As such, there is a large and heterogeneous ecosystem of methods for avatar reconstruction. Techniques based on multi-view stereo or neural rendering achieve the highest quality results, but require hundreds of reference images. Recent generative models produce convincing avatars from a single reference image, but visual fidelity yet lags behind multi-view techniques. Here, we present CAP4D: an approach that uses a morphable multi-view diffusion model to reconstruct photoreal 4D (dynamic 3D) portrait avatars from any number of reference images (i.e., one to 100) and animate and render them in real time. Our approach demonstrates state-of-the-art performance for single-, few-, and multi-image 4D portrait avatar reconstruction, and takes steps to bridge the gap in visual fidelity between single-image and multi-view reconstruction techniques.

cs.CV↗

arXiv2023

Resource Optimization Using A Step-by-step Scheme in Wireless Sensing and Localization Networks

Due to the lack of wireless spectrum resources, people are focusing on the versatile wireless networks. Wireless localization and target sensing both rely on precise extraction of parameters such as signal amplitude, propagation delay and Doppler shift from the received signals. Due to the high multi-path resolution and strong penetration of UWB signals, both localization and sensing can be achieved through the same UWB waveform. Practical networks are often resource-constrained, in order to improve the accuracy of integrated networks, we need to optimize the allocation of resources in the networks. Considering the complexity of the multi-slot networks, this paper derives the Fisher Information Matrix (FIM) expressions for single-slot and dual-slot integrated sensing and localization (ISAL) networks respectively, and proposes two resource optimization schemes, namely step-by-step scheme and integrated scheme. The numerical results show that: (i) for the sensing-resource-deficient networks with relatively uniform node distribution, the energy allocated to each step in the step-by-step scheme satisfies the relationship: energy for clock offset < energy for radar localization < energy for target sensing. (ii) In the multi-slot ISAL networks, the system will allocate more energy to the time slots where the networks are relatively sensing-resource-deficient. (iii) The step-by-step scheme is more suitable for the sensing-resource-abundant networks, while the integrated scheme is more suitable for the sensing-resource-deficient networks.

cs.IT↗