arXiv subjects

Weitao Song

Publications and source records attributed to Weitao Song.

5 recordsLinked to original sources

arXiv2026

HOG-Layout: Hierarchical 3D Scene Generation, Optimization and Editing via Vision-Language Models

3D layout generation and editing play a crucial role in Embodied AI and immersive VR interaction. However, manual creation requires tedious labor, while data-driven generation often lacks diversity. The emergence of large models introduces new possibilities for 3D scene synthesis. We present HOG-Layout that enables text-driven hierarchical scene generation, optimization and real-time scene editing with large language models (LLMs) and vision-language models (VLMs). HOG-Layout improves scene semantic consistency and plausibility through retrieval-augmented generation (RAG) technology, incorporates an optimization module to enhance physical consistency, and adopts a hierarchical representation to enhance inference and optimization, achieving real-time editing. Experimental results demonstrate that HOG-Layout produces more reasonable environments compared with existing baselines, while supporting fast and intuitive scene editing.

cs.CV↗

arXiv2025

Towards spatial computing: recent advances in multimodal natural interaction for XR headsets

With the widespread adoption of Extended Reality (XR) headsets, spatial computing technologies are gaining increasing attention. Spatial computing enables interaction with virtual elements through natural input methods such as eye tracking, hand gestures, and voice commands, thus placing natural human-computer interaction at its core. While previous surveys have reviewed conventional XR interaction techniques, recent advancements in natural interaction, particularly driven by artificial intelligence (AI) and large language models (LLMs), have introduced new paradigms and technologies. In this paper, we review research on multimodal natural interaction for wearable XR, focusing on papers published between 2022 and 2024 in six top venues: ACM CHI, UIST, IMWUT (Ubicomp), IEEE VR, ISMAR, and TVCG. We classify and analyze these studies based on application scenarios, operation types, and interaction modalities. This analysis provides a structured framework for understanding how researchers are designing advanced natural interaction techniques in XR. Based on these findings, we discuss the challenges in natural interaction techniques and suggest potential directions for future research. This review provides valuable insights for researchers aiming to design natural and efficient interaction systems for XR, ultimately contributing to the advancement of spatial computing.

cs.HC↗

arXiv2025

A Physiologically-based simulation model of color appearance for red-green color vision deficiency

Various simulation methods of color appearance for dichromats or anomalous trichromats have been proposed over the years. To further improve the performance of the simulation model and extend the application range to both dichromats or anomalous trichromats, we have proposed a simulation model of cone fundamentals specifically designed for individuals with red-green type color vision deficiency (CVD) based on the CIE 2006 physiological observer model. By utilizing the simulated cone fundamentals, it becomes possible to predict the color appearance of real scenes and digital images for CVD. The fundamental premise of the new model is rooted in the hypothesis that CVD arises from a shift in the peak wavelength of the photopigment absorption spectrum of the L or M cone. Instead of simply maintaining the waveform without alteration as observed in prior studies, we altered waveforms of the absorption spectra of anomalous L/M cone photopigments when adjusting their peak wavelengths. Regarding different shapes in the absorption spectrum between the L and M cone, the absorption spectrum of the anomalous L/M cone was obtained by combining the peak wavenumber shift and linear interpolation of spectral quantal absorption curves between L- and M-photopigments in the wavenumber domain. The performance of the proposed model was substantiated through experimental validation by the pseudoisochromatic plates and Farnsworth Munsell 100 Hue test (FM-100). The findings revealed a high level of consistency between the model prediction and the actual perception reported by individuals with CVD.

physics.optics↗

arXiv2024

Color-Name Aware Optimization to Enhance the Perception of Transparent Overlapped Charts

Transparency is commonly utilized in visualizations to overlay color-coded histograms or sets, thereby facilitating the visual comparison of categorical data. However, these charts often suffer from significant overlap between objects, resulting in substantial color interactions. Existing color blending models struggle in these scenarios, frequently leading to ambiguous color mappings and the introduction of false colors. To address these challenges, we propose an automated approach for generating optimal color encodings to enhance the perception of translucent charts. Our method harnesses color nameability to maximize the association between composite colors and their respective class labels. We introduce a color-name aware (CNA) optimization framework that generates maximally coherent color assignments and transparency settings while ensuring perceptual discriminability for all segments in the visualization. We demonstrate the effectiveness of our technique through crowdsourced experiments with composite histograms, showing how our technique can significantly outperform both standard and visualization-specific color blending models. Furthermore, we illustrate how our approach can be generalized to other visualizations, including parallel coordinates and Venn diagrams. We provide an open-source implementation of our technique as a web-based tool.

cs.GR↗

arXiv2020

Large-scale Huygens metasurfaces for holographic 3D near-eye displays

Novel display technologies aim at providing the users with increasingly immersive experiences. In this regard, it is a long-sought dream to generate three-dimensional (3D) scenes with high resolution and continuous depth, which can be overlaid with the real world. Current attempts to do so, however, fail in providing either truly 3D information, or a large viewing area and angle, strongly limiting the user immersion. Here, we report a proof-of-concept solution for this problem, and realize a compact holographic 3D near-eye display with a large exit pupil of 10mm x 8.66mm. The 3D image is generated from a highly transparent Huygens metasurface hologram with large (>10^8) pixel count and subwavelength pixels, fabricated via deep-ultraviolet immersion photolithography on 300 mm glass wafers. We experimentally demonstrate high quality virtual 3D scenes with ~50k active data points and continuous depth ranging from 0.5m to 2m, overlaid with the real world and easily viewed by naked eye. To do so, we introduce a new design method for holographic near-eye displays that, inherently, is able to provide both parallax and accommodation cues, fundamentally solving the vergence-accommodation conflict that exists in current commercial 3D displays.

physics.optics↗