arXiv subjects

Yueyi Sun

Publications and source records attributed to Yueyi Sun.

9 recordsLinked to original sources

arXiv2026

PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models

Multimodal large language models (MLLMs) have achieved remarkable progress in visual understanding tasks. However, most existing MLLMs rely on autoregressive generation, which limits their efficiency for perception tasks that require captioning multiple regions. In this work, we propose PerceptionDLM, a multimodal diffusion language model optimized for efficient parallel region perception. Built upon PerceptionDLM-Base, a strong foundational baseline that achieves state-of-the-art performance among open-source diffusion MLLMs, our architecture fully leverages the parallel decoding nature of DLMs. Specifically, we introduce efficient prompting and structured attention masking to enable simultaneous perception of multiple masked regions, allowing the model to generate region descriptions in parallel at both the sequence and token levels. This design significantly improves inference efficiency compared with existing approaches that process regions sequentially. To systematically evaluate the parallelism property of visual perception capability for DLMs, we construct a new Parallel Detailed Localized Captioning Benchmark (ParaDLC-Bench) by scaling the DLC-Bench to include multiple region masks per image, enabling joint evaluation of both caption quality and inference efficiency. Experiments demonstrate that PerceptionDLM maintains competitive performance in region captioning while achieving substantial speed improvements for multi-region perception tasks. Our results highlight the potential of multimodal diffusion language models for efficient, parallel visual perception. To the best of our knowledge, we are the first to achieve parallel region caption and perception by leveraging the advantages of diffusion language models. Code, models, and datasets are released.

cs.CV↗

arXiv2025

Visual Reasoning Tracer: Object-Level Grounded Reasoning Benchmark

Recent advances in Multimodal Large Language Models (MLLMs) have significantly improved performance on tasks such as visual grounding and visual question answering. However, the reasoning processes of these models remain largely opaque; they typically output only final predictions without revealing the intermediate steps or fine-grained evidence (e.g., pixels, locations) that lead to the result. This contrasts with human intelligence, which naturally operates through a chain of visual reasoning. To address this limitation, we introduce the Visual Reasoning Tracer (VRT) task, which requires models to not only localize the target object but also explicitly predict the intermediate objects that form the reasoning path. To advance research in this area, we contribute: (1) VRT-Bench, a human-annotated benchmark for evaluating visual reasoning; (2) a new metric for assessing the quality of reasoning traces; and (3) VRT-80k, a large-scale dataset for reasoning model training. Our experiments reveal that while existing models often produce the correct final output, they struggle to ground their intermediate reasoning. In contrast, models trained on VRT-80k achieve substantial improvements in tracing the reasoning path.

cs.CV↗

arXiv2025

Contactless Modulation of Intralayer and Interlayer Excitons in MoS2/WSe2 heterostructures with Acoustoelectric Fields

This work presents a platform that enables surface acoustic wave (SAW) modulation of both intralayer and interlayer excitons in MoS2/WSe2 heterostructures. Harnessing the coupled piezoelectric and strain fields of SAWs, this integrated approach allows for dynamic, precise, and fully contactless control of excitonic properties, a capability essential for the realization of next generation optoelectronic, quantum photonic, and excitonic devices. We identify two distinct modulable interlayer excitons in optical communication bands: IX$_{K\Gamma}$ in the O band (around 1300 nm) and IX$_{K\!-\!K}$ in the S band (around 1500 nm); these two excitons display a robust twist-angle-independent energy splitting of 120 meV, in agreement with density functional theory (DFT) calculations. The type-II band alignment induced by the SAW not only promotes efficient exciton dissociation but also enables direct and tunable modulation of photoluminescence via the formation of confined piezoelectric potential wells. Furthermore, by simultaneously generating in-plane and out-of-plane SAW fields, the platform achieves selective manipulation of intralayer and interlayer excitons, inducing quadratic Stark effects for intralayer excitons and linear Stark effects for interlayer excitons. These findings provide new insights into SAWexciton interactions in van der Waals heterostructures, broaden the operational spectral range, and establish pathways toward on-chip acousto-optic and quantum optoelectronic devices with advanced excitonic functionality.

physics.optics↗

arXiv2025

DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World

Multimodal Large Language Models (MLLMs) demonstrate a complex understanding of scenes, benefiting from large-scale and high-quality datasets. Most existing caption datasets lack the ground locations and relations for visual entities. Several grounded caption datasets face the problems of missing detailed descriptions, relations, and massive object descriptions on high-resolution images. To fill this gap for the community, we present DenseWorld-1M, the first massive, detailed, dense grounded caption dataset in the real world. We design a three-stage labeling pipeline, containing open-world perception, detailed object caption generation, and dense caption merging. The first stage obtains entity-level masks and labels. The second stage generates the object-level, detailed captions with the guidance of masks and labels from the first stage. The final stage merges object captions and masks into spatial and relational dense captions. To accelerate the labeling process and improve caption quality, we present two VLM models: the Detailed Region Caption model and the Spatial Caption Merging model. Extensive experiments on various settings, including vision-language understanding, visual grounding, and region caption generation, demonstrate the effectiveness of our DenseWorld-1M dataset and labeling models.

cs.CV↗

arXiv2025

Mixed-R1: Unified Reward Perspective For Reasoning Capability in Multimodal Large Language Models

Recent works on large language models (LLMs) have successfully demonstrated the emergence of reasoning capabilities via reinforcement learning (RL). Although recent efforts leverage group relative policy optimization (GRPO) for MLLMs post-training, they constantly explore one specific aspect, such as grounding tasks, math problems, or chart analysis. There are no works that can leverage multi-source MLLM tasks for stable reinforcement learning. In this work, we present a unified perspective to solve this problem. We present Mixed-R1, a unified yet straightforward framework that contains a mixed reward function design (Mixed-Reward) and a mixed post-training dataset (Mixed-45K). We first design a data engine to select high-quality examples to build the Mixed-45K post-training dataset. Then, we present a Mixed-Reward design, which contains various reward functions for various MLLM tasks. In particular, it has four different reward functions: matching reward for binary answer or multiple-choice problems, chart reward for chart-aware datasets, IoU reward for grounding problems, and open-ended reward for long-form text responses such as caption datasets. To handle the various long-form text content, we propose a new open-ended reward named Bidirectional Max-Average Similarity (BMAS) by leveraging tokenizer embedding matching between the generated response and the ground truth. Extensive experiments show the effectiveness of our proposed method on various MLLMs, including Qwen2.5-VL and Intern-VL on various sizes. Our dataset and model are available at https://github.com/xushilin1/mixed-r1.

cs.CL↗

arXiv2025

Comprehensive Analog Signal Processing Platform Enabled with Acoustic Charge Transport in Two-dimensional Materials

Two-dimensional Acoustic Charge Transport (2D-ACT) devices, which integrate two dimensional semiconductor field-effect transistor (FET) with high-frequency surface acoustic wave (SAW) device provide a potential compact platform for the processing of analog signals in a wireless, non-contact, low-loss and real-time way. It is expected to be used in long-distance space communication and sensing. However, current investigations into 2D-ACT devices are still limited to the observation of DC acoustoelectric currents, and have yet to achieve real-time electronic signal processing capabilities. In this paper, we have designed a hybrid acoustoelectric platform composed of two-dimensional semiconductor FET and SAW device. The platform is capable of processing DC signals, exhibiting ambipolar transport behavior. The sub-wavelength channel length of the FET within the platform allows for the real-time observation of carrier distribution at a microscopic scale in conjunction with the SAW potential, and facilitating the reproduction and intensity regulation of AC signals. By adjusting the relative phase and intensity ratio of two counter-propagating SAWs, the platform also enables the addition and subtraction of AC signals.

physics.app-ph↗

arXiv2025

Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videos

This work presents Sa2VA, the first comprehensive, unified model for dense grounded understanding of both images and videos. Unlike existing multi-modal large language models, which are often limited to specific modalities and tasks, Sa2VA supports a wide range of image and video tasks, including referring segmentation and conversation, with minimal single-stage instruction tuning. Sa2VA combines SAM-2, a foundation video segmentation model, with MLLM, advanced vision-language models, and unifies text, image, and video into a shared LLM token space. Using the LLM, Sa2VA generates instruction tokens that guide SAM-2 in producing precise masks, enabling a grounded, multi-modal understanding of both static and dynamic visual content. Additionally, we introduce Ref-SAV, an auto-labeled dataset containing over 72k object expressions in complex video scenes, designed to boost model performance. We also manually validate 2k video objects in the Ref-SAV datasets to benchmark referring video object segmentation in complex environments. Experiments show that Sa2VA achieves strong performance across multiple tasks, particularly in referring video object segmentation, highlighting its potential for complex real-world applications. In addition, Sa2VA can be easily extended into various MLLMs, including Qwen-VL and Intern-VL, which can be updated with rapid process in current open-sourced MLLMs. Code and models have been provided to the community.

cs.CV↗

arXiv2025

Boosting the Self-driven Properties of 2D Photodetectors through Synergistic Asymmetrical Effects

Self-driven photodetectors (SDPDs) transform photon energy into electrical energy without external voltage, which makes them highly advantageous for applications such as low-power communication and imaging systems. Two-dimensional materials (2DMs) provide ideal platforms for SDPDs thanks to their band structures covering ultraviolet to infrared spectrum, strong light absorption efficiencies, and high carrier mobilities. However, the lack of stable doping methods and the complicated 2DMs multilayer stacking techniques pose tremendous difficulties for 2DMs to adopt the same device structures (i.e. PN junctions) as bulk materials, and the resultant self-driven performance remains at a low level. This work reveals how different asymmetrical effects can be combined to synergistically boost self-driven properties based on typical 2D metal-semiconductor-metal (MSM) photodetectors. Using WSe2 as an exemplary 2D material to build MSM photodetectors, the synergistic effect of asymmetrical contact electrodes and asymmetrical contact geometries is theoretically and experimentally demonstrated. The open-circuit voltage (Voc) of the SDPD reaches 0.58V, with a zero-bias responsivity of 5.77 A/W and an on/off ratio of 1.73*10^5. Additionally, our devices demonstrate potential for visible light communication (VLC) in underwater environments. Our results offer a promising and efficient strategy for building SDPDs based on various 2DMs and pave the way toward low-power optoelectronic applications.

physics.optics↗

arXiv2024

SegSTRONG-C: Segmenting Surgical Tools Robustly On Non-adversarial Generated Corruptions -- An EndoVis'24 Challenge

Surgical data science has seen rapid advancement with the excellent performance of end-to-end deep neural networks (DNNs). Despite their successes, DNNs have been proven susceptible to minor "corruptions," introducing a major concern for the translation of cutting-edge technology, especially in high-stakes scenarios. We introduce the SegSTRONG-C challenge dedicated to better understanding model deterioration under unforeseen but plausible non-adversarial "corruption" and the capabilities of contemporary methods that seek to improve it. Built on a dataset generated through counterfactual robotic replay, SegSTRONG-C provides paired clean and "corrupted" samples, enabling reproducible evaluation of model robustness. Participants are challenged to train tool segmentation algorithms on "uncorrupted" data and evaluate them on "corrupted" test domains for the binary robot tool segmentation task. Through comprehensive baseline experiments and participating submissions from widespread community engagement, SegSTRONG-C reveals key themes for model failure and identifies promising directions for improving robustness. The performance of challenge winners, achieving an average 0.9394 DSC and 0.9301 NSD across the unreleased test sets with "corruption" types: bleeding, smoke, and low brightness. This highlights how prior knowledge, customized training strategies, and architectural choice can be leveraged to improve robustness. In conclusion, the SegSTRONG-C challenge has identified practical approaches for enhancing model robustness. However, most approaches rely on conventional techniques that have known limitations. Looking ahead, we advocate for expanding intellectual diversity and creativity in non-adversarial robustness beyond data augmentation, calling for new paradigms that enhance universal robustness to unforeseen "corruptions" to facilitate richer applications in surgical data science.

cs.CV↗