arXiv subjects

Ye Tian

Publications and source records attributed to Ye Tian.

At least 73 records · Page 4Linked to original sources

arXiv2026

Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs

While Multimodal Large Language Models (MLLMs) excel at holistic understanding, they struggle in capturing the dense world with complex scenes, requiring fine-grained analysis of intricate details and object inter-relationships. Region-level MLLMs have been a promising step. However, previous attempts are generally optimized to understand given regions in isolation, neglecting crucial global contexts. To address this, we introduce Grasp Any Region (GAR) for comprehen- sive region-level visual understanding. Empowered by an effective RoI-aligned feature replay technique, GAR supports (1) precise perception by leveraging necessary global contexts, and (2) modeling interactions between multiple prompts. Together, it then naturally achieves (3) advanced compositional reasoning to answer specific free-form questions about any region, shifting the paradigm from passive description to active dialogue. Moreover, we construct GAR-Bench, which not only provides a more accurate evaluation of single-region comprehension, but also, more importantly, measures interactions and complex reasoning across multiple regions. Extensive experiments have demonstrated that GAR-1B not only maintains the state-of-the-art captioning capabilities, e.g., outperforming DAM-3B +4.5 on DLC-Bench, but also excels at modeling relationships between multiple prompts with advanced comprehension capabilities, even surpassing InternVL3-78B on GAR-Bench-VQA. More importantly, our zero-shot GAR-8B even outperforms in-domain VideoRefer-7B on VideoRefer-BenchQ, indicating its strong capabilities can be easily transferred to videos.

cs.CV↗

arXiv2026

Representation Learning with Blockwise Missingness and Signal Heterogeneity

Unified representation learning for multi-source data integration faces two important challenges: blockwise missingness and blockwise signal heterogeneity. The former arises from sources observing different, yet potentially overlapping, feature sets, while the latter involves varying signal strengths across subject groups and feature sets. While existing methods perform well with fully observed data or uniform signal strength, their performance degenerates when these two challenges coincide, which is common in practice. To address this, we propose Anchor Projected Principal Component Analysis (APPCA), a general framework for representation learning with structured blockwise missingness that is robust to signal heterogeneity. APPCA first recovers robust group-specific column spaces using all observed feature sets, and then aligns them by projecting shared "anchor" features onto these subspaces before performing PCA. This projection step induces a significant denoising effect. We establish estimation error bounds for embedding reconstruction through a fine-grained perturbation analysis. In particular, using a novel spectral slicing technique, our bound eliminates the standard dependency on the signal strength of subject embeddings, relying instead solely on the signal strength of integrated feature sets. We validate the proposed method through extensive simulation studies and an application to multimodal single-cell sequencing data.

stat.ME↗

arXiv2026

Supervise-assisted Multi-modality Fusion Diffusion Model for PET Restoration

Positron emission tomography (PET) offers powerful functional imaging but involves radiation exposure. Efforts to reduce this exposure by lowering the radiotracer dose or scan time can degrade image quality. While using magnetic resonance (MR) images with clearer anatomical information to restore standard-dose PET (SPET) from low-dose PET (LPET) is a promising approach, it faces challenges with the inconsistencies in the structure and texture of multi-modality fusion, as well as the mismatch in out-of-distribution (OOD) data. In this paper, we propose a supervise-assisted multi-modality fusion diffusion model (MFdiff) for addressing these challenges for high-quality PET restoration. Firstly, to fully utilize auxiliary MR images without introducing extraneous details in the restored image, a multi-modality feature fusion module is designed to learn an optimized fusion feature. Secondly, using the fusion feature as an additional condition, high-quality SPET images are iteratively generated based on the diffusion model. Furthermore, we introduce a two-stage supervise-assisted learning strategy that harnesses both generalized priors from simulated in-distribution datasets and specific priors tailored to in-vivo OOD data. Experiments demonstrate that the proposed MFdiff effectively restores high-quality SPET images from multi-modality inputs and outperforms state-of-the-art methods both qualitatively and quantitatively.

cs.CV↗

arXiv2026

PairUni: Pairwise Training for Unified Multimodal Language Models

Unified Vision-Language Models (UVLMs) perform both understanding and generation within a single architecture. Since these models rely on heterogeneous data and supervision, balancing both generation and understanding in reinforcement learning (RL) is challenging. To address this challenge, we propose PairUni, a unified framework that reorganizes data into understanding-generation (UG) pairs and aligns optimization accordingly. Specifically, we construct a unified paired dataset by synthesizing aligned instances via cross-modal semantic completion and retrieving semantically related samples. These paired structures expose cross-task semantic correspondences and support consistent policy learning. To leverage this structure, we present PairGRPO, a pair-aware variant based on Group Relative Policy Optimization. It assigns a similarity score to each pair to modulate the advantage, strengthening learning from well-aligned examples and reducing task interference. Extensive experiments across diverse UVLM architectures (Autoregressive and Discrete Diffusion) and scales (1B to 14B) demonstrate that PairUni yields consistent improvements over strong baselines. Notably, our method also demonstrates strong generalization by improving performance on image editing tasks without using any editing-specific data. Codes are available at https://github.com/Haochen-Wang409/PairUni.

cs.CL↗

arXiv2026

Full-span reversible space-time birefringence

Birefringence, the polarization-dependent splitting of light in anisotropic crystals, enables diverse optical phenomena and advanced functionalities such as optical communication, nonlinear optics, and quantum optics. However, conventional methods for controlling birefringence typically rely on engineering the optical crystal structure or applying external stimuli such as electric fields, mechanical stress or thermal variations, which are often constrained by limited tunability, challenges in integration with compact photonic devices or slow response time. Here, we introduce a new degree of freedom to manipulate the birefringence of light propagation in optical crystals through programming the spatiotemporal spectral phase of the incident light wave. We demonstrate this approach achieves continuous tuning of birefringence across a spectrum more than 100 times broader than that achievable with conventional birefringence tuning, spanning from positive through zero to negative values, irrespective of the crystal's optical sign and without inherent physical limitations. This unique optical behavior provides a versatile platform for investigating the complex dynamics of wave flow in anisotropic media, while the broad tunability of this space-time birefringence will spur innovations in ultrafast optical manipulation, optical computation, and quantum information processing-applications that demand rapid and flexible device reconfiguration.

physics.optics↗

arXiv2026

ClueTracer: Question-to-Vision Clue Tracing for Training-Free Hallucination Suppression in Multimodal Reasoning

Large multimodal reasoning models solve challenging visual problems via explicit long-chain inference: they gather visual clues from images and decode clues into textual tokens. Yet this capability also increases hallucinations, where the model generates content that is not supported by the input image or the question. To understand this failure mode, we identify \emph{reasoning drift}: during clue gathering, the model over-focuses on question-irrelevant entities, diluting focus on task-relevant cues and gradually decoupling the reasoning trace from visual grounding. As a consequence, many inference-time localization or intervention methods developed for non-reasoning models fail to pinpoint the true clues in reasoning settings. Motivated by these insights, we introduce ClueRecall, a metric for assessing visual clue retrieval, and present ClueTracer, a training-free, parameter-free, and architecture-agnostic plugin for hallucination suppression. ClueTracer starts from the question and traces how key clues propagate along the model's reasoning pathway (question $\rightarrow$ outputs $\rightarrow$ visual tokens), thereby localizing task-relevant patches while suppressing spurious attention to irrelevant regions. Remarkably, \textbf{without any additional training}, ClueTracer improves all \textbf{reasoning} architectures (including \texttt{R1-OneVision}, \texttt{Ocean-R1}, \texttt{MM-Eureka}, \emph{etc}.) by $\mathbf{1.21\times}$ on reasoning benchmarks. When transferred to \textbf{non-reasoning} settings, it yields a $\mathbf{1.14\times}$ gain.

cs.CV↗

arXiv2026

Should Benevolent Deception be Allowed in EHMI? A Mechanism Explanation Based on Game Theory

The application of external human-machine interface (EHMI) on autonomous vehicles (AVs) facilitates information exchange. Existing research fails to consider the impact of the sequence of actions, as well as the effects of EHMI applications and deception, raising the question of whether benevolent, well-intentioned deception should be permitted (i.e., misleading statements that are intended to benefit both parties). We established a game theory based EHMI information disclosure framework for AVs in this study. In considering benevolent deception, this framework divided the decision-making process into three stages, respectively encompassing three key questions: whether to disclose, when to disclose, and what type of intention information to disclose. The results show that theoretical advantages of deception exist in certain cases when AV expects to maximize the safety of the interaction. In 40 out of 484 cases (8.3%), safety can be enhanced through successful deception. Those successful deceptions fall into two categories: 1) In 28 of these cases, the straight-going AV expected the left-turning human-driven vehicle (HV) to yield, while HV exhibited lower speed and higher acceleration; 2) In 12 of these cases, AV expected HV to proceed first, while HV exhibited higher speed and lower acceleration. We also conducted a VR-based driving simulation experiment, and the results confirmed our conclusion. Additionally, we found that when participants had low trust in the EHMI, its use negatively impacted interaction efficiency instead. This study serves as an exploratory behavioral mechanism study based on specific hypotheses for future EHMI design and ethical decision-making of autonomous driving system.

cs.HC↗

arXiv2026

Lip-Siri: Contactless Open-Sentence Silent Speech with Wi-Fi Backscatter

Silent speech interfaces (SSIs) enable silent interaction in noise-sensitive or privacy-sensitive settings. However, existing SSIs face practical deployment trade-offs among privacy, user experience, and energy consumption, and most remain limited to closed-set recognition over small, pre-defined vocabularies of words or sentences, which restricts real-world expressiveness. In this paper, we present Lip-Siri, to the best of our knowledge, the first Wi-Fi backscatter--based SSI that supports open-vocabulary sentence recognition via lexicon-guided subword decoding. Lip-Siri designs a frequency-shifted backscatter tag to isolate tag-modulated reflections and suppress interference from non-target motions, enabling reliable extraction of lip-motion traces from ubiquitous Wi-Fi signals. We then segment continuous traces into lip-motion units, cluster them, learn robust unit representations via cluster-based self-supervision, and finally propose a lexicon-guided Transformer encoder--decoder with beam search to decode variable-length sentence sequences. We implement an end-to-end prototype and evaluate it with 15 participants on 340 sentences and 3,398 words across multiple scenarios. Lip-Siri achieves 85.61% accuracy on word prediction and a WER of 36.87% on continuous sentence recognition, approaching the performance of representative vision-based lip-reading systems.

cs.HC↗

arXiv2026

Quantum tunnelling-integrated optoplasmonic nanotrap enables conductance visualisation of individual proteins

Biological electron transfer (ET) relies on quantum mechanical tunnelling through a dynamically folded protein. Yet, the spatiotemporal coupling between structural fluctuations and electron flux remains poorly understood, largely due to limitations in existing experimental techniques, such as ensemble averaging and non-physiological operating conditions. Here, we introduce a quantum tunnelling-integrated optoplasmonic nanotrap (QTOP-trap), an optoelectronic platform that combines plasmonic optical trapping with real-time quantum tunnelling measurements. This label-free approach enables single-molecule resolution of protein conductance in physiological electrolytes, achieving sub-3 nm spatial precision and 10-μs temporal resolution. By synchronising optoelectronic measurements, QTOP-trap resolves protein-specific conductance signatures and directly correlates tertiary structure dynamics with conductance using a "protein switch" strategy. This methodology establishes a universal framework for dissecting non-equilibrium ET mechanisms in individual conformational-active proteins, with broad implications for bioenergetics research and biomimetic quantum device design.

physics.bio-ph↗

arXiv2025

Robust Unsupervised Multi-task and Transfer Learning on Gaussian Mixture Models

Unsupervised learning has been widely used in many real-world applications. One of the simplest and most important unsupervised learning models is the Gaussian mixture model (GMM). In this work, we study the multi-task learning problem on GMMs, which aims to leverage potentially similar GMM parameter structures among tasks to obtain improved learning performance compared to single-task learning. We propose a multi-task GMM learning procedure based on the EM algorithm that effectively utilizes unknown similarities between related tasks and is robust against a fraction of outlier tasks from arbitrary distributions. The proposed procedure is shown to achieve the minimax optimal rate of convergence for both parameter estimation error and the excess mis-clustering error, in a wide range of regimes. Moreover, we generalize our approach to tackle the problem of transfer learning for GMMs, where similar theoretical results are derived. Additionally, iterative unsupervised multi-task and transfer learning methods may suffer from an initialization alignment problem, and two alignment algorithms are proposed to resolve the issue. Finally, we demonstrate the effectiveness of our methods through simulations and real data examples. To the best of our knowledge, this is the first work studying multi-task and transfer learning on GMMs with theoretical guarantees.

stat.ML↗

arXiv2025

Study of SIDIS Unpolarized Cross Sections from a $^3$He Target with the Solenoidal Large Intensity Device at JLab

In this paper we present a detailed impact study of semi-inclusive deep inelastic scattering unpolarized cross sections' measurements using the proposed SoLID apparatus at Jefferson Lab. This type of data, collected at large Bjorken $x_{bj}$, moderate values of $Q^2$ and small values of the transverse momentum of produced hadrons, $P_{hT}$, allows to study transverse momentum dependent (TMD) parton distribution and fragmentation functions in a still poorly explored region. We present the projected results for charged light mesons based on simulated data. For the azimuthal-angle integrated cross sections we adopt the TMD framework up to the next-to-next-to-next-to-leading-logarithmic (N3LL) accuracy, while a simpler TMD parton model is employed for the study of azimuthal angular dependencies.

nucl-ex↗

arXiv2025

An Fluid Antenna Array-Enabled DOA Estimation Method: End-Fire Effect Suppression

Direction of Arrival (DOA) estimation serves as a critical sensing technology poised to play a vital role in future intelligent and ubiquitous communication systems. Despite the development of numerous mature super-resolution algorithms, the inherent end-fire effect problem in fixed antenna arrays remains inadequately addressed. This work proposed a novel array architecture composed of fluid antennas. By exploiting the spatial reconfigurability of their positions to equivalently modulate the array steering vector and integrating it with the classical MUSIC algorithm, this approach achieved high-precision DOA estimation. Simulation results demonstrated that the proposed method delivers outstanding estimation performance even in highly challenging end-fire regions.

eess.SP↗

arXiv2025

ESearch-R1: Learning Cost-Aware MLLM Agents for Interactive Embodied Search via Reinforcement Learning

Multimodal Large Language Models (MLLMs) have empowered embodied agents with remarkable capabilities in planning and reasoning. However, when facing ambiguous natural language instructions (e.g., "fetch the tool" in a cluttered room), current agents often fail to balance the high cost of physical exploration against the cognitive cost of human interaction. They typically treat disambiguation as a passive perception problem, lacking the strategic reasoning to minimize total task execution costs. To bridge this gap, we propose ESearch-R1, a cost-aware embodied reasoning framework that unifies interactive dialogue (Ask), episodic memory retrieval (GetMemory), and physical navigation (Navigate) into a single decision process. We introduce HC-GRPO (Heterogeneous Cost-Aware Group Relative Policy Optimization). Unlike traditional PPO which relies on a separate value critic, HC-GRPO optimizes the MLLM by sampling groups of reasoning trajectories and reinforcing those that achieve the optimal trade-off between information gain and heterogeneous costs (e.g., navigate time, and human attention). Extensive experiments in AI2-THOR demonstrate that ESearch-R1 significantly outperforms standard ReAct-based agents. It improves task success rates while reducing total operational costs by approximately 50\%, validating the effectiveness of GRPO in aligning MLLM agents with physical world constraints.

cs.AI↗

arXiv2025

CoPHo: Classifier-guided Conditional Topology Generation with Persistent Homology

The structure of topology underpins much of the research on performance and robustness, yet available topology data are typically scarce, necessitating the generation of synthetic graphs with desired properties for testing or release. Prior diffusion-based approaches either embed conditions into the diffusion model, requiring retraining for each attribute and hindering real-time applicability, or use classifier-based guidance post-training, which does not account for topology scale and practical constraints. In this paper, we show from a discrete perspective that gradients from a pre-trained graph-level classifier can be incorporated into the discrete reverse diffusion posterior to steer generation toward specified structural properties. Based on this insight, we propose Classifier-guided Conditional Topology Generation with Persistent Homology (CoPHo), which builds a persistent homology filtration over intermediate graphs and interprets features as guidance signals that steer generation toward the desired properties at each denoising step. Experiments on four generic/network datasets demonstrate that CoPHo outperforms existing methods at matching target metrics, and we further validate its transferability on the QM9 molecular dataset.

cs.LG↗

arXiv2025

Direct Generation of an Array with 78400 Optical Tweezers Using a Single Metasurface

Scalability remains a major challenge in building practical fault-tolerant quantum computers. Currently, the largest number of qubits achieved across leading quantum platforms ranges from hundreds to thousands. In atom arrays, scalability is primarily constrained by the capacity to generate large numbers of optical tweezers, and conventional techniques using acousto-optic deflectors or spatial light modulators struggle to produce arrays much beyond $\sim 10,000$ tweezers. Moreover, these methods require additional microscope objectives to focus the light into micrometer-sized spots, which further complicates system integration and scalability. Here, we demonstrate the experimental generation of an optical tweezer array containing $280\times 280$ spots using a metasurface, nearly an order of magnitude more than most existing systems. The metasurface leverages a large number of subwavelength phase-control pixels to engineer the wavefront of the incident light, enabling both large-scale tweezer generation and direct focusing into micron-scale spots without the need for a microscope. This result shifts the scalability bottleneck for atom arrays from the tweezer generation hardware to the available laser power. Furthermore, the array shows excellent intensity uniformity exceeding $90\%$, making it suitable for homogeneous single-atom loading and paving the way for trapping arrays of more than $10,000$ atoms in the near future.

cond-mat.quant-gas↗

arXiv2025

Structured Light at the Extreme: Harnessing Spatiotemporal Control for High-Field Laser-Matter Interactions

This review charts the emerging paradigm of intelligent structured light for high-field laser-matter interactions, where the precise spatiotemporal and vectorial control of light is a critical degree of freedom. We outline a transformative framework built upon three synergistic pillars. First, we survey the advanced electromagnetic toolkit, moving beyond conventional spatial light modulators to include robust static optics and the promising frontier of plasma light modulators. Second, we detail the optimization engine for this high-dimensional design space, focusing on physics-informed digital twins and AI-driven inverse design to automate the discovery of optimal light structures. Finally, we explore the groundbreaking applications enabled by this integrated approach, including programmable electron beams, orbital-angular-momentum-carrying γ-rays, compact THz accelerators, and robust communications. The path forward necessitates overcoming grand challenges in material science, real-time adaptive control at MHz rates, and the extension of these principles to the quantum realm. This review serves as a call to action for a coordinated, interdisciplinary effort to command, rather than merely observe, light-matter interactions at the extreme.

physics.optics↗

arXiv2025

Does Hearing Help Seeing? Investigating Audio-Video Joint Denoising for Video Generation

Recent audio-video generative systems suggest that coupling modalities benefits not only audio-video synchrony but also the video modality itself. We pose a fundamental question: Does audio-video joint denoising training improve video generation, even when we only care about video quality? To study this, we introduce a parameter-efficient Audio-Video Full DiT (AVFullDiT) architecture that leverages pre-trained text-to-video (T2V) and text-to-audio (T2A) modules for joint denoising. We train (i) a T2AV model with AVFullDiT and (ii) a T2V-only counterpart under identical settings. Our results provide the first systematic evidence that audio-video joint denoising can deliver more than synchrony. We observe consistent improvements on challenging subsets featuring large and object contact motions. We hypothesize that predicting audio acts as a privileged signal, encouraging the model to internalize causal relationships between visual events and their acoustic consequences (e.g., collision $\times$ impact sound), which in turn regularizes video dynamics. Our findings suggest that cross-modal co-training is a promising approach to developing stronger, more physically grounded world models. Code and dataset will be made publicly available.

cs.CV↗

arXiv2025

Hardware-aware Lightweight Photonic Spiking Neural Network for Pattern Classification

There exists a significant scale gap between photonic neural network integrated chips and neural networks, which hinders the deployment and application of photonic neural network. Here, we propose hardware-aware lightweight spiking neural networks (SNNs) architecture tailored to our photonic neuromorphic chips, and conducts hardware-software collaborative computing for solving patter classification tasks. Here, we employed a simplified Mach-Zehnder interferometer (MZI) mesh for performing linear computation, and 16-channel distributed feedback lasers with saturable absorber (DFB-SA) array for performing nonlinear spike activation. Both photonic neuromorphic chips based on the MZI mesh and DFB-SA array were designed, optimized and fabricated. Furthermore, we propose a lightweight spiking neural network (SNN) with discrete cosine transform to reduce input dimension and match the input/output ports number of the photonic neuromorphic chips. We demonstrated an end-to-end inference of an entire layer of the lightweight photonic SNN. The hardware-software collaborative inference accuracy is 90% and 80.5% for MNIST and Fashion-MNIST datasets, respectively. The energy efficiency is 1.39 TOPS/W for the MZI mesh, and is 987.65 GOPS/W for the DFB-SA array. The lightweight architecture and experimental demonstration address the challenge of scale mismatch between the photonic chip and SNN, paving the way for the hardware deployment of photonic SNNs.

physics.optics↗