arXiv subjects

Yang Wu

Publications and source records attributed to Yang Wu.

At least 73 records · Page 4Linked to original sources

arXiv2025

What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilities

As multimodal large language models (MLLMs) advance, MLLM-based virtual agents have demonstrated remarkable performance. However, existing benchmarks face significant limitations, including uncontrollable task complexity, extensive manual annotation with limited scenarios, and a lack of multidimensional evaluation. In response to these challenges, we introduce OmniBench, a self-generating, cross-platform, graph-based benchmark with an automated pipeline for synthesizing tasks of controllable complexity through subtask composition. To evaluate the diverse capabilities of virtual agents on the graph, we further present OmniEval, a multidimensional evaluation framework that includes subtask-level evaluation, graph-based metrics, and comprehensive tests across 10 capabilities. Our synthesized dataset contains 36k graph-structured tasks across 20 scenarios, achieving a 91\% human acceptance rate. Training on our graph-structured data shows that it can more efficiently guide agents compared to manually annotated data. We conduct multidimensional evaluations for various open-source and closed-source models, revealing their performance across various capabilities and paving the way for future advancements. Our project is available at https://omni-bench.github.io/.

cs.CV↗

arXiv2025

FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL

Recent studies extend the autoregression paradigm to text-to-image generation, achieving performance comparable to diffusion models. However, our new PairComp benchmark -- featuring test cases of paired prompts with similar syntax but different fine-grained semantics -- reveals that existing models struggle with fine-grained text-image alignment thus failing to realize precise control over visual tokens. To address this, we propose FocusDiff, which enhances fine-grained text-image semantic alignment by focusing on subtle differences between similar text-image pairs. We construct a new dataset of paired texts and images with similar overall expressions but distinct local semantics, further introducing a novel reinforcement learning algorithm to emphasize such fine-grained semantic differences for desired image generation. Our approach achieves state-of-the-art performance on existing text-to-image benchmarks and significantly outperforms prior methods on PairComp.

cs.CV↗

arXiv2025

Uncertainty-Aware Metabolic Stability Prediction with Dual-View Contrastive Learning

Accurate prediction of molecular metabolic stability (MS) is critical for drug research and development but remains challenging due to the complex interplay of molecular interactions. Despite recent advances in graph neural networks (GNNs) for MS prediction, current approaches face two critical limitations: (1) incomplete molecular modeling due to atom-centric message-passing mechanisms that disregard bond-level topological features, and (2) prediction frameworks that lack reliable uncertainty quantification. To address these challenges, we propose TrustworthyMS, a novel contrastive learning framework designed for uncertainty-aware metabolic stability prediction. First, a molecular graph topology remapping mechanism synchronizes atom-bond interactions through edge-induced feature propagation, capturing both localized electronic effects and global conformational constraints. Second, contrastive topology-bond alignment enforces consistency between molecular topology views and bond patterns via feature alignment, enhancing representation robustness. Third, uncertainty modeling through Beta-Binomial uncertainty quantification enables simultaneous prediction and confidence calibration under epistemic uncertainty. Through extensive experiments, our results demonstrate that TrustworthyMS outperforms current state-of-the-art methods in terms of predictive performance.

cs.LG↗

arXiv2025

Spa-VLM: Stealthy Poisoning Attacks on RAG-based VLM

With the rapid development of the Vision-Language Model (VLM), significant progress has been made in Visual Question Answering (VQA) tasks. However, existing VLM often generate inaccurate answers due to a lack of up-to-date knowledge. To address this issue, recent research has introduced Retrieval-Augmented Generation (RAG) techniques, commonly used in Large Language Models (LLM), into VLM, incorporating external multi-modal knowledge to enhance the accuracy and practicality of VLM systems. Nevertheless, the RAG in LLM may be susceptible to data poisoning attacks. RAG-based VLM may also face the threat of this attack. This paper first reveals the vulnerabilities of the RAG-based large model under poisoning attack, showing that existing single-modal RAG poisoning attacks have a 100\% failure rate in multi-modal RAG scenarios. To address this gap, we propose Spa-VLM (Stealthy Poisoning Attack on RAG-based VLM), a new paradigm for poisoning attacks on large models. We carefully craft malicious multi-modal knowledge entries, including adversarial images and misleading text, which are then injected into the RAG's knowledge base. When users access the VLM service, the system may generate misleading outputs. We evaluate Spa-VLM on two Wikipedia datasets and across two different RAGs. Results demonstrate that our method achieves highly stealthy poisoning, with the attack success rate exceeding 0.8 after injecting just 5 malicious entries into knowledge bases with 100K and 2M entries, outperforming state-of-the-art poisoning attacks designed for RAG-based LLMs. Additionally, we evaluated several defense mechanisms, all of which ultimately proved ineffective against Spa-VLM, underscoring the effectiveness and robustness of our attack.

cs.CR↗

arXiv2025

Vulnerability of Text-to-Image Models to Prompt Template Stealing: A Differential Evolution Approach

Prompt trading has emerged as a significant intellectual property concern in recent years, where vendors entice users by showcasing sample images before selling prompt templates that can generate similar images. This work investigates a critical security vulnerability: attackers can steal prompt templates using only a limited number of sample images. To investigate this threat, we introduce Prism, a prompt-stealing benchmark consisting of 50 templates and 450 images, organized into Easy and Hard difficulty levels. To identify the vulnerabity of VLMs to prompt stealing, we propose EvoStealer, a novel template stealing method that operates without model fine-tuning by leveraging differential evolution algorithms. The system first initializes population sets using multimodal large language models (MLLMs) based on predefined patterns, then iteratively generates enhanced offspring through MLLMs. During evolution, EvoStealer identifies common features across offspring to derive generalized templates. Our comprehensive evaluation conducted across open-source (INTERNVL2-26B) and closed-source models (GPT-4o and GPT-4o-mini) demonstrates that EvoStealer's stolen templates can reproduce images highly similar to originals and effectively generalize to other subjects, significantly outperforming baseline methods with an average improvement of over 10%. Moreover, our cost analysis reveals that EvoStealer achieves template stealing with negligible computational expenses. Our code and dataset are available at https://github.com/whitepagewu/evostealer.

cs.CL↗

arXiv2025

WeatherGen: A Unified Diverse Weather Generator for LiDAR Point Clouds via Spider Mamba Diffusion

3D scene perception demands a large amount of adverse-weather LiDAR data, yet the cost of LiDAR data collection presents a significant scaling-up challenge. To this end, a series of LiDAR simulators have been proposed. Yet, they can only simulate a single adverse weather with a single physical model, and the fidelity of the generated data is quite limited. This paper presents WeatherGen, the first unified diverse-weather LiDAR data diffusion generation framework, significantly improving fidelity. Specifically, we first design a map-based data producer, which can provide a vast amount of high-quality diverse-weather data for training purposes. Then, we utilize the diffusion-denoising paradigm to construct a diffusion model. Among them, we propose a spider mamba generator to restore the disturbed diverse weather data gradually. The spider mamba models the feature interactions by scanning the LiDAR beam circle or central ray, excellently maintaining the physical structure of the LiDAR data. Subsequently, following the generator to transfer real-world knowledge, we design a latent feature aligner. Afterward, we devise a contrastive learning-based controller, which equips weather control signals with compact semantic knowledge through language supervision, guiding the diffusion model to generate more discriminative data. Extensive evaluations demonstrate the high generation quality of WeatherGen. Through WeatherGen, we construct the mini-weather dataset, promoting the performance of the downstream task under adverse weather conditions. Code is available: https://github.com/wuyang98/weathergen

cs.CV↗

arXiv2025

Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program

Recent advancements in reward signal usage for Large Language Models (LLMs) are remarkable. However, significant challenges exist when transitioning reward signal to the multimodal domain, including labor-intensive annotations, over-reliance on one-step rewards, and inadequate evaluation. To address these issues, we propose SVIP, a novel approach to train a step-level multi-dimensional Chain-of-Thought~(CoT) reward model automatically. It generates code for solving visual tasks and transforms the analysis of code blocks into the evaluation of CoT step as training samples. Then, we train SVIP-Reward model using a multi-head attention mechanism called TriAtt-CoT. The advantages of SVIP-Reward are evident throughout the entire process of MLLM. We also introduce a benchmark for CoT reward model training and testing. Experimental results demonstrate that SVIP-Reward improves MLLM performance across training and inference-time scaling, yielding better results on benchmarks while reducing hallucinations and enhancing reasoning ability.

cs.CV↗

arXiv2025

AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea

Instruction-based image editing aims to modify specific image elements with natural language instructions. However, current models in this domain often struggle to accurately execute complex user instructions, as they are trained on low-quality data with limited editing types. We present AnyEdit, a comprehensive multi-modal instruction editing dataset, comprising 2.5 million high-quality editing pairs spanning over 20 editing types and five domains. We ensure the diversity and quality of the AnyEdit collection through three aspects: initial data diversity, adaptive editing process, and automated selection of editing results. Using the dataset, we further train a novel AnyEdit Stable Diffusion with task-aware routing and learnable task embedding for unified image editing. Comprehensive experiments on three benchmark datasets show that AnyEdit consistently boosts the performance of diffusion-based editing models. This presents prospects for developing instruction-driven image editing models that support human creativity.

cs.CV↗

arXiv2025

Non-Hermitian non-Abelian topological transition in the S=1 electron spin system of a nitrogen vacancy centre in diamond

Topological phases and transitions are of fundamental importance in physics, which provide a deep insight into the understanding of materials. Recently, non-Abelian topological transitions have been investigated in Hermitian systems, revealing important topological features. With non-Hermiticity introduced, non-Hermitian non-Abelian topological transitions bring about more intriguing topological features, yet has not been experimentally explored. In this work, we report the observation of the non-Hermitian non-Abelian topological transition at the atomic scale utilizing a nitrogenvacancy center in diamond. While the well-established topological numbers, failed to recognize this transition, we successfully characterized such a transition with the measurement of the complex eigenvalue braids. We obtained the braid invariants from the measured relative phases between eigenvalues. The observed change in braid invariants provides a clear signature of the non-Abelian topological transition. Furthermore, we experimentally revealed an intriguing consequence of this transition, which is the creation of a third-order exceptional point through the collision of two second-order exceptional points with opposite charges. Our experimental findings shed light on the abundant non-Abelian topological phenomena involving non-Hermiticity, and provide insights into manipulating the spectral topology in atomic scale systems to achieve exotic functionalities arising from non-Abelian band braiding.

quant-ph↗

arXiv2025

Experimental observation of Dirac exceptional point

The energy level degeneracies, also known as exceptional points (EPs), are crucial for comprehending emerging phenomena in materials and enabling innovative functionalities for devices. Since EPs were proposed over half a century age, only two types of EPs have been experimentally discovered, revealing intriguing phases of materials such as Dirac and Weyl semimetals. These discoveries have showcased numerous exotic topological properties and novel applications, such as unidirectional energy transfer. Here we report the observation of a novel type of EP, named the Dirac EP, utilizing a nitrogen-vacancy center in diamond. Two of the eigenvalues are measured to be degenerate at the Dirac EP and remain real in its vicinity. This exotic band topology associated with the Dirac EP enables the preservation of the symmetry when passing through, and makes it possible to achieve adiabatic evolution in non-Hermitian systems. We examined the degeneracy between the two eigenstates by quantum state tomography, confirming that the degenerate point is a Dirac EP rather than a Hermitian degeneracy. Our research of the distinct type of EP contributes a fresh perspective on dynamics in non-Hermitian systems and is potentially valuable for applications in quantum control in non-Hermitian systems and the study of the topological properties of EP.

quant-ph↗

arXiv2025

POINT: a web-based platform for pharmacological investigation enhanced by multi-omics networks and knowledge graphs

Network pharmacology (NP) explores pharmacological mechanisms through biological networks. Multi-omics data enable multi-layer network construction under diverse conditions, requiring integration into NP analyses. We developed POINT, a novel NP platform enhanced by multi-omics biological networks, advanced algorithms, and knowledge graphs (KGs) featuring network-based and KG-based analytical functions. In the network-based analysis, users can perform NP studies flexibly using 1,158 multi-omics biological networks encompassing proteins, transcription factors, and non-coding RNAs across diverse cell line-, tissue- and disease-specific conditions. Network-based analysis-including random walk with restart (RWR), GSEA, and diffusion profile (DP) similarity algorithms-supports tasks such as target prediction, functional enrichment, and drug screening. We merged networks from experimental sources to generate a pre-integrated multi-layer human network for evaluation. RWR demonstrated superior performance with a 33.1% average ranking improvement over the second-best algorithm, PageRank, in identifying known targets across 2,002 drugs. Additionally, multi-layer networks significantly improve the ability to identify FDA-approved drug-disease pairs compared to the single-layer network. For KG-based analysis, we compiled three high-quality KGs to construct POINT KG, which cross-references over 90% of network-based predictions. We illustrated the platform's capabilities through two case studies. POINT bridges the gap between multi-omics networks and drug discovery; it is freely accessible at http://point.gene.ac/.

q-bio.MN↗

arXiv2025

EigenActor: Variant Body-Object Interaction Generation Evolved from Invariant Action Basis Reasoning

This paper explores a cross-modality synthesis task that infers 3D human-object interactions (HOIs) from a given text-based instruction. Existing text-to-HOI synthesis methods mainly deploy a direct mapping from texts to object-specific 3D body motions, which may encounter a performance bottleneck since the huge cross-modality gap. In this paper, we observe that those HOI samples with the same interaction intention toward different targets, e.g., "lift a chair" and "lift a cup", always encapsulate similar action-specific body motion patterns while characterizing different object-specific interaction styles. Thus, learning effective action-specific motion priors and object-specific interaction priors is crucial for a text-to-HOI model and dominates its performances on text-HOI semantic consistency and body-object interaction realism. In light of this, we propose a novel body pose generation strategy for the text-to-HOI task: infer object-agnostic canonical body action first and then enrich object-specific interaction styles. Specifically, the first canonical body action inference stage focuses on learning intra-class shareable body motion priors and mapping given text-based semantics to action-specific canonical 3D body motions. Then, in the object-specific interaction inference stage, we focus on object affordance learning and enrich object-specific interaction styles on an inferred action-specific body motion basis. Extensive experiments verify that our proposed text-to-HOI synthesis system significantly outperforms other SOTA methods on three large-scale datasets with better semantic consistency and interaction realism performances.

cs.CV↗

arXiv2025

Ferroelectric Chirality-Driven Direction-Tunable and Spin-Invertible Corner States in 2D MOF-Based Magnetic Second-Order Topological Insulators

Despite the rapid progress in predicting 2D magnetic second-order topological insulators (SOTIs), effective strategies for manipulating their spin-polarized corner states remain largely unexplored. The interplay between ferroelectricity, chirality, magnetism, and topology presents an untapped opportunity for controlling these corner states. Here, we propose a novel approach for tuning spin-polarized corner states in 2D magnetic SOTIs by inducing ferroelectric chirality in 2D metal-organic frameworks (MOFs) with intrinsic structural flexibility. Through symmetry analysis, we strategically replace pyrazine (pyz) ligands with 2-pyrazinolate (2-pyzol) ligands in the 2D MOF Cr(pyz)2, leading to the emergence of a new 2D magnetic SOTI, Cr(2-pyzol)2, which facilitates ferroelectric chirality controlled spin-polarized corner states in both spin channels. Through first-principle calculations, we demonstrate that Cr(2-pyzol)2 belongs to ferroelectric chiral systems, and its corner states can be directionally tuned in real space and spin-inverted in spin space upon ferroelectric chirality switching. Our work represents the first attempt to simultaneously manipulate corner states in both real space and spin space, offering a new strategy for integrating ferroelectric chirality into 2D MOF-based magnetic SOTIs.

cond-mat.mtrl-sci↗

arXiv2025

Intelligent Legal Assistant: An Interactive Clarification System for Legal Question Answering

The rise of large language models has opened new avenues for users seeking legal advice. However, users often lack professional legal knowledge, which can lead to questions that omit critical information. This deficiency makes it challenging for traditional legal question-answering systems to accurately identify users' actual needs, often resulting in imprecise or generalized advice. In this work, we develop a legal question-answering system called Intelligent Legal Assistant, which interacts with users to precisely capture their needs. When a user poses a question, the system requests that the user select their geographical location to pinpoint the applicable laws. It then generates clarifying questions and options based on the key information missing from the user's initial question. This allows the user to select and provide the necessary details. Once all necessary information is provided, the system produces an in-depth legal analysis encompassing three aspects: overall conclusion, jurisprudential analysis, and resolution suggestions.

cs.CL↗

arXiv2025

Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning

Large Language Models (LLMs) have achieved impressive results across numerous domains, yet they experience notable deficiencies in legal question-answering tasks. LLMs often generate generalized responses that lack the logical specificity required for expert legal advice and are prone to hallucination, providing answers that appear correct but are unreliable. Retrieval-Augmented Generation (RAG) techniques offer partial solutions to address this challenge, but existing approaches typically focus only on semantic similarity, neglecting the logical structure essential to legal reasoning. In this paper, we propose the Logical-Semantic Integration Model (LSIM), a novel supervised framework that bridges semantic and logical coherence. LSIM comprises three components: reinforcement learning predicts a structured fact-rule chain for each question, a trainable Deep Structured Semantic Model (DSSM) retrieves the most relevant candidate questions by integrating semantic and logical features, and in-context learning generates the final answer using the retrieved content. Our experiments on a real-world legal QA dataset-validated through both automated metrics and human evaluation-demonstrate that LSIM significantly enhances accuracy and reliability compared to existing methods.

cs.CL↗

arXiv2024

ViDTA: Enhanced Drug-Target Affinity Prediction via Virtual Graph Nodes and Attention-based Feature Fusion

Drug-target interaction is fundamental in understanding how drugs affect biological systems, and accurately predicting drug-target affinity (DTA) is vital for drug discovery. Recently, deep learning methods have emerged as a significant approach for estimating the binding strength between drugs and target proteins. However, existing methods simply utilize the drug's local information from molecular topology rather than global information. Additionally, the features of drugs and proteins are usually fused with a simple concatenation operation, limiting their effectiveness. To address these challenges, we proposed ViDTA, an enhanced DTA prediction framework. We introduce virtual nodes into the Graph Neural Network (GNN)-based drug feature extraction network, which acts as a global memory to exchange messages more efficiently. By incorporating virtual graph nodes, we seamlessly integrate local and global features of drug molecular structures, expanding the GNN's receptive field. Additionally, we propose an attention-based linear feature fusion network for better capturing the interaction information between drugs and proteins. Experimental results evaluated on various benchmarks including Davis, Metz, and KIBA demonstrate that our proposed ViDTA outperforms the state-of-the-art baselines.

cs.LG↗

arXiv2024

Programmable simulation of high-order exceptional point with a trapped ion

The nontrivial degeneracies in non-Hermitian systems, exceptional points (EPs), have attracted extensive attention due to intriguing phenomena. Compared with commonly observed second-order EPs, high-order EPs show rich physics due to their extended dimension and parameter space, ranging from the coalescence of EPs into higher order to potential applications in topological properties. However, these features also pose challenges in controlling multiple coherent and dissipative elements in a scaled system. Here we experimentally demonstrate a native programmable control to simulate a high-order non-Hermitian Hamiltonian in a multi-dimensional trapped ion system. We simulate a series of non-Hermitian systems with varied parameters and observe the coalescence of second-order EPs into a fourth-order EP. Our results pave the way for scalable quantum simulation of high-dimensional dissipative systems and can be beneficial for the application of high-order EPs in quantum sensing and quantum control.

quant-ph↗

arXiv2024

ProGDF: Progressive Gaussian Differential Field for Controllable and Flexible 3D Editing

3D editing plays a crucial role in editing and reusing existing 3D assets, thereby enhancing productivity. Recently, 3DGS-based methods have gained increasing attention due to their efficient rendering and flexibility. However, achieving desired 3D editing results often requires multiple adjustments in an iterative loop, resulting in tens of minutes of training time cost for each attempt and a cumbersome trial-and-error cycle for users. This in-the-loop training paradigm results in a poor user experience. To address this issue, we introduce the concept of process-oriented modelling for 3D editing and propose the Progressive Gaussian Differential Field (ProGDF), an out-of-loop training approach that requires only a single training session to provide users with controllable editing capability and variable editing results through a user-friendly interface in real-time. ProGDF consists of two key components: Progressive Gaussian Splatting (PGS) and Gaussian Differential Field (GDF). PGS introduces the progressive constraint to extract the diverse intermediate results of the editing process and employs rendering quality regularization to improve the quality of these results. Based on these intermediate results, GDF leverages a lightweight neural network to model the editing process. Extensive results on two novel applications, namely controllable 3D editing and flexible fine-grained 3D manipulation, demonstrate the effectiveness, practicality and flexibility of the proposed ProGDF.

cs.GR↗