Searcharxiv⌕ Search

arXiv subjects

Xingyu Zhu

Publications and source records attributed to Xingyu Zhu.

At least 55 records · Page 3Linked to original sources
arXiv2026

Contextual Drag: How Errors in the Context Affect LLM Reasoning

Central to many self-improvement pipelines for large language models (LLMs) is the assumption that models can improve by reflecting on past mistakes. We study a phenomenon termed contextual drag: the presence of failed attempts in the context biases subsequent generations toward structurally similar errors. Across evaluations of 11 proprietary and open-weight models on 8 reasoning tasks, contextual drag induces 10-20% performance drops, and iterative self-refinement in models with severe contextual drag can collapse into self-deterioration. Structural analysis using tree edit distance reveals that subsequent reasoning trajectories inherit structurally similar error patterns from the context. We demonstrate that neither external feedback nor successful self-verification suffices to eliminate this effect. While mitigation strategies such as fallback-behavior fine-tuning and context denoising yield partial improvements, they fail to fully restore baseline performance, positioning contextual drag as a persistent failure mode in current reasoning architectures.

cs.CL↗
arXiv2026

GuardAlign: Test-time Safety Alignment in Multimodal Large Language Models

Large vision-language models (LVLMs) have achieved remarkable progress in vision-language reasoning tasks, yet ensuring their safety remains a critical challenge. Recent input-side defenses detect unsafe images with CLIP and prepend safety prefixes to prompts, but they still suffer from inaccurate detection in complex scenes and unstable safety signals during decoding. To address these issues, we propose GuardAlign, a training-free defense framework that integrates two strategies. First, OT-enhanced safety detection leverages optimal transport to measure distribution distances between image patches and unsafe semantics, enabling accurate identification of malicious regions without additional computational cost. Second, cross-modal attentive calibration strengthens the influence of safety prefixes by adaptively reallocating attention across layers, ensuring that safety signals remain consistently activated throughout generation. Extensive evaluations on six representative MLLMs demonstrate that GuardAlign reduces unsafe response rates by up to 39% on SPA-VL, while preserving utility, achieving an improvement on VQAv2 from 78.51% to 79.21%.

cs.CV↗
arXiv2026

Look Carefully: Adaptive Visual Reinforcements in Multimodal Large Language Models for Hallucination Mitigation

Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language reasoning, yet they remain vulnerable to hallucination, where generated content deviates from visual evidence. Existing mitigation strategies either require costly supervision during training or introduce additional latency at inference time. Recent vision enhancement methods attempt to address this issue by reinforcing visual tokens during decoding, but they typically inject all tokens indiscriminately, which causes interference from background regions and distracts the model from critical cues. To overcome this challenge, we propose Adaptive Visual Reinforcement (AIR), a training-free framework for MLLMs. AIR consists of two components. Prototype-based token reduction condenses the large pool of visual tokens into a compact subset to suppress redundancy. OT-guided patch reinforcement quantifies the alignment between hidden states and patch embeddings to selectively integrate the most consistent patches into feed-forward layers. As a result, AIR enhances the model's reliance on salient visual information and effectively mitigates hallucination. Extensive experiments across representative MLLMs demonstrate that AIR substantially reduces hallucination while preserving general capabilities, establishing it as an effective solution for building reliable MLLMs.

cs.CV↗
arXiv2026

Spider-Sense: Intrinsic Risk Sensing for Efficient Agent Defense with Hierarchical Adaptive Screening

As large language models (LLMs) evolve into autonomous agents, their real-world applicability has expanded significantly, accompanied by new security challenges. Most existing agent defense mechanisms adopt a mandatory checking paradigm, in which security validation is forcibly triggered at predefined stages of the agent lifecycle. In this work, we argue that effective agent security should be intrinsic and selective rather than architecturally decoupled and mandatory. We propose Spider-Sense framework, an event-driven defense framework based on Intrinsic Risk Sensing (IRS), which allows agents to maintain latent vigilance and trigger defenses only upon risk perception. Once triggered, the Spider-Sense invokes a hierarchical defence mechanism that trades off efficiency and precision: it resolves known patterns via lightweight similarity matching while escalating ambiguous cases to deep internal reasoning, thereby eliminating reliance on external models. To facilitate rigorous evaluation, we introduce S$^2$Bench, a lifecycle-aware benchmark featuring realistic tool execution and multi-stage attacks. Extensive experiments demonstrate that Spider-Sense achieves competitive or superior defense performance, attaining the lowest Attack Success Rate (ASR) and False Positive Rate (FPR), with only a marginal latency overhead of 8.3\%.

cs.CR↗
arXiv2026

Dual AGNs on 100 kpc Scales from the Million Quasar Catalog

Research on dual active galactic nuclei (AGNs) is crucial for understanding the coevolution of galaxies and supermassive black holes. However, the current number of dual AGNs remains scarce. In this work, we selected 173 new dual AGNs, 4 AGN triplets, and 1 AGN quadruplet from the Million Quasars Catalog, all with low redshift ($z < 0.5$), a projected distance ($r_p$) of no more than 100 kpc, and a line-of-sight velocity difference ($|Δv|$) of less than 600 km s$^{-1}$, thus supplementing existing low-redshift dual AGNs demographics. Visual inspection of the optical images from the Dark Energy Spectroscopic Instrument Legacy Survey was performed for each pair, revealing that $\sim$16\% of pairs exhibit tidal features. Statistical analyses show an increasing number of dual AGNs with decreasing redshift, with velocity difference primarily at $|Δv| < $ 300 km s$^{-1}$, which is likely an artifact of our selection strategy. The tidal sample peaks as having 13 pairs at 5-20$h^{-1}_{70}$ kpc, but drops to 1 pair $> 55\,h^{-1}_{70}$ kpc. Our study also explores thewide separation ($r_p>10$ kpc) dual AGNs, finding 165 such systems, with 25 displaying clear tidal features. Furthermore, some extra galaxies, AGNs, and/or their candidates were found in the same regions of the pairs or multiplets forming interacting systems with these pairs or multiplets.

astro-ph.GA↗
arXiv2026

Uniqueness of the asymptotic limits for Ricci-flat manifolds with linear volume growth II

We relate the uniqueness of asymptotic limits for noncollapsed Ricci flat manifolds with linear volume growth to the existence of a harmonic function asymptotic to a Busemann function. Parallel to the work of Colding--Minicozzi in the Euclidean volume growth setting, we prove uniqueness of the asymptotic limit and establish a quantitative polynomial convergence rate via a monotone quantity associated with this harmonic function, assuming such harmonic function exists and one asymptotic limit is smooth. Conversely, for an open manifold with nonnegative Ricci curvature, we show that uniqueness of the asymptotic limit implies the existence of the desired harmonic function, without assuming smoothness of the cross section.

math.DG↗
arXiv2025

Topological rigidity of small RCD(K,N) spaces with maximal rank

For a polycyclic group $Λ$, $\text{rank} (Λ)$ is defined as the number of $\mathbb{Z}$ factors in a polycyclic decomposition of $Λ$. For a finitely generated group $G$, $\text{rank} (G)$ is defined as the infimum of $ \text{rank} (Λ)$ among finite index polycyclic subgroups $Λ\leq G$. For a compact $ \text{RCD} (K,N)$ space $(X,\mathsf{d}, \mathfrak{m})$ with $ \text{diam} (X) \leq \varepsilon (K,N)$, the rank of $π_1(X)$ is at most $N$. We show that in case of equality, $X$ is homeomorphic to an infranilmanifold, generalizing a result by Kapovitch--Wilking to the non-smooth setting.

math.DG↗
arXiv2025

Dynamic Multimodal Prototype Learning in Vision-Language Models

With the increasing attention to pre-trained vision-language models (VLMs), \eg, CLIP, substantial efforts have been devoted to many downstream tasks, especially in test-time adaptation (TTA). However, previous works focus on learning prototypes only in the textual modality while overlooking the ambiguous semantics in class names. These ambiguities lead to textual prototypes that are insufficient to capture visual concepts, resulting in limited performance. To address this issue, we introduce \textbf{ProtoMM}, a training-free framework that constructs multimodal prototypes to adapt VLMs during the test time. By viewing the prototype as a discrete distribution over the textual descriptions and visual particles, ProtoMM has the ability to combine the multimodal features for comprehensive prototype learning. More importantly, the visual particles are dynamically updated as the testing stream flows. This allows our multimodal prototypes to continually learn from the data, enhancing their generalizability in unseen scenarios. In addition, we quantify the importance of the prototypes and test images by formulating their semantic distance as an optimal transport problem. Extensive experiments on 15 zero-shot benchmarks demonstrate the effectiveness of our method, achieving a 1.03\% average accuracy improvement over state-of-the-art methods on ImageNet and its variant datasets.

cs.CV↗
arXiv2025

Hierarchical Semantic Alignment for Image Clustering

Image clustering is a classic problem in computer vision, which categorizes images into different groups. Recent studies utilize nouns as external semantic knowledge to improve clustering performance. However, these methods often overlook the inherent ambiguity of nouns, which can distort semantic representations and degrade clustering quality. To address this issue, we propose a hierarChical semAntic alignmEnt method for image clustering, dubbed CAE, which improves clustering performance in a training-free manner. In our approach, we incorporate two complementary types of textual semantics: caption-level descriptions, which convey fine-grained attributes of image content, and noun-level concepts, which represent high-level object categories. We first select relevant nouns from WordNet and descriptions from caption datasets to construct a semantic space aligned with image features. Then, we align image features with selected nouns and captions via optimal transport to obtain a more discriminative semantic space. Finally, we combine the enhanced semantic and image features to perform clustering. Extensive experiments across 8 datasets demonstrate the effectiveness of our method, notably surpassing the state-of-the-art training-free approach with a 4.2% improvement in accuracy and a 2.9% improvement in adjusted rand index (ARI) on the ImageNet-1K dataset.

cs.CV↗
arXiv2025

Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Input

Multimodal Large Language Models (MLLMs) increasingly support dynamic image resolutions. However, current evaluation paradigms primarily assess semantic performance, overlooking the critical question of resolution robustness - whether performance remains stable across varying input resolutions. To address this gap, we introduce \textbf{Res-Bench}, a comprehensive benchmark comprising 14,400 samples across 12 resolution levels and six core capability dimensions. We designed a novel evaluation framework that goes beyond traditional accuracy metrics to capture performance stability. This framework introduces multiple robustness metrics: Spearman's correlation for assessing resolution-performance trends, and Absolute/Relative Continuous Error (ACE/RCE) for measuring performance volatility. Using these metrics, we conducted a large-scale evaluation of leading MLLMs. Our analysis encompasses: (1) model-centric and task-centric robustness examination, (2) investigation of preprocessing strategies including padding and super-resolution, and (3) exploration of fine-tuning for stability enhancement.

cs.CV↗
arXiv2025

Enhancing CLIP Robustness via Cross-Modality Alignment

Vision-language models (VLMs) such as CLIP demonstrate strong generalization in zero-shot classification but remain highly vulnerable to adversarial perturbations. Existing methods primarily focus on adversarial fine-tuning or prompt optimization; they often overlook the gaps in CLIP's encoded features, which is shown as the text and image features lie far apart from each other. This misalignment is significantly amplified under adversarial perturbations, leading to severe degradation in classification performance. To address this problem, we propose Cross-modality Alignment, dubbed COLA, an optimal transport-based framework that explicitly addresses adversarial misalignment by restoring both global image-text alignment and local structural consistency in the feature space. (1) COLA first projects adversarial image embeddings onto a subspace spanned by class text features, effectively filtering out non-semantic distortions while preserving discriminative information. (2) It then models images and texts as discrete distributions over multiple augmented views and refines their alignment via OT, with the subspace projection seamlessly integrated into the cost computation. This design ensures stable cross-modal alignment even under adversarial conditions. COLA is training-free and compatible with existing fine-tuned models. Extensive evaluations across 14 zero-shot classification benchmarks demonstrate the effectiveness of COLA, especially with an average improvement of 6.7% on ImageNet and its variants under PGD adversarial attacks, while maintaining high accuracy on clean samples.

cs.CV↗
arXiv2025

Parallel and perpendicular diffusion of energetic particles in the near-Sun solar wind observed by Parker Solar Probe

We investigate energetic particle diffusion in the inner heliosphere (approximately 0.06-0.3 AU) explored by Parker Solar Probe (PSP). Parallel (kappa_parallel) and perpendicular (kappa_perp) diffusion coefficients are calculated using second-order quasi-linear theory (SOQLT) and unified nonlinear transport (UNLT) theory, respectively. PSP's in situ measurements of magnetic turbulence spectra, including sub-Alfvenic solar wind, are decomposed into parallel and perpendicular wavenumber spectra via a composite two-component turbulence model. These spectra are then used to compute kappa_parallel and kappa_perp across energies ranging from sub-GeV to GeV. Our results reveal a strong energy and radial distance dependence in kappa_parallel. While kappa_perp remains much smaller, it can increase in regions with relatively high turbulence levels delta B / B0. To validate our results, we estimate kappa_parallel using the upstream time-intensity profile of a solar energetic particle event observed by PSP and compare it with theoretical values from different diffusion models. Our results suggest that the SOQLT-calculated parallel diffusion generally shows better agreement with SEP intensity-derived estimates than the classic QLT model. This indicates that the SOQLT framework, which incorporates resonance broadening and nonlinear corrections and does not require an ad hoc pitch-angle cutoff, may provide a more physically motivated description of energetic particle diffusion near the Sun.

physics.space-ph↗
arXiv2025

Accelerating Diffusion Transformer via Error-Optimized Cache

Diffusion Transformer (DiT) is a crucial method for content generation. However, it needs a lot of time to sample. Many studies have attempted to use caching to reduce the time consumption of sampling. Existing caching methods accelerate generation by reusing DiT features from the previous time step and skipping calculations in the next, but they tend to locate and cache low-error modules without focusing on reducing caching-induced errors, resulting in a sharp decline in generated content quality when increasing caching intensity. To solve this problem, we propose the \textbf{E}rror-\textbf{O}ptimized \textbf{C}ache (\textbf{EOC}). This method introduces three key improvements: \textbf{(1)} Prior knowledge extraction: Extract and process the caching differences; \textbf{(2)} A judgment method for cache optimization: Determine whether certain caching steps need to be optimized; \textbf{(3)} Cache optimization: reduce caching errors. Experiments show that this algorithm significantly reduces the error accumulation caused by caching, especially excessive caching. On the ImageNet dataset, without substantially increasing the computational load, this method improves the FID of the generated images when the rule-based model FORA has a caching level of \textbf{75}\%, \textbf{50}\%, and \textbf{25}\%, and the training-based model Learning-to-cache has a caching level of \textbf{22}\%. Specifically, the FID values change from 30.454 to 21.690 (\textbf{28.8}\%), from 6.857 to 5.821 (\textbf{15.1}\%), from 3.870 to 3.692 (\textbf{4.6}\%), and from 3.539 to 3.451 (\textbf{2.5}\%) respectively. Code is available at https://github.com/qiujx0520/EOC_MM2025.git.

cs.CV↗
arXiv2025

On the Power of Context-Enhanced Learning in LLMs

We formalize a new concept for LLMs, context-enhanced learning. It involves standard gradient-based learning on text except that the context is enhanced with additional data on which no auto-regressive gradients are computed. This setting is a gradient-based analog of usual in-context learning (ICL) and appears in some recent works. Using a multi-step reasoning task, we prove in a simplified setting that context-enhanced learning can be exponentially more sample-efficient than standard learning when the model is capable of ICL. At a mechanistic level, we find that the benefit of context-enhancement arises from a more accurate gradient learning signal. We also experimentally demonstrate that it appears hard to detect or recover learning materials that were used in the context during training. This may have implications for data security as well as copyright.

cs.LG↗
arXiv2025

Mesh Watermark Removal Attack and Mitigation: A Novel Perspective of Function Space

Mesh watermark embeds secret messages in 3D meshes and decodes the message from watermarked meshes for ownership verification. Current watermarking methods directly hide secret messages in vertex and face sets of meshes. However, mesh is a discrete representation that uses vertex and face sets to describe a continuous signal, which can be discretized in other discrete representations with different vertex and face sets. This raises the question of whether the watermark can still be verified on the different discrete representations of the watermarked mesh. We conduct this research in an attack-then-defense manner by proposing a novel function space mesh watermark removal attack FuncEvade and then mitigating it through function space mesh watermarking FuncMark. In detail, FuncEvade generates a different discrete representation of a watermarked mesh by extracting it from the signed distance function of the watermarked mesh. We observe that the generated mesh can evade ALL previous watermarking methods. FuncMark mitigates FuncEvade by watermarking signed distance function through message-guided deformation. Such deformation can survive isosurfacing and thus be inherited by the extracted meshes for further watermark decoding. Extensive experiments demonstrate that FuncEvade achieves 100% evasion rate among all previous watermarking methods while achieving only 0.3% evasion rate on FuncMark. Besides, our FuncMark performs similarly on other metrics compared to state-of-the-art mesh watermarking methods.

cs.CV↗
arXiv2024

DreaMark: Rooting Watermark in Score Distillation Sampling Generated Neural Radiance Fields

Recent advancements in text-to-3D generation can generate neural radiance fields (NeRFs) with score distillation sampling, enabling 3D asset creation without real-world data capture. With the rapid advancement in NeRF generation quality, protecting the copyright of the generated NeRF has become increasingly important. While prior works can watermark NeRFs in a post-generation way, they suffer from two vulnerabilities. First, a delay lies between NeRF generation and watermarking because the secret message is embedded into the NeRF model post-generation through fine-tuning. Second, generating a non-watermarked NeRF as an intermediate creates a potential vulnerability for theft. To address both issues, we propose Dreamark to embed a secret message by backdooring the NeRF during NeRF generation. In detail, we first pre-train a watermark decoder. Then, the Dreamark generates backdoored NeRFs in a way that the target secret message can be verified by the pre-trained watermark decoder on an arbitrary trigger viewport. We evaluate the generation quality and watermark robustness against image- and model-level attacks. Extensive experiments show that the watermarking process will not degrade the generation quality, and the watermark achieves 90+% accuracy among both image-level attacks (e.g., Gaussian noise) and model-level attacks (e.g., pruning attack).

cs.GR↗
arXiv2024

Two-dimension vanishing, splitting and positive scalar curvature

We prove several analogs of Gromov's macroscopic dimension conjecture with extra curvature assumptions. More explicitly, we show that for an open Riemannian $n$-manifold $(M,g)$ of nonnegative Ricci (resp. sectional) curvature, if it has uniformly positive scalar curvature and it is uniformly volume noncollapsed, then the essential (resp. Hausdorff) dimension of an asymptotic cone, as a notion of largeness, has a sharp upper bound $n-2$, which is $2$ less than the upper bound for an open Riemannian manifold with only nonnegative Ricci curvature. As a consequence, the dimension of space of linear growth harmonic functions of $M$ has upper bound $n-1$ which is also $2$ less than the sharp bound $n+1$ when $M$ only has nonnegative Ricci curvature. We also prove the first Betti number upper bound is $n-2$ if $M$ is compact, and $n-3$ if $M$ is non-compact. When $M$ is compact we show a fibration theorem over torus, and a rigidity theorem for the fiber when the first Betti number upper bound is achieved.

math.DG↗
arXiv2024

Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting

Vision-language models, such as CLIP, have shown impressive generalization capacities when using appropriate text descriptions. While optimizing prompts on downstream labeled data has proven effective in improving performance, these methods entail labor costs for annotations and are limited by their quality. Additionally, since CLIP is pre-trained on highly imbalanced Web-scale data, it suffers from inherent label bias that leads to suboptimal performance. To tackle the above challenges, we propose a label-Free prompt distribution learning and bias correction framework, dubbed as **Frolic**, which boosts zero-shot performance without the need for labeled data. Specifically, our Frolic learns distributions over prompt prototypes to capture diverse visual representations and adaptively fuses these with the original CLIP through confidence matching. This fused model is further enhanced by correcting label bias via a label-free logit adjustment. Notably, our method is not only training-free but also circumvents the necessity for hyper-parameter tuning. Extensive experimental results across 16 datasets demonstrate the efficacy of our approach, particularly outperforming the state-of-the-art by an average of $2.6\%$ on 10 datasets with CLIP ViT-B/16 and achieving an average margin of $1.5\%$ on ImageNet and its five distribution shifts with CLIP ViT-B/16. Codes are available in https://github.com/zhuhsingyuu/Frolic.

cs.CV↗