arXiv subjects

Yue Cao

Publications and source records attributed to Yue Cao.

At least 55 records · Page 3Linked to original sources

arXiv2025

Sequential Diffusion Language Models

Diffusion language models (DLMs) have strong theoretical efficiency but are limited by fixed-length decoding and incompatibility with key-value (KV) caches. Block diffusion mitigates these issues, yet still enforces a fixed block size and requires expensive training. We introduce Next Sequence Prediction (NSP), which unifies next-token and next-block prediction, enabling the model to adaptively determine the generation length at each step. When the length is fixed to 1, NSP reduces to standard next-token prediction. Building on NSP, we propose Sequential Diffusion Language Model (SDLM), which can retrofit pre-trained autoregressive language models (ALMs) at minimal cost. Specifically, SDLM performs diffusion inference within fixed-size mask blocks, but dynamically decodes consecutive subsequences based on model confidence, thereby preserving KV-cache compatibility and improving robustness to varying uncertainty and semantics across the sequence. Experiments show that SDLM matches or surpasses strong autoregressive baselines using only 3.5M training samples, while achieving 2.1 higher throughput than Qwen-2.5. Notably, the SDLM-32B model delivers even more pronounced efficiency gains, demonstrating the strong scalability potential of our modeling paradigm. Project page and codes: https://github.com/OpenGVLab/SDLM

cs.CL↗

arXiv2025

Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling

We introduce InternVL 2.5, an advanced multimodal large language model (MLLM) series that builds upon InternVL 2.0, maintaining its core model architecture while introducing significant enhancements in training and testing strategies as well as data quality. In this work, we delve into the relationship between model scaling and performance, systematically exploring the performance trends in vision encoders, language models, dataset sizes, and test-time configurations. Through extensive evaluations on a wide range of benchmarks, including multi-discipline reasoning, document understanding, multi-image / video understanding, real-world comprehension, multimodal hallucination detection, visual grounding, multilingual capabilities, and pure language processing, InternVL 2.5 exhibits competitive performance, rivaling leading commercial models such as GPT-4o and Claude-3.5-Sonnet. Notably, our model is the first open-source MLLMs to surpass 70% on the MMMU benchmark, achieving a 3.7-point improvement through Chain-of-Thought (CoT) reasoning and showcasing strong potential for test-time scaling. We hope this model contributes to the open-source community by setting new standards for developing and applying multimodal AI systems. HuggingFace demo see https://huggingface.co/spaces/OpenGVLab/InternVL

cs.CV↗

arXiv2025

FilterFL: Knowledge Filtering-based Data-Free Backdoor Defense for Federated Learning

As a distributed machine learning paradigm, Federated Learning (FL) enables large-scale clients to collaboratively train a model without sharing their raw data. However, due to the lack of data auditing for untrusted clients, FL is vulnerable to poisoning attacks, especially backdoor attacks. By using poisoned data for local training or directly changing the model parameters, attackers can easily inject backdoors into the model, which can trigger the model to make misclassification of targeted patterns in images. To address these issues, we propose a novel data-free trigger-generation-based defense approach based on the two characteristics of backdoor attacks: i) triggers are learned faster than normal knowledge, and ii) trigger patterns have a greater effect on image classification than normal class patterns. Our approach generates the images with newly learned knowledge by identifying the differences between the old and new global models, and filters trigger images by evaluating the effect of these generated images. By using these trigger images, our approach eliminates poisoned models to ensure the updated global model is benign. Comprehensive experiments demonstrate that our approach can defend against almost all the existing types of backdoor attacks and outperform all the seven state-of-the-art defense methods with both IID and non-IID scenarios. Especially, our approach can successfully defend against the backdoor attack even when 80\% of the clients are malicious.

cs.LG↗

arXiv2025

Terahertz-field activation of polar skyrons

Unraveling collective modes arising from coupled degrees of freedom is crucial for understanding complex interactions in solids and developing new functionalities. Unique collective behaviors emerge when two degrees of freedom, ordered on distinct length scales, interact. Polar skyrmions, three-dimensional electric polarization textures in ferroelectric superlattices, disrupt the lattice continuity at the nanometer scale with nontrivial topology, leading to previously unexplored collective modes. Here, using terahertz-field excitation and femtosecond x-ray diffraction, we discovered subterahertz collective modes, dubbed 'skyrons', which appear as swirling patterns of atomic displacements functioning as atomic-scale gearsets. Momentum-resolved time-domain measurements of diffuse scattering revealed an avoided crossing in the dispersion relation of skyrons. We further demonstrated that the amplitude and dispersion of skyrons can be controlled by sample temperature and electric-field bias. Atomistic simulations and dynamical phase-field modeling provided microscopic insights into the three-dimensional crystallographic and polarization dynamics. The discovery of skyrons and their coupling with terahertz fields opens avenues for ultrafast control of topological polar structures.

cond-mat.mtrl-sci↗

arXiv2025

Generating Cylindrical Vector γ Rays via Beam-Target Interactions: Towards Structured Light at High Energies

Structured γ rays, particularly cylindrical vector γ rays, offer promising tools for sub-nuclear imaging and polarization-sensitive probes in fundamental research and applications, but conventional optical methods face great challenges at such photon energy. Here, we put forward a novel method generating such γ rays through relativistic beam-target interactions. For instance, radially polarized γ rays can be generated by using a dense electron beam striking a multifoil target. We find that the radial polarization is transferred from the generated coherent transition radiation (CTR) fields to $γ$ photons through nonlinear Compton scattering, with the high polarization preserved by phase matching. Three-dimensional spin-resolved simulations demonstrate radial polarization degrees approaching 60\%. Furthermore, these γ rays can decay into azimuthally spin-polarized positrons via the nonlinear Breit-Wheeler process, with their spins aligning along the CTR magnetic field. Our work extends the concept of structured light into the γ-ray regime, offering new prospects for broad fields such as nuclear structure probing, fundamental symmetries tests, polarization-sensitive studies in extreme conditions, and laboratory astrophysical observations.

physics.optics↗

arXiv2025

Lattice-charge coupling in a trilayer nickelate with intertwined density wave order

Intertwined charge and spin correlations are ubiquitous in a wide range of transition metal oxides and are often perceived as intimately related to unconventional superconductivity. Theoretically envisioned as driven by strong electronic correlations, the intertwined order is usually found to be strongly coupled to the lattice as signaled by pronounced phonon softening. Recently, both charge/spin density waves (CDW/SDW) and superconductivity have been discovered in several Ruddlesden-Popper (RP) nickelates, in particular trilayer nickelates RE4Ni3O10 (RE=Pr, La). The nature of the intertwined order and the role of lattice-charge coupling are at the heart of the debate about these materials. Using inelastic X-ray scattering, we mapped the phonon dispersions in RE4Ni3O10 and found no evidence of phonon softening near the CDW wavevector over a wide temperature range. Calculations of the electronic susceptibility revealed a peak at the observed SDW ordering vector but not at the CDW wavevector. The absence of phonon softening is in sharp contrast to that in canonical oxide materials, notably cuprates. Our experimental and theoretical findings highlight the crucial role of the spin degree of freedom and establish a foundation for understanding the interplay between superconductivity and density-wave transitions in RP nickelate superconductors and beyond.

cond-mat.str-el↗

arXiv2025

Trotterization, Operator Scrambling, and Entanglement

Operator scrambling, which governs the spread of quantum information in many-body systems, is a central concept in both condensed matter and high-energy physics. Accurately capturing the emergent properties of these systems remains a formidable challenge for classical computation, while quantum simulators have emerged as a powerful tool to address this complexity. In this work, we reveal a fundamental connection between operator scrambling and the reliability of quantum simulations. We show that the Trotter error in simulating operator dynamics is bounded by the degree of operator scrambling, providing the most refined analysis of Trotter errors in operator dynamics so far. Furthermore, we investigate the entanglement properties of the evolved states, revealing that sufficient entanglement can lead to error scaling governed by the normalized Frobenius norms of both the observables of interest and the error operator, thereby enhancing simulation robustness and efficiency compared to previous works. We also show that even in regimes where the system's entanglement remains low, operator-induced entanglement can still emerge and suppress simulation errors. Our results unveil a comprehensive relationship between Trotterization, operator scrambling, and entanglement, offering new perspectives for optimizing quantum simulations.

quant-ph↗

arXiv2025

SurfAAV: Design and Implementation of a Novel Multimodal Surfing Aquatic-Aerial Vehicle

Despite significant advancements in the research of aquatic-aerial robots, existing configurations struggle to efficiently perform underwater, surface, and aerial movement simultaneously. In this paper, we propose a novel multimodal surfing aquatic-aerial vehicle, SurfAAV, which efficiently integrates underwater navigation, surface gliding, and aerial flying capabilities. Thanks to the design of the novel differential thrust vectoring hydrofoil, SurfAAV can achieve efficient surface gliding and underwater navigation without the need for a buoyancy adjustment system. This design provides flexible operational capabilities for both surface and underwater tasks, enabling the robot to quickly carry out underwater monitoring activities. Additionally, when it is necessary to reach another water body, SurfAAV can switch to aerial mode through a gliding takeoff, flying to the target water area to perform corresponding tasks. The main contribution of this letter lies in proposing a new solution for underwater, surface, and aerial movement, designing a novel hybrid prototype concept, developing the required control laws, and validating the robot's ability to successfully perform surface gliding and gliding takeoff. SurfAAV achieves a maximum surface gliding speed of 7.96 m/s and a maximum underwater speed of 3.1 m/s. The prototype's surface gliding maneuverability and underwater cruising maneuverability both exceed those of existing aquatic-aerial vehicles.

cs.RO↗

arXiv2025

Accuracy and Precision of Random Walk with Barrier Model Fitting: Simulations and Applications in Head and Neck Cancers

Promising results have been reported in quantifying microstructural parameters (free diffusivity, cell size, and membrane permeability) in head and neck cancers (HNCs) using time-dependent diffusion MRI fitted to Random Walk with Barrier Model (RWBM). However, model fitting remains challenging due to limited number of measurements, low signal-to-noise ratio and complex nonlinear biophysical model. In this work, we comprehensively investigated and elucidated the dependence of RWBM fitting performance on tissue property, data acquisition and processing, and provided insights on improving data acquisition and model fitting. We numerically evaluated the accuracy and precision of RWBM fitting using non-linear least squares as a function of model parameters, noise levels, and maximum effective diffusion times over a wide range of microstructural parameter values. We then elucidated these results by examining the model's degeneracy and fitting landscape. In vivo fitting results on patients with HNCs were analyzed and were interpreted using the numerical results. We observed that free diffusivity estimates were accurate and precise over a wide range of microstructural parameters, whereas the accuracy and precision of surface-to-volume ratio and membrane permeability depended on the values of the parameters. In addition, a maximum effective diffusion time of 200 ms provided the lowest bias and variance in membrane permeability estimations. By fitting to the short-time limit expression, we observed that the variance of parameter estimation was reduced, and that a more accurate and precise estimation of membrane permeability was achieved. In vivo fitting results were consistent with the numerical findings. In conclusion, this work provides a comprehensive analysis of RWBM fitting in clinical settings and has the potential to guide further optimizations of data acquisition and model fitting methods.

physics.med-ph↗

arXiv2025

The Future of the Correlated Electron Problem

A central problem in modern condensed matter physics is the understanding of materials with strong electron correlations. Despite extensive work, the essential physics of many of these systems is not understood and there is very little ability to make predictions in this class of materials. In this manuscript we share our personal views on the major open problems in the field of correlated electron systems. We discuss some possible routes to make progress in this rich and fascinating field. This manuscript is the result of the vigorous discussions and deliberations that took place at Johns Hopkins University during a three-day workshop January 27, 28, and 29, 2020 that brought together six senior scientists and 46 more junior scientists. Our hope, is that the topics we have presented will provide inspiration for others working in this field and motivation for the idea that significant progress can be made on very hard problems if we focus our collective energies.

cond-mat.str-el↗

arXiv2025

Learning Novel Skills from Language-Generated Demonstrations

Robots are increasingly deployed across diverse domains to tackle tasks requiring novel skills. However, current robot learning algorithms for acquiring novel skills often rely on demonstration datasets or environment interactions, resulting in high labor costs and potential safety risks. To address these challenges, this study proposes DemoGen, a skill-learning framework that enables robots to acquire novel skills from natural language instructions. DemoGen leverages the vision-language model and the video diffusion model to generate demonstration videos of novel skills, which enabling robots to learn new skills effectively. Experimental evaluations in the MetaWorld simulation environments demonstrate the pipeline's capability to generate high-fidelity and reliable demonstrations. Using the generated demonstrations, various skill learning algorithms achieve an accomplishment rate three times the original on novel tasks. These results highlight a novel approach to robot learning, offering a foundation for the intuitive and intelligent acquisition of novel robotic skills. (Project website: https://aoqunjin.github.io/LNSLGD/)

cs.RO↗

arXiv2025

MAGI-1: Autoregressive Video Generation at Scale

We present MAGI-1, a world model that generates videos by autoregressively predicting a sequence of video chunks, defined as fixed-length segments of consecutive frames. Trained to denoise per-chunk noise that increases monotonically over time, MAGI-1 enables causal temporal modeling and naturally supports streaming generation. It achieves strong performance on image-to-video (I2V) tasks conditioned on text instructions, providing high temporal consistency and scalability, which are made possible by several algorithmic innovations and a dedicated infrastructure stack. MAGI-1 facilitates controllable generation via chunk-wise prompting and supports real-time, memory-efficient deployment by maintaining constant peak inference cost, regardless of video length. The largest variant of MAGI-1 comprises 24 billion parameters and supports context lengths of up to 4 million tokens, demonstrating the scalability and robustness of our approach. The code and models are available at https://github.com/SandAI-org/MAGI-1 and https://github.com/SandAI-org/MagiAttention. The product can be accessed at https://sand.ai.

cs.CV↗

arXiv2025

InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

We introduce InternVL3, a significant advancement in the InternVL series featuring a native multimodal pre-training paradigm. Rather than adapting a text-only large language model (LLM) into a multimodal large language model (MLLM) that supports visual inputs, InternVL3 jointly acquires multimodal and linguistic capabilities from both diverse multimodal data and pure-text corpora during a single pre-training stage. This unified training paradigm effectively addresses the complexities and alignment challenges commonly encountered in conventional post-hoc training pipelines for MLLMs. To further improve performance and scalability, InternVL3 incorporates variable visual position encoding (V2PE) to support extended multimodal contexts, employs advanced post-training techniques such as supervised fine-tuning (SFT) and mixed preference optimization (MPO), and adopts test-time scaling strategies alongside an optimized training infrastructure. Extensive empirical evaluations demonstrate that InternVL3 delivers superior performance across a wide range of multi-modal tasks. In particular, InternVL3-78B achieves a score of 72.2 on the MMMU benchmark, setting a new state-of-the-art among open-source MLLMs. Its capabilities remain highly competitive with leading proprietary models, including ChatGPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Pro, while also maintaining strong pure-language proficiency. In pursuit of open-science principles, we will publicly release both the training data and model weights to foster further research and development in next-generation MLLMs.

cs.CV↗

arXiv2025

Local-in-time well-posedness for the regular solution to the 2D full compressible Navier-Stokes equations with degenerate viscosities and heat conductivity

This paper considers the two-dimensional Cauchy problem of the full compressible Navier-Stokes equations with far-field vacuum in $\mathbb{R}^2$, where the viscosity and heat-conductivity coefficients depend on the absolute temperature $θ$ in the form of $θ^ν$ with $ν>0$. Due to the appearance of the vacuum, the momentum equation are both degenerate in the time evolution and spatial dissipation, which makes the study on the well-posedness challenged. By establishing some new singular-weighted (negative powers of the density $ρ$) estimates of the solution, we establish the local-in-time well-posedness of the regular solution with far-field vacuum in terms of $ρ$, the velocity $u$ and the entropy $S$.

math.AP↗

arXiv2025

SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments

Large vision-language models (LVLMs) have shown remarkable capabilities in interpreting visual content. While existing works demonstrate these models' vulnerability to deliberately placed adversarial texts, such texts are often easily identifiable as anomalous. In this paper, we present the first approach to generate scene-coherent typographic adversarial attacks that mislead advanced LVLMs while maintaining visual naturalness through the capability of the LLM-based agent. Our approach addresses three critical questions: what adversarial text to generate, where to place it within the scene, and how to integrate it seamlessly. We propose a training-free, multi-modal LLM-driven scene-coherent typographic adversarial planning (SceneTAP) that employs a three-stage process: scene understanding, adversarial planning, and seamless integration. The SceneTAP utilizes chain-of-thought reasoning to comprehend the scene, formulate effective adversarial text, strategically plan its placement, and provide detailed instructions for natural integration within the image. This is followed by a scene-coherent TextDiffuser that executes the attack using a local diffusion mechanism. We extend our method to real-world scenarios by printing and placing generated patches in physical environments, demonstrating its practical implications. Extensive experiments show that our scene-coherent adversarial text successfully misleads state-of-the-art LVLMs, including ChatGPT-4o, even after capturing new images of physical setups. Our evaluations demonstrate a significant increase in attack success rates while maintaining visual naturalness and contextual appropriateness. This work highlights vulnerabilities in current vision-language models to sophisticated, scene-coherent adversarial attacks and provides insights into potential defense mechanisms.

cs.CV↗

arXiv2025

Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization

Existing open-source multimodal large language models (MLLMs) generally follow a training process involving pre-training and supervised fine-tuning. However, these models suffer from distribution shifts, which limit their multimodal reasoning, particularly in the Chain-of-Thought (CoT) performance. To address this, we introduce a preference optimization (PO) process to enhance the multimodal reasoning capabilities of MLLMs. Specifically, (1) on the data side, we design an automated preference data construction pipeline to create MMPR, a high-quality, large-scale multimodal reasoning preference dataset; and (2) on the model side, we explore integrating PO with MLLMs, developing a simple yet effective method, termed Mixed Preference Optimization (MPO), which boosts multimodal CoT performance. Our approach enhances the multimodal reasoning abilities of both InternVL2-8B and InternVL2-76B. Notably, our model, InternVL2-8B-MPO, achieves an accuracy of 67.0 on MathVista, outperforming InternVL2-8B by 8.7 points and achieving performance comparable to the 10$\times$ larger InternVL2-76B. We hope this study could inspire further advancements in MLLMs. Code, data, and model are released.

cs.CL↗

arXiv2025

VisualPRM: An Effective Process Reward Model for Multimodal Reasoning

We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and families with Best-of-N (BoN) evaluation strategies. Specifically, our model improves the reasoning performance of three types of MLLMs and four different model scales. Even when applied to the highly capable InternVL2.5-78B, it achieves a 5.9-point improvement across seven multimodal reasoning benchmarks. Experimental results show that our model exhibits superior performance compared to Outcome Reward Models and Self-Consistency during BoN evaluation. To facilitate the training of multimodal PRMs, we construct a multimodal process supervision dataset VisualPRM400K using an automated data pipeline. For the evaluation of multimodal PRMs, we propose VisualProcessBench, a benchmark with human-annotated step-wise correctness labels, to measure the abilities of PRMs to detect erroneous steps in multimodal reasoning tasks. We hope that our work can inspire more future research and contribute to the development of MLLMs. Our model, data, and benchmark are released in https://internvl.github.io/blog/2025-03-13-VisualPRM/.

cs.CV↗

arXiv2025

MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents

Physical adversarial attacks in driving scenarios can expose critical vulnerabilities in visual perception models. However, developing such attacks remains challenging due to diverse real-world environments and the requirement for maintaining visual naturality. Building upon this challenge, we reformulate physical adversarial attacks as a one-shot patch generation problem. Our approach generates adversarial patches through a deep generative model that considers the specific scene context, enabling direct physical deployment in matching environments. The primary challenge lies in simultaneously achieving two objectives: generating adversarial patches that effectively mislead object detection systems while determining contextually appropriate deployment within the scene. We propose MAGIC (Mastering Physical Adversarial Generation In Context), a novel framework powered by multi-modal LLM agents to address these challenges. MAGIC automatically understands scene context and generates adversarial patch through the synergistic interaction of language and vision capabilities. In particular, MAGIC orchestrates three specialized LLM agents: The adv-patch generation agent (GAgent) masters the creation of deceptive patches through strategic prompt engineering for text-to-image models. The adv-patch deployment agent (DAgent) ensures contextual coherence by determining optimal deployment strategies based on scene understanding. The self-examination agent (EAgent) completes this trilogy by providing critical oversight and iterative refinement of both processes. We validate our method on both digital and physical levels, i.e., nuImage and manually captured real-world scenes, where both statistical and visual results prove that our MAGIC is powerful and effective for attacking widely applied object detection systems, i.e., YOLO and DETR series.

cs.CV↗