arXiv subjects

Jingru Li

Publications and source records attributed to Jingru Li.

8 recordsLinked to original sources

arXiv2026

Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijacking

Large Vision-Language Models (LVLMs) rely on attention-based retrieval of safety instructions to maintain alignment during generation. Existing attacks typically optimize image perturbations to maximize harmful output likelihood, but suffer from slow convergence due to gradient conflict between adversarial objectives and the model's safety-retrieval mechanism. We propose Attention-Guided Visual Jailbreaking, which circumvents rather than overpowers safety alignment by directly manipulating attention patterns. Our method introduces two simple auxiliary objectives: (1) suppressing attention to alignment-relevant prefix tokens and (2) anchoring generation on adversarial image features. This simple yet effective push-pull formulation reduces gradient conflict by 45% and achieves 94.4% attack success rate on Qwen-VL (vs. 68.8% baseline) with 40% fewer iterations. At tighter perturbation budgets ($\epsilon=8/255$), we maintain 59.0% ASR compared to 45.7% for standard methods. Mechanistic analysis reveals a failure mode we term safety blindness: successful attacks suppress system-prompt attention by 80%, causing models to generate harmful content not by overriding safety rules, but by failing to retrieve them.

cs.CV↗

arXiv2026

Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining

Optimizer design plays a central role in efficient language model pretraining, directly affecting optimization dynamics, convergence speed, and compute cost under fixed training budgets. Muon has emerged as a strong optimizer by orthogonalizing momentum updates, yielding a matrix-valued analogue of sign-based normalization. However, unlike Adam-style methods, Muon does not explicitly incorporate gradient-variance information into its updates. Motivated by Adam's variance-adaptive interpretation, we propose Muon-NSR and Muon-VS, two variance-adaptive Muon variants for language model pretraining. Muon-NSR applies noise-to-signal ratio (NSR) modulation before Newton--Schulz orthogonalization, whereas Muon-VS uses variance scaling (VS) without introducing any additional hyperparameters beyond those of Muon. Both methods preserve Muon's spectral normalization structure while requiring only one additional variance buffer. Experiments on Llama-style and GPT-2 pretraining across model scales from 125M to 1.2B parameters show that our methods improve over well-tuned Muon baselines and remain competitive with representative adaptive Muon-family baselines. On Llama-1.2B, Muon-VS achieves a 1.33$\times$ step-to-target speedup over a well-tuned Muon baseline, with Muon's final validation loss as the target. These results indicate that variance-adaptive modulation is a simple and effective mechanism for improving Muon-style optimizers in language model pretraining.

cs.LG↗

arXiv2025

Giant field-tunable nonlinear Hall effect by Lorentz skew scattering in a graphene moire superlattice

The nonlinear Hall effect (NHE) can enable rectification and energy harvesting, and its control by external fields, including gate, strain and magnetic field, has been pursued intensively. However, existing tuning pathways rely predominantly on fully quantum mechanical effects and are typically inefficient, resulting in weak NHE signals that limit further progress. In this work, we report the discovery of a distinct type of NHE in a graphene-hBN moire superlattice, which arises from a classical-quantum cooperative effect called Lorentz skew scattering (LSK), induced by a perpendicular magnetic field. This field-driven NHE exhibits a linear dependence on magnetic field and a pronounced unidirectional angular dependence. Remarkably, its magnitude reaches up to 32% of the linear Hall signal. We show that this giant, field-tunable NHE originating from LSK follows a unique quartic scaling law and produces a record-high nonlinear Hall conductivity (36000 {\mu}mV-1{\Omega}-1) near van Hove singularities of moire minibands, which is over an order of magnitude larger than all previously reported NHEs. Our findings establish an efficient, magnetic-field-driven route to giant Hall rectification in high-mobility materials, offering a broadly applicable paradigm for modulating the NHE beyond electrostatic gating.

cond-mat.mes-hall↗

arXiv2025

RainFusion: Adaptive Video Generation Acceleration via Multi-Dimensional Visual Redundancy

Video generation using diffusion models is highly computationally intensive, with 3D attention in Diffusion Transformer (DiT) models accounting for over 80\% of the total computational resources. In this work, we introduce {\bf RainFusion}, a novel training-free sparse attention method that exploits inherent sparsity nature in visual data to accelerate attention computation while preserving video quality. Specifically, we identify three unique sparse patterns in video generation attention calculations--Spatial Pattern, Temporal Pattern and Textural Pattern. The sparse pattern for each attention head is determined online with negligible overhead (\textasciitilde\,0.2\%) with our proposed {\bf ARM} (Adaptive Recognition Module) during inference. Our proposed {\bf RainFusion} is a plug-and-play method, that can be seamlessly integrated into state-of-the-art 3D-attention video generation models without additional training or calibration. We evaluate our method on leading open-sourced models including HunyuanVideo, OpenSoraPlan-1.2 and CogVideoX-5B, demonstrating its broad applicability and effectiveness. Experimental results show that RainFusion achieves over {\bf 2$\times$} speedup in attention computation while maintaining video quality, with only a minimal impact on VBench scores (-0.2\%).

cs.CV↗

arXiv2025

AFD-STA: Adaptive Filtering Denoising with Spatiotemporal Attention for Chaotic System Prediction

This paper presents AFD-STA Net, a neural framework integrating adaptive filtering and spatiotemporal dynamics learning for predicting high-dimensional chaotic systems governed by partial differential equations. The architecture combines: 1) An adaptive exponential smoothing module with position-aware decay coefficients for robust attractor reconstruction, 2) Parallel attention mechanisms capturing cross-temporal and spatial dependencies, 3) Dynamic gated fusion of multiscale features, and 4) Deep projection networks with dimension-scaling capabilities. Numerical experiments on nonlinear PDE systems demonstrate the model's effectiveness in maintaining prediction accuracy under both smooth and strongly chaotic regimes while exhibiting noise tolerance through adaptive filtering. Component ablation studies confirm critical contributions from each module, particularly highlighting the essential role of spatiotemporal attention in learning complex dynamical interactions. The framework shows promising potential for real-world applications requiring simultaneous handling of measurement uncertainties and high-dimensional nonlinear dynamics.

cs.LG↗

arXiv2025

Observation of giant nonlinear valley Hall effect

The valley Hall effect (VHE) holds great promise for valleytronic applications by leveraging the valley degree of freedom. To date, research on VHE has focused on its linear response to an applied current, leaving nonlinear valley responses undetected and nonlinear valleytronic devices undeveloped. Here, we report the experimental observation of a nonlinear VHE in a graphene-hBN moire superlattice, evidenced by the generation of second-harmonic nonlocal voltages under AC currents. Remarkably, the nonlinear VHE has magnitude surpassing the linear VHE and is highly tunable via a gate voltage, which exhibits a pair of opposite peaks on the two sides of a Dirac gap. The nonlinear signal shows quadratic scaling with driving current and quartic scaling with local resistance, setting it apart from the linear counterpart. These experimental features are consistent with the theoretical picture of nonlocal transport mediated by nonlinear VHE and linear inverse VHE. We further reveal a nonlinear inverse VHE by observing the third- and fourth-harmonic nonlocal voltages. The nonlinear VHE provides a novel mechanism for valley manipulation and enables a novel valleytronic device, the valley rectifier, that converts AC charge current into DC valley current.

cond-mat.mes-hall↗

arXiv2023

Dynamic Data-Free Knowledge Distillation by Easy-to-Hard Learning Strategy

Data-free knowledge distillation (DFKD) is a widely-used strategy for Knowledge Distillation (KD) whose training data is not available. It trains a lightweight student model with the aid of a large pretrained teacher model without any access to training data. However, existing DFKD methods suffer from inadequate and unstable training process, as they do not adjust the generation target dynamically based on the status of the student model during learning. To address this limitation, we propose a novel DFKD method called CuDFKD. It teaches students by a dynamic strategy that gradually generates easy-to-hard pseudo samples, mirroring how humans learn. Besides, CuDFKD adapts the generation target dynamically according to the status of student model. Moreover, We provide a theoretical analysis of the majorization minimization (MM) algorithm and explain the convergence of CuDFKD. To measure the robustness and fidelity of DFKD methods, we propose two more metrics, and experiments shows CuDFKD has comparable performance to state-of-the-art (SOTA) DFKD methods on all datasets. Experiments also present that our CuDFKD has the fastest convergence and best robustness over other SOTA DFKD methods.

cs.CV↗

arXiv2019

Musical Instrument Playing Technique Detection Based on FCN: Using Chinese Bowed-Stringed Instrument as an Example

Unlike melody extraction and other aspects of music transcription, research on playing technique detection is still in its early stages. Compared to existing work mostly focused on playing technique detection for individual single notes, we propose a general end-to-end method based on Sound Event Detection by FCN for musical instrument playing technique detection. In our case, we choose Erhu, a well-known Chinese bowed-stringed instrument, to experiment with our method. Because of the limitation of FCN, we present an algorithm to detect on variable length audio. The effectiveness of the proposed framework is tested on a new dataset, its categorization of techniques is similar to our training dataset. The highest accuracy of our 3 experiments on the new test set is 87.31%. Furthermore, we also evaluate the performance of the proposed framework on 10 real-world studio music (produced by midi) and 7 real-world recording samples to address the ability of generalization on our model.

cs.SD↗