arXiv subjects

Yan Yang

Publications and source records attributed to Yan Yang.

At least 73 records · Page 4Linked to original sources

arXiv2025

NTIRE 2025 Image Shadow Removal Challenge Report

This work examines the findings of the NTIRE 2025 Shadow Removal Challenge. A total of 306 participants have registered, with 17 teams successfully submitting their solutions during the final evaluation phase. Following the last two editions, this challenge had two evaluation tracks: one focusing on reconstruction fidelity and the other on visual perception through a user study. Both tracks were evaluated with images from the WSRD+ dataset, simulating interactions between self- and cast-shadows with a large number of diverse objects, textures, and materials.

cs.CV↗

arXiv2025

ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs

With the proliferation of task-specific large language models, delta compression has emerged as a method to mitigate the resource challenges of deploying numerous such models by effectively compressing the delta model parameters. Previous delta-sparsification methods either remove parameters randomly or truncate singular vectors directly after singular value decomposition (SVD). However, these methods either disregard parameter importance entirely or evaluate it with too coarse a granularity. In this work, we introduce ImPart, a novel importance-aware delta sparsification approach. Leveraging SVD, it dynamically adjusts sparsity ratios of different singular vectors based on their importance, effectively retaining crucial task-specific knowledge even at high sparsity ratios. Experiments show that ImPart achieves state-of-the-art delta sparsification performance, demonstrating $2\times$ higher compression ratio than baselines at the same performance level. When integrated with existing methods, ImPart sets a new state-of-the-art on delta quantization and model merging.

cs.CL↗

arXiv2025

Revisiting compressible and incompressible pressure-strain interaction in kinetic plasma turbulence

In this study, we revisit the pressure-strain interaction in kinetic turbulence, and in particular we re-examine the decomposition of pressure-strain interaction into compressive and incompressive parts. The pressure dilatation ingredient is clearly due to plasma compressions, but here using kinetic particle-in-cell (PIC) simulations of plasma turbulence, it is demonstrated that the remaining anisotropic part, often called Pi-D, also contains contributions due to compressive, non solenoidal velocities of the particle species. The compressive Pi-D can play a significant role in systems with low plasma $β$ even if the system starts with small density variations. The compressive ingredient of Pi-D is found to be anticorrelated with both incompressive Pi-D and pressure dilatation.

physics.plasm-ph↗

arXiv2025

ProBench: Judging Multimodal Foundation Models on Open-ended Multi-domain Expert Tasks

Solving expert-level multimodal tasks is a key milestone towards general intelligence. As the capabilities of multimodal large language models (MLLMs) continue to improve, evaluation of such advanced multimodal intelligence becomes necessary yet challenging. In this work, we introduce ProBench, a benchmark of open-ended user queries that require professional expertise and advanced reasoning. ProBench consists of 4,000 high-quality samples independently submitted by professionals based on their daily productivity demands. It spans across 10 fields and 56 sub-fields, including science, arts, humanities, coding, mathematics, and creative writing. Experimentally, we evaluate and compare 24 latest models using MLLM-as-a-Judge. Our results reveal that although the best open-source models rival the proprietary ones, ProBench presents significant challenges in visual perception, textual understanding, domain knowledge and advanced reasoning, thus providing valuable directions for future multimodal AI research efforts.

cs.CV↗

arXiv2025

Dirichlet Spaces In Balls And Half-spaces of $\R^n$

The present paper studies the Dirichlet spaces in balls and upper-half Euclidean spaces. As main results, we give identical characterizations of the Dirichlet norms in the respective contexts as for the classical 2-D disc case proved by Douglas and Ahlfors.

math.FA↗

arXiv2025

SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters

The widespread applications of large language models (LLMs) have brought about concerns regarding their potential misuse. Although aligned with human preference data before release, LLMs remain vulnerable to various malicious attacks. In this paper, we adopt a red-teaming strategy to enhance LLM safety and introduce SeqAR, a simple yet effective framework to design jailbreak prompts automatically. The SeqAR framework generates and optimizes multiple jailbreak characters and then applies sequential jailbreak characters in a single query to bypass the guardrails of the target LLM. Different from previous work which relies on proprietary LLMs or seed jailbreak templates crafted by human expertise, SeqAR can generate and optimize the jailbreak prompt in a cold-start scenario using open-sourced LLMs without any seed jailbreak templates. Experimental results show that SeqAR achieves attack success rates of 88% and 60% in bypassing the safety alignment of GPT-3.5-1106 and GPT-4, respectively. Furthermore, we extensively evaluate the transferability of the generated templates across different LLMs and held-out malicious requests, while also exploring defense strategies against the jailbreak attack designed by SeqAR.

cs.CR↗

arXiv2025

Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexity

Bilevel reinforcement learning (RL), which features intertwined two-level problems, has attracted growing interest recently. The inherent non-convexity of the lower-level RL problem is, however, to be an impediment to developing bilevel optimization methods. By employing the fixed point equation associated with the regularized RL, we characterize the hyper-gradient via fully first-order information, thus circumventing the assumption of lower-level convexity. This, remarkably, distinguishes our development of hyper-gradient from the general AID-based bilevel frameworks since we take advantage of the specific structure of RL problems. Moreover, we design both model-based and model-free bilevel reinforcement learning algorithms, facilitated by access to the fully first-order hyper-gradient. Both algorithms enjoy the convergence rate $O(ε^{-1})$. To extend the applicability, a stochastic version of the model-free algorithm is proposed, along with results on its iteration and sample complexity. In addition, numerical experiments demonstrate that the hyper-gradient indeed serves as an integration of exploitation and exploration.

math.OC↗

arXiv2025

LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspace

Bilevel optimization, with broad applications in machine learning, has an intricate hierarchical structure. Gradient-based methods have emerged as a common approach to large-scale bilevel problems. However, the computation of the hyper-gradient, which involves a Hessian inverse vector product, confines the efficiency and is regarded as a bottleneck. To circumvent the inverse, we construct a sequence of low-dimensional approximate Krylov subspaces with the aid of the Lanczos process. As a result, the constructed subspace is able to dynamically and incrementally approximate the Hessian inverse vector product with less effort and thus leads to a favorable estimate of the hyper-gradient. Moreover, we propose a provable subspace-based framework for bilevel problems where one central step is to solve a small-size tridiagonal linear system. To the best of our knowledge, this is the first time that subspace techniques are incorporated into bilevel optimization. This successful trial not only enjoys $\mathcal{O}(ε^{-1})$ convergence rate but also demonstrates efficiency in a synthetic problem and two deep learning tasks.

math.OC↗

arXiv2025

High Resolution Tree Height Mapping of the Amazon Forest using Planet NICFI Images and LiDAR-Informed U-Net Model

Tree canopy height is one of the most important indicators of forest biomass, productivity, and ecosystem structure, but it is challenging to measure accurately from the ground and from space. Here, we used a U-Net model adapted for regression to map the mean tree canopy height in the Amazon forest from Planet NICFI images at ~4.78 m spatial resolution for the period 2020-2024. The U-Net model was trained using canopy height models computed from aerial LiDAR data as a reference, along with their corresponding Planet NICFI images. Predictions of tree heights on the validation sample exhibited a mean error of 3.68 m and showed relatively low systematic bias across the entire range of tree heights present in the Amazon forest. Our model successfully estimated canopy heights up to 40-50 m without much saturation, outperforming existing canopy height products from global models in this region. We determined that the Amazon forest has an average canopy height of ~22 m. Events such as logging or deforestation could be detected from changes in tree height, and encouraging results were obtained to monitor the height of regenerating forests. These findings demonstrate the potential for large-scale mapping and monitoring of tree height for old and regenerating Amazon forests using Planet NICFI imagery.

cs.CV↗

arXiv2024

Exploring Temporal Event Cues for Dense Video Captioning in Cyclic Co-learning

Dense video captioning aims to detect and describe all events in untrimmed videos. This paper presents a dense video captioning network called Multi-Concept Cyclic Learning (MCCL), which aims to: (1) detect multiple concepts at the frame level, using these concepts to enhance video features and provide temporal event cues; and (2) design cyclic co-learning between the generator and the localizer within the captioning network to promote semantic perception and event localization. Specifically, we perform weakly supervised concept detection for each frame, and the detected concept embeddings are integrated into the video features to provide event cues. Additionally, video-level concept contrastive learning is introduced to obtain more discriminative concept embeddings. In the captioning network, we establish a cyclic co-learning strategy where the generator guides the localizer for event localization through semantic matching, while the localizer enhances the generator's event semantic perception through location matching, making semantic perception and event localization mutually beneficial. MCCL achieves state-of-the-art performance on the ActivityNet Captions and YouCook2 datasets. Extensive experiments demonstrate its effectiveness and interpretability.

cs.CV↗

arXiv2024

$1/f$ Noise in the Heliosphere: A Target for PUNCH Science

We present a broad review of 1/f noise observations in the heliosphere, and discuss and complement the theoretical background of generic 1/f models as relevant to NASA's PUNCH mission. First observed in the voltage fluctuations of vacuum tubes, the scale-invariant 1/f spectrum has since been identified across a wide array of natural and artificial systems, including heart rate fluctuations and loudness patterns in musical compositions. In the solar wind, the interplanetary magnetic field trace spectrum exhibits 1/f scaling within the frequency range from around 2e-6 Hz to around 1e-3 Hz at 1 au. One compelling mechanism for the generation of 1/f noise is the superposition principle, where a composite 1/f spectrum arises from the superposition of a collection of individual power-law spectra characterized by a scale-invariant distribution of correlation times. In the context of the solar wind, such a superposition could originate from scale-invariant reconnection processes in the corona. Further observations have detected 1/f signatures in the photosphere and corona at frequency ranges compatible with those observed at 1 au, suggesting an even lower altitude origin of 1/f spectrum in the solar dynamo itself. This hypothesis is bolstered by dynamo experiments and simulations that indicate inverse cascade activities, which can be linked to successive flux tube reconnections beneath the corona, and are known to generate 1/f noise possibly through nonlocal interactions at the largest scales. Conversely, models positing in situ generation of $1/f$ signals face causality issues in explaining the low-frequency portion of the 1/f spectrum. Understanding 1/f noise in the solar wind may inform central problems in heliospheric physics, such as the solar dynamo, coronal heating, the origin of the solar wind, and the nature of interplanetary turbulence.

astro-ph.SR↗

arXiv2024

EZSR: Event-based Zero-Shot Recognition

This paper studies zero-shot object recognition using event camera data. Guided by CLIP, which is pre-trained on RGB images, existing approaches achieve zero-shot object recognition by optimizing embedding similarities between event data and RGB images respectively encoded by an event encoder and the CLIP image encoder. Alternatively, several methods learn RGB frame reconstructions from event data for the CLIP image encoder. However, they often result in suboptimal zero-shot performance. This study develops an event encoder without relying on additional reconstruction networks. We theoretically analyze the performance bottlenecks of previous approaches: the embedding optimization objectives are prone to suffer from the spatial sparsity of event data, causing semantic misalignments between the learned event embedding space and the CLIP text embedding space. To mitigate the issue, we explore a scalar-wise modulation strategy. Furthermore, to scale up the number of events and RGB data pairs for training, we also study a pipeline for synthesizing event data from static RGB images in mass. Experimentally, we demonstrate an attractive scaling property in the number of parameters and synthesized data. We achieve superior zero-shot object recognition performance on extensive standard benchmark datasets, even compared with past supervised learning approaches. For example, our model with a ViT/B-16 backbone achieves 47.84% zero-shot accuracy on the N-ImageNet dataset.

cs.CV↗

arXiv2024

Higher Dimensional Versions of the Douglas-Ahlfors Identities

Denote by ${\mathcal D}$ the open unit disc in the complex plane and $\partial {\mathcal D}$ its boundary. Douglas showed through an identical quantity represented by the Fourier coefficients of the concerned function $u$ that \begin{eqnarray}\label{abs} A(u)=\int_{\mathcal D}|\bigtriangledown U|^2dxdy&=&\frac{1}{2π}\int\int_{\partial {\mathcal D}\times \partial {\mathcal D}} \left|\frac{u(z_1)-u(z_2)}{z_1-z_2}\right|^2|dz_1||dz_2|,\end{eqnarray} \end{abstract} where $u\in L^2(\partial {\mathcal D}), U$ is the harmonic extension of $u$ into ${\mathcal D}$. Ahlfors gave a fourth equivalence form of $A(u)$ in (\ref{more}) via a different proof. The present article studies relations between the counterpart quantities in higher dimensional spheres with several different but commonly adopted settings, namely, harmonic functions in the Euclidean ${\mathbb R}^n, n\ge 2,$ regular functions in the quaternionic algebra, and Clifford monogenic functions with the real-Clifford algebra ${\mathcal{CL}}_{0, n-1},$ the latter being generated by the multiplication anti-commutative basic imaginary units ${\e}_1, {\e}_2, \cdots , {\e}_{n-1}$ with ${\e}_j^2=-1, j=1, 2, \cdots, n-1.$ It is noted that, while exactly the same equivalence relations hold for harmonic functions in ${\mathbb R}^n$ and regular functions in the quaternionic algebra, for the Clifford algebra setting $n>2,$ the relation (\ref{more}) has to be replaced by essentially a different rule.

math.CV↗

arXiv2024

SpikMamba: When SNN meets Mamba in Event-based Human Action Recognition

Human action recognition (HAR) plays a key role in various applications such as video analysis, surveillance, autonomous driving, robotics, and healthcare. Most HAR algorithms are developed from RGB images, which capture detailed visual information. However, these algorithms raise concerns in privacy-sensitive environments due to the recording of identifiable features. Event cameras offer a promising solution by capturing scene brightness changes sparsely at the pixel level, without capturing full images. Moreover, event cameras have high dynamic ranges that can effectively handle scenarios with complex lighting conditions, such as low light or high contrast environments. However, using event cameras introduces challenges in modeling the spatially sparse and high temporal resolution event data for HAR. To address these issues, we propose the SpikMamba framework, which combines the energy efficiency of spiking neural networks and the long sequence modeling capability of Mamba to efficiently capture global features from spatially sparse and high a temporal resolution event data. Additionally, to improve the locality of modeling, a spiking window-based linear attention mechanism is used. Extensive experiments show that SpikMamba achieves remarkable recognition performance, surpassing the previous state-of-the-art by 1.45%, 7.22%, 0.15%, and 3.92% on the PAF, HARDVS, DVS128, and E-FAction datasets, respectively. The code is available at https://github.com/Typistchen/SpikMamba.

cs.CV↗

arXiv2024

LMHaze: Intensity-aware Image Dehazing with a Large-scale Multi-intensity Real Haze Dataset

Image dehazing has drawn a significant attention in recent years. Learning-based methods usually require paired hazy and corresponding ground truth (haze-free) images for training. However, it is difficult to collect real-world image pairs, which prevents developments of existing methods. Although several works partially alleviate this issue by using synthetic datasets or small-scale real datasets. The haze intensity distribution bias and scene homogeneity in existing datasets limit the generalization ability of these methods, particularly when encountering images with previously unseen haze intensities. In this work, we present LMHaze, a large-scale, high-quality real-world dataset. LMHaze comprises paired hazy and haze-free images captured in diverse indoor and outdoor environments, spanning multiple scenarios and haze intensities. It contains over 5K high-resolution image pairs, surpassing the size of the biggest existing real-world dehazing dataset by over 25 times. Meanwhile, to better handle images with different haze intensities, we propose a mixture-of-experts model based on Mamba (MoE-Mamba) for dehazing, which dynamically adjusts the model parameters according to the haze intensity. Moreover, with our proposed dataset, we conduct a new large multimodal model (LMM)-based benchmark study to simulate human perception for evaluating dehazed images. Experiments demonstrate that LMHaze dataset improves the dehazing performance in real scenarios and our dehazing method provides better results compared to state-of-the-art methods.

cs.CV↗

arXiv2024

Storyboard guided Alignment for Fine-grained Video Action Recognition

Fine-grained video action recognition can be conceptualized as a video-text matching problem. Previous approaches often rely on global video semantics to consolidate video embeddings, which can lead to misalignment in video-text pairs due to a lack of understanding of action semantics at an atomic granularity level. To tackle this challenge, we propose a multi-granularity framework based on two observations: (i) videos with different global semantics may share similar atomic actions or appearances, and (ii) atomic actions within a video can be momentary, slow, or even non-directly related to the global video semantics. Inspired by the concept of storyboarding, which disassembles a script into individual shots, we enhance global video semantics by generating fine-grained descriptions using a pre-trained large language model. These detailed descriptions capture common atomic actions depicted in videos. A filtering metric is proposed to select the descriptions that correspond to the atomic actions present in both the videos and the descriptions. By employing global semantics and fine-grained descriptions, we can identify key frames in videos and utilize them to aggregate embeddings, thereby making the embedding more accurate. Extensive experiments on various video action recognition datasets demonstrate superior performance of our proposed method in supervised, few-shot, and zero-shot settings.

cs.CV↗

arXiv2024

Distract Large Language Models for Automatic Jailbreak Attack

Extensive efforts have been made before the public release of Large language models (LLMs) to align their behaviors with human values. However, even meticulously aligned LLMs remain vulnerable to malicious manipulations such as jailbreaking, leading to unintended behaviors. In this work, we propose a novel black-box jailbreak framework for automated red teaming of LLMs. We designed malicious content concealing and memory reframing with an iterative optimization algorithm to jailbreak LLMs, motivated by the research about the distractibility and over-confidence phenomenon of LLMs. Extensive experiments of jailbreaking both open-source and proprietary LLMs demonstrate the superiority of our framework in terms of effectiveness, scalability and transferability. We also evaluate the effectiveness of existing jailbreak defense methods against our attack and highlight the crucial need to develop more effective and practical defense strategies.

cs.CR↗

arXiv2024

Event Camera Data Dense Pre-training

This paper introduces a self-supervised learning framework designed for pre-training neural networks tailored to dense prediction tasks using event camera data. Our approach utilizes solely event data for training. Transferring achievements from dense RGB pre-training directly to event camera data yields subpar performance. This is attributed to the spatial sparsity inherent in an event image (converted from event data), where many pixels do not contain information. To mitigate this sparsity issue, we encode an event image into event patch features, automatically mine contextual similarity relationships among patches, group the patch features into distinctive contexts, and enforce context-to-context similarities to learn discriminative event features. For training our framework, we curate a synthetic event camera dataset featuring diverse scene and motion patterns. Transfer learning performance on downstream dense prediction tasks illustrates the superiority of our method over state-of-the-art approaches.

cs.CV↗