arXiv subjects

Yu Zhang

Publications and source records attributed to Yu Zhang.

At least 577 records · Page 32Linked to original sources

arXiv2025

TCSinger: Zero-Shot Singing Voice Synthesis with Style Transfer and Multi-Level Style Control

Zero-shot singing voice synthesis (SVS) with style transfer and style control aims to generate high-quality singing voices with unseen timbres and styles (including singing method, emotion, rhythm, technique, and pronunciation) from audio and text prompts. However, the multifaceted nature of singing styles poses a significant challenge for effective modeling, transfer, and control. Furthermore, current SVS models often fail to generate singing voices rich in stylistic nuances for unseen singers. To address these challenges, we introduce TCSinger, the first zero-shot SVS model for style transfer across cross-lingual speech and singing styles, along with multi-level style control. Specifically, TCSinger proposes three primary modules: 1) the clustering style encoder employs a clustering vector quantization model to stably condense style information into a compact latent space; 2) the Style and Duration Language Model (S\&D-LM) concurrently predicts style information and phoneme duration, which benefits both; 3) the style adaptive decoder uses a novel mel-style adaptive normalization method to generate singing voices with enhanced details. Experimental results show that TCSinger outperforms all baseline models in synthesis quality, singer similarity, and style controllability across various tasks, including zero-shot style transfer, multi-level style control, cross-lingual style transfer, and speech-to-singing style transfer. Singing voice samples can be accessed at https://aaronz345.github.io/TCSingerDemo/.

eess.AS↗

arXiv2025

TCSinger 2: Customizable Multilingual Zero-shot Singing Voice Synthesis

Customizable multilingual zero-shot singing voice synthesis (SVS) has various potential applications in music composition and short video dubbing. However, existing SVS models overly depend on phoneme and note boundary annotations, limiting their robustness in zero-shot scenarios and producing poor transitions between phonemes and notes. Moreover, they also lack effective multi-level style control via diverse prompts. To overcome these challenges, we introduce TCSinger 2, a multi-task multilingual zero-shot SVS model with style transfer and style control based on various prompts. TCSinger 2 mainly includes three key modules: 1) Blurred Boundary Content (BBC) Encoder, predicts duration, extends content embedding, and applies masking to the boundaries to enable smooth transitions. 2) Custom Audio Encoder, uses contrastive learning to extract aligned representations from singing, speech, and textual prompts. 3) Flow-based Custom Transformer, leverages Cus-MOE, with F0 supervision, enhancing both the synthesis quality and style modeling of the generated singing voice. Experimental results show that TCSinger 2 outperforms baseline models in both subjective and objective metrics across multiple related tasks. Singing voice samples are available at https://aaronz345.github.io/TCSinger2Demo/.

eess.AS↗

arXiv2025

Adversarial Semantic and Label Perturbation Attack for Pedestrian Attribute Recognition

Pedestrian Attribute Recognition (PAR) is an indispensable task in human-centered research and has made great progress in recent years with the development of deep neural networks. However, the potential vulnerability and anti-interference ability have still not been fully explored. To bridge this gap, this paper proposes the first adversarial attack and defense framework for pedestrian attribute recognition. Specifically, we exploit both global- and patch-level attacks on the pedestrian images, based on the pre-trained CLIP-based PAR framework. It first divides the input pedestrian image into non-overlapping patches and embeds them into feature embeddings using a projection layer. Meanwhile, the attribute set is expanded into sentences using prompts and embedded into attribute features using a pre-trained CLIP text encoder. A multi-modal Transformer is adopted to fuse the obtained vision and text tokens, and a feed-forward network is utilized for attribute recognition. Based on the aforementioned PAR framework, we adopt the adversarial semantic and label-perturbation to generate the adversarial noise, termed ASL-PAR. We also design a semantic offset defense strategy to suppress the influence of adversarial attacks. Extensive experiments conducted on both digital domains (i.e., PETA, PA100K, MSP60K, RAPv2) and physical domains fully validated the effectiveness of our proposed adversarial attack and defense strategies for the pedestrian attribute recognition. The source code of this paper will be released on https://github.com/Event-AHU/OpenPAR.

cs.CV↗

arXiv2025

AnchorAttention: Difference-Aware Sparse Attention with Stripe Granularity

Large Language Models (LLMs) with extended context lengths face significant computational challenges during the pre-filling phase, primarily due to the quadratic complexity of self-attention. Existing methods typically employ dynamic pattern matching and block-sparse low-level implementations. However, their reliance on local information for pattern identification fails to capture global contexts, and the coarse granularity of blocks leads to persistent internal sparsity, resulting in suboptimal accuracy and efficiency. To address these limitations, we propose \textbf{AnchorAttention}, a difference-aware, dynamic sparse attention mechanism that efficiently identifies critical attention regions at a finer stripe granularity while adapting to global contextual information, achieving superior speed and accuracy. AnchorAttention comprises three key components: (1) \textbf{Pattern-based Anchor Computation}, leveraging the commonalities present across all inputs to rapidly compute a set of near-maximum scores as the anchor; (2) \textbf{Difference-aware Stripe Sparsity Identification}, performing difference-aware comparisons with the anchor to quickly obtain discrete coordinates of significant regions in a stripe-like sparsity pattern; (3) \textbf{Fine-grained Sparse Computation}, replacing the traditional contiguous KV block loading approach with simultaneous discrete KV position loading to maximize sparsity rates while preserving full hardware computational potential. With its finer-grained sparsity strategy, \textbf{AnchorAttention} achieves higher sparsity rates at the same recall level, significantly reducing computation time. Compared to previous state-of-the-art methods, at a text length of 128k, it achieves a speedup of 1.44$\times$ while maintaining higher recall rates.

cs.LG↗

arXiv2025

DarkDiff: Advancing Low-Light Raw Enhancement by Retasking Diffusion Models for Camera ISP

High-quality photography in extreme low-light conditions is challenging but impactful for digital cameras. With advanced computing hardware, traditional camera image signal processor (ISP) algorithms are gradually being replaced by efficient deep networks that enhance noisy raw images more intelligently. However, existing regression-based models often minimize pixel errors and result in oversmoothing of low-light photos or deep shadows. Recent work has attempted to address this limitation by training a diffusion model from scratch, yet those models still struggle to recover sharp image details and accurate colors. We introduce a novel framework to enhance low-light raw images by retasking pre-trained generative diffusion models with the camera ISP. Extensive experiments demonstrate that our method outperforms the state-of-the-art in perceptual quality across three challenging low-light raw image benchmarks.

cs.CV↗

arXiv2025

AI instructional agent improves student's perceived learner control and learning outcome: empirical evidence from a randomized controlled trial

This study examines the impact of an AI instructional agent on students' perceived learner control and academic performance in a medium demanding course with lecturing as the main teaching strategy. Based on a randomized controlled trial, three instructional conditions were compared: a traditional human teacher, a self-paced MOOC with chatbot support, and an AI instructional agent capable of delivering lectures and responding to questions in real time. Students in the AI instructional agent group reported significantly higher levels of perceived learner control compared to the other groups. They also completed the learning task more efficiently and engaged in more frequent interactions with the instructional system. Regression analyzes showed that perceived learner control positively predicted post-test performance, with behavioral indicators such as reduced learning time and higher interaction frequency supporting this relationship. These findings suggest that AI instructional agents, when designed to support personalized pace and responsive interaction, can enhance both students' learning experience and learning outcomes.

cs.CY↗

arXiv2025

Corrupted but Not Broken: Understanding and Mitigating the Negative Impacts of Corrupted Data in Visual Instruction Tuning

Visual Instruction Tuning (VIT) aims to enhance Multimodal Large Language Models (MLLMs), yet its effectiveness is often compromised by corrupted datasets with issues such as hallucinated content, incorrect responses, and poor OCR quality. Previous approaches to address these challenges have focused on refining datasets through high-quality data collection or rule-based filtering that can be costly or limited in scope. In this paper, we conduct a systematic investigation into the impact of corrupted data on MLLMs and discover that, although corrupted data degrade model performance, such adverse effects are largely reversible, and MLLMs are {\bf corrupted but not broken}. Specifically, we find that disabling a small subset of parameters can almost fully restore performance. Moreover, corrupted MLLMs inherently possess the capability to differentiate between clean and corrupted samples, facilitating dataset cleaning without external intervention. Building on these insights, we introduce a corruption-robust training paradigm that significantly surpasses existing strategies for mitigating the effects of corrupted data.

cs.CV↗

arXiv2025

Large Language Model-enhanced Reinforcement Learning for Low-Altitude Economy Networking

Low-Altitude Economic Networking (LAENet) aims to support diverse flying applications below 1,000 meters by deploying various aerial vehicles for flexible and cost-effective aerial networking. However, complex decision-making, resource constraints, and environmental uncertainty pose significant challenges to the development of the LAENet. Reinforcement learning (RL) offers a potential solution in response to these challenges but has limitations in generalization, reward design, and model stability. The emergence of large language models (LLMs) offers new opportunities for RL to mitigate these limitations. In this paper, we first present a tutorial about integrating LLMs into RL by using the capacities of generation, contextual understanding, and structured reasoning of LLMs. We then propose an LLM-enhanced RL framework for the LAENet in terms of serving the LLM as information processor, reward designer, decision-maker, and generator. Moreover, we conduct a case study by using LLMs to design a reward function to improve the learning performance of RL in the LAENet. Finally, we provide a conclusion and discuss future work.

cs.AI↗

arXiv2025

Latent label distribution grid representation for modeling uncertainty

Although \textbf{L}abel \textbf{D}istribution \textbf{L}earning (LDL) has promising representation capabilities for characterizing the polysemy of an instance, the complexity and high cost of the label distribution annotation lead to inexact in the construction of the label space. The existence of a large number of inexact labels generates a label space with uncertainty, which misleads the LDL algorithm to yield incorrect decisions. To alleviate this problem, we model the uncertainty of label distributions by constructing a \textbf{L}atent \textbf{L}abel \textbf{D}istribution \textbf{G}rid (LLDG) to form a low-noise representation space. Specifically, we first construct a label correlation matrix based on the differences between labels, and then expand each value of the matrix into a vector that obeys a Gaussian distribution, thus building a LLDG to model the uncertainty of the label space. Finally, the LLDG is reconstructed by the LLDG-Mixer to generate an accurate label distribution. Note that we enforce a customized low-rank scheme on this grid, which assumes that the label relations may be noisy and it needs to perform noise-reduction with the help of a Tucker reconstruction technique. Furthermore, we attempt to evaluate the effectiveness of the LLDG by considering its generation as an upstream task to achieve the classification of the objects. Extensive experimental results show that our approach performs competitively on several benchmarks.

cs.LG↗

arXiv2025

Burst Image Super-Resolution via Multi-Cross Attention Encoding and Multi-Scan State-Space Decoding

Multi-image super-resolution (MISR) can achieve higher image quality than single-image super-resolution (SISR) by aggregating sub-pixel information from multiple spatially shifted frames. Among MISR tasks, burst super-resolution (BurstSR) has gained significant attention due to its wide range of applications. Recent methods have increasingly adopted Transformers over convolutional neural networks (CNNs) in super-resolution tasks, due to their superior ability to capture both local and global context. However, most existing approaches still rely on fixed and narrow attention windows that restrict the perception of features beyond the local field. This limitation hampers alignment and feature aggregation, both of which are crucial for high-quality super-resolution. To address these limitations, we propose a novel feature extractor that incorporates two newly designed attention mechanisms: overlapping cross-window attention and cross-frame attention, enabling more precise and efficient extraction of sub-pixel information across multiple frames. Furthermore, we introduce a Multi-scan State-Space Module with the cross-frame attention mechanism to enhance feature aggregation. Extensive experiments on both synthetic and real-world benchmarks demonstrate the superiority of our approach. Additional evaluations on ISO 12233 resolution test charts further confirm its enhanced super-resolution performance.

cs.CV↗

arXiv2025

HaloGS: Loose Coupling of Compact Geometry and Gaussian Splats for 3D Scenes

High fidelity 3D reconstruction and rendering hinge on capturing precise geometry while preserving photo realistic detail. Most existing methods either fuse these goals into a single cumbersome model or adopt hybrid schemes whose uniform primitives lead to a trade off between efficiency and fidelity. In this paper, we introduce HaloGS, a dual representation that loosely couples coarse triangles for geometry with Gaussian primitives for appearance, motivated by the lightweight classic geometry representations and their proven efficiency in real world applications. Our design yields a compact yet expressive model capable of photo realistic rendering across both indoor and outdoor environments, seamlessly adapting to varying levels of scene complexity. Experiments on multiple benchmark datasets demonstrate that our method yields both compact, accurate geometry and high fidelity renderings, especially in challenging scenarios where robust geometric structure make a clear difference.

cs.CV↗

arXiv2025

MOSLIM:Align with diverse preferences in prompts through reward classification

The multi-objective alignment of Large Language Models (LLMs) is essential for ensuring foundational models conform to diverse human preferences. Current research in this field typically involves either multiple policies or multiple reward models customized for various preferences, or the need to train a preference-specific supervised fine-tuning (SFT) model. In this work, we introduce a novel multi-objective alignment method, MOSLIM, which utilizes a single reward model and policy model to address diverse objectives. MOSLIM provides a flexible way to control these objectives through prompting and does not require preference training during SFT phase, allowing thousands of off-the-shelf models to be directly utilized within this training framework. MOSLIM leverages a multi-head reward model that classifies question-answer pairs instead of scoring them and then optimize policy model with a scalar reward derived from a mapping function that converts classification results from reward model into reward scores. We demonstrate the efficacy of our proposed method across several multi-objective benchmarks and conduct ablation studies on various reward model sizes and policy optimization methods. The MOSLIM method outperforms current multi-objective approaches in most results while requiring significantly fewer GPU computing resources compared with existing policy optimization methods.

cs.CL↗

arXiv2025

A Unified Taxonomy-Guided Instruction Tuning Framework for Entity Set Expansion and Taxonomy Expansion

Entity set expansion, taxonomy expansion, and seed-guided taxonomy construction are three representative tasks that can be applied to automatically populate an existing taxonomy with emerging concepts. Previous studies view them as three separate tasks. Therefore, their proposed techniques usually work for one specific task only, lacking generalizability and a holistic perspective. In this paper, we aim at a unified solution to the three tasks. To be specific, we identify two common skills needed for entity set expansion, taxonomy expansion, and seed-guided taxonomy construction: finding "siblings" and finding "parents". We propose a taxonomy-guided instruction tuning framework to teach a large language model to generate siblings and parents for query entities, where the joint pre-training process facilitates the mutual enhancement of the two skills. Extensive experiments on multiple benchmark datasets demonstrate the efficacy of our proposed TaxoInstruct framework, which outperforms task-specific baselines across all three tasks.

cs.CL↗

arXiv2025

Make Silence Speak for Itself: a multi-modal learning analytic approach with neurophysiological data

Background: Silence is a common phenomenon in classrooms, yet its implicit nature limits a clear understanding of students' underlying learning statuses. Aim: This study proposed a nuanced framework to classify classroom silence based on class events and student status, and examined neurophysiological markers to reveal similarities and differences in silent states across achievement groups. Sample: The study involved 54 middle school students during 34 math lessons, with simultaneous recordings of electroencephalogram (EEG), electrodermal activity (EDA), and heart rate signals, alongside video coding of classroom behaviors. Results: We found that high-achieving students showed no significant difference in mean EDA features between strategic silence (i.e., students choose silence deliberately) and active speaking during open questioning but exhibited higher EEG high-frequency relative power spectral density (RPSD) during strategic silence. In structural silence (i.e., students maintain silence following an external command) during directed questioning, they demonstrated significantly higher heart rates while listening to lectures compared to group activities, indicating heightened engagement. Both high- and medium-achieving students displayed elevated heart rates and EDA tonic components in structural silence during questioning compared to teaching. Furthermore, high-achieving students exhibited lower high-frequency RPSD during structural silence than strategic silence, a pattern not observed in other groups, highlighting group heterogeneity. Conclusions: The findings contribute to validating the complexity of silence, challenge its traditional association with passivity, and offer a novel classification framework along with preliminary empirical evidence to deepen the understanding of silent learning behaviors in classroom contexts.

q-bio.NC↗

arXiv2025

Sampling of Graph Signals Based on Joint Time-Vertex Fractional Fourier Transform

With the growing demand for non-Euclidean data analysis, graph signal processing (GSP) has gained significant attention for its capability to handle complex time-varying data. This paper introduces a novel sampling method based on the joint time-vertex fractional Fourier transform (JFRFT), enhancing signal representation in time-frequency analysis and GSP. The JFRFT sampling theory is established by deriving conditions for the perfect recovery of jointly bandlimited signals, along with an optimal sampling set selection strategy. To further enhance the efficiency of large-scale time-vertex signal processing, the design of localized sampling operators is investigated. Numerical simulations and real data experiments validate the superior performance of the proposed methods in terms of recovery accuracy and computational efficiency, offering new insights into efficient time-varying signal processing.

math.GM↗

arXiv2025

Auxiliary Field Quantum Monte Carlo for Electron-Photon Correlation

Hybrid light-matter polaritonic states have shown great promise for altering already known and enabling novel chemical reactions and controlling photophysical phenomena. This field has recently become one of the most prominent and active areas of research that connects the communities of chemistry and quantum optics. The ab initio modeling of such polaritonic phenomena has led to updating commonly used electronic structure methods, such as Hartree-Fock, density functional, and coupled cluster theories, to explicitly include Bosonic degrees of freedom. In this work, we explore the quantum electrodynamic auxiliary field quantum Monte Carlo (QED-AFQMC) method to accurately capture the polaritonic ground state of representative quantum chemical benchmark systems to explore electron-photon correlations. We analyze these correlations across multiple examples and benchmark the QED-AFQMC results against other ab initio quantum electrodynamics methods, including QED-coupled cluster and QED-full configuration interaction, demonstrating the method's accuracy and its potential for scalable simulations of strongly coupled light-matter systems.

quant-ph↗

arXiv2025

Internal and External Impacts of Natural Language Processing Papers

We investigate the impacts of NLP research published in top-tier conferences (i.e., ACL, EMNLP, and NAACL) from 1979 to 2024. By analyzing citations from research articles and external sources such as patents, media, and policy documents, we examine how different NLP topics are consumed both within the academic community and by the broader public. Our findings reveal that language modeling has the widest internal and external influence, while linguistic foundations have lower impacts. We also observe that internal and external impacts generally align, but topics like ethics, bias, and fairness show significant attention in policy documents with much fewer academic citations. Additionally, external domains exhibit distinct preferences, with patents focusing on practical NLP applications and media and policy documents engaging more with the societal implications of NLP models.

cs.CL↗

arXiv2025

Reconstruction of Graph Signals on Complex Manifolds with Kernel Methods

Graph signals are widely used to describe vertex attributes or features in graph-structured data, with applications spanning the internet, social media, transportation, sensor networks, and biomedicine. Graph signal processing (GSP) has emerged to facilitate the analysis, processing, and sampling of such signals. While kernel methods have been extensively studied for estimating graph signals from samples provided on a subset of vertices, their application to complex-valued graph signals remains largely unexplored. This paper introduces a novel framework for reconstructing graph signals using kernel methods on complex manifolds. By embedding graph vertices into a higher-dimensional complex ambient space that approximates a lower-dimensional manifold, the framework extends the reproducing kernel Hilbert space to complex manifolds. It leverages Hermitian metrics and geometric measures to characterize kernels and graph signals. Additionally, several traditional kernels and graph topology-driven kernels are proposed for reconstructing complex graph signals. Finally, experimental results on synthetic and real-world datasets demonstrate the effectiveness of this framework in accurately reconstructing complex graph signals, outperforming conventional kernel-based approaches. This work lays a foundational basis for integrating complex geometry and kernel methods in GSP.

eess.SP↗