arXiv subjects

Xuedong Zhang

Publications and source records attributed to Xuedong Zhang.

15 recordsLinked to original sources

arXiv2026

Movable Antenna-Enabled Integrated Sensing and Communication in Low-Altitude UAV Networks

This paper investigates a multiple unmanned aerial vehicle (UAV)-assisted integrated sensing and communication (ISAC) system equipped with movable antenna (MA) arrays. To align with practical scenarios, we simulate the dynamic roaming of ground users and the three-dimensional deployment of UAVs in the airspace. We aim to maximize the total data rate by jointly optimizing key operational variables, including UAV trajectories, user association, antenna positions, and beamforming. This formulated problem is subject to constraints on transmission power and the sensing signal-to-noise ratio. To address the challenge of dynamically unknown state transitions due to user mobility, the original problem is decomposed into two steps and solved using different algorithms. First, we utilize the hierarchical density-based spatial clustering of applications with noise (HDBSCAN) algorithm to address the ground-to-air association problem, periodically updating clusters and re-associating during training. The clustering hotspots are used to suggest flight directions for the UAVs. Second, we develop the soft actor-critic algorithm to solve the joint optimization problem of UAV trajectories, antenna positions, and beamforming. Experimental results demonstrate that UAVs equipped with MA arrays outperform those with traditional fixed antenna arrays in ISAC systems, and the proposed optimization strategy effectively enhances communication rates while ensuring sensing performance.

eess.SP↗

arXiv2026

Beyond Words: Measuring User Experience through Speech Analysis in Voice User Interfaces

Voice assistants (VAs) are typically evaluated through task performance metrics and self-report questionnaires, but people's voices themselves carry rich paralinguistic cues that reveal affect, effort, and interaction breakdowns. We present a within-subjects study (N=49) that systematically compared three VA personas across three usage scenarios to investigate whether speech-derived audio features can serve as a proxy for user experience (UX). Participants' speech was analyzed for temporal, spectral, and linguistic markers, alongside standardized UX measures, brief mood and stress ratings, and a post-study questionnaire. We found correlations between specific speech features and self-reported satisfaction and experience. Furthermore, a machine learning model trained on speech features achieved promising accuracy in classifying UX levels, indicating that this might be a reasonable alternative to self-report instruments. Our findings establish speech as a viable, real-time signal for implicitly measuring UX and point toward adaptive VUIs that respond dynamically to emotional and usability-related vocal cues.

cs.HC↗

arXiv2026

VowelPrompt: Hearing Speech Emotions from Text via Vowel-level Prosodic Augmentation

Emotion recognition in speech presents a complex multimodal challenge, requiring comprehension of both linguistic content and vocal expressivity, particularly prosodic features such as fundamental frequency, intensity, and temporal dynamics. Although large language models (LLMs) have shown promise in reasoning over textual transcriptions for emotion recognition, they typically neglect fine-grained prosodic information, limiting their effectiveness and interpretability. In this work, we propose VowelPrompt, a linguistically grounded framework that augments LLM-based emotion recognition with interpretable, fine-grained vowel-level prosodic cues. Drawing on phonetic evidence that vowels serve as primary carriers of affective prosody, VowelPrompt extracts pitch-, energy-, and duration-based descriptors from time-aligned vowel segments, and converts these features into natural language descriptions for better interpretability. Such a design enables LLMs to jointly reason over lexical semantics and fine-grained prosodic variation. Moreover, we adopt a two-stage adaptation procedure comprising supervised fine-tuning (SFT) followed by Reinforcement Learning with Verifiable Reward (RLVR), implemented via Group Relative Policy Optimization (GRPO), to enhance reasoning capability, enforce structured output adherence, and improve generalization across domains and speaker variations. Extensive evaluations across diverse benchmark datasets demonstrate that VowelPrompt consistently outperforms state-of-the-art emotion recognition methods under zero-shot, fine-tuned, cross-domain, and cross-linguistic conditions, while enabling the generation of interpretable explanations that are jointly grounded in contextual semantics and fine-grained prosodic structure.

cs.CL↗

arXiv2025

Super-bunching light with giant high-order correlations and extreme multi-photon events

Non-classical light sources emitting bundles of N-photons with strong correlation represent versatile resources of interdisciplinary importance with applications ranging from fundamental tests of quantum mechanics to quantum information processing. Yet, high-order correlations, gN(0),quantifying photon correlation, are still limited to hundreds. Here, we report the generation of a super-bunching light source in photonic crystal fiber with g2(0) reaching 5.86*104 and g5(0) up to 2.72*108, through measuring its photon number probability distributions. under giant g2(0) values, the super-bunching light source presents upturned-tail photon distributions and ubiquitous extreme multi-photon events, where 31 photons from a single light pulse at a mean of 1.99*10-4 photons per pulse have been determined. The probability of this extreme event has been enhanced by 10139 folds compared to a coherent laser with Poissonian distribution. By varying the power of the pumping laser, both photon number distributions and corresponding high-order correlations of this light source can be substantially tailored from Poissonian to super-bunching distributions. These phenomena are attributed to the synchronized nonlinear interactions in photonic crystal fibers pumping by bright squeezed light, and the theoretical simulations agree well with the experimental results. Our research showcases the ability to achieve non-classical light sources with giant high-order correlations and extreme multi-photon events, paving the way for high-order correlation imaging, extreme nonlinear optical effects, quantum information processing, and exploring light-matter interactions with multi-photon physics.

quant-ph↗

arXiv2025

SignDiff: Diffusion Model for American Sign Language Production

In this paper, we propose a dual-condition diffusion pre-training model named SignDiff that can generate human sign language speakers from a skeleton pose. SignDiff has a novel Frame Reinforcement Network called FR-Net, similar to dense human pose estimation work, which enhances the correspondence between text lexical symbols and sign language dense pose frames, reduces the occurrence of multiple fingers in the diffusion model. In addition, we propose a new method for American Sign Language Production (ASLP), which can generate ASL skeletal pose videos from text input, integrating two new improved modules and a new loss function to improve the accuracy and quality of sign language skeletal posture and enhance the ability of the model to train on large-scale data. We propose the first baseline for ASL production and report the scores of 17.19 and 12.85 on BLEU-4 on the How2Sign dev/test sets. We evaluated our model on the previous mainstream dataset PHOENIX14T, and the experiments achieved the SOTA results. In addition, our image quality far exceeds all previous results by 10 percentage points in terms of SSIM.

cs.CV↗

arXiv2025

Measuring User Experience Through Speech Analysis: Insights from HCI Interviews

User satisfaction plays a crucial role in user experience (UX) evaluation. Traditionally, UX measurements are based on subjective scales, such as questionnaires. However, these evaluations may suffer from subjective bias. In this paper, we explore the acoustic and prosodic features of speech to differentiate between positive and neutral UX during interactive sessions. By analyzing speech features such as root-mean-square (RMS), zero-crossing rate(ZCR), jitter, and shimmer, we identified significant differences between the positive and neutral user groups. In addition, social speech features such as activity and engagement also show notable variations between these groups. Our findings underscore the potential of speech analysis as an objective and reliable tool for UX measurement, contributing to more robust and bias-resistant evaluation methodologies. This work offers a novel approach to integrating speech features into UX evaluation and opens avenues for further research in HCI.

cs.HC↗

arXiv2025

On-demand manipulation of superbunching emission from colloidal quantum dots and its application in noise-resistance correlated biphoton imaging

Superbunching effect with second-order correlations larger than 2, $g^{(2)}(0)>2$, indicating the N-photon bundles emission and strong correlation among photons, has a broad range of fascinating applications in quantum illumination, communication, and computation. However, the on-demand manipulation of the superbunching effect in colloidal quantum dots (QDs) under pulsed excitation, which is beneficial to integrated photonics and lab-on-a-chip quantum devices, is still challenging. Here, we disclosed the evolution of $g^{(2)}(0)$ with the parameters of colloidal QDs by Mento Carlo simulations and performed second-order correlation measurements on CdSe/ZnS core/shell QDs under both continuous wave (CW) and pulsed lasers. The photon statistics of a single colloidal QD have been substantially tailored from sub-Poissonian distribution $g^{(2)}(0) <1$) to superbunching emission, with the maximum $g^{(2)}(0)$ reaching 69 and 20 under CW and pulsed excitation, respectively. We have achieved correlated biphoton imaging (CPI), employing the coincidence of the biexciton and bright exciton in one laser pulse, with the stray light and background noise up to 53 times stronger than PL emission of single colloidal QDs. By modulating the PL intensity, the Fourier-domain CPI with reasonably good contrast has been determined, with the stray light noise up to 107 times stronger than PL emission and 75600 times stronger than the counts of biphotons. Our noise-resistance CPI may enable laboratory-based quantum imaging to be applied to real-world applications with the highly desired suppression of strong background noise and stray light.

physics.optics↗

arXiv2025

A Domain Adaptation Framework for Speech Recognition Systems with Only Synthetic data

We introduce DAS (Domain Adaptation with Synthetic data), a novel domain adaptation framework for pre-trained ASR model, designed to efficiently adapt to various language-defined domains without requiring any real data. In particular, DAS first prompts large language models (LLMs) to generate domain-specific texts before converting these texts to speech via text-to-speech technology. The synthetic data is used to fine-tune Whisper with Low-Rank Adapters (LoRAs) for targeted domains such as music, weather, and sports. We introduce a novel one-pass decoding strategy that merges predictions from multiple LoRA adapters efficiently during the auto-regressive text generation process. Experimental results show significant improvements, reducing the Word Error Rate (WER) by 10% to 17% across all target domains compared to the original model, with minimal performance regression in out-of-domain settings (e.g., -1% on Librispeech test sets). We also demonstrate that DAS operates efficiently during inference, introducing an additional 9% increase in Real Time Factor (RTF) compared to the original model when inferring with three LoRA adapters.

eess.AS↗

arXiv2024

LLaMA based Punctuation Restoration With Forward Pass Only Decoding

This paper introduces two advancements in the field of Large Language Model Annotation with a focus on punctuation restoration tasks. Our first contribution is the application of LLaMA for punctuation restoration, which demonstrates superior performance compared to the established benchmark. Despite its impressive quality, LLaMA faces challenges regarding inference speed and hallucinations. To address this, our second contribution presents Forward Pass Only Decoding (FPOD), a novel decoding approach for annotation tasks. This innovative method results in a substantial 19.8x improvement in inference speed, effectively addressing a critical bottleneck and enhancing the practical utility of LLaMA for large-scale data annotation tasks without hallucinations. The combination of these contributions not only solidifies LLaMA as a powerful tool for punctuation restoration but also highlights FPOD as a crucial strategy for overcoming speed constraints.

cs.CL↗

arXiv2024

Towards scalable efficient on-device ASR with transfer learning

Multilingual pretraining for transfer learning significantly boosts the robustness of low-resource monolingual ASR models. This study systematically investigates three main aspects: (a) the impact of transfer learning on model performance during initial training or fine-tuning, (b) the influence of transfer learning across dataset domains and languages, and (c) the effect on rare-word recognition compared to non-rare words. Our finding suggests that RNNT-loss pretraining, followed by monolingual fine-tuning with Minimum Word Error Rate (MinWER) loss, consistently reduces Word Error Rates (WER) across languages like Italian and French. WER Reductions (WERR) reach 36.2% and 42.8% compared to monolingual baselines for MLS and in-house datasets. Out-of-domain pretraining leads to 28% higher WERR than in-domain pretraining. Both rare and non-rare words benefit, with rare words showing greater improvements with out-of-domain pretraining, and non-rare words with in-domain pretraining.

cs.CL↗

arXiv2023

Language Agnostic Data-Driven Inverse Text Normalization

With the emergence of automatic speech recognition (ASR) models, converting the spoken form text (from ASR) to the written form is in urgent need. This inverse text normalization (ITN) problem attracts the attention of researchers from various fields. Recently, several works show that data-driven ITN methods can output high-quality written form text. Due to the scarcity of labeled spoken-written datasets, the studies on non-English data-driven ITN are quite limited. In this work, we propose a language-agnostic data-driven ITN framework to fill this gap. Specifically, we leverage the data augmentation in conjunction with neural machine translated data for low resource languages. Moreover, we design an evaluation method for language agnostic ITN model when only English data is available. Our empirical evaluation shows this language agnostic modeling approach is effective for low resource languages while preserving the performance for high resource languages.

cs.CL↗

arXiv2022

Adaptive Multi-Corpora Language Model Training for Speech Recognition

Neural network language model (NNLM) plays an essential role in automatic speech recognition (ASR) systems, especially in adaptation tasks when text-only data is available. In practice, an NNLM is typically trained on a combination of data sampled from multiple corpora. Thus, the data sampling strategy is important to the adaptation performance. Most existing works focus on designing static sampling strategies. However, each corpus may show varying impacts at different NNLM training stages. In this paper, we introduce a novel adaptive multi-corpora training algorithm that dynamically learns and adjusts the sampling probability of each corpus along the training process. The algorithm is robust to corpora sizes and domain relevance. Compared with static sampling strategy baselines, the proposed approach yields remarkable improvement by achieving up to relative 7% and 9% word error rate (WER) reductions on in-domain and out-of-domain adaptation tasks, respectively.

eess.AS↗

arXiv2022

Mitigating Unintended Memorization in Language Models via Alternating Teaching

Recent research has shown that language models have a tendency to memorize rare or unique sequences in the training corpora which can thus leak sensitive attributes of user data. We employ a teacher-student framework and propose a novel approach called alternating teaching to mitigate unintended memorization in sequential modeling. In our method, multiple teachers are trained on disjoint training sets whose privacy one wishes to protect, and teachers' predictions supervise the training of a student model in an alternating manner at each time step. Experiments on LibriSpeech datasets show that the proposed method achieves superior privacy-preserving results than other counterparts. In comparison with no prevention for unintended memorization, the overall utility loss is small when training records are sufficient.

cs.CL↗

arXiv2022

Improving Data Driven Inverse Text Normalization using Data Augmentation

Inverse text normalization (ITN) is used to convert the spoken form output of an automatic speech recognition (ASR) system to a written form. Traditional handcrafted ITN rules can be complex to transcribe and maintain. Meanwhile neural modeling approaches require quality large-scale spoken-written pair examples in the same or similar domain as the ASR system (in-domain data), to train. Both these approaches require costly and complex annotations. In this paper, we present a data augmentation technique that effectively generates rich spoken-written numeric pairs from out-of-domain textual data with minimal human annotation. We empirically demonstrate that ITN model trained using our data augmentation technique consistently outperform ITN model trained using only in-domain data across all numeric surfaces like cardinal, currency, and fraction, by an overall accuracy of 14.44%.

cs.CL↗

arXiv2018

Speech recognition for medical conversations

In this work we explored building automatic speech recognition models for transcribing doctor patient conversation. We collected a large scale dataset of clinical conversations ($14,000$ hr), designed the task to represent the real word scenario, and explored several alignment approaches to iteratively improve data quality. We explored both CTC and LAS systems for building speech recognition models. The LAS was more resilient to noisy data and CTC required more data clean up. A detailed analysis is provided for understanding the performance for clinical tasks. Our analysis showed the speech recognition models performed well on important medical utterances, while errors occurred in causal conversations. Overall we believe the resulting models can provide reasonable quality in practice.

cs.CL↗