Han-Gil Moon | Searcharxiv

arXiv2026

FlowW2N: Whispered-to-Normal Speech Conversion via Flow-Matching

Whispered-to-normal (W2N) speech conversion aims to reconstruct missing phonation from whispered input while preserving content and speaker identity. This task is challenging due to temporal misalignment between whisper and voiced recordings and lack of paired data. We propose FlowW2N, a conditional flow matching approach that trains exclusively on synthetic, time-aligned whisper-normal pairs and conditions on domain-invariant features. We exploit high-level ASR embeddings that exhibits strong invariance between synthetic and real whispered speech, enabling generalization to real whispers despite never observing it during training. We verify this invariance across ASR layers and propose a selection criterion optimizing content informativeness and cross-domain invariance. Our method achieves SOTA intelligibility on the CHAINS and wTIMIT datasets, reducing Word Error Rate by 26-46% relative to prior work while using only 10 steps at inference and requiring no real paired data.

eess.AS↗

arXiv2022

Instance-level loss based multiple-instance learning framework for acoustic scene classification

In the acoustic scene classification (ASC) task, an acoustic scene consists of diverse sounds and is inferred by identifying combinations of distinct attributes among them. This study aims to extract and cluster these attributes effectively using an improved multiple-instance learning (MIL) framework for ASC. MIL, known as a weakly supervised learning method, is a strategy for extracting an instance from a bundle of frames composing an input audio clip and inferring a scene corresponding to the input data using these unlabeled instances. However, many studies pointed out an underestimation problem of MIL. In this study, we develop a MIL framework more suitable for ASC systems by defining instance-level labels and loss to extract and cluster instances effectively. Furthermore, we design a fully separated convolutional module, which is a lightweight neural network comprising pointwise, frequency-sided depthwise, and temporal-sided depthwise convolutional filters. As a result, compared to vanilla MIL, the confidence and proportion of positive instances increase significantly, overcoming the underestimation problem and improving the classification accuracy up to 11%. The proposed system achieved a performance of 81.1% and 72.3% on the TAU urban acoustic scenes 2019 and 2020 mobile datasets with 139 K parameters, respectively. Especially, it achieves the highest performance among the systems having under the 1 M parameters on the TAU urban acoustic scenes 2019 dataset.

cs.SD↗