SPADE: A Multilingual Dataset for Speech Partial Deepfake Detection and Localization

Abstract

Recent improvements in voice-cloning speech generation systems raise concerns about misuse by malicious actors to impersonate others and spread misinformation. Detecting such tampering is difficult, since deepfakes in the wild may be created by different generative models in a wide range of languages. Furthermore, the speech audio may also only be partially modified, presenting a different and potentially more challenging task than detecting fully-synthetic speech waveforms. To enable further research in this direction, we propose a multilingual dataset for detection and localization of partially edited speech samples. Our dataset includes speech in 12 languages, generated by up to five systems per language, and includes both a training set as well as an evaluation benchmark. To showcase the utility of our proposed dataset, we train localization models of existing architectures and study generalization across three axes: across different languages, across different speech synthesis and editing systems, and across different acoustic environments. Our results show that localization models almost always generalize poorly to speech edited by systems not seen during training. On the other hand, generalization to edited speech in unseen languages still degrades performance but to a lesser extent. We also augment our testing sets with noise to evaluate generalization across acoustic environments, and find that performance of localization models degrade significantly when tested on different acoustic conditions. All together, our results imply that existing deepfake speech detection methods are insufficient for reliably detecting edit-based speech deepfakes in various scenarios unseen during training. SPADE is publicly available on HuggingFace.