Goodbye Equal Error Rate, Hello Local Information Disclosure: Evaluating Voice Anonymisation against 1-to-N Linkage Threats

Abstract

Voice anonymisation aims to protect speaker identity. Currently, its empirical privacy evaluation heavily relies on the Equal Error Rate (EER). Originally designed for biometric verification, EER aggregates scores globally, implicitly assuming an attacker is only trying to verify if two specific voice samples match (a 1-to-1 comparison). This introduces a threat model mismatch with real-world database linkage attacks, where an attacker searches across a fixed set of N enrolled identities (a 1-to-N closed-set search), allowing global averages to obscure localised privacy failures. While recent 1-to-N metrics address this aggregation issue, they abstract away the magnitude of the biometric evidence. In this paper, we propose a modular, information-theoretic evaluation framework explicitly designed for the 1-to-N linkage threat model. Within this framework, our core metric, Local Information Disclosure (LID), provides a principled estimate of the identity information disclosed by a single trial utterance in bits by mapping raw similarity scores to an estimated posterior distribution over the enrolled identities. Demonstrating this framework on VoicePrivacy 2024 Challenge systems reveals how global metrics can obscure privacy vulnerabilities. Even for top-performing systems where standard evaluations report near-perfect EERs (48%), our metric exposes that attackers gain a statistical advantage in at least 63% of trials with maximum observed disclosures reaching 1 bit (effectively doubling the attacker's confidence in the target speaker after observing a single trial utterance). We conclude that shifting toward explainable metrics and appropriate threat models is a practical step toward identifying worst-case vulnerabilities and aligning with strict privacy regulations.