GDBR: Label Recovery Attack Against Partial Gradient Encryption in Federated Learning

Abstract

The increasing demand for data privacy, alongside the benefits of aggregating data from networked devices, has catalyzed the emergence of federated learning (FL). In FL, clients jointly train a global model by sharing gradients computed over private data. While this paradigm eliminates the need to exchange raw data, inference attacks can still be launched to extract sensitive information from gradients. To this end, partial gradient encryption has emerged as a promising design for balancing privacy and efficiency in practical FL systems, as encrypting only the classification-head gradients is believed to prevent known inference attacks while avoiding the high computational cost of encrypting the entire model. However, this design provides a false sense of privacy. By proposing GDBR, we show that sharing even a single unencrypted layer of gradients can lead to serious privacy leakage. GDBR is the first attack capable of high-fidelity label recovery with partial access to the gradients. It exploits a vulnerability in a commonly used neural building block, constructs a gradient bridge from the unencrypted layer to the final output layer, and approximates the logits information for accurate inference of private labels. These inferred labels not only reveal sensitive information about a client's private dataset but also serve as a prerequisite for many downstream attacks, such as data reconstruction and membership inference. GDBR brings these threats squarely into scope for FL systems employing partial encryption. In addition to theoretical analysis, extensive experiments demonstrate the severity of the problem across a wide variety of datasets and model architectures, including convolutional and transformer-based networks. Overall, our findings challenge the widespread assumption that encrypting only the output layer suffices for privacy protection.