Logit Poisoning Attack in Distillation-based Federated Learning and its Countermeasures

Abstract

Distillation-based federated learning has emerged as a promising collaborative learning approach, where clients share the output logit vectors of a public dataset rather than their private model parameters. This practice reduces the risk of privacy invasion attacks and facilitates heterogeneous learning. The landscape of poisoning attacks within distillation-based federated learning is complex, with existing research employing traditional data poisoning strategies targeting the models' parameters. However, these attack schemes primarily have shortcomings rooted in their original designs, which target the model parameters rather than the logit vectors. Furthermore, they do not adequately consider the role of logit vectors in carrying information during the knowledge transfer process. This misalignment results in less efficiency in the context of distillation-based federated learning. Due to the limitations of existing methodologies, our research delves into the intrinsic properties of the logit vector, striving for a more nuanced understanding. We introduce a two-stage scheme for logit poisoning attacks, addressing previous shortcomings. Initially, we collect the local logits, generate the representative vectors, categorize the logit elements within the vector, and design a shuffling table to maximize information entropy. Then, we intentionally scale the shuffled logit vectors to enhance the magnitude of the target vectors. Concurrently, we propose an efficient defense algorithm to counter this new poisoning scheme by calculating the distance between estimated benign vectors and vectors uploaded by users. Through extensive experiments, our study illustrates the significant threat posed by the proposed logit poisoning attack and highlights the effectiveness of our defense algorithm.