SketchGuard: Scaling Byzantine-Robust Decentralized Federated Learning via Sketch-Based Screening

Abstract

Byzantine-robust decentralized federated learning (DFL) protects peer-to-peer training from malicious clients. The dominant defenses rely on similarity-based filtering, in which each client exchanges full model vectors with every neighbor before any filtering decision; this communication grows with the model dimension and scales poorly as models grow. We propose SketchGuard, which decouples screening from aggregation: clients screen neighbors in a compact Count Sketch domain and fetch full models only from those that pass the screen. We show this idea is insecure when implemented naively. Because the sketch is a fixed, publicly known linear map, an adaptive adversary can hide an arbitrarily large perturbation in its null space, so the poisoned model passes both the sketch-domain filter and the re-sketch verification. We prove this vulnerability and close it with commit-then-sketch, a one-message protocol that draws the sketch seed only after models are committed, restoring the oblivious setting in which Count Sketch provably preserves screening decisions. We then establish convergence in strongly convex and non-convex settings, with explicit dependence on network connectivity and data heterogeneity. Empirically, secured SketchGuard matches state-of-the-art full-precision robustness, up to a small threshold inflation, across six attacks including the adaptive null-space attack, a range of network topologies and heterogeneity settings, and a decentralized fine-tuning task on an 11-million-parameter language model, while reducing per-neighbor screening communication to a size independent of the model dimension.