arXiv · 2201.06699
AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference
Abstract
Hybrid private inference (PI) protocol, which synergistically utilizes both multi-party computation (MPC) and homomorphic encryption, is one of the most prominent techniques for PI. However, even the state-of-the-art PI protocols are bottlenecked by the non-linear layers, especially the activation functions. Although a standard non-linear activation function can generate higher model accuracy, it must be processed via a costly garbled-circuit MPC primitive. A polynomial activation can be processed via Beaver's multiplication triples MPC primitive but has been incurring severe accuracy drops so far. In this paper, we propose an accuracy preserving low-degree polynomial activation function (AESPA) that exploits the Hermite expansion of the ReLU and basis-wise normalization. We apply AESPA to popular ML models, such as VGGNet, ResNet, and pre-activation ResNet, to show an inference accuracy comparable to those of the standard models with ReLU activation, achieving superior accuracy over prior low-degree polynomial studies. When applied to the all-RELU baseline on the state-of-the-art Delphi PI protocol, AESPA shows up to 42.1x and 28.3x lower online latency and communication cost.
Explore related subjects
Keep this discovery
Jaiyoung Park, Michael Jaemin Kim, Wonkyung Jung, Jung Ho Ahn. 2022-01-18. AESPA: Accuracy Preserving Low-degree Polynomial Activation for Fast Private Inference. https://arxiv.org/abs/2201.06699
Cite the original work for its findings. Save a collection to share your selection of sources.