Lipschitz Bounds for Neural Networks Using a Two-Copy Linear Fractional Transformation

Abstract

This paper proposes a new method to bound the global Lipschitz constant of a neural network (NN). Our formulation uses a two-copy linear fractional transformation (LFT) representation of the NN with stacked signals. We use biased quadratic constraints (QCs) to bound the input/output behavior of the nonlinear activation functions. We then formulate a linear matrix inequality condition, using these biased QCs, to bound the Lipschitz constant of the NN. The best (smallest) Lipschitz bound is computed via a convex, semidefinite program. There are two key distinctions of the proposed two-copy formulation in comparison to existing methods that rely on a single incremental representation with incremental signals. First, the NN bias terms are eliminated in the standard incremental representation, but appear in our formulation. Second, there is a larger class of QCs that can be expressed for the nonlinearities in our formulation than in the incremental representation. We show that our proposed formulation yields less conservative Lipschitz bounds due to these two differences. We illustrate these points via several simple, motivating examples. We also demonstrate the proposed method using a benchmark problem with a NN controller for an inverted pendulum.