arXiv subjects

Saber Jafarpour

Publications and source records attributed to Saber Jafarpour.

At least 37 records · Page 2Linked to original sources

arXiv2023

A Toolbox for Fast Interval Arithmetic in numpy with an Application to Formal Verification of Neural Network Controlled Systems

In this paper, we present a toolbox for interval analysis in numpy, with an application to formal verification of neural network controlled systems. Using the notion of natural inclusion functions, we systematically construct interval bounds for a general class of mappings. The toolbox offers efficient computation of natural inclusion functions using compiled C code, as well as a familiar interface in numpy with its canonical features, such as n-dimensional arrays, matrix/vector operations, and vectorization. We then use this toolbox in formal verification of dynamical systems with neural network controllers, through the composition of their inclusion functions.

eess.SY↗

arXiv2022

Multistability and anomalies in oscillator models of lossy power grids

The analysis of dissipatively coupled oscillators is challenging and highly relevant in power grids. Standard mathematical methods are not applicable, due to the lack of network symmetry induced by dissipative couplings. Here we demonstrate a close correspondence between stable synchronous states in dissipatively coupled oscillators, and the {winding partition} of their state space, a geometric notion induced by the network topology. Leveraging this winding partition, we accompany this article with an algorithms to compute all synchronous solutions of complex networks of dissipatively coupled oscillators. These geometric and computational tools allow us to identify anomalous behaviors of lossy networked systems. Counterintuitively, we show that loop flows and dissipation can increase the system's transfer capacity, and that dissipation can promote multistability. We apply our geometric framework to compute power flows on the IEEE RTS-96 test system, where we identify two high voltage solutions with distinct loop flows.

nlin.AO↗

arXiv2022

Non-Euclidean Contraction Theory for Monotone and Positive Systems

In this note we study contractivity of monotone systems and exponential convergence of positive systems using non-Euclidean norms. We first introduce the notion of conic matrix measure as a framework to study stability of monotone and positive systems. We study properties of the conic matrix measures and investigate their connection with weak pairings and standard matrix measures. Using conic matrix measures and weak pairings, we characterize contractivity and incremental stability of monotone systems with respect to non-Euclidean norms. Moreover, we use conic matrix measures to provide sufficient conditions for exponential convergence of positive systems to their equilibria. We show that our framework leads to novel results on (i) the contractivity of excitatory Hopfield neural networks, and (ii) the stability of interconnected systems using non-monotone positive comparison systems.

math.OC↗

arXiv2022

Network Critical Slowing Down: Data-Driven Detection of Critical Transitions in Nonlinear Networks

In a Nature article, Scheffer et al. presented a novel data-driven framework to predict critical transitions in complex systems. These transitions, which may stem from failures, degradation, or adversarial actions, have been attributed to bifurcations in the nonlinear dynamics. Their approach was built upon the phenomenon of critical slowing down, i.e., slow recovery in response to small perturbations near bifurcations. We extend their approach to detect and localize critical transitions in nonlinear networks. By introducing the notion of network critical slowing down, the objective of this paper is to detect that the network is undergoing a bifurcation only by analyzing its signatures from measurement data. We focus on two classes of widely-used nonlinear networks: (1) Kuramoto model for the synchronization of coupled oscillators and (2) attraction-repulsion dynamics in swarms, each of which presents a specific type of bifurcation. Based on the phenomenon of critical slowing down, we study the asymptotic behavior of the perturbed system away and close to the bifurcation and leverage this fact to develop a deterministic method to detect and identify critical transitions in nonlinear networks. Furthermore, we study the state covariance matrix subject to a stochastic noise process away and close to the bifurcation and use it to develop a stochastic framework for detecting critical transitions. Our simulation results show the strengths and limitations of the methods.

math.OC↗

arXiv2022

Robust Training and Verification of Implicit Neural Networks: A Non-Euclidean Contractive Approach

This paper proposes a theoretical and computational framework for training and robustness verification of implicit neural networks based upon non-Euclidean contraction theory. The basic idea is to cast the robustness analysis of a neural network as a reachability problem and use (i) the $\ell_{\infty}$-norm input-output Lipschitz constant and (ii) the tight inclusion function of the network to over-approximate its reachable sets. First, for a given implicit neural network, we use $\ell_{\infty}$-matrix measures to propose sufficient conditions for its well-posedness, design an iterative algorithm to compute its fixed points, and provide upper bounds for its $\ell_\infty$-norm input-output Lipschitz constant. Second, we introduce a related embedded network and show that the embedded network can be used to provide an $\ell_\infty$-norm box over-approximation of the reachable sets of the original network. Moreover, we use the embedded network to design an iterative algorithm for computing the upper bounds of the original system's tight inclusion function. Third, we use the upper bounds of the Lipschitz constants and the upper bounds of the tight inclusion functions to design two algorithms for the training and robustness verification of implicit neural networks. Finally, we apply our algorithms to train implicit neural networks on the MNIST dataset and compare the robustness of our models with the models trained via existing approaches in the literature.

cs.LG↗

arXiv2022

Non-Euclidean Contraction Theory for Robust Nonlinear Stability

We study necessary and sufficient conditions for contraction and incremental stability of dynamical systems with respect to non-Euclidean norms. First, we introduce weak pairings as a framework to study contractivity with respect to arbitrary norms, and characterize their properties. We introduce and study the sign and max pairings for the $\ell_1$ and $\ell_\infty$ norms, respectively. Using weak pairings, we establish five equivalent characterizations for contraction, including the one-sided Lipschitz condition for the vector field as well as matrix measure and Demidovich conditions for the corresponding Jacobian. Third, we extend our contraction framework in two directions: we prove equivalences for contraction of continuous vector fields and we formalize the weaker notion of equilibrium contraction, which ensures exponential convergence to an equilibrium. Finally, as an application, we provide (i) incremental input-to-state stability and finite input-state gain properties for contracting systems, and (ii) a general theorem about the Lipschitz interconnection of contracting systems, whereby the Hurwitzness of a gain matrix implies the contractivity of the interconnected system.

math.OC↗

arXiv2022

Non-Euclidean Monotone Operator Theory with Applications to Recurrent Neural Networks

We provide a novel transcription of monotone operator theory to the non-Euclidean finite-dimensional spaces $\ell_1$ and $\ell_{\infty}$. We first establish properties of mappings which are monotone with respect to the non-Euclidean norms $\ell_1$ or $\ell_{\infty}$. In analogy with their Euclidean counterparts, mappings which are monotone with respect to a non-Euclidean norm are amenable to numerous algorithms for computing their zeros. We demonstrate that several classic iterative methods for computing zeros of monotone operators are directly applicable in the non-Euclidean framework. We present a case-study in the equilibrium computation of recurrent neural networks and demonstrate that casting the computation as a suitable operator splitting problem improves convergence rates.

math.OC↗

arXiv2022

Minimax Flow over Acyclic Networks: Distributed Algorithms and Microgrid Application

Given a flow network with variable suppliers and fixed consumers, the minimax flow problem consists in minimizing the maximum flow between nodes, subject to flow conservation and capacity constraints. We solve this problem over acyclic graphs in a distributed manner by showing that it can be recast as a consensus problem between the maximum downstream flows, which we define here for the first time. Additionally, we present a distributed algorithm to estimate these quantities. Finally, exploiting our theoretical results, we design an online distributed controller to prevent overcurrent in microgrids consisting of loads and droop-controlled inverters. Our results are validated numerically on the CIGRE benchmark microgrid.

eess.SY↗

arXiv2022

Comparative Analysis of Interval Reachability for Robust Implicit and Feedforward Neural Networks

We use interval reachability analysis to obtain robustness guarantees for implicit neural networks (INNs). INNs are a class of implicit learning models that use implicit equations as layers and have been shown to exhibit several notable benefits over traditional deep neural networks. We first establish that tight inclusion functions of neural networks, which provide the tightest rectangular over-approximation of an input-output map, lead to sharper robustness guarantees than the well-studied robustness measures of local Lipschitz constants. Like Lipschitz constants, tight inclusions functions are computationally challenging to obtain, and we thus propose using mixed monotonicity and contraction theory to obtain computationally efficient estimates of tight inclusion functions for INNs. We show that our approach performs at least as well as, and generally better than, applying state-of-the-art interval bound propagation methods to INNs. We design a novel optimization problem for training robust INNs and we provide empirical evidence that suitably-trained INNs can be more robust than comparably-trained feedforward networks.

cs.LG↗

arXiv2022

Resilience of Input Metering in Dynamic Flow Networks

In this paper, we study robustness of input metering policies in dynamic flow networks in the presence of transient disturbances and attacks. We consider a compartmental model for dynamic flow networks with a First-In-First-Out (FIFO) routing rule as found in, e.g., transportation networks. We model the effect of the transient disturbance as an abrupt change to the state of the network and use the notion of the region of attraction to measure the resilience of the network to these changes. For constant and periodic input metering, we introduce the notion of monotone-invariant points to establish inner-estimates for the regions of attraction of free-flow equilibrium points and free-flow periodic orbits using monotone systems theory. These results are applicable to, e.g., networks with cycles, which have not been considered in prior literature on dynamic flow networks with FIFO routing. Finally, we propose two approaches for finding suitable monotone-invariant points in the flow networks with FIFO rules.

math.OC↗

arXiv2022

Robust Implicit Networks via Non-Euclidean Contractions

Implicit neural networks, a.k.a., deep equilibrium networks, are a class of implicit-depth learning models where function evaluation is performed by solving a fixed point equation. They generalize classic feedforward models and are equivalent to infinite-depth weight-tied feedforward networks. While implicit models show improved accuracy and significant reduction in memory consumption, they can suffer from ill-posedness and convergence instability. This paper provides a new framework, which we call Non-Euclidean Monotone Operator Network (NEMON), to design well-posed and robust implicit neural networks based upon contraction theory for the non-Euclidean norm $\ell_{\infty}$. Our framework includes (i) a novel condition for well-posedness based on one-sided Lipschitz constants, (ii) an average iteration for computing fixed-points, and (iii) explicit estimates on input-output Lipschitz constants. Additionally, we design a training problem with the well-posedness condition and the average iteration as constraints and, to achieve robust models, with the input-output Lipschitz constant as a regularizer. Our $\ell_{\infty}$ well-posedness condition leads to a larger polytopic training search space than existing conditions and our average iteration enjoys accelerated convergence. Finally, we evaluate our framework in image classification through the MNIST and the CIFAR-10 datasets. Our numerical results demonstrate improved accuracy and robustness of the implicit models with smaller input-output Lipschitz bounds. Code is available at https://github.com/davydovalexander/Non-Euclidean_Mon_Op_Net.

cs.LG↗

arXiv2021

Robustness Certificates for Implicit Neural Networks: A Mixed Monotone Contractive Approach

Implicit neural networks are a general class of learning models that replace the layers in traditional feedforward models with implicit algebraic equations. Compared to traditional learning models, implicit networks offer competitive performance and reduced memory consumption. However, they can remain brittle with respect to input adversarial perturbations. This paper proposes a theoretical and computational framework for robustness verification of implicit neural networks; our framework blends together mixed monotone systems theory and contraction theory. First, given an implicit neural network, we introduce a related embedded network and show that, given an $\ell_\infty$-norm box constraint on the input, the embedded network provides an $\ell_\infty$-norm box overapproximation for the output of the given network. Second, using $\ell_{\infty}$-matrix measures, we propose sufficient conditions for well-posedness of both the original and embedded system and design an iterative algorithm to compute the $\ell_{\infty}$-norm box robustness margins for reachability and classification problems. Third, of independent value, we propose a novel relative classifier variable that leads to tighter bounds on the certified adversarial robustness in classification problems. Finally, we perform numerical simulations on a Non-Euclidean Monotone Operator Network (NEMON) trained on the MNIST dataset. In these simulations, we compare the accuracy and run time of our mixed monotone contractive approach with the existing robustness verification approaches in the literature for estimating the certified adversarial robustness.

cs.LG↗

arXiv2021

From Contraction Theory to Fixed Point Algorithms on Riemannian and Non-Euclidean Spaces

The design of fixed point algorithms is at the heart of monotone operator theory, convex analysis, and of many modern optimization problems arising in machine learning and control. This tutorial reviews recent advances in understanding the relationship between Demidovich conditions, one-sided Lipschitz conditions, and contractivity theorems. We review the standard contraction theory on Euclidean spaces as well as little-known results for Riemannian manifolds. Special emphasis is placed on the setting of non-Euclidean norms and the recently introduced weak pairings for the $\ell_1$ and $\ell_\infty$ norms. We highlight recent results on explicit and implicit fixed point schemes for non-Euclidean contracting systems.

math.OC↗

arXiv2021

Distributed and time-varying primal-dual dynamics via contraction analysis

In this note, we provide an overarching analysis of primal-dual dynamics associated to linear equality-constrained optimization problems using contraction analysis. For the well-known standard version of the problem: we establish convergence under convexity and the contracting rate under strong convexity. Then, for a canonical distributed optimization problem, we use partial contractivity to establish global exponential convergence of its primal-dual dynamics. As an application, we propose a new distributed solver for the least-squares problem with the same convergence guarantees. Finally, for time-varying versions of both centralized and distributed primal-dual dynamics, we exploit their contractive nature to establish bounds on their tracking error. To support our analyses, we introduce novel results on contraction theory.

eess.SY↗

arXiv2021

Topology Inference with Multivariate Cumulants: The Möbius Inference Algorithm

Many tasks regarding the monitoring, management, and design of communication networks rely on knowledge of the routing topology. However, the standard approach to topology mapping--namely, active probing with traceroutes--relies on cooperation from increasingly non-cooperative routers, leading to missing information. Network tomography, which uses end-to-end measurements of additive link metrics (like delays or log packet loss rates) across monitor paths, is a possible remedy. Network tomography does not require that routers cooperate with traceroute probes, and it has already been used to infer the structure of multicast trees. This paper goes a step further. We provide a tomographic method to infer the underlying routing topology of an arbitrary set of monitor paths using the joint distribution of end-to-end measurements, without making any assumptions on routing behavior. Our approach, called the Möbius Inference Algorithm (MIA), uses cumulants of this distribution to quantify high-order interactions among monitor paths, and it applies Möbius inversion to "disentangle" these interactions. In addition to MIA, we provide a more practical variant called Sparse Möbius Inference, which uses various sparsity heuristics to reduce the number and order of cumulants required to be estimated. We show the viability of our approach using synthetic case studies based on real-world ISP topologies.

eess.SP↗

arXiv2021

Contraction Theory for Dynamical Systems on Hilbert Spaces

Contraction theory for dynamical systems on Euclidean spaces is well-established. For contractive (resp. semi-contractive) systems, the distance (resp. semi-distance) between any two trajectories decreases exponentially fast. For partially contractive systems, each trajectory converges exponentially fast to an invariant subspace. In this note, we develop contraction theory on Hilbert spaces. First, we provide a novel integral condition for contractivity, and for time-invariant systems, we establish the existence of a unique globally exponentially stable equilibrium. Second, we introduce the notions of partial and semi-contraction and we provide various sufficient conditions for time-varying and time-invariant systems. Finally, we apply the theory on a classic reaction-diffusion system.

math.OC↗

arXiv2020

Weak and Semi-Contraction for Network Systems and Diffusively-Coupled Oscillators

We develop two generalizations of contraction theory, namely, semi-contraction and weak-contraction theory. First, using the notion of semi-norm, we propose a geometric framework for semi-contraction theory. We introduce matrix semi-measures and characterize their properties. We show that the spectral abscissa of a matrix is the infimum over weighted semi-measures. For dynamical systems, we use the semi-measure of their Jacobian to characterize the contractivity properties of their trajectories. Second, for weakly contracting systems, we prove a dichotomy for the asymptotic behavior of their trajectories and novel sufficient conditions for convergence to an equilibrium. Third, we show that every trajectory of a doubly-contracting system, i.e., a system that is both weakly and semi-contracting, converges to an equilibrium point. Finally, we apply our results to various important network systems including affine averaging and affine flow systems, continuous-time distributed primal-dual algorithms, and networks of diffusively-coupled dynamical systems. For diffusively-coupled systems, the semi-contraction theory leads to a sufficient condition for synchronization that is sharper, in general, than previously-known tests.

eess.SY↗

arXiv2020

Flow and Elastic Networks on the $n$-torus: Geometry, Analysis, and Computation

Networks with phase-valued nodal variables are central in modeling several important societal and physical systems, including power grids, biological systems, and coupled oscillator networks. One of the distinctive features of phase-valued networks is the existence of multiple operating conditions corresponding to critical points of an energy function or feasible flows of a balance equation. For networks with phase-valued states, it is not yet fully understood how many operating conditions exist, how to characterize them, and how to compute them efficiently. A deeper understanding of feasible operating conditions, including their dependence upon network structures, may lead to more reliable and efficient network systems. This paper introduces flow and elastic network problems on the $n$-torus and provides a rigorous and comprehensive framework for their study. Based on a monotonicity assumption, this framework localizes the solutions, bounds their number, and leads to an algorithm to compute them. Our analysis is based on a novel winding partition of the $n$-torus into winding cells, induced by Kirchhoff's angle law for undirected graphs. The winding partition has several useful properties, including notably that, each winding cell contains at most one solution. The proposed algorithm is based on a novel contraction mapping and is guaranteed to compute all solutions. Finally, we apply our results to numerically study the active power flow equations in several test cases and estimate power capacity and congestion of a power network.

math.OC↗