arXiv subjects

George J. Pappas

Publications and source records attributed to George J. Pappas.

At least 127 records · Page 7Linked to original sources

arXiv2021

Scalable Reinforcement Learning Policies for Multi-Agent Control

We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observations (range and bearing) about targets which move using fixed, unknown policies. An attention mechanism is used to parameterize the value function of the agents; this mechanism allows us to handle an arbitrary number of targets. Entropy-regularized off-policy RL methods are used to train a stochastic policy, and we discuss how it enables a hedging behavior between pursuers that leads to a weak form of cooperation in spite of completely decentralized control execution. We further develop a masking heuristic that allows training on smaller problems with few pursuers-targets and execution on much larger problems. Thorough simulation experiments, ablation studies, and comparisons to state of the art algorithms are performed to study the scalability of the approach and robustness of performance to varying numbers of agents and targets.

cs.MA↗

arXiv2021

Adversarial Robustness with Semi-Infinite Constrained Learning

Despite strong performance in numerous applications, the fragility of deep learning to input perturbations has raised serious questions about its use in safety-critical domains. While adversarial training can mitigate this issue in practice, state-of-the-art methods are increasingly application-dependent, heuristic in nature, and suffer from fundamental trade-offs between nominal performance and robustness. Moreover, the problem of finding worst-case perturbations is non-convex and underparameterized, both of which engender a non-favorable optimization landscape. Thus, there is a gap between the theory and practice of adversarial training, particularly with respect to when and why adversarial training works. In this paper, we take a constrained learning approach to address these questions and to provide a theoretical foundation for robust learning. In particular, we leverage semi-infinite optimization and non-convex duality theory to show that adversarial training is equivalent to a statistical problem over perturbation distributions, which we characterize completely. Notably, we show that a myriad of previous robust training techniques can be recovered for particular, sub-optimal choices of these distributions. Using these insights, we then propose a hybrid Langevin Monte Carlo approach of which several common algorithms (e.g., PGD) are special cases. Finally, we show that our approach can mitigate the trade-off between nominal and robust performance, yielding state-of-the-art results on MNIST and CIFAR-10. Our code is available at: https://github.com/arobey1/advbench.

stat.ML↗

arXiv2021

Safe Pontryagin Differentiable Programming

We propose a Safe Pontryagin Differentiable Programming (Safe PDP) methodology, which establishes a theoretical and algorithmic framework to solve a broad class of safety-critical learning and control tasks -- problems that require the guarantee of safety constraint satisfaction at any stage of the learning and control progress. In the spirit of interior-point methods, Safe PDP handles different types of system constraints on states and inputs by incorporating them into the cost or loss through barrier functions. We prove three fundamentals of the proposed Safe PDP: first, both the solution and its gradient in the backward pass can be approximated by solving their more efficient unconstrained counterparts; second, the approximation for both the solution and its gradient can be controlled for arbitrary accuracy by a barrier parameter; and third, importantly, all intermediate results throughout the approximation and optimization strictly respect the constraints, thus guaranteeing safety throughout the entire learning and control process. We demonstrate the capabilities of Safe PDP in solving various safety-critical tasks, including safe policy optimization, safe motion planning, and learning MPCs from demonstrations, on different challenging systems such as 6-DoF maneuvering quadrotor and 6-DoF rocket powered landing.

cs.LG↗

arXiv2021

Learning Region of Attraction for Nonlinear Systems

Estimating the region of attraction (ROA) of general nonlinear autonomous systems remains a challenging problem and requires a case-by-case analysis. Leveraging the universal approximation property of neural networks, in this paper, we propose a counterexample-guided method to estimate the ROA of general nonlinear dynamical systems provided that they can be approximated by piecewise linear neural networks and that the approximation error can be bounded. Specifically, our method searches for robust Lyapunov functions using counterexamples, i.e., the states at which the Lyapunov conditions fail. We generate the counterexamples using Mixed-Integer Quadratic Programming. Our method is guaranteed to find a robust Lyapunov function in the parameterized function class, if exists, after collecting a finite number of counterexamples. We illustrate our method through numerical examples.

eess.SY↗

arXiv2021

Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming

Certifying the safety or robustness of neural networks against input uncertainties and adversarial attacks is an emerging challenge in the area of safe machine learning and control. To provide such a guarantee, one must be able to bound the output of neural networks when their input changes within a bounded set. In this paper, we propose a semidefinite programming (SDP) framework to address this problem for feed-forward neural networks with general activation functions and input uncertainty sets. Our main idea is to abstract various properties of activation functions (e.g., monotonicity, bounded slope, bounded values, and repetition across layers) with the formalism of quadratic constraints. We then analyze the safety properties of the abstracted network via the S-procedure and semidefinite programming. Our framework spans the trade-off between conservatism and computational efficiency and applies to problems beyond safety verification. We evaluate the performance of our approach via numerical problem instances of various sizes.

math.OC↗

arXiv2021

Online learning-based trajectory tracking for underactuated vehicles with uncertain dynamics

Underactuated vehicles have gained much attention in the recent years due to the increasing amount of aerial and underwater vehicles as well as nanosatellites. Trajectory tracking control of these vehicles is a substantial aspect for an increasing range of application domains. However, external disturbances and parts of the internal dynamics are often unknown or very time-consuming to model. To overcome this issue, we present a tracking control law for underactuated rigid-body dynamics using an online learning-based oracle for the prediction of the unknown dynamics. We show that Gaussian process models are of particular interest for the role of the oracle. The presented approach guarantees a bounded tracking error with high probability where the bound is explicitly given. A numerical example highlights the effectiveness of the proposed control law.

eess.SY↗

arXiv2021

Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients

We consider a standard federated learning (FL) architecture where a group of clients periodically coordinate with a central server to train a statistical model. We develop a general algorithmic framework called FedLin to tackle some of the key challenges intrinsic to FL, namely objective heterogeneity, systems heterogeneity, and infrequent and imprecise communication. Our framework is motivated by the observation that under these challenges, various existing FL algorithms suffer from a fundamental speed-accuracy conflict: they either guarantee linear convergence but to an incorrect point, or convergence to the global minimum but at a sub-linear rate, i.e., fast convergence comes at the expense of accuracy. In contrast, when the clients' local loss functions are smooth and strongly convex, we show that FedLin guarantees linear convergence to the global minimum, despite arbitrary objective and systems heterogeneity. We then establish matching upper and lower bounds on the convergence rate of FedLin that highlight the effects of intermittent communication. Finally, we show that FedLin preserves linear convergence rates under aggressive gradient sparsification, and quantify the effect of the compression level on the convergence rate. Our work is the first to provide tight linear convergence rate guarantees, and constitutes the first comprehensive analysis of gradient sparsification in FL.

cs.LG↗

arXiv2021

Reactive and Risk-Aware Control for Signal Temporal Logic

The deployment of autonomous systems in uncertain and dynamic environments has raised fundamental questions. Addressing these is pivotal to build fully autonomous systems and requires a systematic integration of planning and control. We first propose reactive risk signal interval temporal logic (ReRiSITL) as an extension of signal temporal logic (STL) to formulate complex spatiotemporal specifications. Unlike STL, ReRiSITL allows to consider uncontrollable propositions that may model humans as well as random environmental events such as sensor failures. Additionally, ReRiSITL allows to incorporate risk measures, such as (but not limited to) the Conditional Value-at-Risk, to measure the risk of violating certain spatial specifications. Second, we propose an algorithm to check if an ReRiSITL specification is satisfiable. For this purpose, we abstract the ReRiSITL specification into a timed signal transducer and devise a game-based approach. Third, we propose a reactive planning and control framework for dynamical control systems under ReRiSITL specifications.

eess.SY↗

arXiv2021

Robust Motion Planning in the Presence of Estimation Uncertainty

Motion planning is a fundamental problem and focuses on finding control inputs that enable a robot to reach a goal region while safely avoiding obstacles. However, in many situations, the state of the system may not be known but only estimated using, for instance, a Kalman filter. This results in a novel motion planning problem where safety must be ensured in the presence of state estimation uncertainty. Previous approaches to this problem are either conservative or integrate state estimates optimistically which leads to non-robust solutions. Optimistic solutions require frequent replanning to not endanger the safety of the system. We propose a new formulation to this problem with the aim to be robust to state estimation errors while not being overly conservative. In particular, we formulate a stochastic optimal control problem that contains robustified risk-aware safety constraints by incorporating robustness margins to account for state estimation errors. We propose a novel sampling-based approach that builds trees exploring the reachable space of Gaussian distributions that capture uncertainty both in state estimation and in future measurements. We provide robustness guarantees and show, both in theory and simulations, that the induced robustness margins constitute a trade-off between conservatism and robustness for planning under estimation uncertainty that allows to control the frequency of replanning.

cs.RO↗

arXiv2021

Is the brain macroscopically linear? A system identification of resting state dynamics

A central challenge in the computational modeling of neural dynamics is the trade-off between accuracy and simplicity. At the level of individual neurons, nonlinear dynamics are both experimentally established and essential for neuronal functioning. An implicit assumption has thus formed that an accurate computational model of whole-brain dynamics must also be highly nonlinear, whereas linear models may provide a first-order approximation. Here, we provide a rigorous and data-driven investigation of this hypothesis at the level of whole-brain blood-oxygen-level-dependent (BOLD) and macroscopic field potential dynamics by leveraging the theory of system identification. Using functional MRI (fMRI) and intracranial EEG (iEEG), we model the resting state activity of 700 subjects in the Human Connectome Project (HCP) and 122 subjects from the Restoring Active Memory (RAM) project using state-of-the-art linear and nonlinear model families. We assess relative model fit using predictive power, computational complexity, and the extent of residual dynamics unexplained by the model. Contrary to our expectations, linear auto-regressive models achieve the best measures across all three metrics, eliminating the trade-off between accuracy and simplicity. To understand and explain this linearity, we highlight four properties of macroscopic neurodynamics which can counteract or mask microscopic nonlinear dynamics: averaging over space, averaging over time, observation noise, and limited data samples. Whereas the latter two are technological limitations and can improve in the future, the former two are inherent to aggregated macroscopic brain activity. Our results, together with the unparalleled interpretability of linear models, can greatly facilitate our understanding of macroscopic neural dynamics and the principled design of model-based interventions for the treatment of neuropsychiatric disorders.

q-bio.NC↗

arXiv2021

Technical Report: Distributed Sampling-based Planning for Non-Myopic Active Information Gathering

This paper addresses the problem of active information gathering for multi-robot systems. Specifically, we consider scenarios where robots are tasked with reducing uncertainty of dynamical hidden states evolving in complex environments. The majority of existing information gathering approaches are centralized and, therefore, they cannot be applied to distributed robot teams where communication to a central user is not available. To address this challenge, we propose a novel distributed sampling-based planning algorithm that can significantly increase robot and target scalability while decreasing computational cost. In our non-myopic approach, all robots build in parallel local trees exploring the information space and their corresponding motion space. As the robots construct their respective local trees, they communicate with their neighbors to exchange and aggregate their local beliefs about the hidden state through a distributed Kalman filter. We show that the proposed algorithm is probabilistically complete and asymptotically optimal. We provide extensive simulation results that demonstrate the scalability of the proposed algorithm and that it can address large-scale, multi-robot information gathering tasks, that are computationally challenging for centralized methods.

cs.RO↗

arXiv2021

Data-driven control on encrypted data

We provide an efficient and private solution to the problem of encryption-aware data-driven control. We investigate a Control as a Service scenario, where a client employs a specialized outsourced control solution from a service provider. The privacy-sensitive model parameters of the client's system are either not available or variable. Hence, we require the service provider to perform data-driven control in a privacy-preserving manner on the input-output data samples from the client. To this end, we co-design the control scheme with respect to both control performance and privacy specifications. First, we formulate our control algorithm based on recent results from the behavioral framework, and we prove closeness between the classical formulation and our formulation that accounts for noise and precision errors arising from encryption. Second, we use a state-of-the-art leveled homomorphic encryption scheme to enable the service provider to perform high complexity computations on the client's encrypted data, ensuring privacy. Finally, we streamline our solution by exploiting the rich structure of data, and meticulously employing ciphertext batching and rearranging operations to enable parallelization. This solution achieves more than twofold runtime and memory improvements compared to our prior work.

cs.CR↗

arXiv2021

Encrypted Distributed Lasso for Sparse Data Predictive Control

The least squares problem with L1-regularized regressors, called Lasso, is a widely used approach in optimization problems where sparsity of the regressors is desired. This formulation is fundamental for many applications in signal processing, machine learning and control. As a motivating problem, we investigate a sparse data predictive control problem, run at a cloud service to control a system with unknown model, using L1-regularization to limit the behavior complexity. The input-output data collected for the system is privacy-sensitive, hence, we design a privacy-preserving solution using homomorphically encrypted data. The main challenges are the non-smoothness of the L1-norm, which is difficult to evaluate on encrypted data, as well as the iterative nature of the Lasso problem. We use a distributed ADMM formulation that enables us to exchange substantial local computation for little communication between multiple servers. We first give an encrypted multi-party protocol for solving the distributed Lasso problem, by approximating the non-smooth part with a Chebyshev polynomial, evaluating it on encrypted data, and using a more cost effective distributed bootstrapping operation. For the example of data predictive control, we prefer a non-homogeneous splitting of the data for better convergence. We give an encrypted multi-party protocol for this non-homogeneous splitting of the Lasso problem to a non-homogeneous set of servers: one powerful server and a few less powerful devices, added for security reasons. Finally, we provide numerical results for our proposed solutions.

math.OC↗

arXiv2021

Linear Systems can be Hard to Learn

In this paper, we investigate when system identification is statistically easy or hard, in the finite sample regime. Statistically easy to learn linear system classes have sample complexity that is polynomial with the system dimension. Most prior research in the finite sample regime falls in this category, focusing on systems that are directly excited by process noise. Statistically hard to learn linear system classes have worst-case sample complexity that is at least exponential with the system dimension, regardless of the identification algorithm. Using tools from minimax theory, we show that classes of linear systems can be hard to learn. Such classes include, for example, under-actuated or under-excited systems with weak coupling among the states. Having classified some systems as easy or hard to learn, a natural question arises as to what system properties fundamentally affect the hardness of system identifiability. Towards this direction, we characterize how the controllability index of linear systems affects the sample complexity of identification. More specifically, we show that the sample complexity of robustly controllable linear systems is upper bounded by an exponential function of the controllability index. This implies that identification is easy for classes of linear systems with small controllability index and potentially hard if the controllability index is large. Our analysis is based on recent statistical tools for finite sample analysis of system identification as well as a novel lower bound that relates controllability index with the least singular value of the controllability Gramian.

eess.SY↗

arXiv2021

Non-Monotone Energy-Aware Information Gathering for Heterogeneous Robot Teams

This paper considers the problem of planning trajectories for a team of sensor-equipped robots to reduce uncertainty about a dynamical process. Optimizing the trade-off between information gain and energy cost (e.g., control effort, distance travelled) is desirable but leads to a non-monotone objective function in the set of robot trajectories. Therefore, common multi-robot planning algorithms based on techniques such as coordinate descent lose their performance guarantees. Methods based on local search provide performance guarantees for optimizing a non-monotone submodular function, but require access to all robots' trajectories, making it not suitable for distributed execution. This work proposes a distributed planning approach based on local search and shows how lazy/greedy methods can be adopted to reduce the computation and communication of the approach. We demonstrate the efficacy of the proposed method by coordinating robot teams composed of both ground and aerial vehicles with different sensing/control profiles and evaluate the algorithm's performance in two target tracking scenarios. Compared to the naive distributed execution of local search, our approach saves up to 60% communication and 80--92% computation on average when coordinating up to 10 robots, while outperforming the coordinate descent based algorithm in achieving a desirable trade-off between sensing and energy cost.

cs.RO↗

arXiv2021

Technical Report: Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments

Complex manipulation tasks, such as rearrangement planning of numerous objects, are combinatorially hard problems. Existing algorithms either do not scale well or assume a great deal of prior knowledge about the environment, and few offer any rigorous guarantees. In this paper, we propose a novel hybrid control architecture for achieving such tasks with mobile manipulators. On the discrete side, we enrich a temporal logic specification with mobile manipulation primitives such as moving to a point, and grasping or moving an object. Such specifications are translated to an automaton representation, which orchestrates the physical grounding of the task to mobility or manipulation controllers. The grounding from the discrete to the continuous reactive controller is online and can respond to the discovery of unknown obstacles or decide to push out of the way movable objects that prohibit task accomplishment. Despite the problem complexity, we prove that, under specific conditions, our architecture enjoys provable completeness on the discrete side, provable termination on the continuous side, and avoids all obstacles in the environment. Simulations illustrate the efficiency of our architecture that can handle tasks of increased complexity while also responding to unknown obstacles or unanticipated adverse configurations.

cs.RO↗

arXiv2021

Resilient Active Information Acquisition with Teams of Robots

Emerging applications of collaborative autonomy, such as Multi-Target Tracking, Unknown Map Exploration, and Persistent Surveillance, require robots plan paths to navigate an environment while maximizing the information collected via on-board sensors. In this paper, we consider such information acquisition tasks but in adversarial environments, where attacks may temporarily disable the robots' sensors. We propose the first receding horizon algorithm, aiming for robust and adaptive multi-robot planning against any number of attacks, which we call Resilient Active Information acquisitioN (RAIN). RAIN calls, in an online fashion, a Robust Trajectory Planning (RTP) subroutine which plans attack-robust control inputs over a look-ahead planning horizon. We quantify RTP's performance by bounding its suboptimality. We base our theoretical analysis on notions of curvature introduced in combinatorial optimization. We evaluate RAIN in three information acquisition scenarios: Multi-Target Tracking, Occupancy Grid Mapping, and Persistent Surveillance. The scenarios are simulated in C++ and a Unity-based simulator. In all simulations, RAIN runs in real-time, and exhibits superior performance against a state-of-the-art baseline information acquisition algorithm, even in the presence of a high number of attacks. We also demonstrate RAIN's robustness and effectiveness against varying models of attacks (worst-case and random), as well as, varying replanning rates.

cs.RO↗

arXiv2021

Technical Report: Scalable Active Information Acquisition for Multi-Robot Systems

This paper proposes a novel highly scalable non-myopic planning algorithm for multi-robot Active Information Acquisition (AIA) tasks. AIA scenarios include target localization and tracking, active SLAM, surveillance, environmental monitoring and others. The objective is to compute control policies for multiple robots which minimize the accumulated uncertainty of a static hidden state over an a priori unknown horizon. The majority of existing AIA approaches are centralized and, therefore, face scaling challenges. To mitigate this issue, we propose an online algorithm that relies on decomposing the AIA task into local tasks via a dynamic space-partitioning method. The local subtasks are formulated online and require the robots to switch between exploration and active information gathering roles depending on their functionality in the environment. The switching process is tightly integrated with optimizing information gathering giving rise to a hybrid control approach. We show that the proposed decomposition-based algorithm is probabilistically complete for homogeneous sensor teams and under linearity and Gaussian assumptions. We provide extensive simulation results that show that the proposed algorithm can address large-scale estimation tasks that are computationally challenging to solve using existing centralized approaches.

cs.RO↗