arXiv subjects

Paulo Tabuada

Publications and source records attributed to Paulo Tabuada.

At least 37 records · Page 2Linked to original sources

arXiv2024

Meanings and Feelings of Large Language Models: Observability of Latent States in Generative AI

We tackle the question of whether Large Language Models (LLMs), viewed as dynamical systems with state evolving in the embedding space of symbolic tokens, are observable. That is, whether there exist multiple 'mental' state trajectories that yield the same sequence of generated tokens, or sequences that belong to the same Nerode equivalence class ('meaning'). If not observable, mental state trajectories ('experiences') evoked by an input ('perception') or by feedback from the model's own state ('thoughts') could remain self-contained and evolve unbeknown to the user while being potentially accessible to the model provider. Such "self-contained experiences evoked by perception or thought" are akin to what the American Psychological Association (APA) defines as 'feelings'. Beyond the lexical curiosity, we show that current LLMs implemented by autoregressive Transformers cannot have 'feelings' according to this definition: The set of state trajectories indistinguishable from the tokenized output is a singleton. But if there are 'system prompts' not visible to the user, then the set of indistinguishable trajectories becomes non-trivial, and there can be multiple state trajectories that yield the same verbalized output. We prove these claims analytically, and show examples of modifications to standard LLMs that engender such 'feelings.' Our analysis sheds light on possible designs that would enable a model to perform non-trivial computation that is not visible to the user, as well as on controls that the provider of services using the model could take to prevent unintended behavior.

cs.AI↗

arXiv2024

A Framework for Time-Varying Optimization via Derivative Estimation

Optimization algorithms have a rich and fundamental relationship with ordinary differential equations given by its continuous-time limit. When the cost function varies with time -- typically in response to a dynamically changing environment -- online optimization becomes a continuous-time trajectory tracking problem. To accommodate these time variations, one typically requires some inherent knowledge about their nature such as a time derivative. In this paper, we propose a novel construction and analysis of a continuous-time derivative estimation scheme based on "dirty-derivatives", and show how it naturally interfaces with continuous-time optimization algorithms using the language of ISS (Input-to-State Stability). More generally, we show how a simple Lyapunov redesign technique leads to provable suboptimality guarantees when composing this estimator with any well-behaved optimization algorithm for time-varying costs.

math.OC↗

arXiv2024

Universal Approximation Power of Deep Residual Neural Networks via Nonlinear Control Theory

In this paper, we explain the universal approximation capabilities of deep residual neural networks through geometric nonlinear control. Inspired by recent work establishing links between residual networks and control systems, we provide a general sufficient condition for a residual network to have the power of universal approximation by asking the activation function, or one of its derivatives, to satisfy a quadratic differential equation. Many activation functions used in practice satisfy this assumption, exactly or approximately, and we show this property to be sufficient for an adequately deep neural network with $n+1$ neurons per layer to approximate arbitrarily well, on a compact set and with respect to the supremum norm, any continuous function from $\mathbb{R}^n$ to $\mathbb{R}^n$. We further show this result to hold for very simple architectures for which the weights only need to assume two values. The first key technical contribution consists of relating the universal approximation problem to controllability of an ensemble of control systems corresponding to a residual network and to leverage classical Lie algebraic techniques to characterize controllability. The second technical contribution is to identify monotonicity as the bridge between controllability of finite ensembles and uniform approximability on compact sets.

cs.LG↗

arXiv2023

Taming AI Bots: Controllability of Neural States in Large Language Models

We tackle the question of whether an agent can, by suitable choice of prompts, control an AI bot to any state. To that end, we first introduce a formal definition of ``meaning'' that is amenable to analysis. Then, we characterize ``meaningful data'' on which large language models (LLMs) are ostensibly trained, and ``well-trained LLMs'' through conditions that are largely met by today's LLMs. While a well-trained LLM constructs an embedding space of meanings that is Euclidean, meanings themselves do not form a vector (linear) subspace, but rather a quotient space within. We then characterize the subset of meanings that can be reached by the state of the LLMs for some input prompt, and show that a well-trained bot can reach any meaning albeit with small probability. We then introduce a stronger notion of controllability as {\em almost certain reachability}, and show that, when restricted to the space of meanings, an AI bot is controllable. We do so after introducing a functional characterization of attentive AI bots, and finally derive necessary and sufficient conditions for controllability. The fact that AI bots are controllable means that an adversary could steer them towards any state. However, the sampling process can be designed to counteract adverse actions and avoid reaching undesirable regions of state space before their boundary is crossed.

cs.AI↗

arXiv2022

Joint Continuous and Discrete Model Selection via Submodularity

In model selection problems for machine learning, the desire for a well-performing model with meaningful structure is typically expressed through a regularized optimization problem. In many scenarios, however, the meaningful structure is specified in some discrete space, leading to difficult nonconvex optimization problems. In this paper, we connect the model selection problem with structure-promoting regularizers to submodular function minimization with continuous and discrete arguments. In particular, we leverage the theory of submodular functions to identify a class of these problems that can be solved exactly and efficiently with an agnostic combination of discrete and continuous optimization routines. We show how simple continuous or discrete constraints can also be handled for certain problem classes and extend these ideas to a robust optimization framework. We also show how some problems outside of this class can be embedded within the class, further extending the class of problems our framework can accommodate. Finally, we numerically validate our theoretical results with several proof-of-concept examples with synthetic and real-world data, comparing against state-of-the-art algorithms.

math.OC↗

arXiv2022

Let's Talk Through Physics! Covert Cyber-Physical Data Exfiltration on Air-Gapped Edge Devices

Although organizations are continuously making concerted efforts to harden their systems against network attacks by air-gapping critical systems, attackers continuously adapt and uncover covert channels to exfiltrate data from air-gapped systems. For instance, attackers have demonstrated the feasibility of exfiltrating data from a computer sitting in a Faraday cage by exfiltrating data using magnetic fields. Although a large body of work has recently emerged highlighting various physical covert channels, these attacks have mostly targeted open-loop cyber-physical systems where the covert channels exist on physical channels that are not being monitored by the victim. Network architectures such as fog computing push sensitive data to cyber-physical edge devices--whose physical side channels are typically monitored via state estimation. In this paper, we formalize covert data exfiltration that uses existing cyber-physical models and infrastructure of individual devices to exfiltrate data in a stealthy manner, i.e., we propose a method to circumvent cyber-physical state estimation intrusion detection techniques while exfiltrating sensitive data from the network. We propose a generalized model for encoding and decoding sensitive data within cyber-physical control loops. We evaluate our approach on a distributed IoT network that includes computation nodes residing on physical drones as well as on an industrial control system for the control of a robotic arm. Unlike prior works, we formalize the constraints of covert cyber-physical channel exfiltration in the presence of a defender performing state estimation.

cs.CR↗

arXiv2022

From LTL to rLTL Monitoring: Improved Monitorability through Robust Semantics

Runtime monitoring is commonly used to detect the violation of desired properties in safety critical cyber-physical systems by observing its executions. Bauer et al. introduced an influential framework for monitoring Linear Temporal Logic (LTL) properties based on a three-valued semantics: the formula is already satisfied by the given prefix, it is already violated, or it is still undetermined, i.e., it can still be satisfied and violated by appropriate extensions. However, a wide range of formulas are not monitorable under this approach, meaning that they have a prefix for which satisfaction and violation will always remain undetermined no matter how it is extended. In particular, Bauer et al. report that 44% of the formulas they consider in their experiments fall into this category. Recently, a robust semantics for LTL was introduced to capture different degrees by which a property can be violated. In this paper we introduce a robust semantics for finite strings and show its potential in monitoring: every formula considered by Bauer et al. is monitorable under our approach. Furthermore, we discuss which properties that come naturally in LTL monitoring - such as the realizability of all truth values - can be transferred to the robust setting. Lastly, we show that LTL formulas with robust semantics can be monitored by deterministic automata and report on a prototype implementation.

cs.FL↗

arXiv2022

Controlled invariant sets: implicit closed-form representations and applications

We revisit the problem of computing (robust) controlled invariant sets for discrete-time linear systems. Departing from previous approaches, we consider implicit, rather than explicit, representations for controlled invariant sets. Moreover, by considering such representations in the space of states and finite input sequences we obtain closed-form expressions for controlled invariant sets. An immediate advantage is the ability to handle high-dimensional systems since the closed-form expression is computed in a single step rather than iteratively. To validate the proposed method, we present thorough case studies illustrating that in safety-critical scenarios the implicit representation suffices in place of the explicit invariant set. The proposed method is complete in the absence of disturbances, and we provide a weak completeness result when disturbances are present.

math.OC↗

arXiv2022

Learning to control from expert demonstrations

In this paper, we revisit the problem of learning a stabilizing controller from a finite number of demonstrations by an expert. By first focusing on feedback linearizable systems, we show how to combine expert demonstrations into a stabilizing controller, provided that demonstrations are sufficiently long and there are at least $n+1$ of them, where $n$ is the number of states of the system being controlled. When we have more than $n+1$ demonstrations, we discuss how to optimally choose the best $n+1$ demonstrations to construct the stabilizing controller. We then extend these results to a class of systems that can be embedded into a higher-dimensional system containing a chain of integrators. The feasibility of the proposed algorithm is demonstrated by applying it on a CrazyFlie 2.0 quadrotor.

eess.SY↗

arXiv2022

Dirty derivatives for output feedback stabilization

Dirty derivatives are routinely used in industrial settings, particularly in the implementation of the derivative term in PID control, and are especially appealing due to their noise-attenuation and model-free characteristics. In this paper, we provide a Lyapunov-based proof for the stability of linear time-invariant control systems in controller canonical form when utilizing dirty derivatives in place of observers for the purpose of output feedback. This is, to the best of the authors' knowledge, the first time that stability proofs are provided for the use of dirty derivatives in lieu of derivatives of different orders. In the spirit of adaptive control, we also show how dirty derivatives can be used for output feedback control when the control gain is unknown.

math.OC↗

arXiv2021

Being correct is not enough: efficient verification using robust linear temporal logic

While most approaches in formal methods address system correctness, ensuring robustness has remained a challenge. In this paper we present and study the logic rLTL which provides a means to formally reason about both correctness and robustness in system design. Furthermore, we identify a large fragment of rLTL for which the verification problem can be efficiently solved, i.e., verification can be done by using an automaton, recognizing the behaviors described by the rLTL formula $φ$, of size at most $\mathcal{O} \left( 3^{ |φ|} \right)$, where $|φ|$ is the length of $φ$. This result improves upon the previously known bound of $\mathcal{O}\left(5^{|φ|} \right)$ for rLTL verification and is closer to the LTL bound of $\mathcal{O}\left( 2^{|φ|} \right)$. The usefulness of this fragment is demonstrated by a number of case studies showing its practical significance in terms of expressiveness, the ability to describe robustness, and the fine-grained information that rLTL brings to the process of system verification. Moreover, these advantages come at a low computational overhead with respect to LTL verification.

cs.LO↗

arXiv2021

Learned Uncertainty Calibration for Visual Inertial Localization

The widely-used Extended Kalman Filter (EKF) provides a straightforward recipe to estimate the mean and covariance of the state given all past measurements in a causal and recursive fashion. For a wide variety of applications, the EKF is known to produce accurate estimates of the mean and typically inaccurate estimates of the covariance. For applications in visual inertial localization, we show that inaccuracies in the covariance estimates are \emph{systematic}, i.e. it is possible to learn a nonlinear map from the empirical ground truth to the estimated one. This is demonstrated on both a standard EKF in simulation and a Visual Inertial Odometry system on real-world data.

cs.RO↗

arXiv2021

Automaton-based Implicit Controlled Invariant Set Computation for Discrete-Time Linear Systems

In this paper, we derive closed-form expressions for implicit controlled invariant sets for discrete-time controllable linear systems with measurable disturbances. In particular, a disturbance-reactive (or disturbance feedback) controller in the form of a parameterized finite automaton is considered. We show that, for a class of automata, the robust positively invariant sets of the corresponding closed-loop systems can be expressed by a set of linear inequality constraints in the joint space of system states and controller parameters. This leads to an implicit representation of the invariant set in a lifted space. We further show how the same parameterization can be used to compute invariant sets when the disturbance is not available for measurement.

eess.SY↗

arXiv2021

Decentralized Secure State-Tracking in Multi-Agent Systems

This paper addresses the problem of decentralized state-tracking in the presence of sensor attacks. We consider a network of nodes where each node has the objective of tracking the state of a linear dynamical system based on its measurements and messages exchanged with neighboring nodes notwithstanding some measurements being spoofed by an adversary. We propose a novel decentralized attack-resilient state-tracking algorithm based on the simple observation that a compressed version of all the network measurements suffices to reconstruct the state. This motivates a 2-step solution to the decentralized secure state-tracking problem: (1) each node tracks the compressed version of all the network measurements, and (2) each node asymptotically reconstructs the state from the output of step (1). We prove that, under mild technical assumptions, our algorithm enables each node to track the state of the linear system and thus solves the decentralized secure state-tracking problem.

eess.SY↗

arXiv2021

On the Computational Complexity of the Secure State-Reconstruction Problem

In this paper, we discuss the computational complexity of reconstructing the state of a linear system from sensor measurements that have been corrupted by an adversary. The first result establishes that the problem is, in general, NP-hard. We then introduce the notion of eigenvalue observability and show that the state can be reconstructed in polynomial time when each eigenvalue is observable by at least $2s+1$ sensors and at most $s$ sensors are corrupted by an adversary. However, there is a gap between eigenvalue observability and the possibility of reconstructing the state despite attacks - this gap has been characterized in the literature by the notion of sparse observability. To better understand this, we show that when the $\mathbf{A}$ matrix of the linear system has unitary geometric multiplicity, the gap disappears, i.e., eigenvalue observability coincides with sparse observability, and there exists a polynomial time algorithm to reconstruct the state provided the state can be reconstructed.

eess.SY↗

arXiv2021

Data-driven Stabilization of SISO Feedback Linearizable Systems

In this paper we propose a methodology for stabilizing single-input single-output feedback linearizable systems when no system model is known and no prior data is available to identify a model. Conceptually, we have been greatly inspired by the work of Fliess and Join on intelligent PID controllers and the results in this paper provide sufficient conditions under which a modified version of their approach is guaranteed to result in asymptotically stable behavior. One of the key advantages of the proposed results is that, contrary to other approaches to controlling systems without a model (or with a partial model), such as reinforcement learning, there is no need for extensive training nor large amounts of data. Technically, our results draw heavily from the work of Nesic and co-workers on observer and controller design based on approximate models. Along the way we also make connections with other well established results such as high-gain observers and adaptive control. Although we focus on the simple setting of single-input single-output feedback linearizable systems we believe the presented results are already theoretically insightful and practically useful, the last point being substantiated by experimental evidence.

eess.SY↗

arXiv2021

Sampled-Data Stabilization with Control Lyapunov Functions via Quadratically Constrained Quadratic Programs

Controller design for nonlinear systems with Control Lyapunov Function (CLF) based quadratic programs has recently been successfully applied to a diverse set of difficult control tasks. These existing formulations do not address the gap between design with continuous time models and the discrete time sampled implementation of the resulting controllers, often leading to poor performance on hardware platforms. We propose an approach to close this gap by synthesizing sampled-data counterparts to these CLF-based controllers, specified as quadratically constrained quadratic programs (QCQPs). Assuming feedback linearizability and stable zero-dynamics of a system's continuous time model, we derive practical stability guarantees for the resulting sampled-data system. We demonstrate improved performance of the proposed approach over continuous time counterparts in simulation.

eess.SY↗

arXiv2020

Distortion based Light-weight Security for Cyber-Physical Systems

In Cyber-Physical Systems (CPS), inference based on communicated data is of critical significance as it can be used to manipulate or damage the control operations by adversaries. This calls for efficient mechanisms for secure transmission of data since control systems are becoming increasingly distributed over larger geographical areas. Distortion based security, recently proposed as one candidate for secure transmissions in CPS, is not only more appropriate for these applications but also quite frugal in terms of prior requirements on shared keys. In this paper, we propose distortion-based metrics to protect CPS communication and show that it is possible to confuse adversaries with just a few bits of pre-shared keys. In particular, we will show that a linear dynamical system can communicate its state in a manner that prevents an eavesdropper from accurately learning the state.

cs.IT↗