arXiv subjects

Karl H. Johansson

Publications and source records attributed to Karl H. Johansson.

At least 163 records · Page 9Linked to original sources

arXiv2023

Differentially Private Set-Based Estimation Using Zonotopes

For large-scale cyber-physical systems, the collaboration of spatially distributed sensors is often needed to perform the state estimation process. Privacy concerns naturally arise from disclosing sensitive measurement signals to a cloud estimator that predicts the system state. To solve this issue, we propose a differentially private set-based estimation protocol that preserves the privacy of the measurement signals. Compared to existing research, our approach achieves less privacy loss and utility loss using a numerically optimized truncated noise distribution. The proposed estimator is perturbed by weaker noise than the analytical approaches in the literature to guarantee the same level of privacy, therefore improving the estimation utility. Numerical and comparison experiments with truncated Laplace noise are presented to support our approach. Zonotopes, a less conservative form of set representation, are used to represent estimation sets, giving set operations a computational advantage. The privacy-preserving noise anonymizes the centers of these estimated zonotopes, concealing the precise positions of the estimated zonotopes.

cs.CR↗

arXiv2023

Adaptive Sampling of Algal Blooms Using Autonomous Underwater Vehicle and Satellite Imagery: Experimental Validation in the Baltic Sea

This paper investigates using satellite data to improve adaptive sampling missions, particularly for front tracking scenarios such as with algal blooms. Our proposed solution to find and track algal bloom fronts uses an Autonomous Underwater Vehicle (AUV) equipped with a sensor that measures the concentration of chlorophyll a and satellite data. The proposed method learns the kernel parameters for a Gaussian process model using satellite images of chlorophyll a from the previous days. Then, using the data collected by the AUV, it models chlorophyll a concentration online. We take the gradient of this model to obtain the direction of the algal bloom front and feed it to our control algorithm. The performance of this method is evaluated through realistic simulations for an algal bloom front in the Baltic sea, using the models of the AUV and the chlorophyll a sensor. We compare the performance of different estimation methods, from GP to curve interpolation using least squares. Sensitivity analysis is performed to evaluate the impact of sensor noise on the methods performance. We implement our method on an AUV and run experiments in the Stockholm archipelago in the summer of 2022.

eess.SY↗

arXiv2023

Hierarchical Finite State Machines for Efficient Optimal Planning in Large-scale Systems

In this paper, we consider a planning problem for a hierarchical finite state machine (HFSM) and develop an algorithm for efficiently computing optimal plans between any two states. The algorithm consists of an offline and an online step. In the offline step, one computes exit costs for each machine in the HFSM. It needs to be done only once for a given HFSM, and it is shown to have time complexity scaling linearly with the number of machines in the HFSM. In the online step, one computes an optimal plan from an initial state to a goal state, by first reducing the HFSM (using the exit costs), computing an optimal trajectory for the reduced HFSM, and then expand this trajectory to an optimal plan for the original HFSM. The time complexity is near-linearly with the depth of the HFSM. It is argued that HFSMs arise naturally for large-scale control systems, exemplified by an application where a robot moves between houses to complete tasks. We compare our algorithm with Dijkstra's algorithm on HFSMs consisting of up to 2 million states, where our algorithm outperforms the latter, being several orders of magnitude faster.

eess.SY↗

arXiv2023

What is the Expected Transient Behavior of Opinion Evolution for Two Communities?

We study the transient behavior of a gossip model, in which agents randomly interact pairwise over a weighted graph with two communities. Edges within each community have identical weights, different from the weights between communities. It is shown that, at the early stage of the opinion evolution, the expected agent states in the same community have identical sign, despite influence of stubborn agents. Moreover, it is shown that the expected states of the agents in the same community concentrate around the initial average opinion of that community, if the weights within communities are larger than between. In contrast, if the edge weights between communities are larger, then the expected states of all agents concentrate around everyone's initial average opinion. Different from the traditional asymptotic analysis in the opinion dynamics literature, these results focus on the initial phase of opinion evolution and establish a correspondence between community structure and transient behavior of the gossip model. The results are illustrated by numerical examples.

eess.SY↗

arXiv2023

Complexity reduction for resilient state estimation of uniformly observable nonlinear systems

A resilient state estimation scheme for uniformly observable nonlinear systems, based on a method for local identification of sensor attacks, is presented. The estimation problem is combinatorial in nature, and so many methods require substantial computational and storage resources as the number of sensors increases. To reduce the complexity, the proposed method performs the attack identification with local subsets of the measurements, not with the set of all measurements. A condition for nonlinear attack identification is introduced as a relaxed version of existing redundant observability condition. It is shown that an attack identification can be performed even when the state cannot be recovered from the measurements. As a result, although a portion of measurements are compromised, they can be locally identified and excluded from the state estimation, and thus the true state can be recovered. Simulation results demonstrate the effectiveness of the proposed scheme.

eess.SY↗

arXiv2023

Learning Flow Functions from Data with Applications to Nonlinear Oscillators

We describe a recurrent neural network (RNN) based architecture to learn the flow function of a causal, time-invariant and continuous-time control system from trajectory data. By restricting the class of control inputs to piecewise constant functions, we show that learning the flow function is equivalent to learning the input-to-state map of a discrete-time dynamical system. This motivates the use of an RNN together with encoder and decoder networks which map the state of the system to the hidden state of the RNN and back. We show that the proposed architecture is able to approximate the flow function by exploiting the system's causality and time-invariance. The output of the learned flow function model can be queried at any time instant. We experimentally validate the proposed method using models of the Van der Pol and FitzHugh Nagumo oscillators. In both cases, the results demonstrate that the architecture is able to closely reproduce the trajectories of these two systems. For the Van der Pol oscillator, we further show that the trained model generalises to the system's response with a prolonged prediction time horizon as well as control inputs outside the training distribution. For the FitzHugh-Nagumo oscillator, we show that the model accurately captures the input-dependent phenomena of excitability.

eess.SY↗

arXiv2023

Parameterization-Free Observer Design for Nonlinear Systems: Application to the State Estimation of Networked SIR Epidemics

Traditional observer design methods rely on certain properties of the system's nonlinearity, such as Lipschitz continuity, one-sided Lipschitzness, a bounded Jacobian, or quadratic boundedness. These properties are described by parameterized inequalities. However, enforcing these inequalities globally can lead to very large parameters, resulting in overly conservative observer design criteria. These criteria become infeasible for highly nonlinear applications, such as networked epidemic processes. In this paper, we present an observer design approach for estimating the state of nonlinear systems, without requiring any parameterization of the system's nonlinearities. The proposed observer design depends only on systems' matrices and applies to systems with any nonlinearity. We establish different design criteria for ensuring both asymptotic and exponential convergence of the estimation error to zero. To demonstrate the efficacy of our approach, we employ it for estimating the state of a networked SIR epidemic model. We show that, even in the presence of measurement noise, the observer can accurately estimate the epidemic state of each node in the network. To the best of our knowledge, the proposed observer is the first that is capable of estimating the state of networked SIR models.

eess.SY↗

arXiv2023

Feedback Design for Devising Optimal Epidemic Control Policies

This paper proposes a feedback design that effectively copes with uncertainties for reliable epidemic monitoring and control. There are several optimization-based methods to estimate the parameters of an epidemic model by utilizing past reported data. However, due to the possibility of noise in the data, the estimated parameters may not be accurate, thereby exacerbating the model uncertainty. To address this issue, we provide an observer design that enables robust state estimation of epidemic processes, even in the presence of uncertain models and noisy measurements. Using the estimated model and state, we then devise optimal control policies by minimizing a predicted cost functional. To demonstrate the effectiveness of our approach, we implement it on a modified SIR epidemic model. The results show that our proposed method is efficient in mitigating the uncertainties that may arise in epidemic monitoring and control.

math.OC↗

arXiv2023

Distributed Optimization for Quadratic Cost Functions over Large-Scale Networks with Quantized Communication and Finite-Time Convergence

We propose two distributed iterative algorithms that can be used to solve, in finite time, the distributed optimization problem over quadratic local cost functions in large-scale networks. The first algorithm exhibits synchronous operation whereas the second one exhibits asynchronous operation. Both algorithms share salient features. Specifically, the algorithms operate exclusively with quantized values, which means that the information stored, processed and exchanged between neighboring nodes is subject to deterministic uniform quantization. The algorithms rely on event-driven updates in order to reduce energy consumption, communication bandwidth, network congestion, and/or processor usage. Finally, once the algorithms converge, nodes distributively terminate their operation. We prove that our algorithms converge in a finite number of iterations to the exact optimal solution depending on the quantization level, and we present applications of our algorithms to (i) optimal task scheduling for data centers, and (ii) global model aggregation for distributed federated learning. We provide simulations of these applications to illustrate the operation, performance, and advantages of the proposed algorithms. Additionally, it is shown that our proposed algorithms compare favorably to algorithms in the current literature. Quantized communication and asynchronous updates increase the required time to completion, but finite-time operation is maintained.

eess.SY↗

arXiv2023

Secure State Estimation with Asynchronous Measurements against Malicious Measurement-data and Time-stamp Manipulation

This paper proposes a secure state estimation scheme with non-periodic asynchronous measurements for linear continuous-time systems under false data attacks on the measurement transmit channel. After sampling the output of the system, a sensor transmits the measurement information in a triple composed of sensor index, time-stamp, and measurement value to the fusion center via vulnerable communication channels. The malicious attacker can corrupt a subset of the sensors through (i) manipulating the time-stamp and measurement value; (ii) blocking transmitted measurement triples; or (iii) injecting fake measurement triples. To deal with such attacks, we propose the design of local estimators based on observability space decomposition, where each local estimator updates the local state and sends it to the fusion center after sampling a measurement. Whenever there is a local update, the fusion center combines all the local states and generates a secure state estimate by adopting the median operator. We prove that local estimators of benign sensors are unbiased with stable covariance. Moreover, the fused central estimation error has bounded expectation and covariance against at most $p$ corrupted sensors as long as the system is $2p$-sparse observable. The efficacy of the proposed scheme is demonstrated through an application on a benchmark example of the IEEE 14-bus system.

eess.SY↗

arXiv2023

Policy Evaluation in Distributional LQR

Distributional reinforcement learning (DRL) enhances the understanding of the effects of the randomness in the environment by letting agents learn the distribution of a random return, rather than its expected value as in standard RL. At the same time, a main challenge in DRL is that policy evaluation in DRL typically relies on the representation of the return distribution, which needs to be carefully designed. In this paper, we address this challenge for a special class of DRL problems that rely on linear quadratic regulator (LQR) for control, advocating for a new distributional approach to LQR, which we call \emph{distributional LQR}. Specifically, we provide a closed-form expression of the distribution of the random return which, remarkably, is applicable to all exogenous disturbances on the dynamics, as long as they are independent and identically distributed (i.i.d.). While the proposed exact return distribution consists of infinitely many random variables, we show that this distribution can be approximated by a finite number of random variables, and the associated approximation error can be analytically bounded under mild assumptions. Using the approximate return distribution, we propose a zeroth-order policy gradient algorithm for risk-averse LQR using the Conditional Value at Risk (CVaR) as a measure of risk. Numerical experiments are provided to illustrate our theoretical results.

math.OC↗

arXiv2023

Online Control Synthesis for Uncertain Systems under Signal Temporal Logic Specifications

This paper studies the online control synthesis problem for uncertain discrete-time systems subject to signal temporal logic (STL) specifications. Different from existing techniques, this work proposes an approach based on STL, reachability analysis, and temporal logic trees. Firstly, a real-time version of STL semantics and a tube-based temporal logic tree (tTLT) are proposed. We show that the tTLT is an underapproximation for the STL formula, in the sense that a trajectory satisfying an tTLT also satisfies the corresponding STL formula. Secondly, an online control synthesis algorithm is designed. It is shown that when the STL formula is robustly satisfiable and the initial state of the system belongs to the initial root node of the tTLT, it is guaranteed that the trajectory generated by the control synthesis algorithm satisfies the STL formula. The effectiveness of the proposed approach is verified by a simulation example and a practical experiment.

eess.SY↗

arXiv2023

Competitive epidemic networks with multiple survival-of-the-fittest outcomes

We use a deterministic model to study two competing viruses spreading over a two-layer network in the Susceptible--Infected--Susceptible (SIS) framework, and address a central problem of identifying the winning virus in a "survival-of-the-fittest" battle. Existing sufficient conditions ensure that the same virus always wins regardless of initial states. For networks with an arbitrary but finite number of nodes, there exists a necessary and sufficient condition that guarantees local exponential stability of the two equilibria corresponding to each virus winning the battle, meaning that either of the viruses can win, depending on the initial states. However, establishing existence and finding examples of networks with more than three nodes that satisfy such a condition has remained unaddressed. In this paper, we prove that, for any arbitrary number of nodes, such networks exist. We do this by proving that given almost any network layer of one virus, there exists a network layer for the other virus such that the resulting two-layer network satisfies the aforementioned condition. To operationalize our findings, a four-step procedure is developed to reliably and consistently design one of the network layers, when given the other layer. Conclusions from numerical case studies, including a real-world mobility network that captures the commuting patterns for people between $107$ provinces in Italy, extend on the theoretical result and its consequences.

math.DS↗

arXiv2022

Regret and Cumulative Constraint Violation Analysis for Distributed Online Constrained Convex Optimization

This paper considers the distributed online convex optimization problem with time-varying constraints over a network of agents. This is a sequential decision making problem with two sequences of arbitrarily varying convex loss and constraint functions. At each round, each agent selects a decision from the decision set, and then only a portion of the loss function and a coordinate block of the constraint function at this round are privately revealed to this agent. The goal of the network is to minimize the network-wide loss accumulated over time. Two distributed online algorithms with full-information and bandit feedback are proposed. Both dynamic and static network regret bounds are analyzed for the proposed algorithms, and network cumulative constraint violation is used to measure constraint violation, which excludes the situation that strictly feasible constraints can compensate the effects of violated constraints. In particular, we show that the proposed algorithms achieve $\mathcal{O}(T^{\max\{κ,1-κ\}})$ static network regret and $\mathcal{O}(T^{1-κ/2})$ network cumulative constraint violation, where $T$ is the time horizon and $κ\in(0,1)$ is a user-defined trade-off parameter. Moreover, if the loss functions are strongly convex, then the static network regret bound can be reduced to $\mathcal{O}(T^κ)$. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

math.OC↗

arXiv2022

Distributed Computation of Exact Average Degree and Network Size in Finite Number of Steps under Quantized Communication

We consider the problems of computing the average degree and the size of a given network in a distributed fashion under quantized communication. We present two distributed algorithms which rely on quantized operation (i.e., nodes process and transmit quantized messages), and are able to calculate the exact solutions in a finite number of steps. Furthermore, during the operation of our algorithms, each node is able to determine in a distributed manner whether convergence has been achieved and thus terminate its operation. To the best of the authors' knowledge these algorithms are the first to find the exact solutions under quantized communication (i.e., there is no error in the final calculation). Additionally, note that our network size calculation algorithm is the first in the literature which calculates the exact size of a network in a finite number of steps without introducing a final error. This error in other algorithms can be either due to quantization or asymptotic convergence. In our case, no error is introduced since the desired result is calculated in the form of a fraction involving a quantized numerator and a quantized denominator. Finally, we demonstrate the operation of our algorithms and their potential advantages.

eess.SY↗

arXiv2022

Safe Reinforcement Learning Using Black-Box Reachability Analysis

Reinforcement learning (RL) is capable of sophisticated motion planning and control for robots in uncertain environments. However, state-of-the-art deep RL approaches typically lack safety guarantees, especially when the robot and environment models are unknown. To justify widespread deployment, robots must respect safety constraints without sacrificing performance. Thus, we propose a Black-box Reachability-based Safety Layer (BRSL) with three main components: (1) data-driven reachability analysis for a black-box robot model, (2) a trajectory rollout planner that predicts future actions and observations using an ensemble of neural networks trained online, and (3) a differentiable polytope collision check between the reachable set and obstacles that enables correcting unsafe actions. In simulation, BRSL outperforms other state-of-the-art safe RL methods on a Turtlebot 3, a quadrotor, a trajectory-tracking point mass, and a hexarotor in wind with an unsafe set adjacent to the area of highest reward.

cs.RO↗

arXiv2022

Safe Reinforcement Learning using Data-Driven Predictive Control

Reinforcement learning (RL) algorithms can achieve state-of-the-art performance in decision-making and continuous control tasks. However, applying RL algorithms on safety-critical systems still needs to be well justified due to the exploration nature of many RL algorithms, especially when the model of the robot and the environment are unknown. To address this challenge, we propose a data-driven safety layer that acts as a filter for unsafe actions. The safety layer uses a data-driven predictive controller to enforce safety guarantees for RL policies during training and after deployment. The RL agent proposes an action that is verified by computing the data-driven reachability analysis. If there is an intersection between the reachable set of the robot using the proposed action, we call the data-driven predictive controller to find the closest safe action to the proposed unsafe action. The safety layer penalizes the RL agent if the proposed action is unsafe and replaces it with the closest safe one. In the simulation, we show that our method outperforms state-of-the-art safe RL methods on the robotics navigation problem for a Turtlebot 3 in Gazebo and a quadrotor in Unreal Engine 4 (UE4).

cs.RO↗

arXiv2022

Rate analysis of dual averaging for nonconvex distributed optimization

This work studies nonconvex distributed constrained optimization over stochastic communication networks. We revisit the distributed dual averaging algorithm, which is known to converge for convex problems. We start from the centralized case, for which the change of two consecutive updates is taken as the suboptimality measure. We validate the use of such a measure by showing that it is closely related to stationarity. This equips us with a handle to study the convergence of dual averaging in nonconvex optimization. We prove that the squared norm of this suboptimality measure converges at rate $\mathcal{O}(1/t)$. Then, for the distributed setup we show convergence to the stationary point at rate $\mathcal{O}(1/t)$. Finally, a numerical example is given to illustrate our theoretical results.

math.OC↗