arXiv subjects

Kyriakos G. Vamvoudakis

Publications and source records attributed to Kyriakos G. Vamvoudakis.

At least 19 recordsLinked to original sources

arXiv2026

Deception Against Data-Driven Linear-Quadratic Control

Deception is a common defense mechanism against adversaries with an information disadvantage. It can force such adversaries to select suboptimal policies for a defender's benefit. We consider a setting where an adversary tries to learn the optimal linear-quadratic attack against a system, the dynamics of which it does not know. On the other end, a defender who knows its dynamics exploits its information advantage and injects a deceptive input into the system to mislead the adversary. The defender's aim is to then strategically design this deceptive input: it should force the adversary to learn, as closely as possible, a pre-selected attack that is different from the optimal one. We show that this deception design problem boils down to a solution of a coupled algebraic Riccati and a Lyapunov equation which, however, are challenging to tackle analytically. Nevertheless, we use a block successive over-relaxation algorithm to extract their solution numerically and prove the algorithm's convergence under certain conditions. We perform simulations on a benchmark aircraft, where we showcase how the proposed algorithm can mislead adversaries into learning attacks that are less performance-degrading.

eess.SY↗

arXiv2026

Meta-Multi-Agent Reinforcement Learning for Fast Adaptation of Interactive Policies with Applications to Autonomous Driving

This paper develops a meta-multi-agent reinforcement learning (meta-MARL) framework to enable fast adaptation of interactive policies in a multi-agent system (MAS). Meta-reinforcement learning (meta-RL) enables agents to rapidly adapt to new tasks/environments using a bi-level optimization mechanism. However, existing meta-RL generally focuses on single-agent systems. Extending these frameworks and algorithms to multi-agent systems poses additional challenges, as tasks are characterized by not only the environment but also agents' strategic interactions. To address these challenges, we model multi-agent reinforcement learning (MARL) problems as Markov games (MGs) and develop a meta-MARL framework for rapid interactive policy adaptation across a distribution of MGs. A new concept, called meta-NE, is defined to describe the desired solution concept in a meta-MARL problem. Sufficient conditions for the equivalence between a meta-NE and a stationary point of the gradient-play-based meta-MARL algorithm are established. Our evaluation on autonomous-driving tasks demonstrates that the proposed meta-MARL method achieves faster adaptation than pretrained MARL baselines, validating the effectiveness of our framework.

cs.AI↗

arXiv2026

Anytime Primal--Dual Certification of the Maximum Disturbance Radius in Robust MPC

Adjustable-set robust model predictive control (MPC) characterizes a state-dependent maximum disturbance radius, which can be interpreted as a certified robustness reserve. Existing methods primarily focus on optimizing and propagating this reserve under in-set disturbances. This letter investigates how much reserve remains after a finite out-of-set disturbance without immediately re-solving the full optimization problem. To this end, we propose a two-sided reserve-depletion envelope formed by independent primal and dual correction hierarchies, which supply monotone lower and upper bounds, respectively. The envelope remains valid across active-set changes, has an online-computable width, and contracts monotonically under subspace expansion. Leveraging its finite-step exactness, we present a basis-first adaptive algorithm that operates in an anytime manner: every completed reduced solve returns a valid certificate, enabling early termination once a prescribed tolerance is reached. Across the tested cases, numerical studies report no certificate violations and median speedups of up to 4.97x over warm-started full re-optimization at a 2% certificate-width tolerance.

eess.SY↗

arXiv2026

Over-Approximating Minimizer Sets of Constrained Convex Programs with Parametric Uncertainty via Reachability Analysis

We study the set of solutions to a parameterized, strongly convex optimization problem whose cost depends on uncertain, bounded parameters. We compute a certified outer approximation of the corresponding set of optimizers, using convergence properties of the projected gradient descent (PGD) algorithm for convex programs. Concretely, by treating the cost parameter as constant but unknown, we interpret the PGD iterates as an uncertain dynamical system and analyze its forward reachable sets. Since PGD converges exponentially to the unique optimizer for each fixed parameter, these reachable sets provide outer approximations of the optimizer set, with an explicit error bound that decays exponentially with the iteration count. We apply system-level synthesis (SLS) on the PGD dynamics to optimize the step-size sequence and obtain reachable-set over-approximations. Our method outperforms existing baselines in over-approximating, with low conservativeness, the minimizer sets of convex programs with uncertain costs and high-dimensional decision variables.

math.OC↗

arXiv2026

Cost-Matching Model Predictive Control for Efficient Reinforcement Learning in Humanoid Locomotion

In this paper, we propose a cost-matching approach for optimal humanoid locomotion within a Model Predictive Control (MPC)-based Reinforcement Learning (RL) framework. A parameterized MPC formulation with centroidal dynamics is trained to approximate the action-value function obtained from high-fidelity closed-loop data. Specifically, the MPC cost-to-go is evaluated along recorded state-action trajectories, and the parameters are updated to minimize the discrepancy between MPC-predicted values and measured returns. This formulation enables efficient gradient-based learning while avoiding the computational burden of repeatedly solving the MPC problem during training. The proposed method is validated in simulation using a commercial humanoid platform. Results demonstrate improved locomotion performance and robustness to model mismatch and external disturbances compared with manually tuned baselines.

cs.RO↗

arXiv2025

Tailoring Reproducing Kernels for Optimal Control via Policy Iteration

This paper presents a novel approach to formulating the actor-critic method for optimal control by casting policy iteration in reproducing kernel Hilbert spaces (RKHSs -- also known as native spaces). By tailoring the reproducing kernel and RKHS to the dynamics of the nonlinear optimal control problem, we leverage recent advancements in characterizing error bounds from statistical and machine learning theory. These approximations define a general strategy to select the bases of the actor-critic networks, and we formally guarantee for the first time that this basis selection procedure leads to closed-form error bounds for the individual steps of policy iteration. These bounds often have a geometric and computable form, making them potentially useful for a priori or a posteriori evaluation of candidate collections of scattered bases. Numerical studies subsequently provide qualitative evidence of the practical performance achieved for the full recursion using the algorithms and theory developed for the single-step error bounds.

math.OC↗

arXiv2025

Quantum Deception: Honey-X Deception using Quantum Games

In this paper, we develop a framework for deception in quantum games, extending the Honey-X paradigm from classical zero-sum settings into the quantum domain. Building on a view of deception in classical games as manipulation of a player's perception of the payoff matrix, we formalize quantum deception as controlled perturbations of the payoff Hamiltonian subject to a deception budget. We show that when victims are aware of possible deception, their equilibrium strategies surprisingly coincide with those of naive victims who fully trust the deceptive Hamiltonian. This equivalence allows us to cast quantum deception as a bilevel optimization problem, which can be reformulated into a bilinear semidefinite program. To illustrate the framework, we present simulations on quantum versions of the Penny Flip game, demonstrating how quantum strategy spaces and non-classical payoffs can amplify the impact of deception relative to classical formulations.

eess.SY↗

arXiv2025

Input-Output Data-Driven Sensor Selection for Cyber-Physical Systems

In this paper, we consider the problem of input-output data-driven sensor selection for unknown cyber-physical systems (CPS). In particular, out of a large set of sensors available for use, we choose a subset of them that maximizes a metric of observability of the CPS. The considered observability metric is related to the $\mathcal{H}_2$ system norm, which quantifies the average output energy of the selected sensors over a finite or an infinite horizon. However, its computation inherently requires knowledge of the unknown matrices of the system, so we draw connections from the reinforcement learning literature and design an input-output data-driven algorithm to compute it in a model-free manner. We then use the derived data-driven metric expression to choose the best sensors of the system in polynomial time, effectively obtaining a provably convergent model-free sensor selection process. Additionally, we show how the proposed data-driven approach can be exploited to select sensors that optimize volumetric measures of observability, while also noting its applicability to the dual problem of actuator selection. Simulations are performed to demonstrate the validity and effectiveness of the proposed approach.

eess.SY↗

arXiv2025

A Physics-Informed Learning Framework to Solve the Infinite-Horizon Optimal Control Problem

We propose a physics-informed neural networks (PINNs) framework to solve the infinite-horizon optimal control problem of nonlinear systems. In particular, since PINNs are generally able to solve a class of partial differential equations (PDEs), they can be employed to learn the value function of the infinite-horizon optimal control problem via solving the associated steady-state Hamilton-Jacobi-Bellman (HJB) equation. However, an issue here is that the steady-state HJB equation generally yields multiple solutions; hence if PINNs are directly employed to it, they may end up approximating a solution that is different from the optimal value function of the problem. We tackle this by instead applying PINNs to a finite-horizon variant of the steady-state HJB that has a unique solution, and which uniformly approximates the optimal value function as the horizon increases. An algorithm to verify if the chosen horizon is large enough is also given, as well as a method to extend it -- with reduced computations and robustness to approximation errors -- in case it is not. Unlike many existing methods, the proposed technique works well with non-polynomial basis functions, does not require prior knowledge of a stabilizing controller, and does not perform iterative policy evaluations. Simulations are performed, which verify and clarify theoretical findings.

eess.SY↗

arXiv2024

Resilient Learning-Based Control Under Denial-of-Service Attacks

In this paper, we have proposed a resilient reinforcement learning method for discrete-time linear systems with unknown parameters, under denial-of-service (DoS) attacks. The proposed method is based on policy iteration that learns the optimal controller from input-state data amidst DoS attacks. We achieve an upper bound for the DoS duration to ensure closed-loop stability. The resilience of the closed-loop system, when subjected to DoS attacks with the learned controller and an internal model, has been thoroughly examined. The effectiveness of the proposed methodology is demonstrated on an inverted pendulum on a cart.

eess.SY↗

arXiv2024

Actively Learning Reinforcement Learning: A Stochastic Optimal Control Approach

In this paper we propose a framework towards achieving two intertwined objectives: (i) equipping reinforcement learning with active exploration and deliberate information gathering, such that it regulates state and parameter uncertainties resulting from modeling mismatches and noisy sensory; and (ii) overcoming the computational intractability of stochastic optimal control. We approach both objectives by using reinforcement learning to compute the stochastic optimal control law. On one hand, we avoid the curse of dimensionality prohibiting the direct solution of the stochastic dynamic programming equation. On the other hand, the resulting stochastic optimal control reinforcement learning agent admits caution and probing, that is, optimal online exploration and exploitation. Unlike fixed exploration and exploitation balance, caution and probing are employed automatically by the controller in real-time, even after the learning process is terminated. We conclude the paper with a numerical simulation, illustrating how a Linear Quadratic Regulator with the certainty equivalence assumption may lead to poor performance and filter divergence, while our proposed approach is stabilizing, of an acceptable performance, and computationally convenient.

cs.LG↗

arXiv2024

Convergence Rates of Online Critic Value Function Approximation in Native Spaces

In this paper, the evolution equation that defines the online critic for the approximation of the optimal value function is cast in a general class of reproducing kernel Hilbert spaces (RKHSs). Exploiting some core tools of RKHS theory, this formulation allows deriving explicit bounds on the performance of the critic in terms of the kernel and definition of the RKHS, the number of basis functions, and the location of centers used to define scattered bases. The performance of the critic is precisely measured in terms of the power function of the scattered basis used in approximations, and it can be used either in an a priori evaluation of potential bases or in an a posteriori assessments of value function error for basis enrichment or pruning. The most concise bounds in the paper describe explicitly how the critic performance depends on the placement of centers, as measured by their fill distance in a subset that contains the trajectory of the critic.

math.OC↗

arXiv2024

A comprehensive survey of research towards AI-enabled unmanned aerial systems in pre-, active-, and post-wildfire management

Wildfires have emerged as one of the most destructive natural disasters worldwide, causing catastrophic losses in both human lives and forest wildlife. Recently, the use of Artificial Intelligence (AI) in wildfires, propelled by the integration of Unmanned Aerial Vehicles (UAVs) and deep learning models, has created an unprecedented momentum to implement and develop more effective wildfire management. Although some of the existing survey papers have explored various learning-based approaches, a comprehensive review emphasizing the application of AI-enabled UAV systems and their subsequent impact on multi-stage wildfire management is notably lacking. This survey aims to bridge these gaps by offering a systematic review of the recent state-of-the-art technologies, highlighting the advancements of UAV systems and AI models from pre-fire, through the active-fire stage, to post-fire management. To this aim, we provide an extensive analysis of the existing remote sensing systems with a particular focus on the UAV advancements, device specifications, and sensor technologies relevant to wildfire management. We also examine the pre-fire and post-fire management approaches, including fuel monitoring, prevention strategies, as well as evacuation planning, damage assessment, and operation strategies. Additionally, we review and summarize a wide range of computer vision techniques in active-fire management, with an emphasis on Machine Learning (ML), Reinforcement Learning (RL), and Deep Learning (DL) algorithms for wildfire classification, segmentation, detection, and monitoring tasks. Ultimately, we underscore the substantial advancement in wildfire modeling through the integration of cutting-edge AI techniques and UAV-based data, providing novel insights and enhanced predictive capabilities to understand dynamic wildfire behavior.

cs.LG↗

arXiv2023

Decentralized and Privacy-Preserving Learning of Approximate Stackelberg Solutions in Energy Trading Games with Demand Response Aggregators

In this work, a novel Stackelberg game theoretic framework is proposed for trading energy bidirectionally between the demand-response (DR) aggregator and the prosumers. This formulation allows for flexible energy arbitrage and additional monetary rewards while ensuring that the prosumers' desired daily energy demand is met. Then, a scalable (linear with the number of prosumers), decentralized, privacy-preserving algorithm is proposed to find approximate equilibria with online sampling and learning of the prosumers' cumulative best response, which finds applications beyond this energy game. Moreover, cost bounds are provided on the quality of the approximate equilibrium solution. Finally, real data from the California day-ahead market and the UC Davis campus building energy demands are utilized to demonstrate the efficacy of the proposed framework and algorithm.

cs.LG↗

arXiv2023

UWB Ranging and IMU Data Fusion: Overview and Nonlinear Stochastic Filter for Inertial Navigation

This paper proposes a nonlinear stochastic complementary filter design for inertial navigation that takes advantage of a fusion of Ultra-wideband (UWB) and Inertial Measurement Unit (IMU) technology ensuring semi-global uniform ultimate boundedness (SGUUB) of the closed loop error signals in mean square. The proposed filter estimates the vehicle's orientation, position, linear velocity, and noise covariance. The filter is designed to mimic the nonlinear navigation motion kinematics and is posed on a matrix Lie Group, the extended form of the Special Euclidean Group $\mathbb{SE}_{2}\left(3\right)$. The Lie Group based structure of the proposed filter provides unique and global representation avoiding singularity (a common shortcoming of Euler angles) as well as non-uniqueness (a common limitation of unit-quaternion). Unlike Kalman-type filters, the proposed filter successfully addresses IMU measurement noise considering unknown upper-bounded covariance. Although the navigation estimator is proposed in a continuous form, the discrete version is also presented. Moreover, the unit-quaternion implementation has been provided in the Appendix. Experimental validation performed using a publicly available real-world six-degrees-of-freedom (6 DoF) flight dataset obtained from an unmanned Micro Aerial Vehicle (MAV) illustrating the robustness of the proposed navigation technique. Keywords: Sensor-fusion, Inertial navigation, Ultra-wideband ranging, Inertial measurement unit, Stochastic differential equation, Stability, Localization, Observer design.

eess.SY↗

arXiv2023

An Observer-Based Reinforcement Learning Solution for Model-Following Problems

In this paper, a multi-objective model-following control problem is solved using an observer-based adaptive learning scheme. The overall goal is to regulate the model-following error dynamics along with optimizing the dynamic variables of a process in a model-free fashion. This solution employs an integral reinforcement learning approach to adapt three strategies. The first strategy observes the states of desired process dynamics, while the second one stabilizes and optimizes the closed-loop system. The third strategy allows the process to follow a desired reference-trajectory. The adaptive learning scheme is implemented using an approximate projection estimation approach under mild conditions about the learning parameters.

eess.SY↗

arXiv2023

Nonlinear Deterministic Observer for Inertial Navigation using Ultra-wideband and IMU Sensor Fusion

Navigation in Global Positioning Systems (GPS)-denied environments requires robust estimators reliant on fusion of inertial sensors able to estimate rigid-body's orientation, position, and linear velocity. Ultra-wideband (UWB) and Inertial Measurement Unit (IMU) represent low-cost measurement technology that can be utilized for successful Inertial Navigation. This paper presents a nonlinear deterministic navigation observer in a continuous form that directly employs UWB and IMU measurements. The estimator is developed on the extended Special Euclidean Group $\mathbb{SE}_{2}\left(3\right)$ and ensures exponential convergence of the closed loop error signals starting from almost any initial condition. The discrete version of the proposed observer is tested using a publicly available real-world dataset of a drone flight. Keywords: Ultra-wideband, Inertial measurement unit, Sensor Fusion, Positioning system, GPS-denied navigation.

eess.SY↗

arXiv2023

An Online Model-Following Projection Mechanism Using Reinforcement Learning

In this paper, we propose a model-free adaptive learning solution for a model-following control problem. This approach employs policy iteration, to find an optimal adaptive control solution. It utilizes a moving finite-horizon of model-following error measurements. In addition, the control strategy is designed by using a projection mechanism that employs Lagrange dynamics. It allows for real-time tuning of derived actor-critic structures to find the optimal model-following strategy and sustain optimized adaptation performance. Finally, the efficacy of the proposed framework is emphasized through a comparison with sliding mode and high-order model-free adaptive control approaches. Keywords: Model Reference Adaptive Systems, Reinforcement Learning, adaptive critics, control system, stochastic, nonlinear system

eess.SY↗