arXiv subjects

Abed AlRahman Al Makdah

Publications and source records attributed to Abed AlRahman Al Makdah.

13 recordsLinked to original sources

arXiv2026

DER Allocation without Load Prediction via Reinforcement Learning

The growing variability of renewable generation increases the need for fast and flexible grid-balancing mechanisms. Existing frameworks for distributed energy resource aggregations (DERAs) rely on short-term forecasts of net demand, making their performance highly sensitive to prediction errors. In this paper we present a forecast-free reinforcement learning (RL) framework for DERA allocation that learns optimal policies directly from operational data. We model the DERA dynamics as a deterministic linear system and the exogenous net load as a feature-based linear Markov process, capturing short-range temporal dependencies without explicit forecasting. We derive a closed-form expression for the optimal policy, which is learned through a least-squares value iteration (LSVI) algorithm using data collected across episodes. The proposed framework preserves the interpretability and constraint satisfaction of DER model while adapting to stochastic demand variations through data-driven updates. Numerical experiments on real California Independent System Operator (CAISO) net-demand data demonstrate that the learned controller achieves high tracking accuracy and stable regulation across heterogeneous DER aggregators without requiring any demand prediction.

eess.SY↗

arXiv2025

Linear Dynamics meets Linear MDPs: Closed-Form Optimal Policies via Reinforcement Learning

Many applications -- including power systems, robotics, and economics -- involve a dynamical system interacting with a stochastic and hard-to-model environment. We adopt a reinforcement learning approach to control such systems. Specifically, we consider a deterministic, discrete-time, linear, time-invariant dynamical system coupled with a feature-based linear Markov process with an unknown transition kernel. The objective is to learn a control policy that optimizes a quadratic cost over the system state, the Markov process, and the control input. Leveraging both components of the system, we derive an explicit parametric form for the optimal state-action value function and the corresponding optimal policy. Our model is distinct in combining aspects of both classical Linear Quadratic Regulator (LQR) and linear Markov decision process (MDP) frameworks. This combination retains the implementation simplicity of LQR, while allowing for sophisticated stochastic modeling afforded by linear MDPs, without estimating the transition probabilities, thereby enabling direct policy improvement. We use tools from control theory to provide theoretical guarantees on the stability of the system under the learned policy and provide a sample complexity analysis for its convergence to the optimal policy. We illustrate our results via a numerical example that demonstrates the effectiveness of our approach in learning the optimal control policy under partially known stochastic dynamics.

math.OC↗

arXiv2025

Online Optimization with Unknown Time-varying Parameters

In this paper, we study optimization problems where the cost function contains time-varying parameters that are unmeasurable and evolve according to linear, yet unknown, dynamics. We propose a solution that leverages control theoretic tools to identify the dynamics of the parameters, predict their evolution, and ultimately compute a solution to the optimization problem. The identification of the dynamics of the time-varying parameters is done online using measurements of the gradient of the cost function. This system identification problem is not standard, since the output matrix is known and the dynamics of the parameters must be estimated in the original coordinates without similarity transformations. Interestingly, our analysis shows that, under mild conditions that we characterize, the identification of the parameters dynamics and, consequently, the computation of a time-varying solution to the optimization problem, requires only a finite number of measurements of the gradient of the cost function. We illustrate the effectiveness of our algorithm on a series of numerical examples.

math.OC↗

arXiv2023

On the Sample Complexity of the Linear Quadratic Gaussian Regulator

In this paper we provide direct data-driven expressions for the Linear Quadratic Regulator (LQR), the Kalman filter, and the Linear Quadratic Gaussian (LQG) controller using a finite dataset of noisy input, state, and output trajectories. We show that our data-driven expressions are consistent, since they converge as the number of experimental trajectories increases, we characterize their convergence rate, and quantify their error as a function of the system and data properties. These results complement the body of literature on data-driven control and finite-sample analysis, and provide new ways to solve canonical control and estimation problems that do not assume, nor require the estimation of, a model of the system and noise and do not rely on solving implicit equations.

math.OC↗

arXiv2023

Imitation and Transfer Learning for LQG Control

In this paper we study an imitation and transfer learning setting for Linear Quadratic Gaussian (LQG) control, where (i) the system dynamics, noise statistics and cost function are unknown and expert data is provided (that is, sequences of optimal inputs and outputs) to learn the LQG controller, and (ii) multiple control tasks are performed for the same system but with different LQG costs. We show that the LQG controller can be learned from a set of expert trajectories of length $n(l+2)-1$, with $n$ and $l$ the dimension of the system state and output, respectively. Further, the controller can be decomposed as the product of an estimation matrix, which depends only on the system dynamics, and a control matrix, which depends on the LQG cost. This data-based separation principle allows us to transfer the estimation matrix across different LQG tasks, and to reduce the length of the expert trajectories needed to learn the LQG controller to~$2n+m-1$ with $m$ the dimension of the inputs (for single-input systems with $l=2$, this yields approximately a $50\%$ reduction of the required expert data).

eess.SY↗

arXiv2023

Model-based and Data-based Dynamic Output Feedback for Externally Positive Systems

In this work, we derive dynamic output-feedback controllers that render the closed-loop system externally positive. We begin by expressing the class of discrete-time, linear, time-invariant systems and the class of dynamic controllers in the space of input-output behaviors, where a dynamic controller can be expressed as a static behavioral feedback gain. We leverage the static form of the controller to derive output-feedback controllers that achieve monotonic output tracking of a constant non-negative reference output. Further, we provide a direct data-driven approach to derive monotonic tracking output-feedback controllers for single-input-single-output (SISO) systems. Our approaches, model-based and data-based, allow us to obtain output-feedback controllers that render the closed-loop system externally positive. Finally, we validate our results numerically in a drone landing control problem.

eess.SY↗

arXiv2022

Behavioral Feedback for Optimal LQG Control

In this work, we revisit the Linear Quadratic Gaussian (LQG) optimal control problem from a behavioral perspective. Motivated by the suitability of behavioral models for data-driven control, we begin with a reformulation of the LQG problem in the space of input-output behaviors and obtain a complete characterization of the optimal solutions. In particular, we show that the optimal LQG controller can be expressed as a static behavioral-feedback gain, thereby eliminating the need for dynamic state estimation characteristic of state space methods. The static form of the optimal LQG gain also makes it amenable to its computation by gradient descent, which we investigate via numerical experiments. Furthermore, we highlight the advantage of this approach in the data-driven control setting of learning the optimal LQG controller from expert demonstrations.

eess.SY↗

arXiv2022

Learning Lipschitz Feedback Policies from Expert Demonstrations: Closed-Loop Guarantees, Generalization and Robustness

In this work, we propose a framework to learn feedback control policies with guarantees on closed-loop generalization and adversarial robustness. These policies are learned directly from expert demonstrations, contained in a dataset of state-control input pairs, without any prior knowledge of the task and system model. We use a Lipschitz-constrained loss minimization scheme to learn feedback policies with certified closed-loop robustness, wherein the Lipschitz constraint serves as a mechanism to tune the generalization performance and robustness to adversarial disturbances. Our analysis exploits the Lipschitz property to obtain closed-loop guarantees on generalization and robustness of the learned policies. In particular, we derive a finite sample bound on the policy learning error and establish robust closed-loop stability under the learned control policy. We also derive bounds on the closed-loop regret with respect to the expert policy and the deterioration of closed-loop performance under bounded (adversarial) disturbances to the state measurements. Numerical results validate our analysis and demonstrate the effectiveness of our robust feedback policy learning framework. Finally, our results suggest the existence of a potential tradeoff between nominal closed-loop performance and adversarial robustness, and that improvements in nominal closed-loop performance can only be made at the expense of robustness to adversarial perturbations.

cs.LG↗

arXiv2021

Robust Adversarial Classification via Abstaining

In this work, we consider a binary classification problem and cast it into a binary hypothesis testing framework, where the observations can be perturbed by an adversary. To improve the adversarial robustness of a classifier, we include an abstain option, where the classifier abstains from making a decision when it has low confidence about the prediction. We propose metrics to quantify the nominal performance of a classifier with an abstain option and its robustness against adversarial perturbations. We show that there exist a tradeoff between the two metrics regardless of what method is used to choose the abstain region. Our results imply that the robustness of a classifier with an abstain option can only be improved at the expense of its nominal performance. Further, we provide necessary conditions to design the abstain region for a 1- dimensional binary classification problem. We validate our theoretical results on the MNIST dataset, where we numerically show that the tradeoff between performance and robustness also exist for the general multi-class classification problems.

cs.LG↗

arXiv2020

Lipschitz Bounds and Provably Robust Training by Laplacian Smoothing

In this work we propose a graph-based learning framework to train models with provable robustness to adversarial perturbations. In contrast to regularization-based approaches, we formulate the adversarially robust learning problem as one of loss minimization with a Lipschitz constraint, and show that the saddle point of the associated Lagrangian is characterized by a Poisson equation with weighted Laplace operator. Further, the weighting for the Laplace operator is given by the Lagrange multiplier for the Lipschitz constraint, which modulates the sensitivity of the minimizer to perturbations. We then design a provably robust training scheme using graph-based discretization of the input space and a primal-dual algorithm to converge to the Lagrangian's saddle point. Our analysis establishes a novel connection between elliptic operators with constraint-enforced weighting and adversarial learning. We also study the complementary problem of improving the robustness of minimizers with a margin on their loss, formulated as a loss-constrained minimization problem of the Lipschitz constant. We propose a technique to obtain robustified minimizers, and evaluate fundamental Lipschitz lower bounds by approaching Lipschitz constant minimization via a sequence of gradient $p$-norm minimization problems. Ultimately, our results show that, for a desired nominal performance, there exists a fundamental lower bound on the sensitivity to adversarial perturbations that depends only on the loss function and the data distribution, and that improvements in robustness beyond this bound can only be made at the expense of nominal performance. Our training schemes provably achieve these bounds both under constraints on performance and~robustness.

cs.LG↗

arXiv2020

Accuracy Prevents Robustness in Perception-based Control

In this paper we prove the existence of a fundamental trade-off between accuracy and robustness in perception-based control, where control decisions rely solely on data-driven, and often incompletely trained, perception maps. In particular, we consider a control problem where the state of the system is estimated from measurements extracted from a high-dimensional sensor, such as a camera. We assume that a map between the camera's readings and the state of the system has been learned from a set of training data of finite size, from which the noise statistics are also estimated. We show that algorithms that maximize the estimation accuracy (as measured by the mean squared error) using the learned perception map tend to perform poorly in practice, where the sensor's statistics often differ from the learned ones. Conversely, increasing the variability and size of the training data leads to robust performance, however limiting the estimation accuracy, and thus the control performance, in nominal conditions. Ultimately, our work proves the existence and the implications of a fundamental trade-off between accuracy and robustness in perception-based control, which, more generally, affects a large class of machine learning and data-driven algorithms.

math.OC↗

arXiv2019

On the Robustness of Data-Driven Controllers for Linear Systems

This paper proposes a new framework and several results to quantify the performance of data-driven state-feedback controllers for linear systems against targeted perturbations of the training data. We focus on the case where subsets of the training data are randomly corrupted by an adversary, and derive lower and upper bounds for the stability of the closed-loop system with compromised controller as a function of the perturbation statistics, size of the training data, sensitivity of the data-driven algorithm to perturbation of the training data, and properties of the nominal closed-loop system. Our stability and convergence bounds are probabilistic in nature, and rely on a first-order approximation of the data-driven procedure that designs the state-feedback controller, which can be computed directly using the training data. We illustrate our findings via multiple numerical studies.

eess.SY↗

arXiv2019

A Fundamental Performance Limitation for Adversarial Classification

Despite the widespread use of machine learning algorithms to solve problems of technological, economic, and social relevance, provable guarantees on the performance of these data-driven algorithms are critically lacking, especially when the data originates from unreliable sources and is transmitted over unprotected and easily accessible channels. In this paper we take an important step to bridge this gap and formally show that, in a quest to optimize their accuracy, binary classification algorithms -- including those based on machine-learning techniques -- inevitably become more sensitive to adversarial manipulation of the data. Further, for a given class of algorithms with the same complexity (i.e., number of classification boundaries), the fundamental tradeoff curve between accuracy and sensitivity depends solely on the statistics of the data, and cannot be improved by tuning the algorithm.

cs.LG↗