Searcharxiv⌕ Search

arXiv subjects

Pol Mestres

Publications and source records attributed to Pol Mestres.

At least 19 recordsLinked to original sources

Towards Probabilistically Safe Reinforcement Learning: Theory and Deployment on Humanoid and Aerial Robots

Reinforcement learning (RL) enables robots to acquire complex skills, but standard RL formulations do not guarantee that learned policies satisfy safety constraints, which limits their use in safety-critical settings. Existing safe RL methods with provable guarantees typically support only restricted classes of constraints and require unbiased estimators of the objective and constraint functions, which prevents their direct integration with state-of-the-art algorithms such as proximal policy optimization (PPO). In this paper, we extend the recently introduced robust safe gradient flow-based reinforcement learning (RSGF-RL) framework to overcome these limitations. Our formulation accommodates a general class of trajectory-level safety constraints, including chance constraints such as collision avoidance with a prescribed probability, and our analysis allows for biased value and gradient estimators. We show that, with high probability, the constraint violation throughout training is bounded by the estimator bias plus a term that vanishes as the number of rollouts grows, and decays to this level if the initial policy is unsafe. We also establish the convergence properties of the iterates to approximate KKT points of the original safe RL problem. Finally, we show that RSGF-RL implemented with PPO-style estimates enables zero-shot sim-to-real deployment of whole-body control policies on a Unitree G1 humanoid and collision-free waypoint navigation on a Crazyflie drone.

cs.RO↗

Regions of Attraction of CBF-Based Safety Filters for Stable Planar Linear Systems

In this paper we study the dynamical properties induced by safety filters based on control barrier functions (CBFs). These are obtained by minimally modifying a nominal controller with the goal of rendering it safe. We consider the simple setting of a planar linear plant with a stable linear nominal controller and an affine constraint. We show that even in this simple setting, the filtered system can oscillate or diverge while remaining safe. We fully characterize the parameters for which the origin is globally (exponentially or asymptotically) stable. For all other parameters, the filter creates a saddle on the constraint boundary, and we characterize the region of attraction of the origin. For underdamped nominal dynamics, its boundary is an unstable periodic orbit, a homoclinic loop, or the saddle's stable manifold, depending on whether the CBF gain is below, at, or above a homoclinic transition value. A vehicle sideslip example and two three-dimensional examples illustrate the results and what changes beyond planar dynamics.

eess.SY↗

Feasibility and Explicit Safety Filters for Control Barrier Functions in Linear Systems

Safety filters based on control barrier functions (CBFs) and high-order control barrier functions (HOCBFs) are commonly implemented through quadratic programs (QPs), which may become infeasible under conflicting safety and actuator constraints. For linear systems with affine safety functions, we exploit repeated input-space directions to obtain an explicit polyhedral feasibility domain, linear-program certificates for regional feasibility and final-gain selection, and conditions identifying gain-independent obstructions and the minimum common final gain. A compatible QP metric yields an exact saturation-based filter and, together with the regional certificate, persistent feasibility and safety. For a single exterior ellipsoidal CBF with unrestricted inputs, we also derive an exact, gain-independent linear matrix inequality (LMI) test. Double-integrator and aircraft examples illustrate the results.

eess.SY↗

Stability Analysis in Multi-Constraint Safety Filters for Linear Systems

Multi-constraint safety filters based on control barrier functions for linear systems with affine state constraints yield continuous piecewise-affine closed-loop dynamics and may introduce boundary equilibria and unstable active-set modes. Although these filters guarantee forward invariance, they can change nominal stability, and it remains unclear when unstable modes cause divergence versus bounded, convergent behavior. This paper develops a geometric framework to analyze these cases: leveraging explicit active-set realizations, we show that equilibria associated with nonempty active sets lie on the corresponding constraint faces and that any unstable directions are tangent to those faces due to exponential enforcement of the active constraints. We characterize mode stability via a minimum-phase test, provide a sufficient recession-cone certificate for divergence under a fixed active set, and derive tractable linear-matrix-inequality conditions for global exponential stability or boundedness using Lyapunov and LaSalle arguments.

eess.SY↗

Functional Observers for Output-Feedback Safety with High-Order Control Barrier Functions

This paper studies output-feedback safety filtering for linear systems using high-order control barrier functions (HOCBFs) when the full state is unavailable. Since the HOCBF conditions depend only on specific state functionals, full-state reconstruction is unnecessary; instead, their implementation requires only functional observability or detectability of these quantities. We show that the HOCBF gains determine the required functionals and hence their observability properties, coupling safety and estimation design. We exploit this structure using scalar functional observers and develop a polynomial co-design framework that characterizes HOCBF observability and detectability, observer existence and order, and convergence-rate limitations. A certified one-sided estimation-error bound then enables safe output-feedback implementation.

eess.SY↗

A Unified Perspective on Conformal Prediction and Wasserstein Distributionally Robust Optimization for Uncertainty Quantification

Uncertainty quantification from finite data is central to machine learning, optimization, and automation systems, where decisions must remain reliable under limited samples and test-time distribution shift. Conformal prediction (CP) and distributionally robust optimization (DRO) offer two complementary approaches: CP constructs data-dependent prediction sets with distribution-free finite-sample validity under exchangeability, while DRO optimizes worst-case performance over an ambiguity set around an empirical distribution. We develop a unified probabilistic perspective on CP and DRO by viewing both as ways to turn finite calibration data into a data-dependent quantile estimator that a test score falls below with high probability. From this perspective, CP and DRO correct the empirical quantile along two coordinates of the same family of estimators: CP inflates the quantile level, whereas DRO shifts the quantile value through an ambiguity radius. Both methods provide the same calibration-conditional guarantee for the true distribution, requiring the target coverage to hold with high probability over the calibration sample. Their constructions differ, however: CP uses a closed-form, distribution-free level correction, while DRO uses a value-space correction whose certified radius depends on properties of the unknown distribution and additionally guarantees coverage uniformly over the ambiguity set. This distinction emerges in the tails of the score distribution. Because CP relies on sparse upper-tail order statistics of the calibration samples, its level inflation barely moves the estimator when those samples are dense near the target quantile but overshoots when they are sparse, whereas a well-chosen DRO radius corrects in value space and may avoid this overshoot.

math.OC↗

Safe Stabilizing Linear Feedback: Necessary and Sufficient Conditions, Optimality, and Margins

Control barrier functions (CBFs) have become an important controller design tool for autonomous systems subject to safety constraints. Despite their popularity, recent works have shown that CBF-based controllers can destabilize the internal dynamics of the system. In this paper, we consider linear systems with affine safety constraints and design linear feedback controllers that satisfy high-order CBF (HOCBF) constraints while rendering the origin globally exponentially stable. We first characterize the exact class of all linear gain matrices that globally satisfy the HOCBF constraints, including necessary and sufficient conditions for when this class is nonempty. Then, by leveraging the recently introduced notion of CBF output dynamics and CBF internal dynamics, we provide the necessary and sufficient conditions for the existence of stabilizing gain matrices within that class. Finally, we show that Linear Quadratic Regulator (LQR) and robust control problems can be solved while being constrained within this class of safe and stabilizing gain matrices, through standard linear control techniques such as algebraic Riccati equations (AREs) and Linear Matrix Inequalities (LMIs). We illustrate our results in a simulation example.

math.OC↗

Dynamical Properties of Safety Filters for Linear Systems and Affine Control Barrier Functions

This letter studies the dynamical properties of safety filters designed based on Control Barrier Functions (CBF). This mechanism, which is popular in safety-critical applications, takes a nominal controller and minimally modifies it to render it safe. Although CBF-based safety filters make the closed-loop system safe, characterizing their additional dynamical properties, such as stability, boundedness, or existence of spurious equilibria, remains a challenging problem. Here, we address this problem for the case of linear systems and an affine CBF constraint. We provide conditions under which the closed-loop system presents undesired equilibria, unbounded trajectories, or the origin is globally exponentially stable.

math.OC↗

Independence of Closed-Loop Equilibria and Stability from the Choice of Control Barrier Function for a Given Safe Set

Control barrier functions (CBFs) play a critical role in the design of safe optimization-based controllers for control-affine systems. Given a CBF associated with a given, predefined ``safe'' set, the typical approach consists in embedding CBF-based constraints into the optimization problem defining the control law to enforce forward invariance of the safe set. While this approach effectively guarantees safety for a given CBF, the CBF-based control law can introduce undesirable equilibrium points (i.e., points that are not equilibria of the original system). Given that there exist many different CBFs associated with a given fixed safe set, open questions remain on how the choice of CBF influences the number and locations of undesirable equilibria and, in general, the dynamics of the closed-loop system. This paper investigates how the choice of CBF impacts the dynamics of the closed-loop system and shows that: (i) The choice of CBF does not affect the number, location, and (local) stability properties of the equilibria in the interior of the safe set; (ii) undesirable equilibria only appear on the boundary of the safe set; and, (iii) the number and location of undesirable equilibria for the closed-loop system do not depend of the choice of the CBF. Additionally, for the well-established \textit{safety filters}, we show that the stability properties of equilibria of the closed-loop system are independent of the choice of the CBF and of the associated extended class-K function, provided that the CBFs are chosen from the same equivalence class.

eess.SY↗

Explicit Control Barrier Function-based Safety Filters and their Resource-Aware Computation

This paper studies the efficient implementation of safety filters that are designed using control barrier functions (CBFs), which minimally modify a nominal controller to render it safe with respect to a prescribed set of states. Although CBF-based safety filters are often implemented by solving a quadratic program (QP) in real time, the use of off-the-shelf solvers for such optimization problems poses a challenge in applications where control actions need to be computed efficiently at very high frequencies. In this paper, we introduce a closed-form expression for controllers obtained through CBF-based safety filters. This expression is obtained by partitioning the state-space into different regions, with a different closed-form solution in each region. We leverage this formula to introduce a resource-aware implementation of CBF-based safety filters that detects changes in the partition region and uses the closed-form expression between changes. We showcase the applicability of our approach in examples ranging from aerospace control to safe reinforcement learning.

eess.SY↗

Universal Formulas for Safe Control and Their Neural Network Approximations

We study the problem of designing a controller that satisfies an arbitrary number of affine inequalities at every point in the state space. This is motivated by the fact that a variety of key control objectives, such as stability, safety, and input saturation, are guaranteed by closed-loop systems whose controllers satisfy such inequalities. Many works in the literature design such controllers as the solution to a state-dependent quadratic program (QP) whose constraints are precisely the inequalities. When the input dimension and number of constraints are high, computing a solution of this QP in real time can become computationally burdensome. Additionally, the solution of such optimization problems is not smooth in general, which can degrade the performance of the system. This paper provides a novel method to design a smooth controller that satisfies an arbitrary number of affine constraints. The controller is given at every state as the minimizer of a strictly convex function. To avoid computing the minimizer of such function in real time, we introduce a method based on neural networks (NN) to approximate the controller. Remarkably, this NN can be used to solve the controller design problem for any task with less than a fixed input dimension and number of affine constraints, and is completely independent of the state dimension. This is why we refer to such NN approximation as a NN-based universal formula for control. Additionally, we show that the NN-based controller only needs to be trained with datapoints from a bounded set in the state space, which significantly simplifies the training process.

math.OC↗

Safety Filtering with an Infinite Number of Constraints

Control barrier functions (CBFs) provide a rigorous framework for designing controllers enforcing safety constraints. While CBF theory is well-developed for a finite number of safety constraints, certain applications, e.g., backup CBFs, require an infinite number of constraints. Despite the practical success of CBFs, several fundamental questions remain unanswered when safe sets are defined with an infinite numbers of constraints, including: necessary and sufficient conditions for forward set invariance, the actual definition of CBFs associated with these sets, the regularity properties of the resulting controllers, and the ability to reduce a collection of infinite constraints to a finite number. This paper addresses these questions by extending CBF theory to the infinite constraint setting. We identify regularity conditions under which Nagumo's Theorem reduces to barrier-like inequalities and when the associated CBF controllers are at least continuous. We further connect these results to optimal-decay CBFs, bridging theoretical conditions for invariance and practical instantiations of the resulting controller. Finally, we illustrate how the developed theory addresses limitations of backup CBFs.

eess.SY↗

Probabilistic Control Barrier Functions for Systems with State Estimation Uncertainty using Sub-Gaussian Concentration

Safety-critical control systems, such as spacecraft performing proximity operations, must provide formal safety guarantees despite stochastic uncertainties from state estimation and unmodeled dynamics. Although Control Barrier Functions (CBFs) have been extended to stochastic systems, existing approaches typically face a trade-off between the tightness of probabilistic guarantees and computational tractability. This paper presents a particle-based probabilistic CBF framework that overcomes this limitation by exploiting the sub-Gaussian structure of the barrier function increment under Gaussian uncertainties. We establish that Gaussian uncertainties propagating through Lipschitz-continuous control-affine dynamics preserve sub-Gaussianity of the barrier function increment, with explicit tail bounds. Leveraging this structure, we derive finite-sample bounds on the approximation error between particle-based Conditional Value at Risk (CVaR) estimates and ground-truth probabilistic constraints; applying this yields a tractable optimization problem formulation with finite-sample safety certificates. We show through numerical experiments how the proposed approach provides tight yet provably valid probabilistic safety guarantees.

eess.SY↗

Stability Margins of CBF-QP Safety Filters: Analysis and Synthesis

Control barrier function (CBF)-QP safety filters enforce safety by minimally modifying a nominal controller. While prior work has mainly addressed robustness of safety under uncertainty, robustness of the resulting closed-loop \emph{stability} is much less understood. This issue is important because once the safety filter becomes active, it modifies the nominal dynamics and can reduce stability margins or even destabilize the system, despite preserving safety. For linear systems with a single affine safety constraint, we show that the active-mode dynamics admit an exact scalar loop representation, leading to a classical robust-control interpretation in terms of gain, phase, and delay margins. This viewpoint yields exact stability-margin characterizations and tractable linear matrix inequality (LMI)-based certificates and synthesis conditions for controllers with certified robustness guarantees. Numerical examples illustrate the proposed analysis and the enlargement of certified stability margins for safety-filtered systems.

eess.SY↗

Safe Policy Optimization via Control Barrier Function-based Safety Filters

Control barrier function (CBF)-based safety filters provide a systematic way to enforce state constraints, but they can significantly alter the closed-loop dynamics induced by a nominal, stabilizing controller. In particular, the resulting safety-filtered system may exhibit undesirable behaviors including limit cycles, unbounded trajectories, and undesired equilibria. This paper develops a policy optimization framework to maximally enhance the stability properties of safety-filtered controllers. Focusing on linear systems with linear nominal controllers, we jointly parameterize the nominal feedback gain and safety-filter components, and optimize them using trajectory-based objectives computed from closed-loop rollouts. To ensure that the nominal controller remains stabilizing throughout training, we encode Lyapunov-based stability conditions as smooth scalar constraints and enforce them using robust safe gradient flow. This guarantees feasibility of the stability constraints along the optimization iterates and therefore avoids instability during training. Numerical experiments on obstacle-avoidance problems show that the proposed approach can remove asymptotically stable undesired equilibria and improve convergence behavior while maintaining forward invariance of the safe set.

eess.SY↗

Safe Feedback Optimization through Control Barrier Functions

Feedback optimization refers to a class of methods that steer a control system to a steady state that solves an optimization problem. Despite tremendous progress on the topic, an important problem remains open: enforcing state constraints at all times. The difficulty in addressing it lies on mediating between the safety enforcement and the closed-loop stability, and ensuring the equivalence between closed-loop equilibria and the optimization problem's critical points. In this work, we present a feedback-optimization method that enforces state constraints at all times employing high-order control-barrier functions. We provide several results on the proposed controller dynamics, including well-posedness, safety guarantees, equivalence between equilibria and critical points, and local and global (in certain convex cases) asymptotic stability of optima. Various simulations illustrate our results.

math.OC↗

Probabilistic Control Barrier Functions: Safety in Probability for Discrete-Time Stochastic Systems

Control systems operating in the real world face countless sources of unpredictable uncertainties. These random disturbances can render deterministic guarantees inapplicable and cause catastrophic safety failures. To overcome this, this paper proposes a method for designing safe controllers for discrete-time stochastic systems that retain probabilistic guarantees of safety. To do this we modify the traditional notion of a control barrier function (CBF) to explicitly account for these stochastic uncertainties and call these new modified functions probabilistic CBFs. We show that probabilistic CBFs can be used to design controllers that guarantee safety over a finite number of time steps with a prescribed probability. Next, by leveraging various uncertainty quantification methods, such as concentration inequalities and the scenario approach, we provide a variety of sufficient conditions that result in computationally tractable controllers with tunable probabilistic guarantees across a plethora of practical scenarios. Finally, we showcase the applicability of our results in simulation and hardware for the control of a quadruped robot.

eess.SY↗

Anytime Safe Reinforcement Learning

This paper considers the problem of solving constrained reinforcement learning problems with anytime guarantees, meaning that the algorithmic solution returns a safe policy regardless of when it is terminated. Drawing inspiration from anytime constrained optimization, we introduce Reinforcement Learning-based Safe Gradient Flow (RL-SGF), an on-policy algorithm which employs estimates of the value functions and their respective gradients associated with the objective and safety constraints for the current policy, and updates the policy parameters by solving a convex quadratically constrained quadratic program. We show that if the estimates are computed with a sufficiently large number of episodes (for which we provide an explicit bound), safe policies are updated to safe policies with a probability higher than a prescribed tolerance. We also show that iterates asymptotically converge to a neighborhood of a KKT point, whose size can be arbitrarily reduced by refining the estimates of the value function and their gradients. We illustrate the performance of RL-SGF in a navigation example.

eess.SY↗