arXiv subjects

Arash Bahari Kordabad

Publications and source records attributed to Arash Bahari Kordabad.

At least 19 recordsLinked to original sources

arXiv2026

Integer-Free Control Synthesis for Signal Temporal Logic via Ranked Implicants

This paper proposes an integer-free method for control synthesis of nonlinear discrete-time systems under finite-horizon Signal Temporal Logic (STL) constraints. Instead of introducing integer variables, we consider the discrete choices of the specification in a ranked order and solve a standard nonlinear program for each selected choice until a satisfying trajectory is found. We first define implicants as sets of timed atomic constraints whose joint satisfaction is sufficient to satisfy the specification. A reference trajectory obtained from a smooth approximation of the STL robustness is used to rank these implicants. Fixing one implicant turns the problem into a standard nonlinear program with no auxiliary weights or bilinear terms. If the solution does not satisfy the specification, the next implicant will be considered. We prove soundness and completeness of our method relative to the nonlinear programming solver. Numerical evaluation on nonlinear multi-agent systems shows that our proposed method identifies satisfying trajectories where the exact continuous reformulation fails, and moreover, it is faster than a mixed-integer formulation of the same specification.

eess.SY↗

arXiv2026

Context-Triggered Robust MPC for Temporal Logic Specifications

We consider the problem of synthesizing robust feedback controllers for discrete-time linear systems that ensure the satisfaction of context-dependent linear temporal logic specifications in the presence of additive bounded disturbances. Building on existing results that reduce context-triggered temporal logic synthesis to the realization of context-dependent reach-avoid-stay (cRAS) objectives, we focus on the corresponding low-level control synthesis problem. We first employ certificate-based conditions for the almost-sure satisfaction of RAS specifications. Based on these conditions, we propose a switching control architecture that combines robust model predictive control (MPC) with a local invariant controller, and show that the resulting MPC value function serves as a reachability certificate while avoidance is enforced through robust constraints and the stay is enforced via the local controller. To obtain computationally tractable formulations for the resulting robust optimizations, we employ convex duality to reformulate the robust constraints into equivalent deterministic optimization problems, yielding convex quadratic and second-order cone programs for relevant geometric settings. The proposed framework is demonstrated on a robot navigation problem with context-triggered logical switches in both static and moving environments. The results show significantly larger feasible sets than Lyapunov-based approaches, while naturally accommodating dynamic environments and online task reconfiguration.

eess.SY↗

arXiv2026

Multi-Agent Temporal Logic Planning via Penalty Functions and Block-Coordinate Optimization

Multi-agent planning under Signal Temporal Logic (STL) is often hindered by collaborative tasks that lead to computational challenges due to the inherent high dimensionality of the problem, preventing scalable synthesis with satisfaction guarantees. To address this, we formulate STL planning as an optimization program under multi-agent STL constraints and introduce a penalty-based unconstrained relaxation that can be efficiently solved via a Block-Coordinate Gradient Descent (BCGD) method, where each block corresponds to a single agent's decision variables, thereby mitigating complexity. By utilizing a quadratic penalty function defined via smooth STL semantics, we show that BCGD iterations converge to a stationary point of the penalized problem under standard regularity assumptions. To enforce feasibility, the BCGD solver is embedded within a two-layer optimization scheme: inner BCGD updates are performed for a fixed penalty parameter, which is then increased in an outer loop to progressively improve multi-agent STL robustness. The proposed framework enables scalable computations and is validated through various complex multi-robot planning scenarios.

eess.SY↗

arXiv2026

Almost Sure Reachability in Continuous-time Stochastic Systems

We provide certificates for almost sure reachability of continuous-time stochastic systems governed by stochastic differential equations (SDEs). We first show that a standard Euler-Maruyama discretization may fail to preserve almost sure reachability property of the system using a double-well Langevin system. This observation motivates us to develop certificates for almost sure reachability directly on the continuous-time system. We introduce a pair of certificates, a drift function and a variant function, and prove necessity and sufficiency for almost sure reachability of an open bounded target set. Using these certificates, for linear SDEs, we give a characterization of almost sure reachability in terms of the spectral structure of the system matrices. For polynomial SDEs, we fix a polynomial template for the drift function and choose the variant function template as an exponential function composed with a polynomial. This allows us to translate the conditions in the certificates into sum-of-squares (SOS) constraints. We then propose an alternating scheme to resolve bilinearities. We illustrate the approach on the double-well Langevin example, showing that continuous-time SOS certificates recover almost sure reachability that is lost under time discretization. Moreover, we verify the SOS approach on a polynomial system.

eess.SY↗

arXiv2025

Quasi-Newton Compatible Actor-Critic for Deterministic Policies

In this paper, we propose a second-order deterministic actor-critic framework in reinforcement learning that extends the classical deterministic policy gradient method to exploit curvature information of the performance function. Building on the concept of compatible function approximation for the critic, we introduce a quadratic critic that simultaneously preserves the true policy gradient and an approximation of the performance Hessian. A least-squares temporal difference learning scheme is then developed to estimate the quadratic critic parameters efficiently. This construction enables a quasi-Newton actor update using information learned by the critic, yielding faster convergence compared to first-order methods. The proposed approach is general and applicable to any differentiable policy class. Numerical examples demonstrate that the method achieves improved convergence and performance over standard deterministic actor-critic baselines.

cs.LG↗

arXiv2025

Second-Order Policy Gradient Methods for the Linear Quadratic Regulator

Policy gradient methods are a powerful family of reinforcement learning algorithms for continuous control that optimize a policy directly. However, standard first-order methods often converge slowly. Second-order methods can accelerate learning by using curvature information, but they are typically expensive to compute. The linear quadratic regulator (LQR) is a practical setting in which key quantities, such as the policy gradient, admit closed-form expressions. In this work, we develop second-order policy gradient algorithms for LQR by deriving explicit formulas for both the approximate and exact Hessians used in Gauss--Newton and Newton methods, respectively. Numerical experiments show a faster convergence rate for the proposed second-order approach over the standard first-order policy gradient baseline.

eess.SY↗

arXiv2025

Sum-of-Squares Certificates for Almost-Sure Reachability of Stochastic Polynomial Systems

In this paper, we present a computational approach to certify almost sure reachability for discrete-time polynomial stochastic systems by turning drift--variant criteria into sum-of-squares (SOS) programs solved with standard semidefinite solvers. Specifically, we provide an SOS method based on two complementary certificates: (i) a drift certificate that enforces a radially unbounded function to be non-increasing in expectation outside a compact set of states; and (ii) a variant certificate that guarantees a one-step decrease with positive probability and ensures the target contains its nonpositive sublevel set. We transform these conditions to SOS constraints. For the variant condition, we enforce a robust decrease over a parameterized disturbance ball with nonzero probability and encode the constraints via an S-procedure with polynomial multipliers. The resulting bilinearities are handled by an alternating scheme that alternates between optimizing multipliers and updating the variant and radius until a positive slack is obtained. Two case studies illustrate the workflow and certifies almost-sure reachability.

math.OC↗

arXiv2025

On Certificates for Almost Sure Reachability in Stochastic Systems

Almost sure reachability refers to the property of a stochastic system whereby, from any initial condition, the system state reaches a given target set with probability one. In this paper, we study the problem of certifying almost sure reachability in discrete-time stochastic systems using drift and variant conditions. While these conditions are both necessary and sufficient in theory, computational approaches often rely on restricting the search to fixed templates, such as polynomial or quadratic functions. We show that this restriction compromises completeness: there exists a polynomial system for which a given target set is almost surely reachable but admits no polynomial certificate, and a linear system for which a neighborhood of the origin is almost surely reachable but admits no quadratic certificate. We then provide a complete characterization of reachability certificates for linear systems with additive noise. Our analysis yields conditions on the system matrices under which valid certificates exist, and shows how the structure and dimension of the system determine the need for non-quadratic templates. Our results generalize the classical random walk behavior to a broader class of stochastic dynamical systems.

eess.SY↗

arXiv2025

Robust Model Predictive Control for Aircraft Intent-Aware Collision Avoidance

This paper presents the use of robust model predictive control for the design of an intent-aware collision avoidance system for multi-agent aircraft engaged in horizontal maneuvering scenarios. We assume that information from other agents is accessible in the form of waypoints or destinations. Consequently, we consider that other agents follow their optimal Dubin's path--a trajectory that connects their current state to their intended state--while accounting for potential uncertainties. We propose using scenario tree model predictive control as a robust approach that demonstrates computational efficiency. We demonstrate that the proposed method can easily integrate intent information and offer a robust scheme that handles different uncertainties. The method is illustrated through simulation results.

eess.SY↗

arXiv2025

Intent-Aware MPC for Aircraft Detect-and-Avoid with Response Delay: A Comparative Study with ACAS Xu

In this paper, we propose an intent-aware Model Predictive Control (MPC) approach for the remain-well-clear (RWC) functionality of a multi-agent aircraft detect-and-avoid (DAA) system and compare its performance with the standardized Airborne Collision Avoidance System Xu (ACAS Xu). The aircraft system is modeled as a linear system for horizontal maneuvering, with advisories on the rate of turn as the control input. Both deterministic and stochastic time delays are considered to account for the lag between control guidance issuance and the response of the aircraft. The capability of the MPC scheme in producing an optimal control profile over the entire horizon is used to mitigate the impact of the delay. We compare the proposed MPC method with ACAS Xu using various evaluation metrics, including loss of DAA well-clear percentage, near mid-air collision percentage, horizontal miss distance, and additional flight distance across different encounter scenarios. It is shown that the MPC scheme achieves better evaluation metrics than ACAS Xu for both deterministic and stochastic scenarios.

eess.SY↗

arXiv2025

Data-Driven Distributionally Robust Control for Interacting Agents under Logical Constraints

In this paper, we propose a distributionally robust control synthesis for an agent with stochastic dynamics that interacts with other agents under uncertainties and constraints expressed by signal temporal logic (STL). We formulate the control synthesis as a chance-constrained program (CCP) with STL specifications that must be satisfied with high probability under all uncertainty tubes induced by the other agents. To tackle the CCP, we propose two methods based on concentration of measure (CoM) theory and conditional value at risk (CVaR) and compare the required assumptions and resulting optimizations. These approaches convert the CCP into an expectation-constrained program (ECP), which is simpler to solve than the original CCP. To estimate the expectation using a finite set of observed data, we adopt a distributionally robust optimization (DRO) approach. The underlying DRO can be approximated as a robust data-driven optimization that provides a probabilistic under-approximation to the original ECP, where the probability depends on the number of samples. Therefore, under feasibility, the original STL constraints are satisfied with two layers of designed confidence: the confidence of the chance constraint and the confidence of the approximated data-driven optimization, which depends on the number of samples. We then provide details on solving the resulting robust data-driven optimization numerically. Finally, we compare the two proposed approaches through case studies.

eess.SY↗

arXiv2024

Optimality Conditions for Model Predictive Control: Rethinking Predictive Model Design

Optimality is a critical aspect of Model Predictive Control (MPC), especially in economic MPC. However, achieving optimality in MPC presents significant challenges, and may even be impossible, due to inherent inaccuracies in the predictive models. Predictive models often fail to accurately capture the true system dynamics, such as in the presence of stochasticity, leading to suboptimal MPC policies. In this paper, we establish the necessary and sufficient conditions on the underlying prediction model for an MPC scheme to achieve closed-loop optimality. Interestingly, these conditions are counterintuitive to the traditional approach of building predictive models that best fit the data. These conditions present a mathematical foundation for constructing models that are directly linked to the performance of the resulting MPC scheme.

math.OC↗

arXiv2024

Distributionally Robust Control for Chance-Constrained Signal Temporal Logic Specifications

We consider distributionally robust optimal control of stochastic linear systems under signal temporal logic (STL) chance constraints when the disturbance distribution is unknown. By assuming that the underlying predicate functions are Lipschitz continuous and the noise realizations are drawn from a distribution having a concentration of measure property, we first formulate the underlying chance-constrained control problem as stochastic programming with constraints on expectations and propose a solution using a distributionally robust approach based on the Wasserstein metric. We show that by choosing a proper Wasserstein radius, the original chance-constrained optimization can be satisfied with a user-defined confidence level. A numerical example illustrates the efficacy of the method.

eess.SY↗

arXiv2023

Equivalence of Optimality Criteria for Markov Decision Process and Model Predictive Control

This paper shows that the optimal policy and value functions of a Markov Decision Process (MDP), either discounted or not, can be captured by a finite-horizon undiscounted Optimal Control Problem (OCP), even if based on an inexact model. This can be achieved by selecting a proper stage cost and terminal cost for the OCP. A very useful particular case of OCP is a Model Predictive Control (MPC) scheme where a deterministic (possibly nonlinear) model is used to reduce the computational complexity. This observation leads us to parameterize an MPC scheme fully, including the cost function. In practice, Reinforcement Learning algorithms can then be used to tune the parameterized MPC scheme. We verify the developed theorems analytically in an LQR case and we investigate some other nonlinear examples in simulations.

eess.SY↗

arXiv2022

Functional Stability of Discounted Markov Decision Processes Using Economic MPC Dissipativity Theory

This paper discusses the functional stability of closed-loop Markov Chains under optimal policies resulting from a discounted optimality criterion, forming Markov Decision Processes (MDPs). We investigate the stability of MDPs in the sense of probability measures (densities) underlying the state distributions and extend the dissipativity theory of Economic Model Predictive Control in order to characterize the MDP stability. This theory requires a so-called storage function satisfying a dissipativity inequality. In the probability measures space and for the discounted setting, we introduce new dissipativity conditions ensuring the MDP stability. We then use finite-horizon optimal control problems in order to generate valid storage functionals. In practice, we propose to use Q-learning to compute the storage functionals.

eess.SY↗

arXiv2022

Quasi-Newton Iteration in Deterministic Policy Gradient

This paper presents a model-free approximation for the Hessian of the performance of deterministic policies to use in the context of Reinforcement Learning based on Quasi-Newton steps in the policy parameters. We show that the approximate Hessian converges to the exact Hessian at the optimal policy, and allows for a superlinear convergence in the learning, provided that the policy parametrization is rich. The natural policy gradient method can be interpreted as a particular case of the proposed method. We analytically verify the formulation in a simple linear case and compare the convergence of the proposed method with the natural policy gradient in a nonlinear example.

cs.LG↗

arXiv2021

Verification of Dissipativity and Evaluation of Storage Function in Economic Nonlinear MPC using Q-Learning

In the Economic Nonlinear Model Predictive (ENMPC) context, closed-loop stability relates to the existence of a storage function satisfying a dissipation inequality. Finding the storage function is in general -- for nonlinear dynamics and cost -- challenging, and has attracted attentions recently. Q-Learning is a well-known Reinforcement Learning (RL) techniques that attempts to capture action-value functions based on the state-input transitions and stage cost of the system. In this paper, we present the use of the Q-Learning approach to obtain the storage function and verify the dissipativity for discrete-time systems subject to state-input constraints. We show that undiscounted Q-learning is able to capture the storage function for dissipative problems when the parameterization is rich enough. The efficiency of the proposed method will be illustrated in the different case studies.

eess.SY↗

arXiv2021

Reinforcement Learning based on Scenario-tree MPC for ASVs

In this paper, we present the use of Reinforcement Learning (RL) based on Robust Model Predictive Control (RMPC) for the control of an Autonomous Surface Vehicle (ASV). The RL-MPC strategy is utilized for obstacle avoidance and target (set-point) tracking. A scenario-tree robust MPC is used to handle potential failures of the ship thrusters. Besides, the wind and ocean current are considered as unknown stochastic disturbances in the real system, which are handled via constraints tightening. The tightening and other cost parameters are adjusted by RL, using a Q-learning technique. An economic cost is considered, minimizing the time and energy required to achieve the ship missions. The method is illustrated in simulation on a nonlinear 3-DOF model of a scaled version of the Cybership 2.

eess.SY↗