Searcharxiv⌕ Search

arXiv subjects

Zachary I. Bell

Publications and source records attributed to Zachary I. Bell.

27 records · Page 2Linked to original sources
arXiv2023

State and Parameter Estimation for Affine Nonlinear Systems

Real-world control applications in complex and uncertain environments require adaptability to handle model uncertainties and robustness against disturbances. This paper presents an online, output-feedback, critic-only, model-based reinforcement learning architecture that simultaneously learns and implements an optimal controller while maintaining stability during the learning phase. Using multiplier matrices, a convenient way to search for observer gains is designed along with a controller that learns from simulated experience to ensure stability and convergence of trajectories of the closed-loop system to a neighborhood of the origin. Local uniform ultimate boundedness of the trajectories is established using a Lyapunov-based analysis and demonstrated through simulation results, under mild excitation conditions.

eess.SY↗
arXiv2023

Output Feedback Adaptive Optimal Control of Affine Nonlinear systems with a Linear Measurement Model

Real-world control applications in complex and uncertain environments require adaptability to handle model uncertainties and robustness against disturbances. This paper presents an online, output-feedback, critic-only, model-based reinforcement learning architecture that simultaneously learns and implements an optimal controller while maintaining stability during the learning phase. Using multiplier matrices, a convenient way to search for observer gains is designed along with a controller that learns from simulated experience to ensure stability and convergence of trajectories of the closed-loop system to a neighborhood of the origin. Local uniform ultimate boundedness of the trajectories is established using a Lyapunov-based analysis and demonstrated through simulation results, under mild excitation conditions.

eess.SY↗
arXiv2022

Guarding a Non-Maneuverable Translating Line with an Attached Defender

In this paper we consider a target-guarding differential game where the defender must protect a linearly translating line-segment by intercepting an attacker who tries to reach it. In contrast to common target-guarding problems, we assume that the defender is attached to the target and moves along with it. This assumption affects the defenders' maximum speed in inertial frame, which depends on the target's direction of motion. Zero-sum differential game of degree for both the attacker-win and defender-win scenarios are studied, where the payoff is defined to be the distance between the two agents at the time of game termination. We derive the equilibrium strategies and the Value function by leveraging the solution for the infinite-length target scenario. The zero-level set of this Value function provides the barrier surface that divides the state space into defender-win and attacker-win regions. We present simulation results to demonstrate the theoretical results.

eess.SY↗
arXiv2022

A Zeroth-Order Momentum Method for Risk-Averse Online Convex Games

We consider risk-averse learning in repeated unknown games where the goal of the agents is to minimize their individual risk of incurring significantly high cost. Specifically, the agents use the conditional value at risk (CVaR) as a risk measure and rely on bandit feedback in the form of the cost values of the selected actions at every episode to estimate their CVaR values and update their actions. A major challenge in using bandit feedback to estimate CVaR is that the agents can only access their own cost values, which, however, depend on the actions of all agents. To address this challenge, we propose a new risk-averse learning algorithm with momentum that utilizes the full historical information on the cost values. We show that this algorithm achieves sub-linear regret and matches the best known algorithms in the literature. We provide numerical experiments for a Cournot game that show that our method outperforms existing methods.

cs.LG↗
arXiv2022

Safe Controller for Output Feedback Linear Systems using Model-Based Reinforcement Learning

The objective of this research is to enable safety-critical systems to simultaneously learn and execute optimal control policies in a safe manner to achieve complex autonomy. Learning optimal policies via trial and error, i.e., traditional reinforcement learning, is difficult to implement in safety-critical systems, particularly when task restarts are unavailable. Safe model-based reinforcement learning techniques based on a barrier transformation have recently been developed to address this problem. However, these methods rely on full state feedback, limiting their usability in a real-world environment. In this work, an output-feedback safe model-based reinforcement learning technique based on a novel barrier-aware dynamic state estimator has been designed to address this issue. The developed approach facilitates simultaneous learning and execution of safe control policies for safety-critical linear systems. Simulation results indicate that barrier transformation is an effective approach to achieve online reinforcement learning in safety-critical systems using output feedback.

eess.SY↗
arXiv2021

Safe Model-Based Reinforcement Learning for Systems with Parametric Uncertainties

Reinforcement learning has been established over the past decade as an effective tool to find optimal control policies for dynamical systems, with recent focus on approaches that guarantee safety during the learning and/or execution phases. In general, safety guarantees are critical in reinforcement learning when the system is safety-critical and/or task restarts are not practically feasible. In optimal control theory, safety requirements are often expressed in terms of state and/or control constraints. In recent years, reinforcement learning approaches that rely on persistent excitation have been combined with a barrier transformation to learn the optimal control policies under state constraints. To soften the excitation requirements, model-based reinforcement learning methods that rely on exact model knowledge have also been integrated with the barrier transformation framework. The objective of this paper is to develop safe reinforcement learning method for deterministic nonlinear systems, with parametric uncertainties in the model, to learn approximate constrained optimal policies without relying on stringent excitation conditions. To that end, a model-based reinforcement learning technique that utilizes a novel filtered concurrent learning method, along with a barrier transformation, is developed in this paper to realize simultaneous learning of unknown model parameters and approximate optimal state-constrained control policies for safety-critical systems.

eess.SY↗
arXiv2021

Safety aware model-based reinforcement learning for optimal control of a class of output-feedback nonlinear systems

The ability to learn and execute optimal control policies safely is critical to realization of complex autonomy, especially where task restarts are not available and/or the systems are safety-critical. Safety requirements are often expressed in terms of state and/or control constraints. Methods such as barrier transformation and control barrier functions have been successfully used, in conjunction with model-based reinforcement learning, for safe learning in systems under state constraints, to learn the optimal control policy. However, existing barrier-based safe learning methods rely on full state feedback. In this paper, an output-feedback safe model-based reinforcement learning technique is developed that utilizes a novel dynamic state estimator to implement simultaneous learning and control for a class of safety-critical systems with partially observable state.

eess.SY↗
arXiv2021

Asynchronous Zeroth-Order Distributed Optimization with Residual Feedback

We consider a zeroth-order distributed optimization problem, where the global objective function is a black-box function and, as such, its gradient information is inaccessible to the local agents. Instead, the local agents can only use the values of the objective function to estimate the gradient and update their local decision variables. In this paper, we also assume that these updates are done asynchronously. To solve this problem, we propose an asynchronous zeroth-order distributed optimization method that relies on a one-point residual feedback to estimate the unknown gradient. We show that this estimator is unbiased under asynchronous updating, and theoretically analyze the convergence of the proposed method. We also present numerical experiments that demonstrate that our method outperforms two-point methods under asynchronous updating. To the best of our knowledge, this is the first asynchronous zeroth-order distributed optimization method that is also supported by theoretical guarantees.

math.OC↗
arXiv2018

A Switched Systems Approach to Path Following with Intermittent State Feedback

Autonomous agents are often tasked with operating in an area where feedback is unavailable. Inspired by such applications, this paper develops a novel switched systems-based control method for uncertain nonlinear systems with temporary loss of state feedback. To compensate for intermittent feedback, an observer is used while state feedback is available to reduce the estimation error, and a predictor is utilized to propagate the estimates while state feedback is unavailable. Based on the resulting subsystems, maximum and minimum dwell time conditions are developed via a Lyapunov-based switched systems analysis to relax the constraint of maintaining constant feedback. Using the dwell time conditions, a switching trajectory is developed to enter and exit the feedback denied region in a manner that ensures the overall switched system remains stable. A scheme for designing a switching trajectory with a smooth transition function is provided. Simulation and experimental results are presented to demonstrate the performance of control design.

eess.SY↗