arXiv subjects

Shubhendu Bhasin

Publications and source records attributed to Shubhendu Bhasin.

30 records · Page 2Linked to original sources

arXiv2023

Safe Q-learning for continuous-time linear systems

Q-learning is a promising method for solving optimal control problems for uncertain systems without the explicit need for system identification. However, approaches for continuous-time Q-learning have limited provable safety guarantees, which restrict their applicability to real-time safety-critical systems. This paper proposes a safe Q-learning algorithm for partially unknown linear time-invariant systems to solve the linear quadratic regulator problem with user-defined state constraints. We frame the safe Q-learning problem as a constrained optimal control problem using reciprocal control barrier functions and show that such an extension provides a safety-assured control policy. To the best of our knowledge, Q-learning for continuous-time systems with state constraints has not yet been reported in the literature.

eess.SY↗

arXiv2023

MRAC with Memory for Switched Linear Systems

This work proposes a switched model reference adaptive control (S-MRAC) architecture for a multi-input multi-output (MIMO) switched linear system with memory for enhanced learning. A salient feature of the proposed method that separates it from most previous results is the use of memory that store the estimator states at switching and facilitate parameter learning during both active and inactive phases of a subsystem, thereby improving the tracking performance of the overall switched system. Specifically, the learning experience from the previous active duration of a subsystem is retained in the memory and reused when the subsystem is inactive and when the subsystem becomes active again. Parameter convergence is shown based on an intermittent initial excitation (IIE), which is significantly relaxed than the classical persistence of excitation (PE) condition. A common Lyapunov function is considered to ensure closed-loop stability with S-MRAC. Further under IIE, the exponential stability of tracking and parameter estimation error dynamics are guaranteed.

eess.SY↗

arXiv2022

Adaptive Tracking Control of Uncertain Euler-Lagrange Systems with State and Input Constraints

This paper proposes a novel control architecture for state and input constrained Euler-Lagrange (E-L) systems with parametric uncertainties. A simple saturated controller is strategically coupled with a Barrier Lyapunov Function (BLF) based controller to ensure state and input constraint satisfaction. To the best of the authors' knowledge, this is the first result for E-L systems that guarantee asymptotic tracking with user-specified state and input constraints. The proposed controller also ensures that all the closed-loop signals remain bounded. The efficacy of the proposed controller in terms of constraint satisfaction and tracking performance is verified using simulation on a robot manipulator system.

eess.SY↗

arXiv2022

Adaptive Observers for MIMO Discrete-Time LTI Systems

In this paper, an adaptive observer is proposed for multi-input multi-output (MIMO) discrete-time linear time-invariant (LTI) systems. Unlike existing MIMO adaptive observer designs, the proposed approach is applicable to LTI systems in their general form. Further, the proposed method uses recursive least square (RLS) with covariance resetting for adaptation that is shown to guarantee that the estimates are bounded, irrespective of any excitation condition, even in the presence of a vanishing perturbation term in the error used for updation in RLS. Detailed analysis for convergence and boundedness has been provided along with simulation results for illustrating the performance of the developed theory.

eess.SY↗

arXiv2022

Adaptive Output Feedback Model Predictive Control

Model predictive control (MPC) for uncertain systems in the presence of hard constraints on state and input is a non-trivial problem, and the challenge is increased manyfold in the absence of state measurements. In this paper, we propose an adaptive output feedback MPC technique, based on a novel combination of an adaptive observer and robust MPC, for single-input single-output discrete-time linear time-invariant systems. At each time instant, the adaptive observer provides estimates of the states and the system parameters that are then leveraged in the MPC optimization routine while robustly accounting for the estimation errors. The solution to the optimization problem results in a homothetic tube where the state estimate trajectory lies. The true state evolves inside a larger outer tube obtained by augmenting a set, invariant to the state estimation error, around the homothetic tube sections. The proof for recursive feasibility for the proposed `homothetic and invariant' two-tube approach is provided, along with simulation results on an academic system.

eess.SY↗

arXiv2022

Safe reinforcement learning control for continuous-time nonlinear systems without a backup controller

This paper proposes an on-policy reinforcement learning (RL) control algorithm that solves the optimal regulation problem for a class of uncertain continuous-time nonlinear systems under user-defined state constraints. We formulate the safe RL problem as the minimization of the Hamiltonian subject to a constraint on the time-derivative of a barrier Lyapunov function (BLF). We subsequently use the analytical solution of the optimization problem to modify the Actor-Critic-Identifier architecture to learn the optimal control policy safely. The proposed method does not require the presence of external backup controllers, and the RL policy ensures safety for the entire duration. The efficacy of the proposed controller is demonstrated on a class of Euler-Lagrange systems.

eess.SY↗

arXiv2022

State and Input Constrained Model Reference Adaptive Control

Satisfaction of state and input constraints is one of the most critical requirements in control engineering applications. In classical model reference adaptive control (MRAC) formulation, although the states and the input remain bounded, the bound is neither user-defined nor known a-priori. In this paper, an MRAC is developed for multivariable linear time-invariant (LTI) plant with user-defined state and input constraints using a simple saturated control design coupled with a barrier Lyapunov function (BLF). Without any restrictive assumptions that may limit practical implementation, the proposed controller guarantees that both the plant state and the control input remain within a user-defined safe set for all time while simultaneously ensuring that the plant state trajectory tracks the reference model trajectory. The controller ensures that all the closed-loop signals remain bounded and the trajectory tracking error converges to zero asymptotically. Simulation results validate the efficacy of the proposed constrained MRAC in terms of better tracking performance and limited control effort compared to the standard MRAC algorithm.

eess.SY↗

arXiv2022

Online Adaptive Identification of Switched Affine Systems Using a Two-Tier Filter Architecture with Memory

This work proposes an online adaptive identification method for multi-input multi-output (MIMO) switched affine systems with guaranteed parameter convergence. A family of online parameter estimators is used that is equipped with a dual-layer low pass filter architecture to facilitate parameter learning and identification of each subsystem. The filters capture information about the unknown parameters in the form of a prediction error which is used in the parameter estimation algorithm. A salient feature of the proposed method that distinguishes it from most previous results is the use of a memory bank that stores filter values and promotes parameter learning during both active and inactive phases of a subsystem. Specifically, the learnt experience from the previous active phase of a subsystem is retained in the memory and leveraged for parameter learning in its subsequent active and inactive phases. Further, a new notion of intermittent initial excitation (IIE) is introduced that extends the previously established initial excitation (IE) condition to the switched system framework. IIE is shown to be sufficient to ensure exponential convergence of the switched system parameters.

eess.SY↗

arXiv2020

Attractive Ellipsoid Sliding Mode Observer Design for State of Charge Estimation of Lithium-ion Cells

This work investigates the real-time estimation of the state-of-charge (SoC) of Lithium-ion (Li-ion) cells for reliable, safe and efficient utilization. A novel attractive ellipsoid based sliding-mode observer (AESMO) algorithm is designed to estimate the SoC in real-time. The algorithm utilizes standard equivalent circuit model of a Li-ion cell and provides reliable and efficient SoC estimate in the presence of bounded uncertainties in the battery parameters as well as exogenous disturbances. The theoretical framework of the observer design is not limited to the SoC estimation problem of Li-ion cell but applicable to a wider class of nonlinear systems with both matched and mismatched uncertainties. The main advantage of the proposed observer is to provide a fast and optimal SoC estimate based on minimization over the uncertainty bound. The proposed method is experimentally tested and evaluated using the hybrid pulse power characterization test (HPPC)and urban dynamometer driving schedule (UDDS) test data, which demonstrate its effectiveness and feasibility.

eess.SY↗

arXiv2020

Adaptive Control of Time-Varying Parameter Systems with Asymptotic Tracking

A continuous adaptive control design is developed for nonlinear dynamical systems with linearly parameterizable uncertainty involving time-varying uncertain parameters. The key feature of this design is a robust integral of the sign of the error (RISE)-like term in the adaptation law which compensates for potentially destabilizing terms in the closed-loop error system arising from the time-varying nature of uncertain parameters. A Lyapunov-based stability analysis ensures asymptotic tracking, and boundedness of the closed-loop signals.

eess.SY↗

arXiv2016

Memory-Based Data-Driven MRAC Architecture Ensuring Parameter Convergence

Convergence of controller parameters in standard model reference adaptive control (MRAC) requires the system states to be persistently exciting (PE), a restrictive condition to be verified online. A recent data-driven approach, concurrent learning, uses information-rich past data concurrently with the standard parameter update laws to guarantee parameter convergence without the need of the PE condition. This method guarantees exponential convergence of both the tracking and the controller parameter estimation errors to zero, whereas, the classical MRAC merely ensures asymptotic convergence of tracking error to zero. However, the method requires knowledge of the state derivative, at least at the time instances when the state values are stored in memory. The method further assumes knowledge of the control allocation matrix. This paper addresses these limitations by using a memory-based finite-time system identifier in conjunction with a data-driven approach, leading to convergence of both the tracking and the controller parameter estimation errors without the PE condition and knowledge of the system matrices and the state derivative. A Lyapunov based stability proof is included to justify the validity of the proposed data-driven approach. Simulation results demonstrate the efficacy of the suggested method.

eess.SY↗

arXiv2013

Approximate Optimal Trajectory Tracking for Continuous Time Nonlinear Systems

Approximate dynamic programming has been investigated and used as a method to approximately solve optimal regulation problems. However, the extension of this technique to optimal tracking problems for continuous time nonlinear systems has remained a non-trivial open problem. The control development in this paper guarantees ultimately bounded tracking of a desired trajectory, while also ensuring that the controller converges to an approximate optimal policy.

eess.SY↗