arXiv subjects

Yingnan Cui

Publications and source records attributed to Yingnan Cui.

5 recordsLinked to original sources

arXiv2023

Online Algorithms and Policies Using Adaptive and Machine Learning Approaches

This paper considers the problem of real-time control and learning in dynamic systems subjected to parametric uncertainties. We propose a combination of a Reinforcement Learning (RL) based policy in the outer loop suitably chosen to ensure stability and optimality for the nominal dynamics, together with Adaptive Control (AC) in the inner loop so that in real-time AC contracts the closed-loop dynamics towards a stable trajectory traced out by RL. Two classes of nonlinear dynamic systems are considered, both of which are control-affine. The first class of dynamic systems utilizes equilibrium points %with expansion forms around these points and a Lyapunov approach while second class of nonlinear systems uses contraction theory. AC-RL controllers are proposed for both classes of systems and shown to lead to online policies that guarantee stability using a high-order tuner and accommodate parametric uncertainties and magnitude limits on the input. In addition to establishing a stability guarantee with real-time control, the AC-RL controller is also shown to lead to parameter learning with persistent excitation for the first class of systems. Numerical validations of all algorithms are carried out using a quadrotor landing task on a moving platform.

cs.LG↗

arXiv2023

Discrete-Time High Order Tuner With A Time-Varying Learning Rate

We propose a new discrete-time online parameter estimation algorithm that combines two different aspects, one that adds momentum, and another that includes a time-varying learning rate. It is well known that recursive least squares based approaches that include a time-varying gain can lead to exponential convergence of parameter errors under persistent excitation, while momentum-based approaches have demonstrated a fast convergence of tracking error towards zero with constant regressors. The question is when combined, will the filter from the momentum method come in the way of exponential convergence. This paper proves that exponential convergence of parameter is still possible with persistent excitation. Simulation results demonstrated competitive properties of the proposed algorithm compared to the recursive least squares algorithm with forgetting.

math.OC↗

arXiv2022

Accelerated Performance and Accelerated Learning with Discrete-Time High-Order Tuners

We consider two high-order tuners that have been shown to have accelerated performance, one based on Polyak's heavy ball method and another based on Nesterov's acceleration method. We show that parameter estimates are bounded and converge to the true values exponentially fast when the regressors are persistently exciting. Simulation results corroborate the accelerated performance and accelerated learning properties of these high-order tuners in comparison to algorithms based on normalized gradient descent.

math.OC↗

arXiv2022

New Algorithms for Discrete-Time Parameter Estimation

We propose two algorithms for discrete-time parameter estimation, one for time-varying parameters under persistent excitation (PE) condition, another for constant parameters under no PE condition. For the first algorithm, we show that in the presence of time-varying unknown parameters, the parameter estimation error converges uniformly to a compact set under conditions of persistent excitation, with the size of the compact set proportional to the time-variation of unknown parameters. Leveraging a projection operator, the second algorithm is shown to result in boundedness guarantees when the plant has constant unknown parameters. Simulations show better convergence results compared to recursive least squares (RLS) and comparable results to RLS with forgetting factor.

cs.LG↗

arXiv2021

A High-order Tuner for Accelerated Learning and Control

Gradient-descent based iterative algorithms pervade a variety of problems in estimation, prediction, learning, control, and optimization. Recently iterative algorithms based on higher-order information have been explored in an attempt to lead to accelerated learning. In this paper, we explore a specific a high-order tuner that has been shown to result in stability with time-varying regressors in linearly parametrized systems, and accelerated convergence with constant regressors. We show that this tuner continues to provide bounded parameter estimates even if the gradients are corrupted by noise. Additionally, we also show that the parameter estimates converge exponentially to a compact set whose size is dependent on noise statistics. As the HT algorithms can be applied to a wide range of problems in estimation, filtering, control, and machine learning, the result obtained in this paper represents an important extension to the topic of real-time and fast decision making.

cs.LG↗