Searcharxiv⌕ Search

arXiv subjects

Zhong-Ping Jiang

Publications and source records attributed to Zhong-Ping Jiang.

At least 37 records · Page 2Linked to original sources
arXiv2022

Predicting the effects of waning vaccine immunity against COVID-19 through high-resolution agent-based modeling

The potential waning of the vaccination immunity to COVID-19 could pose threats to public health, as it is tenable that the timing of such waning would synchronize with the near-complete restoration of normalcy. Should also testing be relaxed, we might witness a resurgent COVID-19 wave in winter 2021/2022. In response to this risk, an additional vaccine dose, the booster shot, is being administered worldwide. In a projected study with an outlook of six months, we explore the interplay between the rate at which boosters are distributed and the extent to which testing practices are implemented, using a highly granular agent-based model tuned on a medium-sized U.S. town. Theoretical projections indicate that the administration of boosters at the rate at which the vaccine is currently administered could yield a severe resurgence of the pandemic. Projections suggest that the peak levels of mid spring 2021 in the vaccination rate may prevent such a scenario to occur, although exact agreement between observations and projections should not be expected due to continuously evolving nature of the pandemics. Our study highlights the importance of testing, especially to detect asymptomatic individuals in the near future, as the release of the booster reaches full speed.

q-bio.PE↗
arXiv2021

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness.

eess.SY↗
arXiv2021

Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulation

This paper studies the robustness of reinforcement learning algorithms to errors in the learning process. Specifically, we revisit the benchmark problem of discrete-time linear quadratic regulation (LQR) and study the long-standing open question: Under what conditions is the policy iteration method robustly stable from a dynamical systems perspective? Using advanced stability results in control theory, it is shown that policy iteration for LQR is inherently robust to small errors in the learning process and enjoys small-disturbance input-to-state stability: whenever the error in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded, and, moreover, enter and stay in a small neighbourhood of the optimal LQR solution. As an application, a novel off-policy optimistic least-squares policy iteration for the LQR problem is proposed, when the system dynamics are subjected to additive stochastic disturbances. The proposed new results in robust reinforcement learning are validated by a numerical example.

math.OC↗
arXiv2021

High-resolution agent-based modeling of COVID-19 spreading in a small town

Amid the ongoing COVID-19 pandemic, public health authorities and the general population are striving to achieve a balance between safety and normalcy. Ever changing conditions call for the development of theory and simulation tools to finely describe multiple strata of society while supporting the evaluation of "what-if" scenarios. Particularly important is to assess the effectiveness of potential testing approaches and vaccination strategies. Here, an agent-based modeling platform is proposed to simulate the spreading of COVID-19 in small towns and cities, with a single-individual resolution. The platform is validated on real data from New Rochelle, NY -- one of the first outbreaks registered in the United States. Supported by expert knowledge and informed by reported data, the model incorporates detailed elements of the spreading within a statistically realistic population. Along with pertinent functionality such as testing, treatment, and vaccination options, the model accounts for the burden of other illnesses with symptoms similar to COVID-19. Unique to the model is the possibility to explore different testing approaches -- in hospitals or drive-through facilities -- and vaccination strategies that could prioritize vulnerable groups. Decision making by public authorities could benefit from the model, for its fine-grain resolution, open-source nature, and wide range of features.

q-bio.PE↗
arXiv2020

Latency-Robust Control of High-Speed Signal-Free Intersections

High-speed signal-free intersections are a novel urban traffic operations enabled by connected and autonomous vehicles. However, the impact of communication latency on intersection performance has not been well understood. In this paper, we consider vehicle coordination at signal-free intersections with latency. We focus on two questions: (i) how to ensure latency-resiliency of the coordination algorithm, and (ii) how latency affects the intersection's capacity. We consider a trajectory-based model with bounded speed uncertainties. Latency leads to uncertain state observation. We propose a piecewise-linear control law that ensures safety (avoidance of interference) as long as the initial condition is safe. We also analytically quantify the throughput that the proposed control can attain in the face of latency.

eess.SY↗
arXiv2020

Robust Policy Iteration for Continuous-time Linear Quadratic Regulation

This paper studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulation (LQR) problem. It is shown that Kleinman's policy iteration algorithm is inherently robust to small disturbances and enjoys local input-to-state stability in the sense of Sontag. More precisely, whenever the disturbance-induced input term in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded and enter a small neighborhood of the optimal solution of the LQR problem. Based on this result, an off-policy data-driven policy iteration algorithm for the LQR problem is shown to be robust when the system dynamics are subjected to small additive unknown bounded disturbances. The theoretical results are validated by a numerical example.

eess.SY↗
arXiv2020

Temporal-Differential Learning in Continuous Environments

In this paper, a new reinforcement learning (RL) method known as the method of temporal differential is introduced. Compared to the traditional temporal-difference learning method, it plays a crucial role in developing novel RL techniques for continuous environments. In particular, the continuous-time least squares policy evaluation (CT-LSPE) and the continuous-time temporal-differential (CT-TD) learning methods are developed. Both theoretical and empirical evidences are provided to demonstrate the effectiveness of the proposed temporal-differential learning methodology.

cs.LG↗
arXiv2020

Adaptive Optimal Control of Linear Periodic Systems: An Off-Policy Value Iteration Approach

This paper studies the infinite-horizon adaptive optimal control of continuous-time linear periodic (CTLP) systems. A novel value iteration (VI) based off-policy ADP algorithm is proposed for a general class of CTLP systems, so that approximate optimal solutions can be obtained directly from the collected data, without the exact knowledge of system dynamics. Under mild conditions, the proofs on uniform convergence of the proposed algorithm to the optimal solutions are given for both the model-based and model-free cases. The VI-based ADP algorithm is able to find suboptimal controllers without assuming the knowledge of an initial stabilizing controller. Application to the optimal control of a triple inverted pendulum subjected to a periodically varying load demonstrates the feasibility and effectiveness of the proposed method.

eess.SY↗
arXiv2019

Distributed Global Output-Feedback Control for a Class of Euler-Lagrange Systems

This published paper investigates the distributed tracking control problem for a class of Euler-Lagrange multi-agent systems when the agents can only measure the positions. In this case, the lack of the separation principle and the strong nonlinearity in unmeasurable states pose severe technical challenges to global output-feedback control design. To overcome these difficulties, a global nonsingular coordinate transformation matrix in the upper triangular form is firstly proposed such that the nonlinear dynamic model can be partially linearized with respect to the unmeasurable states. And, a new type of velocity observers is designed to estimate the unmeasurable velocities for each system. Then, based on the outputs of the velocity observers, we propose distributed control laws that enable the coordinated tracking control system to achieve uniform global exponential stability (UGES). Both theoretical analysis and numerical simulations are presented to validate the effectiveness of the proposed control scheme. Followed by the original paper, a typo and a mistake is corrected.

eess.SY↗
arXiv2018

Continuous-Time Robust Dynamic Programming

This paper presents a new theory, known as robust dynamic pro- gramming, for a class of continuous-time dynamical systems. Different from traditional dynamic programming (DP) methods, this new theory serves as a fundamental tool to analyze the robustness of DP algorithms, and in par- ticular, to develop novel adaptive optimal control and reinforcement learning methods. In order to demonstrate the potential of this new framework, four illustrative applications in the fields of stochastic optimal control and adaptive DP are presented. Three numerical examples arising from both finance and engineering industries are also given, along with several possible extensions of the proposed framework.

math.OC↗
arXiv2017

Global Adaptive Dynamic Programming for Continuous-Time Nonlinear Systems

This paper presents a novel method of global adaptive dynamic programming (ADP) for the adaptive optimal control of nonlinear polynomial systems. The strategy consists of relaxing the problem of solving the Hamilton-Jacobi-Bellman (HJB) equation to an optimization problem, which is solved via a new policy iteration method. The proposed method distinguishes from previously known nonlinear ADP methods in that the neural network approximation is avoided, giving rise to significant computational improvement. Instead of semiglobally or locally stabilizing, the resultant control policy is globally stabilizing for a general class of nonlinear polynomial systems. Furthermore, in the absence of the a priori knowledge of the system dynamics, an online learning method is devised to implement the proposed policy iteration technique by generalizing the current ADP theory. Finally, three numerical examples are provided to validate the effectiveness of the proposed method.

math.DS↗
arXiv2013

Robust Adaptive Dynamic Programming for Optimal Nonlinear Control Design

This paper studies the robust optimal control design for uncertain nonlinear systems from a perspective of robust adaptive dynamic programming (robust-ADP). The objective is to fill up a gap in the past literature of ADP where dynamic uncertainties or unmodeled dynamics are not addressed. A key strategy is to integrate tools from modern nonlinear control theory, such as the robust redesign and the backstepping techniques as well as the nonlinear small-gain theorem, with the theory of ADP. The proposed robust-ADP methodology can be viewed as a natural extension of ADP to uncertain nonlinear systems. A practical learning algorithm is developed in this paper, and has been applied to a sensorimotor control problem.

math.DS↗
arXiv2012

Global stabilization of nonlinear systems based on vector control lyapunov functions

This paper studies the use of vector Lyapunov functions for the design of globally stabilizing feedback laws for nonlinear systems. Recent results on vector Lyapunov functions are utilized. The main result of the paper shows that the existence of a vector control Lyapunov function is a necessary and sufficient condition for the existence of a smooth globally stabilizing feedback. Applications to nonlinear systems are provided: simple and easily checkable sufficient conditions are proposed to guarantee the existence of a smooth globally stabilizing feedback law. The obtained results are applied to the problem of the stabilization of an equilibrium point of a reaction network taking place in a continuous stirred tank reactor.

math.OC↗
arXiv2010

Reduced Order Dead-Beat Observers for a Bioreactor

This paper studies the strong observability property and the reduced-order dead-beat observer design problem for a continuous bioreactor. New relationships between coexistence and strong observability, and checkable sufficient conditions for strong observability, are established for a chemostat with two competing microbial species. Furthermore, the dynamic output feedback stabilization problem is solved for the case of one species.

math.OC↗
arXiv2010

Hybrid dead-beat observers for a class of nonlinear systems

This paper studies the reduced-order or full-order, dead-beat observer problem for a class of nonlinear systems, linear in the unmeasured states. A novel hybrid observer design strategy is proposed, with the help of the notion of strong observability in finite time. The proposed methodology is applied to a batch reactor, for which a hybrid dead-beat observer is obtained in the absence of the precise measurements of the concentration variables. Moreover, the observer is used for the estimation of the frequency of a sinusoidal signal. The results show that accurate estimations can be provided even if the signal is corrupted by high frequency noise.

math.OC↗
arXiv2010

New Results in Trajectory-Based Small-Gain with Application to the Stabilization of a Chemostat

New trajectory-based small-gain results are obtained for nonlinear feedback systems under relaxed assumptions. Specifically, during a transient period, the solutions of the feedback system may not satisfy some key inequalities that previous small-gain results usually utilize to prove stability properties. The results allow the application of the small-gain perspective to various systems which satisfy less demanding stability notions than the Input-to-Output Stability property. The robust global feedback stabilization problem of an uncertain time-delayed chemostat model is solved by means of the trajectory-based small-gain results.

math.OC↗
arXiv2010

Nash Equilibrium and Robust Stability in Dynamic Games: A Small-Gain Perspective

This paper develops a novel methodology to study robust stability properties of Nash equilibrium points in dynamic games. Small-gain techniques in modern mathematical control theory are used for the first time to derive conditions guaranteeing uniqueness and global asymptotic stability of Nash equilibrium point for economic models described by functional difference equations. Specification to a Cournot oligopoly game is studied in detail to demonstrate the power of the proposed methodology.

math.DS↗