arXiv subjects

Weihai Zhang

Publications and source records attributed to Weihai Zhang.

18 recordsLinked to original sources

arXiv2026

Finite horizon stochastic $H_2/H_\infty$ control for continuous-time mean-field systems with Poisson jumps

The stochastic $H_2/H_\infty$ control problem for continuous-time mean-field stochastic differential equations with Poisson jumps over finite horizon is investigated in this paper. Continuous and jump diffusion terms in the system depend not only on the state but also on the control input, external disturbance, and mean-field components. By employing the quasi-linear technique and the method of completing the square, a mean-field stochastic jump bounded real lemma of the system is derived, which plays a crucial role in solving stochastic $H_2/H_\infty$ control problem. It is demonstrated in this study that the feasibility of the stochastic $H_2/H_\infty$ control problem is equivalent to the solvability of four sets of cross-coupled generalized differential Riccati equations, thus generalizing the previous results to mean-field jump-diffusion systems. To validate the proposed methodology, a numerical simulation example is provided to illustrate the effectiveness of the control strategy. The results establish a systematic approach for designing $H_2/H_\infty$ controllers that simultaneously guarantee the robustness against disturbances and optimal performance for interacting particle systems.

math.OC↗

arXiv2026

Stochastic Bounded Real Lemma and $H_{\infty}$ Control of Difference Systems in Hilbert Spaces

This paper mainly establishes the finite-horizon stochastic bounded real lemma, and then solves the $H_{\infty}$ control problem for discrete-time stochastic linear systems defined on the separable Hilbert spaces, thereby unifying the relevant theoretical results previously confined to the Euclidean space $\mathbb{R}^n$. To achieve these goals, the indefinite linear quadratic (LQ)-optimal control problem is firstly discussed. By employing the bounded linear operator theory and the inner product, a sufficient and necessary condition for the existence of a linear state feedback LQ-optimal control law is derived, which is closely linked with the solvability of the backward Riccati operator equation with a sign condition. Based on this, stochastic bounded real lemma is set up to facilitate the $H_{\infty}$ performance of the disturbed system in Hilbert spaces. Furthermore, the Nash equilibrium problem associated with two parameterized quadratic performance indices is worked out, which enables a uniform treatment of the $H_{\infty}$ and $H_2/H_{\infty}$ control designs by selecting specific values for the parameters. Several examples are supplied to illustrate the effectiveness of the obtained results, especially the practical significance in engineering applications.

math.OC↗

arXiv2025

Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming

This paper investigates a model-free solution to the stochastic linear quadratic regulation (LQR) problem for linear discrete-time systems with both multiplicative and additive noises. We formulate the stochastic LQR problem as a nonconvex optimization problem and rigorously analyze its dual problem structure. By exploiting the inherent convexity of the dual problem and analyzing Karush-Kuhn-Tucker conditions with respect to optimality in convex optimization, we establish an explicit relationship between the optimal point of the dual problem and the parameters of the associated Q-function. This theoretical insight, combined with the technique of the matrix direct sum, makes it possible to develop a novel model-free sample-efficient, non-iterative semidefinite programming algorithm that directly estimates optimal control gain without requiring an initial stabilizing controller, or noises measurability. The robustness of the model-free SDP method to errors is investigated. Our approach provides a new optimization-theoretic framework for understanding Q-learning algorithms while advancing the theoretical foundations of reinforcement learning in stochastic optimal control. Numerical validation on a pulse-width modulated inverter system demonstrates the algorithm's effectiveness, particularly in achieving a single-step non-iterative solution without hyper-parameter tuning.

math.OC↗

arXiv2025

Primal-dual policy learning for mean-field stochastic LQR problem

Integrating data-driven techniques with mechanism-driven insights has recently gained popularity as a powerful learning approach to solving traditional LQR problems for designing intelligent controllers in complex dynamic systems. However, the theoretical understanding of various reinforcement learning algorithms needs further exploration to enhance their efficiency and safety. In this article, by means of primal-dual optimization tools, we study the partially model-free design of the mean-field stochastic LQR (MF-SLQR) controller using a policy learning approach. Firstly, by designing appropriate optimizing variables, the considered MF-SLQR problem is transformed into a new static nonconvex constrained optimization problem with equivalence preserved in certain senses. After that, the equivalent formulation of the duality results is constructed via finding the solution of the generalized Lyapunov equation. Then, the strong duality is analyzed, based on which we establish a primal-dual algorithm by Karush-Kuhn-Tucker conditions. More importantly, a partially model-free implementation is also presented, which has a direct connection with the classical policy iteration algorithm. Finally, we use a high-dimensional example to validate our methods.

math.OC↗

arXiv2025

Learning-based primal-dual optimal control of discrete-time stochastic systems with multiplicative noise

Reinforcement learning (RL) is an effective approach for solving optimal control problems without knowing the exact information of the system model. However, the classical Q-learning method, a model-free RL algorithm, has its limitations, such as lack of strict theoretical analysis and the need for artificial disturbances during implementation. This paper explores the partially model-free stochastic linear quadratic regulator (SLQR) problem for a system with multiplicative noise from the primal-dual perspective to address these challenges. This approach lays a strong theoretical foundation for understanding the intrinsic mechanisms of classical RL algorithms. We reformulate the SLQR into a non-convex primal-dual optimization problem and derive a strong duality result, which enables us to provide model-based and model-free algorithms for SLQR optimal policy design based on the Karush-Kuhn-Tucker (KKT) conditions. An illustrative example demonstrates the proposed model-free algorithm's validity, showcasing the central nervous system's learning mechanism in human arm movement.

math.OC↗

arXiv2025

Small Gain Theorem-Based Robustness Analysis of Discrete-Time MJLSs with the Markov Chain on a Borel Space and Its Application to NCSs

This paper is concerned with the robustness of discrete-time Markov jump linear systems (MJLSs) with the Markov chain on a Borel space. For this general class of MJLSs, a small gain theorem is first established and subsequently applied to derive a lower bound of the stability radius. On this basis, with the aid of the extended bounded real lemma and Schur complements, the robust stability problems for the MJLSs are tackled via linear matrix inequality (LMI) techniques. The novel contribution, primarily founded on the scenario where the state space of the Markov chain is restricted in a continuous set, lies in the formulation of a griding approach. The approach converts the existence problem of solutions of an inequality related to $H_{\infty}$ analysis, which is an infinite-dimensional challenge, into a finite-dimensional LMI feasibility problem. As an application, within the framework of MJLSs, a robustness issue of the sampled-data systems is addressed by using a Markov chain, which is determined by the initial distribution and the stochastic kernel, to model transmission delays existing in networked control systems (NCSs). Finally, the feasibility of the results is verified through two examples.

math.OC↗

arXiv2025

Detectability, Riccati Equations, and the Game-Based Control of Discrete-Time MJLSs with the Markov Chain on a Borel Space

In this paper, detectability is first put forward for discrete-time Markov jump linear systems with the Markov chain on a Borel space ($Θ$, $\mathcal{B}(Θ)$). Under the assumption that the unforced system is detectable, a stability criterion is established relying on the existence of the positive semi-definite solution to the generalized Lyapunov equation. It plays a key role in seeking the conditions that guarantee the existence and uniqueness of the maximal solution and the stabilizing solution for a class of general coupled algebraic Riccati equations (coupled-AREs). Then the nonzero-sum game-based control problem is tackled, and Nash equilibrium strategies are achieved by solving four integral coupled-AREs. As an application of the Nash game approach, the infinite horizon mixed $H_{2}/H_{\infty}$ control problem is studied, along with its solvability conditions. These works unify and generalize those set up in the case where the state space of the Markov chain is restricted to a finite or countably infinite set. Finally, some examples are included to validate the developed results, involving a practical example of the solar thermal receiver.

math.OC↗

arXiv2024

Model-free stochastic linear quadratic design by semidefinite programming

In this article, we study a model-free design approach for stochastic linear quadratic (SLQ) controllers. Based on the convexity of the SLQ dual problem and the Karush-Kuhn-Tucker (KKT) conditions, we find the relationship between the optimal point of the dual problem and the Q-function, which can be used to develop a novel model-free semidefinite programming (SDP) algorithm for deriving optimal control gain. This study provides a new optimization perspective for understanding Q-learning algorithms and lays a theoretical foundation for effective reinforcement learning (RL) algorithms. Finally, the effectiveness of the proposed model-free SDP algorithm is demonstrated by two case simulations.

math.OC↗

arXiv2024

Stability and Bounded Real Lemmas of Discrete-Time MJLSs with the Markov Chain on a Borel Space

In this paper, exponential stability of discrete-time Markov jump linear systems (MJLSs) with the Markov chain on a Borel space $(Θ, \mathcal{B}(Θ))$ is studied, and bounded real lemmas (BRLs) are given. The work generalizes the results from the previous literature that considered only the Markov chain taking values in a countable set to the scenario of an uncountable set and provides unified approaches for describing exponential stability and $H_{\infty}$ performance of MJLSs. This paper covers two kinds of exponential stabilities: one is exponential mean-square stability with conditioning (EMSSy-C), and the other is exponential mean-square stability (EMSSy). First, based on the infinite-dimensional operator theory, the equivalent conditions for determining these two kinds of stabilities are shown respectively by the exponentially stable evolutions generated by the corresponding bounded linear operators on different Banach spaces, which turn out to present the spectral criteria of EMSSy-C and EMSSy. Furthermore, the relationship between these two kinds of stabilities is discussed. Moreover, some easier-to-check criteria are established for EMSSy-C of MJLSs in terms of the existence of uniformly positive definite solutions of Lyapunov-type equations or inequalities. In addition, BRLs are given separately in terms of the existence of solutions of the $Θ$-coupled difference Riccati equation for the finite horizon case and algebraic Riccati equation for the infinite horizon case, which facilitates the $H_{\infty}$ analysis of MJLSs with the Markov chain on a Borel space.

math.OC↗

arXiv2024

Model-free $H_{\infty}$ control of Itô stochastic system via off-policy reinforcement learning

The stochastic $H_{\infty}$ control is studied for a linear stochastic Itô system with an unknown system model. The linear stochastic $H_{\infty}$ control issue is known to be transformable into the problem of solving a so-called generalized algebraic Riccati equation (GARE), which is a nonlinear equation that is typically difficult to solve analytically. Worse, model-based techniques cannot be utilized to approximately solve a GARE when an accurate system model is unavailable or prohibitively expensive to construct in reality. To address these issues, an off-policy reinforcement learning (RL) approach is presented to learn the solution of a GARE from real system data rather than a system model; its convergence is demonstrated, and the robustness of RL to errors in the learning process is investigated. In the off-policy RL approach, the system data may be created with behavior policies rather than the target policies, which is highly significant and promising for use in actual systems. Finally, the proposed off-policy RL approach is validated on a stochastic linear F-16 aircraft system.

math.OC↗

arXiv2023

On stabilizability and exact observability of stochastic systems with their applications

This paper discusses the stabilizability, weak stabilizability, exact observability and robust quadratic stabilizability of linear stochastic control systems. By means of the spectrum technique of the generalized Lyapunov operator, a necessary and sufficient condition is given for stabilizability and weak stabilizability of stochastic systems, respectively. Some new concepts called unremovable spectrums, strong solutions, and weakly feedback stabilizing solutions are introduced. An unremovable spectrum theorem is given, which generalizes the corresponding theorem of deterministic systems to stochastic systems. A stochastic Popov-Belevith-Hautus (PBH) criterion for exact observability is obtained. For applications, we give a comparison theorem for generalized algebraic Riccati equations (GAREs), and two results on Lyapunov-type equations are obtained, which improve the previous works. Finally, we also discuss robust quadratic stabilization of uncertain stochastic systems, and a necessary and sufficient condition is given for quadratic stabilization via a linear matrix inequality (LMI).

math.OC↗

arXiv2022

Improved finite-time stability and instability theorems for stochastic nonlinear systems

This paper studies finite-time stability and instability theorems in probability sense for stochastic nonlinear systems. Firstly, a new sufficient condition is proposed to guarantee that the considered system has a global solution. Secondly, we propose improved finite-time stability and instability criteria that relax the constraints on $\mathcal {L}V$ (the infinitesimal operator of Lyapunov function $V$) by the uniformly asymptotically stable function(UASF). The improved finite-time stability theorems allow $\mathcal {L}V$ to be indefinite (negative or positive) rather than just only allow $\mathcal {L}V$ to be negative. Most existing finite-time stability and instability results can be viewed as special cases of the obtained theorems. Finally, some simulation examples verify the validity of the theoretical results.

math.OC↗

arXiv2018

Robust $H_\infty$ Filtering for Nonlinear Discrete-time Stochastic Systems

This paper mainly discusses the $H_{\infty}$ filtering of general nonlinear discrete time-varying stochastic systems. A nonlinear discrete-time stochastic bounded real lemma (SBRL) is firstly obtained by means of the smoothness of the conditional mathematical expectation, and then, based on the given SBRL and a stochastic LaSalle-type theorem, a sufficient condition for the existence of the $H_\infty$ filtering of general nonlinear discrete time-varying stochastic systems is presented via a new introduced Hamilton-Jacobi inequality (HJI), which is easily verified. When the worst-case disturbance $\{v^*_k\}_{k\in {\mathcal N}}$ is considered, the suboptimal $H_2/H_\infty$ filtering is studied. Two examples including a practical engineering example show the effectiveness of our main results.

math.OC↗

arXiv2018

Mixed $H_-/H_{\infty}$ Fault Detection Filtering for Itô-Type Affine Nonlinear Stochastic Systems

This paper studies the mixed $H_-/H_{\infty}$ fault detection filtering of Itô-type nonlinear stochastic systems. Mixed $H_-/H_{\infty}$ filtering combines the system robustness to the external disturbance and the sensitivity to the fault of the residual signal. Firstly, for Itô-type affine nonlinear stochastic systems, some sufficient criteria are obtained for the existence of $H_-/H_{\infty}$ filter in terms of Hamilton-Jacobi inequalities (HJIs). Secondly, for a class of quasi-linear Itô systems, a sufficient condition is given for the existence of $H_-/H_{\infty}$ filter by means of linear matrix inequalities (LMIs). Finally, a numerical example is presented to illustrate the effectiveness of the proposed results.

math.OC↗

arXiv2018

Finite-time stability and stabilization of linear discrete time-varying stochastic systems

This paper studies the finite-time stability and stabilization of linear discrete time-varying stochastic systems with multiplicative noise. Firstly, necessary and sufficient conditions for finite-time stability are presented via state transition matrix approach. Secondly, this paper also develops the Lyapunov function method to study finite-time stability and stabilization of discrete time-varying stochastic systems based on matrix inequalities and linear matrix inequalities (LMIs), so as to Matlab LMI Toolbox can be used. Two numerical examples are given to illustrate the effectiveness of the proposed results.

math.OC↗

arXiv2017

New Approach to General Nonlinear Discrete-Time Stochastic $H_\infty$ Control

In this paper, a new approach based on convex analysis is introduced to solve the $H_\infty$ problem for discrete-time nonlinear stochastic systems. A stochastic version of bounded real lemma is proved and the state feedback $H_\infty$ control is studied. Two examples are presented to show the effectiveness of our developed theory.

math.OC↗

arXiv2015

Detectability, Observability and Lyapunov-Type Theorems of Linear Discrete Time-Varying Stochastic Systems with Multiplicative Noise

The objective of this paper is to study detectability, observability and related Lyapunov-type theorems of linear discrete-time time-varying stochastic systems with multiplicative noise. Some new concepts such as uniform detectability, ${\cal K}^{\infty}$-exact detectability (resp. ${\cal K}^{WFT}$-exact detectability, ${\cal K}^{FT}$-exact detectability, ${\cal K}^{N}$-exact detectability) and ${\cal K}^{\infty}$-exact observability (resp. ${\cal K}^{WFT}$-exact observability, ${\cal K}^{FT}$-exact observability, ${\cal K}^{N}$-exact observability) are introduced, respectively, and nice properties associated with uniform detectability, exact detectability and exact observability are also obtained. Moreover, some Lyapunov-type theorems associated with generalized Lyapunov equations and exponential stability in mean square sense are presented under uniform detectability, ${\cal K}^{N}$-exact observability and ${\cal K}^{N}$-exact detectability, respectively.

math.OC↗

arXiv2015

New Results on Stability of Singular Stochastic Markov Jump Systems with State-Dependent Noise

This paper aims to develop the stability theory for singular stochastic Markov jump systems with state-dependent noise, including both continuous- and discrete-time cases. The sufficient conditions for the existence and uniqueness of a solution to the system equation are provided. Some new and fundamental concepts such as non-impulsiveness and mean square admissibility are introduced, which are different from those of other existing works. By making use of the $\mathcal{H}$-representation technique and the pseudo inverse $E^+$ of a singular matrix $E$, sufficient conditions ensuring the system to be mean square admissible are established in terms of strict linear matrix inequalities, which can be regarded as extensions of the corresponding results of deterministic singular systems and normal stochastic systems. Practical examples are given to demonstrate the effectiveness of the proposed approaches.

math.OC↗