arXiv subjects

Juanjuan Xu

Publications and source records attributed to Juanjuan Xu.

At least 19 recordsLinked to original sources

arXiv2026

Stochastic Optimal Linear Quadratic Regulation Control of Discrete-time Systems with Delay and Quadratic Constraints

This article explores the discrete-time stochastic optimal LQR control with delay and quadratic constraints. The inclusion of delay, compared to delay-free optimal LQR control with quadratic constraints, significantly increases the complexity of the problem. Using Lagrangian duality, the optimal control is obtained by solving the Riccati-ZXL equation in conjunction with a gradient ascent algorithm. Specifically, the parameterized optimal controller and cost function are derived by solving the Riccati-ZXL equation, with a gradient ascent algorithm determining the optimal parameter. The primary contribution of this work is presenting the optimal control as a feedback mechanism based on the state's conditional expectation, wherein the gain is determined using the Riccati-ZXL equation and the gradient ascent algorithm. Numerical examples demonstrate the effectiveness of the obtained results.

math.OC↗

arXiv2026

Relaxed Control with Entropy Regularization for Itô Stochastic Systems with Input Delay

This paper investigates the infinite-horizon classical stochastic optimal control problem with input delay under an entropy-regularized relaxed control framework. In particular, by constructing a relaxed system and introducing an entropy regularization term, we reformulate the classical optimal control problem into an entropy regularized formulation, and derive the optimal controller that follows a Gaussian distribution. Furthermore, we show that the optimal Gaussian control distribution converges to the optimal Dirac measure as the exploration weight tends to zero. Numerical simulation is provided to validate the effectiveness of the proposed method.

math.OC↗

arXiv2026

Distributed Load Frequency Control of Multi-Area Smart Grid

In this paper, we investigate the distributed load frequency control problem in a multi-area smart grid under external load disturbances and measurement noise. The novelty lies in that the information privacy is fully taken into account, that is, the internal structural parameters and operational states of each area are not shared with non-neighboring areas, which makes traditional distributed optimal control methods ineffective. The main contribution is to propose a distributed algorithm for the global optimal power regulation command under information privacy constraints via distributed approximation of the control Riccati equation, the estimation Riccati equation, and the state estimation. Simulation results show that the proposed algorithm can approximate the performance of centralized optimal control, and the performance index under the proposed distributed controller is smaller than that under the commonly used distributed control.

math.OC↗

arXiv2026

Distributed Algorithm for the Global Optimal Controller of Nonlinear Multi-Agent Systems

In this paper, we investigate the distributed optimal control problem for a kind of nonlinear multi-agent systems. In particular,both the state and the system dynamic structures of each agent are private and can only be shared among communicating agents.This type of information structure is inevitable in fields such as collaborative control for industrial confidentiality, and renders traditional distributed control methods using all systems' dynamic structures ineffective. The primary contribution is the proposal of a distributed algorithm for the global optimal controller under such practical information structure via distributed approximation of the Hamilton-Jacobi-Bellman equation. Practical numerical simulation demonstrates the effectiveness of the proposed algorithm.

math.OC↗

arXiv2025

Distributed Solving of Linear Quadratic Optimal Controller with Terminal State Constraint

This paper is concerned with the linear quadratic (LQ) optimal control of continuous-time system with terminal state constraint. In particular, multiple agents exist in the system which can only access partial information of the matrix parameters. This makes the classical solving method based on Riccati equation with global information suffering. The main contribution is to present a distributed algorithm to derive the optimal controller which is consisting of the distributed iterations for the Riccati equation, a backward differential equation driven by the optimal Lagrange multiplier and the optimal state. Furthermore, the proposed distributed iteration method is extended to solve the consensus control problem for heterogeneous multi-agent systems, achieving the globally optimal performance of the system. The effectiveness of the proposed algorithm is verified by two numerical examples, where the performance index under the proposed distributed controller is smaller than that under the commonly used consensus control.

math.OC↗

arXiv2025

Linear Quadratic Mean Field Stackelberg Games: Open-loop and Feedback Solutions

This paper investigates open-loop and feedback solutions of linear quadratic mean field (MF) games with a leader and a large number of followers. The leader first gives its strategy and then all the followers cooperate to optimize the social cost as the sum of their costs. By variational analysis with MF approximations, we obtain a set of open-loop controls of players in terms of solutions to MF forward-backward stochastic differential equations (FBSDEs), which is further shown be to an asymptotic Stackelberg equilibrium. By applying the matrix maximum principle, a set of decentralized feedback strategies is constructed for all the players. For open-loop and feedback solutions, the corresponding optimal costs of all players are explicitly given by virtue of the solutions to two Riccati equations, respectively. The performances of two solutions are compared by the numerical simulation.

math.OC↗

arXiv2025

Spectrum Assignment of Stochastic Systems with Multiplicative Noise

This paper studies the spectrum assignment of a class of stochastic systems with multiplicative noise. A novel $α$-spectrum assignment is proposed for discrete-time and continuous-time stochastic systems with multiplicative noise. In particular, $0$-spectrum assignment is equivalent to the pole assignment for the deterministic systems. The main contribution is two-fold: On the one hand, we present the conditions for $α$-spectrum assignment and the design of feedback controllers based on the system parameters. On the other hand, when the system parameters are unknown, we present a stochastic approximation algorithm to learn the feedback gains which guarantee the spectrum of the stochastic systems to achieve the predetermined value. Numerical examples are provided to demonstrate the effectiveness of the proposed algorithms.

math.OC↗

arXiv2024

LQ Optimal Control of First-Order Hyperbolic PDE Systems with Final State Constraints

This paper studies the linear-quadratic (LQ) optimal control problem of a class of systems governed by the first-order hyperbolic partial differential equations (PDEs) with final state constraints. The main contribution is to present the solvability condition and the corresponding explicit optimal controller by using the Lagrange multiplier method and the technique of solving forward and backward partial differential equations (FBPDEs). In particular, the result is reduced to the case with zero-valued final state constraints. Several numerical examples are provided to demonstrate the performance of the designed optimal controller.

math.OC↗

arXiv2024

Closed-Loop Stackelberg Strategy for Linear-Quadratic Leader-Follower Game

This paper is concerned with the closed-loop Stackelberg strategy for linear-quadratic leader-follower game. Completely different from the open-loop and feedback Stackelberg strategy, the solvability of the closed-loop solution even the linear case remains challenging. The main contribution of the paper is to derive the explicitly linear closed-loop Stackelberg strategy with one-step memory in terms of Riccati equations. The key technique is to apply the constrained maximum principle to the leader-follower game and explicitly solve the corresponding forward and backward difference equations. Numerical examples verify the effectiveness of the results, which achieves better performance than feedback strategy.

math.OC↗

arXiv2024

Distributed Optimal Control and Application to Consensus of Multi-Agent Systems

This paper develops a novel approach to the consensus problem of multi-agent systems by minimizing a weighted state error with neighbor agents via linear quadratic (LQ) optimal control theory. Existing consensus control algorithms only utilize the current state of each agent, and the design of distributed controller depends on nonzero eigenvalues of the communication topology. The presented optimal consensus controller is obtained by solving Riccati equations and designing appropriate observers to account for agents' historical state information. It is shown that the corresponding cost function under the proposed controllers is asymptotically optimal. Simulation examples demonstrate the effectiveness of the proposed scheme, and a much faster convergence speed than the conventional consensus methods. Moreover, the new method avoids computing nonzero eigenvalues of the communication topology as in the traditional consensus methods.

math.OC↗

arXiv2023

Exact Controllability of Discrete-Time Stochastic System with Multiplicative Noise

This paper is concerned with the exact controllability of discrete-time stochastic system which is one of the basic problems of modern control theory. Though the exact controllability of continuous-time system governed by Ito stochastic differential equations has been well studied in S. Peng, Progress in Natural Science, 1994, the counterpart of the discrete-time case is still open due to the adaptiveness constraint of the controllers and the solvability challenging of stochastic difference equation with terminal value. The main contribution in this paper is to present both the Gramian matrix criterion and the Rank criterion for the exact controllability of discrete-time stochastic system. The novelty lies in the transformation of the forward stochastic difference equation into a novel backward one.

math.OC↗

arXiv2023

Q-Learning for Linear Quadratic Optimal Control with Terminal State Constraint

This paper is concerned with the linear quadratic optimal control of discrete-time time-varying system with terminal state constraint. The main contribution is to propose a Q-learning algorithm for the optimal controller when the time-varying system matrices and input matrices are both unknown. Different from the existing Q-learning algorithms in the literature which are mainly for the unconstrained optimal control problem, the novelty of the proposed algorithm is available to deal with the case with terminal state constraints. A numerical example is illustrated to verify the effectiveness of the proposed algorithm.

math.OC↗

arXiv2023

Decentralized Control of Linear Systems with Private Input and Measurement Information

In this paper, we study the linear quadratic (LQ) optimal control problem of linear systems with private input and measurement information. The main challenging lies in the unavailability of other regulators' historical input information. To overcome this difficulty, we introduce a kind of novel observers by using the private input and measurement information and accordingly design a kind of new decentralized controllers. In particular, it is verified that the corresponding cost function under the proposed decentralized controllers are asymptotically optimal as comparison with the optimal cost under optimal state-feedback controller. The presented results in this paper are new to the best of our knowledge, which represent the fundamental contribution to classical decentralized control.

math.OC↗

arXiv2023

Reinforcement Learning-Based Optimal Control for Multiplicative-Noise Systems with Input Delay

In this paper, the reinforcement learning (RL)-based optimal control problem is studied for multiplicative-noise systems, where input delay is involved and partial system dynamics is unknown. To solve a variant of Riccati-ZXL equations, which is a counterpart of standard Riccati equation and determines the optimal controller, we first develop a necessary and sufficient stabilizing condition in form of several Lyapunov-type equations, a parallelism of the classical Lyapunov theory. Based on the condition, we provide an offline and convergent algorithm for the variant of Riccati-ZXL equations. According to the convergent algorithm, we propose a RL-based optimal control design approach for solving linear quadratic regulation problem with partially unknown system dynamics. Finally, a numerical example is used to evaluate the proposed algorithm.

math.OC↗

arXiv2022

Distributed Q-Learning for Stochastic LQ Control with Unknown Uncertainty

This paper studies a discrete-time stochastic control problem with linear quadratic criteria over an infinite-time horizon. We focus on a class of control systems whose system matrices are associated with random parameters involving unknown statistical properties. In particular, we design a distributed Q-learning algorithm to tackle the Riccati equation and derive the optimal controller stabilizing the system. The key technique is that we convert the problem of solving the Riccati equation into deriving the zero point of a matrix equation and devise a distributed stochastic approximation method to compute the estimates of the zero point. The convergence analysis proves that the distributed Q-learning algorithm converges to the correct value eventually. A numerical example sheds light on that the distributed Q-learning algorithm converges asymptotically.

math.OC↗

arXiv2021

Stabilization of Stackelberg Game-Based Control Systems

In this paper, we are concerned with the stabilizatbility of Stackelberg game-based systems. In particular, two players are involved in the system where one is the follower to minimize the related cost function and the other is the leader to stabilize the system. The main contribution is to derive the necessary and sufficient condition for the stabilization of the game-based system. The key technique is to explicitly solve the forward and backward difference equations (FBDEs) based on the maximum principle and give the optimal feedback gain matrix of the leader by using the matrix maximum principle.

math.OC↗

arXiv2020

A New Approach for Solving Delayed Forward and Backward Stochastic Differential Equations

This paper is concerned with the decoupling of delayed linear forward-backward stochastic differential equations (D-FBSDEs), which is much more involved than the delay-free case due to the infinite dimension caused by the delay. A new approach of `discretization' is proposed to obtain the explicit solution to the D-FBSDEs. Firstly, we transform the continuous-time D-FBSDEs into the discrete-time form by using discretization. Secondly, we derive the solution of the discrete-time D-FBSDEs by applying backward iterative induction. Finally the explicit solution of the continuous-time D-FBSDEs is obtained by taking the limit to the solution of discrete-time form. The proposed approach can be applied to solve more general FBSDEs with delay, which would provide a complete solution to the stochastic LQ control with time delay.

math.OC↗

arXiv2020

The Difference and Unity of Irregular LQ Control and Standard LQ Control and Its Solution

Irregular linear quadratic control (LQ, was called Singular LQ) has been a long-standing problem since 1970s. This paper will show that an irregular LQ control (deterministic) is solvable (for arbitrary initial value) if and only if the LQ cost can be rewritten as a regular one by changing the terminal cost $x'(T)Hx(T)$ to $x'(T)[H+P_1(T)]x(T)$, while the optimal controller can achieve $P_1(T)x(T)=0$ at the same time. In other words, the irregular controller (if exists) needs to do two things at the same time, one thing is to minimize the cost and the other is to achieve the terminal constraint $P_1(T)x(T)=0$, which clarifies the essential difference of irregular LQ from the standard LQ control where the controller is to minimize the cost only. With this breakthrough, we further study the irregular LQ control for stochastic systems with multiplicative noise. A sufficient solving condition and the optimal controller is presented based on Riccati equations.

math.OC↗