arXiv subjects

Ji-Feng Zhang

Publications and source records attributed to Ji-Feng Zhang.

36 records · Page 2Linked to original sources

arXiv2024

Differentially Private Distributed Stochastic Optimization with Time-Varying Sample Sizes

Differentially private distributed stochastic optimization has become a hot topic due to the urgent need of privacy protection in distributed stochastic optimization. In this paper, two-time scale stochastic approximation-type algorithms for differentially private distributed stochastic optimization with time-varying sample sizes are proposed using gradient- and output-perturbation methods. For both gradient- and output-perturbation cases, the convergence of the algorithm and differential privacy with a finite cumulative privacy budget $\varepsilon$ for an infinite number of iterations are simultaneously established, which is substantially different from the existing works. By a time-varying sample sizes method, the privacy level is enhanced, and differential privacy with a finite cumulative privacy budget $\varepsilon$ for an infinite number of iterations is established. By properly choosing a Lyapunov function, the algorithm achieves almost-sure and mean-square convergence even when the added privacy noises have an increasing variance. Furthermore, we rigorously provide the mean-square convergence rates of the algorithm and show how the added privacy noise affects the convergence rate of the algorithm. Finally, numerical examples including distributed training on a benchmark machine learning dataset are presented to demonstrate the efficiency and advantages of the algorithms.

eess.SY↗

arXiv2023

Consensus-Based Distributed Nonlinear Filtering with Kernel Mean Embedding

This paper proposes a consensus-based distributed nonlinear filter with kernel mean embedding (KME). This fills with gap of posterior density approximation with KME for distributed nonlinear dynamic systems. To approximate the posterior density, the system state is embedded into a higher-dimensional reproducing kernel Hilbert space (RKHS), and then the nonlinear measurement function is linearly converted. As a result, an update rule of KME of posterior distribution is established in the RKHS. To show the proposed distributed filter being capable of achieving the centralized estimation accuracy, a centralized filter, serving as an extension of the standard Kalman filter in the state space to the RKHS, is developed first. Benefited from the KME, the proposed distributed filter converges to the centralized one while maintaining the distributed pattern. Two examples are introduced to demonstrate the effectiveness of the developed filters in target tracking scenarios including nearly constantly moving target and turning target, respectively, with bearing-only, range and bearing measurements.

eess.SY↗

arXiv2023

Asymptotically Efficient Quasi-Newton Type Identification with Quantized Observations Under Bounded Persistent Excitations

This paper is concerned with the optimal identification problem of dynamical systems in which only quantized output observations are available under the assumption of fixed thresholds and bounded persistent excitations. Based on a time-varying projection, a weighted Quasi-Newton type projection (WQNP) algorithm is proposed. With some mild conditions on the weight coefficients, the algorithm is proved to be mean square and almost surely convergent, and the convergence rate can be the reciprocal of the number of observations, which is the same order as the optimal estimate under accurate measurements. Furthermore, inspired by the structure of the Cramer-Rao lower bound, an information-based identification (IBID) algorithm is constructed with adaptive design about weight coefficients of the WQNP algorithm, where the weight coefficients are related to the parameter estimates which leads to the essential difficulty of algorithm analysis. Beyond the convergence properties, this paper demonstrates that the IBID algorithm tends asymptotically to the Cramer-Rao lower bound, and hence is asymptotically efficient. Numerical examples are simulated to show the effectiveness of the information-based identification algorithm.

eess.SY↗

arXiv2023

Differentially Private Bipartite Consensus over Signed Networks with Time-Varying Noises

This paper investigates the differentially private bipartite consensus algorithm over signed networks. The proposed algorithm protects each agent's sensitive information by adding noise with time-varying variances to the cooperative-competitive interactive information. In order to achieve privacy protection, the variance of the added noise is allowed to be increased, and substantially different from the existing works. In addition, the variance of the added noise can be either decaying or constant. By using time-varying step-sizes based on the stochastic approximation method, we show that the algorithm converges in mean-square and almost-surely even with an increasing privacy noise. We further develop a method to design the step-size and the noise parameter, affording the algorithm to achieve asymptotically unbiased bipartite consensus with the desired accuracy and the predefined differential privacy level. Moreover, we give the mean-square and almost-sure convergence rate of the algorithm, and the privacy level with different forms of the privacy noises. We also reveal the algorithm's trade-off between the convergence rate and the privacy level. Finally, a numerical example verifies the theoretical results and demonstrates the algorithm's superiority against existing methods.

eess.SY↗

arXiv2023

Nash equilibrium of multi-agent graphical game with a privacy information encrypted learning algorithm

This paper studies the global Nash equilibrium problem of leader-follower multi-agent dynamics, which yields consensus with a privacy information encrypted learning algorithm. With the secure hierarchical structure, the relationship between the secure consensus problem and global Nash equilibrium is discussed under potential packet loss attacks, and the necessary and sufficient condition for the existence of global Nash equilibrium is provided regarding the soft-constrained graphical game. To achieve the optimal policies, the convergence of decentralized learning algorithm is guaranteed with an iteratively updated pair of decoupled gains. By using the developed quantization scheme and additive-multiplicative property, the encryption-decryption is successfully embedded in the data transmission and computation to overcome the potential privacy violation in unreliable networks. A simulation example is provided to verify the effectiveness of the designed algorithm.

eess.SY↗

arXiv2023

A general maximum principle for optimal control of stochastic differential delay systems

In this paper, we solve an open problem and obtain a general maximum principle for a stochastic optimal control problem where the control domain is an arbitrary non-empty set and all the coefficients (especially the diffusion term and the terminal cost) contain the control and state delay. In order to overcome the difficulty of dealing with the cross term of state and its delay in the variational inequality, we propose a new method: transform a delayed variational equation into a Volterra integral equation without delay, and introduce novel first-order, second-order adjoint equations via the backward stochastic Volterra integral equation theory. Finally we express these two kinds of adjoint equations in more compact anticipated backward stochastic differential equation types for several special yet typical control systems.

math.OC↗

arXiv2022

Probabilistic Framework of Howard's Policy Iteration: BML Evaluation and Robust Convergence Analysis

This paper aims to build a probabilistic framework for Howard's policy iteration algorithm using the language of forward-backward stochastic differential equations (FBSDEs). As opposed to conventional formulations based on partial differential equations, our FBSDE-based formulation can be easily implemented by optimizing criteria over sample data, and is therefore less sensitive to the state dimension. In particular, both on-policy and off-policy evaluation methods are discussed by constructing different FBSDEs. The backward-measurability-loss (BML) criterion is then proposed for solving these equations. By choosing specific weight functions in the proposed criterion, we can recover the popular Deep BSDE method or the martingale approach for BSDEs. The convergence results are established under both ideal and practical conditions, depending on whether the optimization criteria are decreased to zero. In the ideal case, we prove that the policy sequences produced by proposed FBSDE-based algorithms and the standard policy iteration have the same performance, and thus have the same convergence rate. In the practical case, the proposed algorithm is still proved to converge robustly under mild assumptions on optimization errors.

math.OC↗

arXiv2020

Kalman-Bucy filtering and minimum mean square estimator under uncertainty

In this paper, we study a generalized Kalman-Bucy filtering problem under uncertainty. The drift uncertainty for both signal process and observation process is considered and the attitude to uncertainty is characterized by a convex operator (convex risk measure). The optimal filter or the minimum mean square estimator (MMSE) is calculated by solving the minimum mean square estimation problem under a convex operator. In the first part of this paper, this estimation problem is studied under g-expectation which is a special convex operator. For this case, we prove that there exists a worst-case prior. Based on this worst-case prior we obtained the Kalman-Bucy filtering equation under g-expectation. In the second part of this paper, we study the minimum mean square estimation problem under general convex operators. The existence and uniqueness results of the MMSE are deduced.

math.OC↗

arXiv2020

Mean Field Linear Quadratic Control: Uniform Stabilization and Social Optimality

This paper is concerned with uniform stabilization and social optimality for general mean field linear quadratic control systems, where subsystems are coupled via individual dynamics and costs, and the state weight is not assumed with the definiteness condition. For the finite-horizon problem, we first obtain a set of forward-backward stochastic differential equations (FBSDEs) from variational analysis, and construct a feedback-type control by decoupling the FBSDEs. For the infinite-horizon problem, by using solutions to two Riccati equations, we design a set of decentralized control laws, which is further proved to be asymptotically social optimal. Some equivalent conditions are given for uniform stabilization of the systems in different cases, respectively. Finally, the proposed decentralized controls are compared to the asymptotic optimal strategies in previous works.

math.OC↗

arXiv2019

Social Optima in Robust Mean Field LQG Control: From Finite to Infinite Horizon

This paper studies social optimal control of mean field LQG (linear-quadratic-Gaussian) models with uncertainty. Specially, the uncertainty is represented by a uncertain drift which is common for all agents. A robust optimization approach is applied by assuming all agents treat the uncertain drift as an adversarial player. In our model, both dynamics and costs of agents are coupled by mean field terms, and both finite- and infinite-time horizon cases are considered. By examining social functional variation and exploiting person-by-person optimality principle, we construct an auxiliary control problem for the generic agent via a class of forward-backward stochastic differential equation system. By solving the auxiliary problem and constructing consistent mean field approximation, a set of decentralized control strategies is designed and shown to be asymptotically optimal.

math.OC↗

arXiv2018

Mixed Equilibrium Solution of Time-Inconsistent Stochastic LQ Problem

In this paper, we propose a novel equilibrium solution notion for the time-inconsistent stochastic linear-quadratic optimal control problem. This notion is called the mixed equilibrium solution, which consists of two parts: a pure-feedback-strategy part and an open-loop-control part. When the pure-feedback-strategy part is zero or the open-loop-control part does not depend on the initial state, the mixed equilibrium solution reduces to the open-loop equilibrium control and the feedback equilibrium strategy, respectively. Using a maximum-principle-like methodology with forward-backward stochastic difference equations, a necessary and sufficient condition is established to characterize the existence of a mixed equilibrium solution. Then, by decoupling the forward-backward stochastic difference equations, three sets of difference equations, which together portray the existence of a mixed equilibrium solution, are obtained. Moreover, the case with a fixed time-state initial pair and the case with all the initial pairs are separately investigated. Furthermore, an example is constructed to show that the mixed equilibrium solution exists for all the initial pairs, although neither the open-loop equilibrium control nor the feedback equilibrium strategy exists for some initial pairs.

math.OC↗

arXiv2018

Consensus conditions of continuous-time multi-agent systems with time-delays and measurement noises

This work is concerned with stochastic consensus conditions of multi-agent systems with both time-delays and measurement noises. For the case of additive noises, we develop some necessary conditions and sufficient conditions for stochastic weak consensus by estimating the differential resolvent function for delay equations. By the martingale convergence theorem, we obtain necessary conditions and sufficient conditions for stochastic strong consensus. For the case of multiplicative noises, we consider two kinds of time-delays, appeared in the measurement term and the noise term, respectively. We first show that stochastic weak consensus with the exponential convergence rate implies stochastic strong consensus. Then by constructing degenerate Lyapunov functional, we find the sufficient consensus conditions and show that stochastic consensus can be achieved by carefully choosing the control gain according to the noise intensities and the time-delay in the measurement term.

eess.SY↗

arXiv2018

Equilibrium Solutions of Multi-Period Mean-Variance Portfolio Selection

This is a companion paper of [Mixed equilibrium solution of time-inconsistent stochastic LQ problem, arXiv:1802.03032], where general theory has been established to characterize the open-loop equilibrium control, feedback equilibrium strategy and mixed equilibrium solution for a time-inconsistent stochastic linear-quadratic problem. This note is, on the one hand to test the developed theory of that paper, and on the other hand to push the solvability of multi-period mean-variance portfolio selection. A nondegenerate assumption has been removed in this note, which is popular in existing literature about multi-period mean-variance portfolio selection; and neat conditions have been obtained to characterize the existence of equilibrium solutions.

math.OC↗

arXiv2017

Delayed Optimal Control of Stochastic LQ Problem

A discrete-time stochastic LQ problem with multiplicative noises and state transmission delay is studied in this paper, which does not require any definiteness constraint on the cost weighting matrices. From some abstract representations of the system and cost functional, the solvability of this LQ problem is characterized by some conditions with operator form. Based on these, necessary and sufficient conditions are derived for the case with a fixed time-state initial pair and the general case with all the time-state initial pairs. For both cases, a set of coupled discrete-time Riccati-like equations can be derived to characterize the existence and the form of the delayed optimal control. In particular, for the general case with all the initial pairs, the existence of the delayed optimal control is equivalent to the solvability of the Riccati-like equations with some algebraic constraints, and both of them are also equivalent to the solvability of a set of coupled linear matrix equality-inequalities. Note that both the constrained Riccati-like equations and the linear matrix equality-inequalities are introduced for the first time in the literature for the proposed LQ problem. Furthermore, the convexity and the {uniform convexity} of the cost functional are fully characterized via certain properties of the solution of the Riccati-like equations.

math.OC↗

arXiv2017

On Time-Consistent Solution to Time-Inconsistent Linear-Quadratic Optimal Control of Discrete-Time Stochastic Systems

In this paper, we investigate a class of time-inconsistent discrete-time stochastic linear-quadratic optimal control problems, whose time-consistent solutions consist of an open-loop equilibrium control and a linear feedback equilibrium strategy. The open-loop equilibrium control is defined for a given initial pair, while the linear feedback equilibrium strategy is defined for all the initial pairs. Maximum-principle-type necessary and sufficient conditions containing stationary and convexity are derived for the existence of these two time-consistent solutions, respectively. Furthermore, for the case where the system matrices are independent of the initial time, we show that the existence of the open-loop equilibrium control for a given initial pair is equivalent to the solvability of a set of nonsymmetric generalized difference Riccati equations and a set of linear difference equations. Moreover, the existence of linear feedback equilibrium strategy is equivalent to the solvability of another set of symmetric generalized difference Riccati equations.

math.OC↗

arXiv2016

Time-Inconsistent Mean-Field Stochastic LQ Problem: Open-Loop Time-Consistent Control

This paper is concerned with the open-loop time-consistent solution of time-inconsistent mean-field stochastic linear-quadratic optimal control. Different from standard stochastic linear-quadratic problems, both the system matrices and the weighting matrices are dependent on the initial times, and the conditional expectations of the control and state enter quadratically into the cost functional. Such features will ruin Bellman's principle of optimality and result in the time-inconsistency of the optimal control. Based on the dynamical nature of the systems involved, a kind of open-loop time-consistent equilibrium control is investigated in this paper. It is shown that the existence of open-loop time-consistent equilibrium control for a fixed initial pair is equivalent to the solvability of a set of forward-backward stochastic difference equations with stationary conditions and convexity conditions. By decoupling the forward-backward stochastic difference equations, necessary and sufficient conditions in terms of linear difference equations and generalized difference Riccati equations are given for the existence of open-loop time-consistent equilibrium control with a fixed initial pair. Moreover, the existence of open-loop time-consistent equilibrium control for all the initial pairs is shown to be equivalent to the solvability of a set of coupled constrained generalized difference Riccati equations and two sets of constrained linear difference equations.

math.OC↗

arXiv2016

Coordination Over Multi-Agent Networks With Unmeasurable States and Finite-Level Quantization

In this note, the coordination of linear discrete-time multi-agent systems over digital networks is investigated with unmeasurable states in agents' dynamics. The quantized-observer based communication protocols and Certainty Equivalence principle based control protocols are proposed to characterize the inter-agent communication and the cooperative control in an integrative framework. By investigating the structural and asymptotic properties of the equations of stabilization and estimation errors nonlinearly coupled by the finite-level quantization scheme, some necessary conditions and sufficient conditions are given for the existence of such communication and control protocols to ensure the inter-agent state observation and cooperative stabilization. It is shown that these conditions come down to the simultaneous stabilizability and the detectability of the dynamics of agents and the structure of the communication network.

eess.SY↗

arXiv2014

Multi-Agent Consensus With Relative-State-Dependent Measurement Noises

In this note, the distributed consensus corrupted by relative-state-dependent measurement noises is considered. Each agent can measure or receive its neighbors' state information with random noises, whose intensity is a vector function of agents' relative states. By investigating the structure of this interaction and the tools of stochastic differential equations, we develop several small consensus gain theorems to give sufficient conditions in terms of the control gain, the number of agents and the noise intensity function to ensure mean square (m. s.) and almost sure (a. s.) consensus and quantify the convergence rate and the steady-state error. Especially, for the case with homogeneous communication and control channels, a necessary and sufficient condition to ensure m. s. consensus on the control gain is given and it is shown that the control gain is independent of the specific network topology, but only depends on the number of nodes and the noise coefficient constant. For symmetric measurement models, the almost sure convergence rate is estimated by the Iterated Logarithm Law of Brownian motions.

eess.SY↗