arXiv subjects

Yiguang Hong

Publications and source records attributed to Yiguang Hong.

At least 91 records · Page 5Linked to original sources

arXiv2020

Input-Feedforward-Passivity-Based Distributed Optimization Over Jointly Connected Balanced Digraphs

In this paper, a distributed optimization problem is investigated via input feedforward passivity. First, an input-feedforward-passivity-based continuous-time distributed algorithm is proposed. It is shown that the error system of the proposed algorithm can be decomposed into a group of individual input feedforward passive (IFP) systems that interact with each other using output feedback information. Based on this IFP framework, convergence conditions of a suitable coupling gain are derived over weight-balanced and uniformly jointly strongly connected (UJSC) topologies. It is also shown that the IFP-based algorithm converges exponentially when the topology is strongly connected. Second, a novel distributed derivative feedback algorithm is proposed based on the passivation of IFP systems. While most works on directed topologies require knowledge of eigenvalues of the graph Laplacian, the derivative feedback algorithm is fully distributed, namely, it is robust against randomly changing weight-balanced digraphs with any positive coupling gain and without knowing any global information. Finally, numerical examples are presented to illustrate the proposed distributed algorithms.

math.OC↗

arXiv2020

Distributed Aggregative Optimization over Multi-Agent Networks

This paper proposes a new framework for distributed optimization, called distributed aggregative optimization, which allows local objective functions to be dependent not only on their own decision variables, but also on the average of summable functions of decision variables of all other agents. To handle this problem, a distributed algorithm, called distributed gradient tracking (DGT), is proposed and analyzed, where the global objective function is strongly convex, and the communication graph is balanced and strongly connected. It is shown that the algorithm can converge to the optimal variable at a linear rate. A numerical example is provided to corroborate the theoretical result.

math.OC↗

arXiv2020

Distributed Mirror Descent for Online Composite Optimization

In this paper, we consider an online distributed composite optimization problem over a time-varying multi-agent network that consists of multiple interacting nodes, where the objective function of each node consists of two parts: a loss function that changes over time and a regularization function. This problem naturally arises in many real-world applications ranging from wireless sensor networks to signal processing. We propose a class of online distributed optimization algorithms that are based on approximate mirror descent, which utilize the Bregman divergence as distance-measuring function that includes the Euclidean distances as a special case. We consider two standard information feedback models when designing the algorithms, that is, full-information feedback and bandit feedback. For the full-information feedback model, the first algorithm attains an average regularized regret of order $\mathcal{O}(1/\sqrt{T})$ with the total number of rounds $T$. The second algorithm, which only requires the information of the values of the loss function at two predicted points instead of the gradient information, achieves the same average regularized regret as that of the first algorithm. Simulation results of a distributed online regularized linear regression problem are provided to illustrate the performance of the proposed algorithms.

math.OC↗

arXiv2020

Linearly Convergent Algorithm with Variance Reduction for Distributed Stochastic Optimization

This paper considers a distributed stochastic strongly convex optimization, where agents connected over a network aim to cooperatively minimize the average of all agents' local cost functions. Due to the stochasticity of gradient estimation and distributedness of local objective, fast linearly convergent distributed algorithms have not been achieved yet. This work proposes a novel distributed stochastic gradient tracking algorithm with variance reduction, where the local gradients are estimated by an increasing batch-size of sampled gradients. With an undirected connected communication graph and a geometrically increasing batch-size, the iterates are shown to converge in mean to the optimal solution at a geometric rate (achieving linear convergence). The iteration, communication, and oracle complexity for obtaining an $ε$-optimal solution are established as well. Particulary, the communication complexity is $\mathcal{O}(\ln (1/ε))$ while the oracle complexity (number of sampled gradients) is $\mathcal{O}(1/ε^2)$, which is of the same order as that of centralized approaches. Hence, the proposed scheme is communication-efficient without requiring extra sampled gradients. Numerical simulations are given to demonstrate the theoretic results.

math.OC↗

arXiv2020

Distributed sub-optimal resource allocation via a projected form of singular perturbation

Distributed optimization for resource allocation problems is investigated and a sub-optimal continuous-time algorithm is proposed. Our algorithm has lower order dynamics than others to reduce burdens of computation and communication, and is applicable to weight-balanced graphs. Moreover, it can deal with both local set constraints and coupled inequality constraints, and remove the requirement of twice differentiability of the cost function in comparison with the existing sub-optimal algorithm. However, this algorithm is not easy to be analyzed since it involves singular perturbation type dynamics with projected non-differentiable right-hand side. We overcome the encountered difficulties and obtain results including the existence of an equilibrium, the sub-optimality, and the convergence of the algorithm.

math.OC↗

arXiv2020

Exponentially Convergent Algorithm Design for Constrained Distributed Optimization via Non-smooth Approach

We consider minimizing a sum of non-smooth objective functions with set constraints in a distributed manner. As to this problem, we propose a distributed algorithm with an exponential convergence rate for the first time. By the exact penalty method, we reformulate the problem equivalently as a standard distributed one without consensus constraints. Then we design a distributed projected subgradient algorithm with the help of differential inclusions. Furthermore, we show that the algorithm converges to the optimal solution exponentially for strongly convex objective functions.

math.OC↗

arXiv2019

Heterogeneous Hegselmann-Krause Dynamics with Environment and Communication Noise

The Hegselmann-Krause (HK) model is a wellknown opinion dynamics, attracting a significant amount of interest from a number of fields. However, the heterogeneous HK model is difficult to analyze - even the most basic property of convergence is still open to prove. For the first time, this paper takes into consideration heterogeneous HK models with environment or communication noise. Under environment noise, it has been revealed that the heterogeneous HK model with or without global information has a phase transition for the upper limit of the maximum opinion difference, and has a critical noise amplitude depending on the minimal confidence threshold for quasi-synchronization. In addition, the convergence time to the quasi-synchronization is bounded by a negative exponential distribution. The heterogeneous HK model with global information and communication noise is also analyzed. Finally, for the basic HK model with communication noise, we show that the heterogeneous case exhibits a different behavior regarding quasi-synchronization from the homogenous case. Interestingly, raising the confidence thresholds of constituent agents may break quasi-synchronization. Our results reveal that the heterogeneity of individuals is harmful to synchronization, which may be the reason why the synchronization of opinions is hard to reach in reality, even within that of a small group.

math.OC↗

arXiv2019

Network Flows that Solve Sylvester Matrix Equations

In this paper, we study distributed methods for solving a Sylvester equation in the form of AX+XB=C for matrices A, B, C$\in R^{n\times n}$ with X being the unknown variable. The entries of A, B and C (called data) are partitioned into a number of pieces (or sometimes we permit these pieces to overlap). Then a network with a given structure is assigned, whose number of nodes is consistent with the partition. Each node has access to the corresponding set of data and holds a dynamic state. Nodes share their states among their neighbors defined from the network structure, and we aim to design flows that can asymptotically converge to a solution of this equation. The decentralized data partitions may be resulted directly from networks consisting of physically isolated subsystems, or indirectly from artificial and strategic design for processing large data sets. Natural partial row/column partitions, full row/column partitions and clustering block partitions of the data A, B and C are assisted by the use of the vectorized matrix equation. We show that the existing "consensus + projection" flow and the "local conservation + global consensus" flow for distributed linear algebraic equations can be used to drive distributed flows that solve this kind of equations. A "consensus + projection + symmetrization" flow is also developed for equations with symmetry constraints on the solution matrices. We reveal some fundamental convergence rate limitations for such flows regardless of the choices of node interaction strengths and network structures. For a special case with B=A$^T$, where the equation mentioned is reduced to a classical Lyapunov equation, we demonstrate that by exploiting the symmetry of data, we can obtain flows with lower complexity for certain partitions.

math.OC↗

arXiv2019

Max-Min Fair Sensor Scheduling: Game-theoretic Perspective and Algorithmic Solution

We consider the design of a fair sensor schedule for a number of sensors monitoring different linear time-invariant processes. The largest average remote estimation error among all processes is to be minimized. We first consider a general setup for the max-min fair allocation problem. By reformulating the problem as its equivalent form, we transform the fair resource allocation problem into a zero-sum game between a "judge" and a resource allocator. We propose an equilibrium seeking procedure and show that there exists a unique Nash equilibrium in pure strategy for this game. We then apply the result to the sensor scheduling problem and show that the max-min fair sensor scheduling policy can be achieved.

eess.SY↗

arXiv2019

Distributed Computation for Solving the Sylvester Equation Based on Optimization

This paper solves the Sylvester equation in the form of AX+XB=C in a distributed way, and proposes three distributed continuous-time algorithms for three cases. We start with the basic algorithm for solving a least squares solution of the equation, and then give a simplified algorithm for the case when there is an exact solution to the equation, followed by an algorithm with regularization case. Based on local information and appropriate communication among neighbor agents, we solve the distributed computation problem of the Sylvester equation from the optimization viewpoint, and we prove the convergence of proposed algorithms to an optimal solution in three different cases, with help of the convex optimization and semi-stability.

math.OC↗

arXiv2019

Distributed Interval Optimization with Stochastic Zeroth-order Oracle

In this paper, we investigate a distributed interval optimization problem which is modeled with optimizing a sum of convex interval-valued objective functions subject to global convex constraints, corresponding to agents over a time-varying network. We first reformulate the distributed interval optimization problem as a distributed constrained optimization problem by scalarization. Then, we design a stochastic zeroth-order algorithm to solve the reformulated distributed problem, optimal solutions of which are also proved to be Pareto optimal solutions of the distributed interval optimization problem. Moreover, we construct the explicit convergence and the convergence rate in expectation of the given algorithm. Finally, a numerical example is given to illustrate the effectiveness of the proposed algorithm.

math.OC↗

arXiv2019

Distributed Kalman Filters with State Equality Constraints: Time-based and Event-triggered Communications

In this paper, we investigate a distributed estimation problem for multi-agent systems with state equality constraints (SEC). First, under a time-based consensus communication protocol, applying a modified projection operator and the covariance intersection fusion method, we propose a distributed Kalman filter with guaranteed consistency and satisfied SEC. Furthermore, we establish the relationship between consensus step, SEC and estimation error covariance in dynamic and steady processes, respectively. Employing a space decomposition method, we show the error covariance in the constraint set can be arbitrarily small by setting a sufficiently large consensus step. Besides, we propose an extended collective observability (ECO) condition based on SEC, which is milder than existing observability conditions. Under the ECO condition, through utilizing a technique of matrix approximation, we prove the boundedness of error covariance and the exponentially asymptotic unbiasedness of state estimate, respectively. Moreover, under the ECO condition for linear time-invariant systems with SEC, we provide a novel event-triggered communication protocol by employing the consistency, and give an offline design principle of triggering thresholds with guaranteed boundedness of error covariance. More importantly, we quantify and analyze the communication rate for the proposed event-triggered distributed Kalman filter, and provide optimization based methods to obtain the minimal (maximal) successive non-triggering (triggering) times. Two simulations are provided to demonstrate the developed theoretical results and the effectiveness of the filters.

eess.SY↗

arXiv2018

Optimal Distributed Stochastic Mirror Descent for Strongly Convex Optimization

In this paper we consider convergence rate problems for stochastic strongly-convex optimization in the non-Euclidean sense with a constraint set over a time-varying multi-agent network. We propose two efficient non-Euclidean stochastic subgradient descent algorithms based on the Bregman divergence as distance-measuring function rather than the Euclidean distances that were employed by the standard distributed stochastic projected subgradient algorithms. For distributed optimization of nonsmooth and strongly convex functions whose only stochastic subgradients are available, the first algorithm recovers the best previous known rate of O(ln(T)/T) (where T is the total number of iterations). The second algorithm is an epoch variant of the first algorithm that attains the optimal convergence rate of O(1/T), matching that of the best previously known centralized stochastic subgradient algorithm. Finally, we report some simulation results to illustrate the proposed algorithms.

math.OC↗

arXiv2018

Optimal Output Consensus of High-Order Multi-Agent Systems with Embedded Technique

In this paper, we study an optimal output consensus problem for a multi-agent network with agents in the form of multi-input multi-output minimum-phase dynamics. Optimal output consensus can be taken as an extended version of the existing output consensus problem for higher-order agents with an optimization requirement, where the output variables of agents are driven to achieve a consensus on the optimal solution of a global cost function. To solve this problem, we first construct an optimal signal generator, and then propose an embedded control scheme by embedding the generator in the feedback loop. We give two kinds of algorithms based on different available information along with both state feedback and output feedback, and prove that these algorithms with the embedded technique can guarantee the solvability of the problem for high-order multi-agent systems under standard assumptions.

math.OC↗

arXiv2018

Subgradient-Free Stochastic Optimization Algorithm for Non-smooth Convex Functions over Time-Varying Networks

In this paper we consider a distributed stochastic optimization problem without the gradient/subgradient information for the local objective functions, subject to local convex constraints. The objective functions may be non-smooth and observed with stochastic noises, and the network for the distributed design is time-varying. By adding the stochastic dithers into the local objective functions and constructing the randomized differences motivated by the Kiefer-Wolfowitz algorithm, we propose a distributed subgradient-free algorithm to find the global minimizer with local observations. Moreover, we prove that the consensus of estimates and global minimization can be achieved with probability one over the time-varying network, and then obtain the convergence rate of the mean average of estimates as well. Finally, we give a numerical example to illustrate the effectiveness of the proposed algorithm.

eess.SY↗

arXiv2018

Distributed Computation of Linear Matrix Equations: An Optimization Perspective

This paper investigates the distributed computation of the well-known linear matrix equation in the form of $AXB = F$, with the matrices A, B, X, and F of appropriate dimensions, over multi-agent networks from an optimization perspective. In this paper, we consider the standard distributed matrix-information structures, where each agent of the considered multi-agent network has access to one of the sub-block matrices of A, B, and F. To be specific, we first propose different decomposition methods to reformulate the matrix equations in standard structures as distributed constrained optimization problems by introducing substitutional variables; we show that the solutions of the reformulated distributed optimization problems are equivalent to least squares solutions to original matrix equations; and we design distributed continuous-time algorithms for the constrained optimization problems, even by using augmented matrices and a derivative feedback technique. Moreover, we prove the exponential convergence of the algorithms to a least squares solution to the matrix equation for any initial condition.

math.OC↗

arXiv2018

An Intrinsic Approach to Formation Control of Regular Polyhedra for Reduced Attitudes

This paper addresses formation control of reduced attitudes in which a continuous control protocol is proposed for achieving and stabilizing all regular polyhedra (also known as Platonic solids) under a unified framework. The protocol contains only relative reduced attitude measurements and does not depend on any particular parametrization as is usually used in the literature. A key feature of the control proposed is that it is intrinsic in the sense that it does not need to incorporate any information of the desired formation. Instead, the achieved formation pattern is totally attributed to the geometric properties of the space and the designed inter-agent connection topology. Using a novel coordinates transformation, asymptotic stability of the desired formations is proven by studying stability of a constrained nonlinear system. In addition, a methodology to investigate stability of such constrained systems is also presented.

math.OC↗

arXiv2017

Distributed Observers Design for Leader-Following Control of Multi-Agent Networks (Extended Version)

This paper is concerned with a leader-follower problem for a multi-agent system with a switching interconnection topology. Distributed observers are designed for the second-order follower-agents, under the common assumption that the velocity of the active leader cannot be measured in real time. Some dynamic neighbor-based rules, consisting of distributed controllers and observers for the autonomous agents, are developed to keep updating the information of the leader. With the help of an explicitly constructed common Lyapunov function (CLF), it is proved that each agent can follow the active leader. Moreover, the tracking error is estimated even in a noisy environment. Finally, a numerical example is given for illustration.

math.OC↗