arXiv subjects

Iyanuoluwa Emiola

Publications and source records attributed to Iyanuoluwa Emiola.

5 recordsLinked to original sources

arXiv2022

Q-linear Convergence of Distributed Optimization with Barzilai-Borwein Step Sizes

The growth in sizes of large-scale systems and data in machine learning have made distributed optimization a naturally appealing technique to solve decision problems in different contexts. In such methods, each agent iteratively carries out computations on its local objective using information received from its neighbors, and shares relevant information with neighboring agents. Though gradient-based methods are widely used because of their simplicity, they are known to have slow convergence rates. On the other hand, though Newton-type methods have better convergence properties, they are not as applicable because of the enormous computation and memory requirements. In this work, we introduce a distributed quasi-Newton method with Barzilai-Borwein step-sizes. We prove a Q-linear convergence to the optimal solution, present conditions under which the algorithm is superlinearly convergent and validate our results via numerical simulations.

math.OC↗

arXiv2022

Quantized and Distributed Subgradient Optimization Method with Malicious Attack

This paper considers a distributed optimization problem in a multi-agent system where a fraction of the agents act in an adversarial manner. Specifically, the malicious agents steer the network of agents away from the optimal solution by sending false information to their neighbors and consume significant bandwidth in the communication process. We propose a distributed gradient-based optimization algorithm in which the non-malicious agents exchange quantized information with one another. We prove convergence of the solution to a neighborhood of the optimal solution, and characterize the solutions obtained under the communication-constrained environment and presence of malicious agents. Numerical simulations to illustrate the results are also presented.

math.OC↗

arXiv2021

Comparison of Minimization Methods for Rosenbrock Functions

This paper gives an in-depth review of the most common iterative methods for unconstrained optimization using two functions that belong to a class of Rosenbrock functions as a performance test. This study covers the Steepest Gradient Descent Method, the Newton-Raphson Method, and the Fletcher-Reeves Conjugate Gradient method. In addition, four different step-size selecting methods including fixed-step-size, variable step-size, quadratic-fit, and golden section method were considered. Due to the computational nature of solving minimization problems, testing the algorithms is an essential part of this paper. Therefore, an extensive set of numerical test results is also provided to present an insightful and a comprehensive comparison of the reviewed algorithms. This study highlights the differences and the trade-offs involved in comparing these algorithms.

math.OC↗

arXiv2021

Sublinear Regret with Barzilai-Borwein Step Sizes

This paper considers the online scenario using the Barzilai-Borwein Quasi-Newton Method. In an online optimization problem, an online agent uses a certain algorithm to decide on an objective at each time step after which a possible loss is encountered. Even though the online player will ideally try to make the best decisions possible at each time step, there is a notion of regret associated with the player's decisions. This study examines the regret of an online player using optimization methods like the Quasi-Newton methods, due to their fast convergent properties. The Barzilai-Borwein (BB) gradient method is chosen in this paper over other Quasi-Newton methods such as the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm because of its less computational complexities. In addition, the BB gradient method is suitable for large-scale optimization problems including the online optimization scenario presented in this paper. To corroborate the effectiveness of the Barzilai-Borwein (BB) gradient algorithm, a greedy online gradient algorithm is used in this study based on the two BB step sizes. Through a rigorous regret analysis on the BB step sizes, it is established that the regret obtained from the BB algorithm is sublinear in time. Moreover, this paper shows that the average regret using the online BB algorithm approaches zero without assuming strong convexity on the cost function to be minimized.

math.OC↗

arXiv2021

On Distributed Optimization in the Presence of Malicious Agents

In this paper, we consider an unconstrained distributed optimization problem over a network of agents, in which some agents are adversarial. We solve the problem via gradient-based distributed optimization algorithm and characterize the effect of the adversarial agents on the convergence of the algorithm to the optimal solution. The attack model considered is such that agents locally perturb their iterates before broadcasting it to neighbors; and we analyze the case in which the adversarial agents cooperate in perturbing their estimates and the case where each adversarial agent acts independently. Based on the attack model adopted in the paper, we show that the solution converges to the neighborhood of the optimal solution and depends on the magnitude of the attack (perturbation) term. The analyses presented establishes conditions under which the malicious agents have enough information to obstruct convergence to the optimal solution by the non-adversarial agents.

math.OC↗