Searcharxiv⌕ Search

arXiv subjects

Woocheol Choi

Publications and source records attributed to Woocheol Choi.

At least 19 recordsLinked to original sources
arXiv2025

A Linear Convergence Result for the Jacobi-Proximal Alternating Direction Method of Multipliers

In this paper, we analyze the convergence rate of the Jacobi-Proximal Alternating Direction Method of Multipliers (ADMM) initially introduced by Deng et al. for the block-structured optimization problem with linear constraint. The algorithm is well-suited for parallel implementation and widely used for large-scale multi-block optimization problems. While the o(1/k) convergence of the Jacobi-Proximal ADMM for the case $N \geq 3$ has been well-established in the previous work, to the best of our knowledge, its linear convergence for $N \geq 3$ remains unproven. We establish the linear convergence of the algorithm when the cost functions are strongly convex and smooth. Numerical experiments are presented supporting the convergence result.

math.OC↗
arXiv2025

Adaptive Consensus with Exponential Decay

This paper addresses the adaptive consensus problem in uncertain multi-agent systems, particularly under challenges posed by quantized communication. We consider agents with general linear dynamics subject to nonlinear uncertainties and propose an adaptive consensus control framework that integrates concurrent learning. Unlike traditional methods relying solely on instantaneous data, concurrent learning leverages stored historical data to enhance parameter estimation without requiring persistent excitation. We establish that the proposed controller ensures exponential convergence of both consensus and parameter estimation. Furthermore, we extend the analysis to scenarios where inter-agent communication is quantized using a uniform quantizer. We prove that the system still achieves consensus up to an error proportional to the quantization level, with exponential convergence rate.

math.OC↗
arXiv2025

On the convergence result of the gradient-push algorithm on directed graphs with constant stepsize

Distributed optimization has recieved a lot of interest due to its wide applications in various fields. It consists of multiple agents that connected by a graph and optimize a total cost in a collaborative way. Often in the applications, the graph of the agents is given by a directed graph. The gradient-push algorithm is a fundamental method for distributed optimization for which the agents are connected by a directed graph. Despite of its wide usage in the literatures, its convergence property has not been established well for the important case that the stepsize is constant and the domain is the entire space. This work proves that the gradient-push algorithm with stepsize $α>0$ converges exponentially fast to an $O(α)$-neighborhood of the optimizer if the stepsize $α$ is less than a specific value. For the result, we assume that each cost is smooth and the total cost is strongly convex. Numerical experiments are provided to support the theoretical convergence result. \textcolor{black}{We also present a numerical test showing that the gradient-push algorithm may approach a small neighborhood of the minimizer faster than the Push-DIGing algorithm which is a variant of the gradient-push algorithm involving the communication of the gradient informations of the agents.

math.OC↗
arXiv2024

Convergence result for the gradient-push algorithm and its application to boost up the Push-DIging algorithm

The gradient-push algorithm is a fundamental algorithm for the distributed optimization problem \begin{equation} \min_{x \in \mathbb{R}^d} f(x) = \sum_{j=1}^n f_j (x), \end{equation} where each local cost $f_j$ is only known to agent $a_i$ for $1 \leq i \leq n$ and the agents are connected by a directed graph. In this paper, we obtain convergence results for the gradient-push algorithm with constant stepsize whose range is sharp in terms the order of the smoothness constant $L>0$. Precisely, under the two settings: 1) Each local cost $f_i$ is strongly convex and $L$-smooth, 2) Each local cost $f_i$ is convex quadratic and $L$-smooth while the aggregate cost $f$ is strongly convex, we show that the gradient-push algorithm with stepsize $α>0$ converges to an $O(α)$-neighborhood of the minimizer of $f$ for a range $α\in (0, c/L]$ with a value $c>0$ independent of $L>0$. As a benefit of the result, we suggest a hybrid algorithm that performs the gradient-push algorithm with a relatively large stepsize $α>0$ for a number of iterations and then go over to perform the Push-DIGing algorithm. It is verified by a numerical test that the hybrid algorithm enhances the performance of the Push-DIGing algorithm significantly. The convergence results of the gradient-push algorithm are also supported by numerical tests.

math.OC↗
arXiv2024

On the convergence analysis of the decentralized projected gradient descent method

In this work, we are concerned with the decentralized optimization problem: \begin{equation*} \min_{x \in Ω}~f(x) = \frac{1}{n} \sum_{i=1}^n f_i (x), \end{equation*} where $Ω\subset \mathbb{R}^d$ is a convex domain and each $f_i : Ω\rightarrow \mathbb{R}$ is a local cost function only known to agent $i$. A fundamental algorithm is the decentralized projected gradient method (DPG) given by \begin{equation*} x_i(t+1)=\mathcal{P}_Ω\Big[\sum^n_{j=1}w_{ij} x_j(t) -α(t)\nabla f_i(x_i(t))\Big] \end{equation*} where $\mathcal{P}_Ω$ is the projection operator to $Ω$ and $ \{w_{ij}\}_{1\leq i,j \leq n}$ are communication weight among the agents. While this method has been widely used in the literature, its convergence property has not been established so far, except for the special case $Ω= \mathbb{R}^n$. This work establishes new convergence estimates of DPG when the aggregate cost $f$ is strongly convex and each function $f_i$ is smooth. If the stepsize is given by constant $α(t) \equivα>0$ and suitably small, we prove that each $x_i (t)$ converges to an $O(\sqrtα)$-neighborhood of the optimal point. In addition, we further improve the convergence result by showing that the point $x_i (t)$ converges to an $O(α)$-neighborhood of the optimal point if the domain is given the half-space $\mathbb{R}^{d-1}\times \mathbb{R}_{+}$ for any dimension $d\in \mathbb{N}$. Also, we obtain new convergence results for decreasing stepsizes. Numerical experiments are provided to support the convergence results.

math.OC↗
arXiv2024

Non-ergodic linear convergence property of the delayed gradient descent under the strongly convexity and the Polyak-Łojasiewicz condition

In this work, we establish the linear convergence estimate for the gradient descent involving the delay $τ\in\mathbb{N}$ when the cost function is $μ$-strongly convex and $L$-smooth. This result improves upon the well-known estimates in Arjevani et al. \cite{ASS} and Stich-Karmireddy \cite{SK} in the sense that it is non-ergodic and is still established in spite of weaker constraint of cost function. Also, the range of learning rate $η$ can be extended from $η\leq 1/(10Lτ)$ to $η\leq 1/(4Lτ)$ for $τ=1$ and $η\leq 3/(10Lτ)$ for $τ\geq 2$, where $L >0$ is the Lipschitz continuity constant of the gradient of cost function. In a further research, we show the linear convergence of cost function under the Polyak-Łojasiewicz\,(PL) condition, for which the available choice of learning rate is further improved as $η\leq 9/(10Lτ)$ for the large delay $τ$. The framework of the proof for this result is also extended to the stochastic gradient descent with time-varying delay under the PL condition. Finally, some numerical experiments are provided in order to confirm the reliability of the analyzed results.

math.OC↗
arXiv2023

On the convergence of the distributed proximal point algorithm

In this work, we establish convergence results for the distributed proximal point algorithm (DPPA) for distributed optimization problems. We consider the problem on the whole domain Rd and find a general condition on the stepsize and cost functions such that the DPPA is stable. We prove that the DPPA with stepsize $η> 0$ exponentially converges to an $O(η)$-neighborhood of the optimizer. Our result clearly explains the advantage of the DPPA with respect to the convergence and stability in comparison with the distributed gradient descent algorithm. We also provide numerical tests supporting the theoretical results.

math.OC↗
arXiv2023

Convergence property of the Quantized Distributed Gradient descent with constant stepsizes and an effective strategy for the stepsize selection

In this paper, we establish new convergence results for the quantized distributed gradient descent and suggest a novel strategy of choosing the stepsizes for the high-performance of the algorithm. Under the strongly convexity assumption on the aggregate cost function and the smoothness assumption on each local cost function, we prove the algorithm converges exponentially fast to a small neighborhood of the optimizer whose radius depends on the stepsizes. Based on our convergence result, we suggest an effective selection of stepsizes which repeats diminishing the stepsizes after a number of specific iterations. Both the convergence results and the effectiveness of the suggested stepsize selection are also verified by the numerical experiments.

math.OC↗
arXiv2023

Inexact Online Proximal Mirror Descent for time-varying composite optimization

In this paper, we consider the online proximal mirror descent for solving the time-varying composite optimization problems. For various applications, the algorithm naturally involves the errors in the gradient and proximal operator. We obtain sharp estimates on the dynamic regret of the algorithm when the regular part of the cost is convex and smooth. If the Bregman distance is given by the Euclidean distance, our result also improves the previous work in two ways: (i) We establish a sharper regret bound compared to the previous work in the sense that our estimate does not involve $O(T)$ term appearing in that work. (ii) We also obtain the result when the domain is the whole space $\mathbb{R}^n$, whereas the previous work was obtained only for bounded domains. We also provide numerical tests for problems involving the errors in the gradient and proximal operator.

math.OC↗
arXiv2023

A tight bound on the stepsize of the decentralized gradient descent

In this paper, we consider the decentralized gradinet descent (DGD) given by \begin{equation*} x_i (t+1) = \sum_{j=1}^m w_{ij} x_j (t) - α(t) \nabla f_i (x_i (t)). \end{equation*} We find a sharp range of the stepsize $α(t)>0$ such that the sequence $\{x_i (t)\}$ is uniformly bounded when the aggregate cost $f$ is assumed be strongly convex with smooth local costs which might be non-convex. Precisely, we find a tight bound $α_0 >0$ such that the states of the DGD algorithm is uniformly bounded for non-increasing sequence $α(t)$ satisfying $α(0) \leq α_0$. The theoretical results are also verified by numerical experiments.

math.OC↗
arXiv2023

Convergence analysis of the splitting method to the nonlinear heat equation

In this paper, we analyze an operator splitting scheme of the nonlinear heat equation in $Ω\subset\mathbb{R}^d$ ($d\geq 1$): $\partial_t u = Δu + λ|u|^{p-1} u$ in $Ω\times(0,\infty)$, $u=0$ in $\partialΩ\times(0,\infty)$, $u ({\bf x},0) =ϕ({\bf x})$ in $Ω$. where $λ\in\{-1,1\}$ and $ϕ\in W^{1,q}(Ω)\cap L^{\infty} (Ω)$ with $2\leq p < \infty$ and $d(p-1)/2 0$. Finally, we give some numerical examples to confirm the reliability of the analyzed result.

math.NA↗
arXiv2022

Convergence results of a nested decentralized gradient method for non-strongly convex problems

We are concerned with the convergence of NEAR-DGD$^+$ (Nested Exact Alternating Recursion Distributed Gradient Descent) method introduced to solve the distributed optimization problems. Under the assumption of the strong convexity of local objective functions and the Lipschitz continuity of their gradients, the linear convergence is established in \cite{BBKW - Near DGD}. In this paper, we investigate the convergence property of NEAR-DGD$^+$ in the absence of strong convexity. More precisely, we establish the convergence results in the following two cases: (1) When only the convexity is assumed on the objective function. (2) When the objective function is represented as a composite function of a strongly convex function and a rank deficient matrix, which falls into the class of convex and quasi-strongly convex functions. Numerical results are provided to support the convergence results.

math.OC↗
arXiv2022

On the convergence of decentralized gradient descent with diminishing stepsize, revisited

Distributed optimization has received a lot of interest in recent years due to its wide applications in various fields. In this work, we revisit the convergence property of the decentralized gradient descent [A. Nedi{ć}-A.Ozdaglar (2009)] on the whole space given by $$ x_i(t+1) = \sum^m_{j=1}w_{ij}x_j(t) - α(t) \nabla f_i(x_i(t)), $$ where the stepsize is given as $α(t) = \frac{a}{(t+w)^p}$ with $0< p\leq 1$. Under the strongly convexity assumption on the total cost function $f$ with local cost functions $f_i$ not necessarily being convex, we show that the sequence converges to the optimizer with rate $O(t^{-p})$ when the values of $a>0$ and $w>0$ are suitably chosen.

math.OC↗
arXiv2021

Event-triggered bipartite consensus for multiagent system with general linear dynamics: an integral-type event triggered control

In this paper, we propose an integral based event-triggering controller for bipartite consensus of the multi-agent systems whose dynamics are described by general linear system. We prove that the system achieves the bipartite consensus in asymptotic regime and there is a positive minimum inter-event time (MIET) between two consecutive triggering times of each agent. The proof of the asymptotive stability involves a novel argument used to bound the norm of the difference between the true state and its estimated state (for each time) by an integration of its square. Numerical results are provided supporting the effectiveness of the proposed controller.

math.OC↗
arXiv2021

Gradient-push algorithm for distributed optimization with event-triggered communications

Decentralized optimization problems consist of multiple agents connected by a network. The agents have each local cost function, and the goal is to minimize the sum of the functions cooperatively. It requires the agents communicate with each other, and reducing the cost for communication is desired for a communication-limited environment. In this work, we propose a gradient-push algorithm involving event-triggered communication on directed network. Each agent sends its state information to its neighbors only when the difference between the latest sent state and the current state is larger than a threshold. The convergence of the algorithm is established under a decay and a summability condition on a stepsize and a triggering threshold. Numerical experiments are presented to support the effectiveness and the convergence results of the algorithm.

math.OC↗
arXiv2021

A unified framework for distributed optimization algorithms over time-varying directed graphs

In this paper, we propose a framework under which the decentralized optimization algorithms suggested in \cite{JKJJ,MA, NO,NO2} can be treated in a unified manner. More precisely, we show that the distributed subgradient descent algorithms \cite{JKJJ, NO}, the subgradient-push algorithm \cite{NO2}, and the distributed algorithm with row-stochastic matrix \cite{MA} can be derived by making suitable choices of consensus matrices, step-size and subgradient from the decentralized subgradient descent proposed in \cite{NO}. As a result of such unified understanding, we provide a convergence proof that covers the algorithms in \cite{JKJJ,MA, NO,NO2} under a novel algebraic condition that is strictly weaker than the conventional graph-theoretic condition in \cite{NO}. This unification also enables us to derive a new distributed optimization scheme.

math.OC↗
arXiv2021

Semi-classical limit of quantum free energy minimizers for the gravitational Hartree equation

For the gravitational Vlasov-Poisson equation, Guo and Rein constructed a class of classical isotropic states as minimizers of free energies (or energy-Casimir functionals) under mass constraints. For the quantum counterpart, that is, the gravitational Hartree equation, isotropic states are constructed as free energy minimizers by Aki, Dolbeault and Sparber. In this paper, we are concerned with the correspondence between quantum and classical isotropic states. Precisely, we prove that as the Planck constant $\hbar$ goes to zero, free energy minimizers for the Hartree equation converge to those for the Vlasov-Poisson equation in terms of potential functions as well as via the Husimi transform and the Töplitz quantization.

math.AP↗