arXiv subjects

Doheon Kim

Publications and source records attributed to Doheon Kim.

13 recordsLinked to original sources

arXiv2026

Deterministic Dynamics of Sampling Processes in Score-Based Diffusion Models with Multiplicative Noise Conditioning

Score-based diffusion models generate new samples by learning the score function associated with a diffusion process. While the effectiveness of these models can be theoretically explained using differential equations related to the sampling process, previous work by Song and Ermon (2020) demonstrated that neural networks using multiplicative noise conditioning can still generate satisfactory samples. In this setup, the model is expressed as the product of two functions: one depending on the spatial variable and the other on the noise magnitude. This structure limits the model's ability to represent a more general relationship between the spatial variable and the noise, indicating that it cannot fully learn the correct score. Despite this limitation, the models perform well in practice. In this work, we provide a theoretical explanation for this phenomenon by studying the deterministic dynamics of the associated differential equations, offering insight into how the model operates.

cs.LG↗

arXiv2023

On the convergence result of the gradient-push algorithm on directed graphs with constant stepsize

Distributed optimization has recieved a lot of interest due to its wide applications in various fields. It consists of multiple agents that connected by a graph and optimize a total cost in a collaborative way. Often in the applications, the graph of the agents is given by a directed graph. The gradient-push algorithm is a fundamental method for distributed optimization for which the agents are connected by a directed graph. Despite of its wide usage in the literatures, its convergence property has not been established well for the important case that the stepsize is constant and the domain is the entire space. This work proves that the gradient-push algorithm with stepsize $\alpha>0$ converges exponentially fast to an $O(\alpha)$-neighborhood of the optimizer if the stepsize $\alpha$ is less than a specific value. For the result, we assume that each cost is smooth and the total cost is strongly convex. Numerical experiments are provided to support the theoretical convergence result. \textcolor{black}{We also present a numerical test showing that the gradient-push algorithm may approach a small neighborhood of the minimizer faster than the Push-DIGing algorithm which is a variant of the gradient-push algorithm involving the communication of the gradient informations of the agents.

math.OC↗

arXiv2021

Convergence results of a nested decentralized gradient method for non-strongly convex problems

We are concerned with the convergence of NEAR-DGD$^+$ (Nested Exact Alternating Recursion Distributed Gradient Descent) method introduced to solve the distributed optimization problems. Under the assumption of the strong convexity of local objective functions and the Lipschitz continuity of their gradients, the linear convergence is established in \cite{BBKW - Near DGD}. In this paper, we investigate the convergence property of NEAR-DGD$^+$ in the absence of strong convexity. More precisely, we establish the convergence results in the following two cases: (1) When only the convexity is assumed on the objective function. (2) When the objective function is represented as a composite function of a strongly convex function and a rank deficient matrix, which falls into the class of convex and quasi-strongly convex functions. Numerical results are provided to support the convergence results.

math.OC↗

arXiv2021

A unified framework for distributed optimization algorithms over time-varying directed graphs

In this paper, we propose a framework under which the decentralized optimization algorithms suggested in \cite{JKJJ,MA, NO,NO2} can be treated in a unified manner. More precisely, we show that the distributed subgradient descent algorithms \cite{JKJJ, NO}, the subgradient-push algorithm \cite{NO2}, and the distributed algorithm with row-stochastic matrix \cite{MA} can be derived by making suitable choices of consensus matrices, step-size and subgradient from the decentralized subgradient descent proposed in \cite{NO}. As a result of such unified understanding, we provide a convergence proof that covers the algorithms in \cite{JKJJ,MA, NO,NO2} under a novel algebraic condition that is strictly weaker than the conventional graph-theoretic condition in \cite{NO}. This unification also enables us to derive a new distributed optimization scheme.

math.OC↗

arXiv2021

Convergence analysis of the discrete consensus-based optimization algorithm with random batch interactions and heterogeneous noises

We present stochastic consensus and convergence of the discrete consensus-based optimization (CBO) algorithm with random batch interactions and heterogeneous external noises. Despite the wide applications and successful performance in many practical simulations, the convergence of the discrete CBO algorithm was not rigorously investigated in such a generality. In this work, we introduce a generalized discrete CBO algorithm with a weighted representative point and random batch interactions, and show that the proposed discrete CBO algorithm exhibits stochastic consensus and convergence toward the common equilibrium state exponentially fast under suitable assumptions on system parameters. For this, we recast the given CBO algorithm with random batch interactions as a discrete consensus model with a random switching network topology, and then we use the mixing property of interactions over sufficiently long time interval to derive stochastic consensus and convergence estimates in mean square and almost sure senses. Our proposed analysis significantly improves earlier works on the convergence analysis of CBO models with full batch interactions and homogeneous external noises.

math.OC↗

arXiv2021

Entropy production estimate for the ES-BGK model with the correct Prandtl number

In this paper, we establish the entropy-entropy production estimate for the ES-BGK model, a generalized version of the BGK model of the Boltzmann equation introduced for better approximation in the Navier-Stokes limit. Our result improves the previous entropy production estimate [39] in that (1) the full range of Prandtl parameters $-1/2\leq\nu <1$ including the critical case $\nu=-1/2$ is covered, and (2) a sharper entropy production bound is obtained. An explicit characterization of the coefficient of the entropy-entropy production estimate is also presented.

math.AP↗

arXiv2020

Convergence and error estimates for time-discrete consensus-based optimization algorithms

We present convergence and error estimates of the time-discrete consensus-based optimization(CBO) algorithms proposed in [arXiv:1909.09249] for general nonconvex functions. In authors' recent work [arxiv: 1910.08239], rigorous error analysis of the first-order consensus-based optimization algorithm proposed in [arXiv:1909.09249] was studied at the particle level without resorting to the kinetic equation via a mean-field limit. However, the error analysis for the corresponding time-discrete algorithm was not done mainly due to lack of discrete analogue of It\^o's stochastic calculus. In this paper, we provide a simple and elementary convergence and error analysis for a general time-discrete consensus-based optimization algorithm, which includes the three discrete algorithms in [arXiv:1909.09249]. Our analysis provides numerical stability and convergence conditions for the three algorithms, as well as error estimates to the global minimum.

math.OC↗

arXiv2020

Stationary BGK models for chemically reacting gas in a slab

We study the boundary value problem of two stationary BGK-type models - the BGK model for fast chemical reaction and the BGK model for slow chemical reaction - and provide a unified argument to establish the existence and uniqueness of stationary flows of reactive BGK models in a slab. For both models, the main difficulty arises in the uniform control of the reactive parameters from above and below, since, unlike the BGK models for non-reactive gases, the reactive parameters for the reactive BGK models are defined through highly nonlinear relations. To overcome this difficulty, we introduce several nonlinear functionals that capture essential structures of such nonlinear relations such as the monotonicity in specific variables, that enable one to derive necessary estimates for the reactive equilibrium coefficients.

math.AP↗

arXiv2019

Emergence of stochastic flocking for the discrete Cucker-Smale model with randomly switching topologies

We study emergent dynamics of the discrete Cucker-Smale (in short, DCS) model with randomly switching network topologies. For this, we provide a sufficient framework leading to the stochastic flocking with probability one. Our sufficient framework is formulated in terms of an admissible set of network topologies realized by digraphs and probability density function for random switching times. As examples for the law of switching times, we use the Poisson process and the geometric process and show that these two processes satisfy the required conditions in a given framework so that we have a stochastic flocking with probability one. As a corollary of our flocking analysis, we improve the earlier result [J.-G. Dong, S.-Y. Ha, J. Jung and D. Kim: On the stochastic flocking of the Cucker-Smale flock with randomly switching topologies. arXiv:1911.07390.] on the continuous C-S model.

math.DS↗

arXiv2019

On the stochastic flocking of the Cucker-Smale flock with randomly switching topologies

We present an emergent stochastic flocking dynamics of the Cucker-Smale (CS) ensemble under randomly switching topologies. The evolution of the CS ensemble with randomly switching topologies involves two random components (switching times and choices of network topologies at switching instant). First, we allow switching times for the network topology to be random so that the successive increments are i.i.d. processes following the common probability distribution. Second, at each switching instant, we choose a network topology randomly from a finite set of admissible network topologies whose union contains a spanning tree. Even for the fixed deterministic network topology, the CS ensemble may not exhibit a mono-cluster flocking depending on the initial data and the decay mode of the communication weight functions measuring the degree of interactions between particles. For the flocking dynamics of the CS ensemble with these two random components, we first use a priori conditions on the network topologies and uniform boundedness of position diameter, and derive the flocking estimates via matrix theory together with a priori conditions, and then replace the a priori condition for the position diameter by some suitable condition on the system parameters and communication weight. The a priori condition on the network topology will be guaranteed by the suitable spanning tree time-blocks with probability one.

math.OC↗

arXiv2019

Convergence of a first-order consensus-based global optimization algorithm

Global optimization of a non-convex objective function often appears in large-scale machine-learning and artificial intelligence applications. Recently, consensus-based optimization (in short CBO) methods have been introduced as one of the gradient-free optimization methods. In this paper, we provide a convergence analysis for the first-order CBO method in \cite{C-J-L-Z}. Prior to the current work, the convergence study was carried out for CBO methods on corresponding mean-field limit, a Fokker-Planck equation, which does not imply the convergence of the CBO method {\it per se}. Based on the consensus estimate directly on the first-order CBO model, we provide a convergence analysis of the first-order CBO method \cite{C-J-L-Z} without resorting to the corresponding mean-field model. Our convergence analysis consists of two steps. In the first step, we show that the CBO model exhibits a global consensus time asymptotically for any initial data, and in the second step, we provide a sufficient condition on system parameters--which is dimension independent-- and initial data which guarantee that the converged consensus state lies in a small neighborhood of the global minimum almost surely.

math.OC↗

arXiv2018

Emergent behaviors of continuous and discrete thermomechanical Cucker-Smale models on general digraphs

We present emergent dynamics of continuous and discrete thermomechanical Cucker-Smale(TCS) models equipped with temperature as an extra observable on general digraph. In previous literature, the emergent behaviors of the TCS models were mainly studied on a complete graph, or symmetric connected graphs. Under this symmetric setting, the total momentum is a conserved quantity. This determines the asymptotic velocity and temperature a priori using the initial data only. Moreover, this conservation law plays a crucial role in the flocking analysis based on the elementary $\ell_2$ energy estimates. In this paper, we consider a more general connection topology which is registered by a general digraph, and the weights between particles are given to be inversely proportional to the metric distance between them. Due to this possible symmetry breaking in communication, the total momentum is not a conserved quantity, and this lack of conservation law makes the asymptotic velocity and temperature depend on the whole history of solutions. To circumvent this lack of conservation laws, we instead employ some tools from matrix theory on the scrambling matrices and some detailed analysis on the state-transition matrices. We present two sufficient frameworks for the emergence of mono-cluster flockings on a digraph for the continuous and discrete models. Our sufficient frameworks are given in terms of system parameters and initial data.

math.CA↗

arXiv2018

Interplay of time-delay and velocity alignment in the Cucker-Smale model on a general digraph

We study dynamic interplay between time-delay and velocity alignment in the ensemble of Cucker-Smale (C-S) particles(or agents) on time-varying networks which are modeled by digraphs containing spanning trees. Time-delayed dynamical systems often appear in mathematical models from biology and control theory, and they have been extensively investigated in literature. In this paper, we provide sufficient frameworks for the mono-cluster flocking to the continuous and discrete C-S models, which are formulated in terms of system parameters and initial data. In our proposed frameworks, we show that the continuous and discrete C-S models exhibit exponential flocking estimates. For the explicit C-S communication weights which decay algebraically, our results exhibit threshold phenomena depending on the decay rate and depth of digraph. We also provide several numerical examples and compare them with our analytical results.

math.CA↗