Searcharxiv⌕ Search

arXiv subjects

Hicham Kouhkouh

Publications and source records attributed to Hicham Kouhkouh.

16 recordsLinked to original sources

Convergence of time-discrete finite particle consensus based optimization in Hilbert spaces

We study a time-discrete, finite-particle Consensus-Based Optimization (CBO) algorithm in a separable Hilbert space. Our analysis provides convergence guarantees directly for the computable particle system, complementing recent continuous-time and mean-field results in infinite dimensions. Using a common-noise formulation with trace-class covariance, we first establish quantitative pairwise contraction, exponential decay of the expected swarm variance, and almost-sure convergence of all particles to a common consensus state. We then combine estimates on the exponentiated objective functional with a quantitative Laplace principle to show that, for sufficiently large inverse temperature and suitably prepared initial data, the energy of the limiting consensus state can be made arbitrarily close to the global minimum over the active subspace. A key feature of the Hilbert-space formulation is that the stochastic contribution to the convergence estimates is controlled by the trace of the covariance operator and is therefore uniform with respect to the Galerkin dimension. Numerical experiments on an elliptic energy minimization problem with mixed boundary conditions and a PDE-constrained inverse source problem validate the theoretical convergence results and demonstrate stable performance under increasing spatial resolution, in contrast with CBO based on isotropic finite-dimensional noise.

math.OC↗

A derivative-free particle method for optimization in Hilbert spaces

We introduce a stochastic interacting particle system in separable Hilbert spaces together with its associated mean-field formulation. The model is shown to retain the characteristic consensus-driven structure of classical Consensus-Based Optimization, while accounting for the analytical challenges of infinite-dimensional dynamics. We establish well-posedness of the proposed dynamics and analyze the associated consensus mechanism. Furthermore, we derive convergence guarantees under suitable assumptions on the objective functional, showing concentration of the dynamics toward the minimizer in the long-time regime. This extends the applicability of the method to a broad class of infinite-dimensional optimization problems. In addition, we study the corresponding finite-particle system relevant for numerical implementation and propose a practical algorithm.

math.OC↗

Non-Convex Global Optimization as an Optimal Stabilization Problem: Convergence Rates

We develop a rigorous framework for global non-convex optimization by reformulating the minimization problem as a discounted infinite-horizon optimal control problem. For non-convex, continuous, and possibly non-smooth objective functions with multiple global minimizers, where classical gradient-based methods lack global convergence guarantees, we establish explicit exponential convergence rates with computable constants. Our analysis proves (i) variational convergence of the value function of the optimal control problem, (ii) convergence in the objective function for the original problem, as well as (iii) pathwise convergence of optimal trajectories to the minimizer set under minimal structural assumptions that require neither convexity, differentiability, nor Łojasiewicz-type conditions on the objective. These quantitative results significantly strengthen the asymptotic theory developed in our previous work (arXiv:2511.10815). Numerical experiments demonstrate the practical effectiveness of the approach on challenging non-convex problems.

math.OC↗

Faithful global convergence for the rescaled Consensus-Based Optimization

We analyze the Consensus-Based Optimization (CBO) algorithm with a consensus point rescaled by a small fixed parameter $κ\in (0,1)$. Under minimal assumptions on the objective function and the initial data, we establish its unconditional convergence to the global minimizer. Our results hold in the asymptotic regime where both the time--horizon $t \to \infty$ and the inverse--temperature $α\to \infty$, providing a rigorous theoretical foundation for the algorithm's global convergence. Furthermore, our findings extend to the case of multiple and non--discrete set of minimizers.

math.OC↗

Non-Convex Global Optimization as an Optimal Stabilization Problem: Dynamical Properties

We study global optimization of non-convex functions through optimal control theory. Our main result establishes that (quasi-)optimal trajectories of a discounted control problem converge globally and practically asymptotically to the set of global minimizers. Specifically, for any tolerance $η> 0$, there exist parameters $λ$ (discount rate) and $t$ (time horizon) such that trajectories remain within an $η$-neighborhood of the global minimizers after some finite time $τ$. This convergence is achieved directly, without solving ergodic Hamilton-Jacobi-Bellman equations. We prove parallel results for three problem formulations: evolutive discounted, stationary discounted, and evolutive non-discounted cases. The analysis relies on occupation measures to quantify the fraction of time trajectories spend away from the minimizer set, establishing both reachability and stability properties.

math.OC↗

Uniform-in-time mean-field limit estimate for the Consensus-Based Optimization

We establish a uniform-in-time estimate for the mean-field convergence of the Consensus-Based Optimization (CBO) algorithm by rescaling the consensus point in the dynamics with a small parameter $κ\in (0,1)$. This uniform-in-time estimate is essential, as CBO convergence relies on a sufficiently large time horizon and is crucial for ensuring stable, reliable long-term convergence, the latter being key to the practical effectiveness of CBO methods.

math.OC↗

A multiscale Consensus-Based algorithm for multi-level optimization

A novel multiscale consensus-based optimization (CBO) algorithm for solving bi- and tri-level optimization problems is introduced. Existing CBO techniques are generalized by the proposed method through the employment of multiple interacting populations of particles, each of which is used to optimize one level of the problem. These particle populations are evolved through multiscale-in-time dynamics, which are formulated as a singularly perturbed system of stochastic differential equations. Theoretical convergence analysis for the multiscale CBO model to an averaged effective dynamics as the time-scale separation parameter approaches zero is provided. The resulting algorithm is presented for both bi-level and tri-level optimization problems. The effectiveness of the approach in tackling complex multi-level optimization tasks is demonstrated through numerical experiments on various benchmark functions. Additionally, it is shown that the proposed method performs well on min-max optimization problems, comparing favorably with existing CBO algorithms for saddle point problems.

math.OC↗

Self-interacting CBO: Existence, uniqueness, and long-time convergence

A self-interacting dynamics that mimics the standard Consensus-Based Optimization (CBO) model is introduced. This single-particle dynamics is shown to converge to a unique invariant measure that approximates the global minimum of a given function. As an application, its connection to CBO with Personal Best introduced by C. Totzeck and M.-T. Wolfram (Math. Biosci. Eng., 2020) has been established.

math.OC↗

Relaxation and asymptotic expansion of controlled stiff differential equations

The control of relaxation-type systems of ordinary differential equations is investigated using the Hamilton-Jacobi-Bellman equation. First, we recast the model as a singularly perturbed dynamics which we embed in a family of controlled systems. Then we study this dynamics together with the value function of the associated optimal control problem. We provide an asymptotic expansion in the relaxation parameter of the value function. We also show that its solution converges toward the solution of a Hamilton-Jacobi-Bellman equation for a reduced control problem. Such systems are motivated by semi-discretisation of kinetic and hyperbolic partial differential equations. Several examples are presented including Jin-Xin relaxation.

math.OC↗

Long-time behaviour of deterministic Mean Field Games with non-monotone interactions

We consider deterministic Mean Field Games (MFG) in all Euclidean space with a cost functional continuous with respect to the distribution of the agents and attaining its minima in a compact set. We first show that the static MFG with such a cost has an equilibrium, and we build from it a solution of the ergodic MFG system of 1st order PDEs with the same cost. Next we address the long-time limit of the solutions to finite horizon MFG with cost functional satisfying various additional assumptions, but not the classical Lasry-Lions monotonicity condition. Instead we assume that the cost has the same set of minima for all measures describing the population. We prove the convergence of the distribution of the agents and of the value function to a solution of the ergodic MFG system as the horizon of the game tends to infinity, extending to this class of MFG some results of weak KAM theory.

math.AP↗

Deep Relaxation of Controlled Stochastic Gradient Descent via Singular Perturbations

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the Entropic Gradient Descent in the optimization of deep neural networks, via homogenisation. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton-Jacobi-Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls, and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modelled as dynamic controls.

math.OC↗

A viscous ergodic problem with unbounded and measurable ingredients. Part 1: HJB Equation

We address the problem of existence and uniqueness of solutions $(c,u(\cdot))$ to ergodic Hamilton-Jacobi-Bellman (HJB) equations of the form $H(x,\nabla u(x), D^{2}u(x)) = c$ in the whole space $\mathbb{R}^{m}$ with unbounded and merely measurable data and where $H$ is a Bellman Hamiltonian. The method we use is different from classical approaches. It relies on duality theory and optimization in abstract Banach spaces together with maximal dissipativity of the diffusion operator.

math.AP↗

A viscous ergodic problem with unbounded and measurable ingredients. Part 2: Mean-Field Games

We address the problem of existence and (non-)uniqueness of solutions $\big(c,u(\cdot),μ\big)$ to ergodic mean-field games in the whole space $\mathbb{R}^{m}$ with unbounded and merely measurable data, and for non-separable Hamiltonian. The payoff functional satisfies a new monotonicity condition, different from the usual one due to Lasry and Lions. The method we use is also different from classical approaches. It relies on duality theory and optimization in abstract Banach spaces together with maximal dissipativity of diffusion operators.

math.AP↗

Singular perturbations in stochastic optimal control with unbounded data

We study singular perturbations of a class of two-scale stochastic control systems with unbounded data. The assumptions are designed to cover some relaxation problems for deep neural networks. We construct effective Hamiltonian and initial data and prove the convergence of the value function to the solution of a limit (effective) Cauchy problem for a parabolic equation of HJB type. We use methods of probability, viscosity solutions and homogenization.

math.OC↗

An Eikonal equation with vanishing Lagrangian arising in Global Optimization

We show a connection between global unconstrained optimization of a continuous function $f$ and weak KAM theory for an eikonal-type equation arising also in ergodic control. A solution $v$ of the critical Hamilton-Jacobi equation is built by a small discount approximation as well as the long time limit of an associated evolutive equation. Then $v$ is represented as the value function of a control problem with target, whose optimal trajectories are driven by a differential inclusion describing the gradient descent of $v$. Such trajectories are proved to converge to the set of minima of $f$, using tools in control theory and occupational measures. We prove also that in some cases the set of minima is reached in finite time.

math.OC↗

The turnpike property and the long-time behavior of the Hamilton-Jacobi-Bellman equation for finite-dimensional LQ control problems

We analyze the consequences that the so-called turnpike property has on the long-time behavior of the value function corresponding to a finite-dimensional linear-quadratic optimal control problem with general terminal cost and constrained controls. We prove that, when the time horizon $T$ tends to infinity, the value function asymptotically behaves as $W(x) + c\, T + λ$, and we provide a control interpretation of each of these three terms, making clear the link with the turnpike property. As a by-product, we obtain the long-time behavior of the solution to the associated Hamilton-Jacobi-Bellman equation in a case where the Hamiltonian is not coercive in the momentum variable. As a result of independent interest, we showed that linear-quadratic optimal control problems with constrained control enjoy a turnpike property, also particularly when the steady optimum may saturate the control constraints.

math.AP↗