Gradient-free optimization via integration

Abstract

We develop and analyse an approach to optimize functions $l\colon \mathbb{R}^d \rightarrow \mathbb{R}$ not assumed to be convex, differentiable or even continuous. The algorithm belongs to the class of model-based search methods. The idea is to fit recursively $l$ to a parametric family of distributions, using a Bayesian update followed by a reprojection back onto the chosen family. Remarkably, reprojection in our scenario boils down to computing expectations, which can be simply approximated through Monte Carlo. We show that when the family of distributions is appropriately chosen this approach can be interpreted as an implicit time-inhomogeneous gradient descent algorithm on a sequence of smoothed approximations of $l$, providing a route to establishing convergence. We establish new results for generic inhomogeneous gradient descent algorithms, which we specialise to the model-based search algorithm in the Gaussian scenario. We illustrate the performance of the algorithm on a challenging classification task in machine learning.