Perpertual Coupled Simulated Annealing for Continuous Optimization

Abstract

Global optimization heuristics are popular to optimize hard non-convex problems. Despite their irrefutably large cost-to-solution, in the lack of other working greedy or convex approaches, global optimization algorithms remain the no-brainer choice. Nevertheless, successful use often requires tedious adjustments to initial parameters to avoid premature convergence. The Coupled Simulated Annealing approach proposed a method based on the coupling of multiple optimizers to escape premature convergence, having achieved success in optimizing hyperparameters of many applications of machine learning; however, a careful choice of the generation temperature is still required. In this paper we propose the Perpetual Orbit technique as a solution to control the generation temperature and avoid search stagnation. In principle, this technique can also be applied to other ensemble- and population-based algorithms that have a dispersion variable. The results of our experiments show superior performance when using the proposed technique because it makes the Couple Simulated Annealing totally parameter-free and capable of reaching equal or better solutions in more than 85\% cases across all functions and competitor methods analyzed.