arXiv subjects

Carsten Witt

Publications and source records attributed to Carsten Witt.

35 records · Page 2Linked to original sources

arXiv2020

Improved Runtime Results for Simple Randomised Search Heuristics on Linear Functions with a Uniform Constraint

In the last decade remarkable progress has been made in development of suitable proof techniques for analysing randomised search heuristics. The theoretical investigation of these algorithms on classes of functions is essential to the understanding of the underlying stochastic process. Linear functions have been traditionally studied in this area resulting in tight bounds on the expected optimisation time of simple randomised search algorithms for this class of problems. Recently, the constrained version of this problem has gained attention and some theoretical results have also been obtained on this class of problems. In this paper we study the class of linear functions under uniform constraint and investigate the expected optimisation time of Randomised Local Search (RLS) and a simple evolutionary algorithm called (1+1) EA. We prove a tight bound of $Θ(n^2)$ for RLS and improve the previously best known upper bound of (1+1) EA from $O(n^2 \log (Bw_{\max}))$ to $O(n^2\log B)$ in expectation and to $O(n^2 \log n)$ with high probability, where $w_{\max}$ and $B$ are the maximum weight of the linear objective function and the bound of the uniform constraint, respectively. Also, we obtain a tight bound of $O(n^2)$ for the (1+1) EA on a special class of instances. We complement our theoretical studies by experimental investigations that consider different values of $B$ and also higher mutation rates that reflect the fact that $2$-bit flips are crucial for dealing with the uniform constraint.

cs.NE↗

arXiv2020

Evolutionary Algorithms with Self-adjusting Asymmetric Mutation

Evolutionary Algorithms (EAs) and other randomized search heuristics are often considered as unbiased algorithms that are invariant with respect to different transformations of the underlying search space. However, if a certain amount of domain knowledge is available the use of biased search operators in EAs becomes viable. We consider a simple (1+1) EA for binary search spaces and analyze an asymmetric mutation operator that can treat zero- and one-bits differently. This operator extends previous work by Jansen and Sudholt (ECJ 18(1), 2010) by allowing the operator asymmetry to vary according to the success rate of the algorithm. Using a self-adjusting scheme that learns an appropriate degree of asymmetry, we show improved runtime results on the class of functions OneMax$_a$ describing the number of matching bits with a fixed target $a\in\{0,1\}^n$.

cs.NE↗

arXiv2020

Improved Fixed-Budget Results via Drift Analysis

Fixed-budget theory is concerned with computing or bounding the fitness value achievable by randomized search heuristics within a given budget of fitness function evaluations. Despite recent progress in fixed-budget theory, there is a lack of general tools to derive such results. We transfer drift theory, the key tool to derive expected optimization times, to the fixed-budged perspective. A first and easy-to-use statement concerned with iterating drift in so-called greed-admitting scenarios immediately translates into bounds on the expected function value. Afterwards, we consider a more general tool based on the well-known variable drift theorem. Applications of this technique to the LeadingOnes benchmark function yield statements that are more precise than the previous state of the art.

cs.NE↗

arXiv2020

Self-Adjusting Evolutionary Algorithms for Multimodal Optimization

Recent theoretical research has shown that self-adjusting and self-adaptive mechanisms can provably outperform static settings in evolutionary algorithms for binary search spaces. However, the vast majority of these studies focuses on unimodal functions which do not require the algorithm to flip several bits simultaneously to make progress. In fact, existing self-adjusting algorithms are not designed to detect local optima and do not have any obvious benefit to cross large Hamming gaps. We suggest a mechanism called stagnation detection that can be added as a module to existing evolutionary algorithms (both with and without prior self-adjusting algorithms). Added to a simple (1+1) EA, we prove an expected runtime on the well-known Jump benchmark that corresponds to an asymptotically optimal parameter setting and outperforms other mechanisms for multimodal optimization like heavy-tailed mutation. We also investigate the module in the context of a self-adjusting (1+$λ$) EA and show that it combines the previous benefits of this algorithm on unimodal problems with more efficient multimodal optimization. To explore the limitations of the approach, we additionally present an example where both self-adjusting mechanisms, including stagnation detection, do not help to find a beneficial setting of the mutation rate. Finally, we investigate our module for stagnation detection experimentally.

cs.NE↗

arXiv2019

Sharp Bounds on the Runtime of the (1+1) EA via Drift Analysis and Analytic Combinatorial Tools

The expected running time of the classical (1+1) EA on the OneMax benchmark function has recently been determined by Hwang et al. (2018) up to additive errors of $O((\log n)/n)$. The same approach proposed there also leads to a full asymptotic expansion with errors of the form $O(n^{-K}\log n)$ for any $K>0$. This precise result is obtained by matched asymptotics with rigorous error analysis (or by solving asymptotically the underlying recurrences via inductive approximation arguments), ideas radically different from well-established techniques for the running time analysis of evolutionary computation such as drift analysis. This paper revisits drift analysis for the (1+1) EA on OneMax and obtains that the expected running time $E(T)$, starting from $\lceil n/2\rceil$ one-bits, is determined by the sum of inverse drifts up to logarithmic error terms, more precisely $$\sum_{k=1}^{\lfloor n/2\rfloor}\frac{1}{Δ(k)} - c_1\log n \le E(T) \le \sum_{k=1}^{\lfloor n/2\rfloor}\frac{1}{Δ(k)} - c_2\log n,$$ where $Δ(k)$ is the drift (expected increase of the number of one-bits from the state of $n-k$ ones) and $c_1,c_2 >0$ are explicitly computed constants. This improves the previous asymptotic error known for the sum of inverse drifts from $\tilde{O}(n^{2/3})$ to a logarithmic error and gives for the first time a non-asymptotic error bound. Using standard asymptotic techniques, the difference between $E(T)$ and the sum of inverse drifts is found to be $(e/2)\log n+O(1)$.

cs.NE↗

arXiv2018

Runtime Analysis for Self-adaptive Mutation Rates

We propose and analyze a self-adaptive version of the $(1,λ)$ evolutionary algorithm in which the current mutation rate is part of the individual and thus also subject to mutation. A rigorous runtime analysis on the OneMax benchmark function reveals that a simple local mutation scheme for the rate leads to an expected optimization time (number of fitness evaluations) of $O(nλ/\logλ+n\log n)$ when $λ$ is at least $C \ln n$ for some constant $C > 0$. For all values of $λ\ge C \ln n$, this performance is asymptotically best possible among all $λ$-parallel mutation-based unbiased black-box algorithms. Our result shows that self-adaptation in evolutionary computation can find complex optimal parameter settings on the fly. At the same time, it proves that a relatively complicated self-adjusting scheme for the mutation rate proposed by Doerr, Gießen, Witt, and Yang~(GECCO~2017) can be replaced by our simple endogenous scheme. On the technical side, the paper contributes new tools for the analysis of two-dimensional drift processes arising in the analysis of dynamic parameter choices in EAs, including bounds on occupation probabilities in processes with non-constant drift.

cs.NE↗

arXiv2018

Theory of Estimation-of-Distribution Algorithms

Estimation-of-distribution algorithms (EDAs) are general metaheuristics used in optimization that represent a more recent alternative to classical approaches like evolutionary algorithms. In a nutshell, EDAs typically do not directly evolve populations of search points but build probabilistic models of promising solutions by repeatedly sampling and selecting points from the underlying search space. Recently, there has been made significant progress in the theoretical understanding of EDAs. This article provides an up-to-date overview of the most commonly analyzed EDAs and the most recent theoretical results in this area. In particular, emphasis is put on the runtime analysis of simple univariate EDAs, including a description of typical benchmark functions and tools for the analysis. Along the way, open problems and directions for future research are described.

cs.NE↗

arXiv2018

Upper Bounds on the Runtime of the Univariate Marginal Distribution Algorithm on OneMax

A runtime analysis of the Univariate Marginal Distribution Algorithm (UMDA) is presented on the OneMax function for wide ranges of its parameters $μ$ and $λ$. If $μ\ge c\log n$ for some constant $c>0$ and $λ=(1+Θ(1))μ$, a general bound $O(μn)$ on the expected runtime is obtained. This bound crucially assumes that all marginal probabilities of the algorithm are confined to the interval $[1/n,1-1/n]$. If $μ\ge c' \sqrt{n}\log n$ for a constant $c'>0$ and $λ=(1+Θ(1))μ$, the behavior of the algorithm changes and the bound on the expected runtime becomes $O(μ\sqrt{n})$, which typically even holds if the borders on the marginal probabilities are omitted. The results supplement the recently derived lower bound $Ω(μ\sqrt{n}+n\log n)$ by Krejca and Witt (FOGA 2017) and turn out as tight for the two very different values $μ=c\log n$ and $μ=c'\sqrt{n}\log n$. They also improve the previously best known upper bound $O(n\log n\log\log n)$ by Dang and Lehre (GECCO 2015).

cs.NE↗

arXiv2018

General Drift Analysis with Tail Bounds

Drift analysis is one of the state-of-the-art techniques for the runtime analysis of randomized search heuristics (RSHs) such as evolutionary algorithms (EAs), simulated annealing etc. The vast majority of existing drift theorems yield bounds on the expected value of the hitting time for a target state, e.g., the set of optimal solutions, without making additional statements on the distribution of this time. We address this lack by providing a general drift theorem that includes bounds on the upper and lower tail of the hitting time distribution. The new tail bounds are applied to prove very precise sharp-concentration results on the running time of a simple EA on standard benchmark problems, including the class of general linear functions. Surprisingly, the probability of deviating by an $r$-factor in lower order terms of the expected time decreases exponentially with $r$ on all these problems. The usefulness of the theorem outside the theory of RSHs is demonstrated by deriving tail bounds on the number of cycles in random permutations. All these results handle a position-dependent (variable) drift that was not covered by previous drift theorems with tail bounds. Moreover, our theorem can be specialized into virtually all existing drift theorems with drift towards the target from the literature. Finally, user-friendly specializations of the general drift theorem are given.

cs.NE↗

arXiv2018

The (1+$λ$) Evolutionary Algorithm with Self-Adjusting Mutation Rate

We propose a new way to self-adjust the mutation rate in population-based evolutionary algorithms in discrete search spaces. Roughly speaking, it consists of creating half the offspring with a mutation rate that is twice the current mutation rate and the other half with half the current rate. The mutation rate is then updated to the rate used in that subpopulation which contains the best offspring. We analyze how the $(1+λ)$ evolutionary algorithm with this self-adjusting mutation rate optimizes the OneMax test function. We prove that this dynamic version of the $(1+λ)$ EA finds the optimum in an expected optimization time (number of fitness evaluations) of $O(nλ/\logλ+n\log n)$. This time is asymptotically smaller than the optimization time of the classic $(1+λ)$ EA. Previous work shows that this performance is best-possible among all $λ$-parallel mutation-based unbiased black-box algorithms. This result shows that the new way of adjusting the mutation rate can find optimal dynamic parameter values on the fly. Since our adjustment mechanism is simpler than the ones previously used for adjusting the mutation rate and does not have parameters itself, we are optimistic that it will find other applications.

cs.NE↗

arXiv2017

Edge-matching Problems with Rotations

Edge-matching problems, also called edge matching puzzles, are abstractions of placement problems with neighborhood conditions. Pieces with colored edges have to be placed on a board such that adjacent edges have the same color. The problem has gained interest recently with the (now terminated) Eternity~II puzzle, and new complexity results. In this paper we consider a number of settings which differ in size of the puzzles and the manipulations allowed on the pieces. We investigate the effect of allowing rotations of the pieces on the complexity of the problem, an aspect that is only marginally treated so far. We show that some problems have polynomial time algorithms while others are NP-complete. Especially we show that allowing rotations in one-row puzzles makes the problem NP-hard. The proofs of the hardness result uses a large number of colors. This is essential because we also show that this problem (and another related one) is fixed-parameter tractable, where the relevant parameter is the number of colors.

cs.DS↗

arXiv2016

Update Strength in EDAs and ACO: How to Avoid Genetic Drift

We provide a rigorous runtime analysis concerning the update strength, a vital parameter in probabilistic model-building GAs such as the step size $1/K$ in the compact Genetic Algorithm (cGA) and the evaporation factor $ρ$ in ACO. While a large update strength is desirable for exploitation, there is a general trade-off: too strong updates can lead to genetic drift and poor performance. We demonstrate this trade-off for the cGA and a simple MMAS ACO algorithm on the OneMax function. More precisely, we obtain lower bounds on the expected runtime of $Ω(K\sqrt{n} + n \log n)$ and $Ω(\sqrt{n}/ρ+ n \log n)$, respectively, showing that the update strength should be limited to $1/K, ρ= O(1/(\sqrt{n} \log n))$. In fact, choosing $1/K, ρ\sim 1/(\sqrt{n}\log n)$ both algorithms efficiently optimize OneMax in expected time $O(n \log n)$. Our analyses provide new insights into the stochastic behavior of probabilistic model-building GAs and propose new guidelines for setting the update strength in global optimization.

cs.NE↗

arXiv2015

On the Runtime of Randomized Local Search and Simple Evolutionary Algorithms for Dynamic Makespan Scheduling

Evolutionary algorithms have been frequently used for dynamic optimization problems. With this paper, we contribute to the theoretical understanding of this research area. We present the first computational complexity analysis of evolutionary algorithms for a dynamic variant of a classical combinatorial optimization problem, namely makespan scheduling. We study the model of a strong adversary which is allowed to change one job at regular intervals. Furthermore, we investigate the setting of random changes. Our results show that randomized local search and a simple evolutionary algorithm are very effective in dynamically tracking changes made to the problem instance.

cs.DS↗

arXiv2013

The Fitness Level Method with Tail Bounds

The fitness-level method, also called the method of f-based partitions, is an intuitive and widely used technique for the running time analysis of randomized search heuristics. It was originally defined to prove upper and lower bounds on the expected running time. Recently, upper tail bounds were added to the technique; however, these tail bounds only apply to running times that are at least twice as large as the expectation. We remove this restriction and supplement the fitness-level method with sharp tail bounds, including lower tails. As an exemplary application, we prove that the running time of randomized local search on OneMax is sharply concentrated around n ln n - 0.1159 n.

cs.NE↗

arXiv2012

Erratum: Simplified Drift Analysis for Proving Lower Bounds in Evolutionary Computation

This erratum points out an error in the simplified drift theorem (SDT) [Algorithmica 59(3), 369-386, 2011]. It is also shown that a minor modification of one of its conditions is sufficient to establish a valid result. In many respects, the new theorem is more general than before. We no longer assume a Markov process nor a finite search space. Furthermore, the proof of the theorem is more compact than the previous ones. Finally, previous applications of the SDT are revisited. It turns out that all of these either meet the modified condition directly or by means of few additional arguments.

cs.NE↗

arXiv2011

Tight Bounds on the Optimization Time of the (1+1) EA on Linear Functions

The analysis of randomized search heuristics on classes of functions is fundamental for the understanding of the underlying stochastic process and the development of suitable proof techniques. Recently, remarkable progress has been made in bounding the expected optimization time of the simple (1+1) EA on the class of linear functions. We improve the best known bound in this setting from $(1.39+o(1))en\ln n$ to $en\ln n+O(n)$ in expectation and with high probability, which is tight up to lower-order terms. Moreover, upper and lower bounds for arbitrary mutations probabilities $p$ are derived, which imply expected polynomial optimization time as long as $p=O((\ln n)/n)$ and which are tight if $p=c/n$ for a constant $c$. As a consequence, the standard mutation probability $p=1/n$ is optimal for all linear functions, and the (1+1) EA is found to be an optimal mutation-based algorithm. The proofs are based on adaptive drift functions and the recent multiplicative drift theorem.

cs.NE↗

arXiv2011

Finite First Hitting Time versus Stochastic Convergence in Particle Swarm Optimisation

We reconsider stochastic convergence analyses of particle swarm optimisation, and point out that previously obtained parameter conditions are not always sufficient to guarantee mean square convergence to a local optimum. We show that stagnation can in fact occur for non-trivial configurations in non-optimal parts of the search space, even for simple functions like SPHERE. The convergence properties of the basic PSO may in these situations be detrimental to the goal of optimisation, to discover a sufficiently good solution within reasonable time. To characterise optimisation ability of algorithms, we suggest the expected first hitting time (FHT), i.e., the time until a search point in the vicinity of the optimum is visited. It is shown that a basic PSO may have infinite expected FHT, while an algorithm introduced here, the Noisy PSO, has finite expected FHT on some functions.

cs.NE↗