arXiv subjects

Olivier Teytaud

Publications and source records attributed to Olivier Teytaud.

At least 37 records · Page 2Linked to original sources

arXiv2021

Transfer of Fully Convolutional Policy-Value Networks Between Games and Game Variants

In this paper, we use fully convolutional architectures in AlphaZero-like self-play training setups to facilitate transfer between variants of board games as well as distinct games. We explore how to transfer trained parameters of these architectures based on shared semantics of channels in the state and action representations of the Ludii general game system. We use Ludii's large library of games and game variants for extensive transfer learning evaluations, in zero-shot transfer experiments as well as experiments with additional fine-tuning time.

cs.LG↗

arXiv2021

Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking

Existing studies in black-box optimization for machine learning suffer from low generalizability, caused by a typically selective choice of problem instances used for training and testing different optimization algorithms. Among other issues, this practice promotes overfitting and poor-performing user guidelines. To address this shortcoming, we propose in this work a benchmark suite, OptimSuite, which covers a broad range of black-box optimization problems, ranging from academic benchmarks to real-world applications, from discrete over numerical to mixed-integer problems, from small to very large-scale problems, from noisy over dynamic to static problems, etc. We demonstrate the advantages of such a broad collection by deriving from it Automated Black Box Optimizer (ABBO), a general-purpose algorithm selection wizard. Using three different types of algorithm selection techniques, ABBO achieves competitive performance on all benchmark suites. It significantly outperforms previous state of the art on some of them, including YABBOB and LSGO. ABBO relies on many high-quality base components. Its excellent performance is obtained without any task-specific parametrization. The OptimSuite benchmark collection, the ABBO wizard and its base solvers have all been merged into the open-source Nevergrad platform, where they are available for reproducible research.

cs.LG↗

arXiv2021

Deep Learning for General Game Playing with Ludii and Polygames

Combinations of Monte-Carlo tree search and Deep Neural Networks, trained through self-play, have produced state-of-the-art results for automated game-playing in many board games. The training and search algorithms are not game-specific, but every individual game that these approaches are applied to still requires domain knowledge for the implementation of the game's rules, and constructing the neural network's architecture -- in particular the shapes of its input and output tensors. Ludii is a general game system that already contains over 500 different games, which can rapidly grow thanks to its powerful and user-friendly game description language. Polygames is a framework with training and search algorithms, which has already produced superhuman players for several board games. This paper describes the implementation of a bridge between Ludii and Polygames, which enables Polygames to train and evaluate models for games that are implemented and run through Ludii. We do not require any game-specific domain knowledge anymore, and instead leverage our domain knowledge of the Ludii system and its abstract state and move representations to write functions that can automatically determine the appropriate shapes for input and output tensors for any game implemented in Ludii. We describe experimental results for short training runs in a wide variety of different board games, and discuss several open problems and avenues for future research.

cs.AI↗

arXiv2020

Adversarial Attacks on Linear Contextual Bandits

Contextual bandit algorithms are applied in a wide range of domains, from advertising to recommender systems, from clinical trials to education. In many of these domains, malicious agents may have incentives to attack the bandit algorithm to induce it to perform a desired behavior. For instance, an unscrupulous ad publisher may try to increase their own revenue at the expense of the advertisers; a seller may want to increase the exposure of their products, or thwart a competitor's advertising campaign. In this paper, we study several attack scenarios and show that a malicious agent can force a linear contextual bandit algorithm to pull any desired arm $T - o(T)$ times over a horizon of $T$ steps, while applying adversarial modifications to either rewards or contexts that only grow logarithmically as $O(\log T)$. We also investigate the case when a malicious agent is interested in affecting the behavior of the bandit algorithm in a single context (e.g., a specific user). We first provide sufficient conditions for the feasibility of the attack and we then propose an efficient algorithm to perform the attack. We validate our theoretical results on experiments performed on both synthetic and real-world datasets.

cs.LG↗

arXiv2020

EvolGAN: Evolutionary Generative Adversarial Networks

We propose to use a quality estimator and evolutionary methods to search the latent space of generative adversarial networks trained on small, difficult datasets, or both. The new method leads to the generation of significantly higher quality images while preserving the original generator's diversity. Human raters preferred an image from the new version with frequency 83.7pc for Cats, 74pc for FashionGen, 70.4pc for Horses, and 69.2pc for Artworks, and minor improvements for the already excellent GANs for faces. This approach applies to any quality scorer and GAN generator.

cs.CV↗

arXiv2020

Tarsier: Evolving Noise Injection in Super-Resolution GANs

Super-resolution aims at increasing the resolution and level of detail within an image. The current state of the art in general single-image super-resolution is held by NESRGAN+, which injects a Gaussian noise after each residual layer at training time. In this paper, we harness evolutionary methods to improve NESRGAN+ by optimizing the noise injection at inference time. More precisely, we use Diagonal CMA to optimize the injected noise according to a novel criterion combining quality assessment and realism. Our results are validated by the PIRM perceptual score and a human study. Our method outperforms NESRGAN+ on several standard super-resolution datasets. More generally, our approach can be used to optimize any method based on noise injection.

cs.CV↗

arXiv2020

On averaging the best samples in evolutionary computation

Choosing the right selection rate is a long standing issue in evolutionary computation. In the continuous unconstrained case, we prove mathematically that a single parent $μ=1$ leads to a sub-optimal simple regret in the case of the sphere function. We provide a theoretically-based selection rate $μ/λ$ that leads to better progress rates. With our choice of selection rate, we get a provable regret of order $O(λ^{-1})$ which has to be compared with $O(λ^{-2/d})$ in the case where $μ=1$. We complete our study with experiments to confirm our theoretical claims.

cs.NE↗

arXiv2020

Population Control meets Doob's Martingale Theorems: the Noise-free Multimodal Case

We study a test-based population size adaptation (TBPSA) method, inspired from population control, in the noise-free multimodal case. In the noisy setting, TBPSA usually recommends, at the end of the run, the center of the Gaussian as an approximation of the optimum. We show that combined with a more naive recommendation, namely recommending the visited point which had the best fitness value so far, TBPSA is also powerful in the noise-free multimodal context. We demonstrate this experimentally and explore this mechanism theoretically: we prove that TBPSA is able to escape plateaus with probability one in spite of the fact that it can converge to local minima. This leads to an algorithm effective in the multimodal setting without resorting to a random restart from scratch.

math.OC↗

arXiv2020

Versatile Black-Box Optimization

Choosing automatically the right algorithm using problem descriptors is a classical component of combinatorial optimization. It is also a good tool for making evolutionary algorithms fast, robust and versatile. We present Shiwa, an algorithm good at both discrete and continuous, noisy and noise-free, sequential and parallel, black-box optimization. Our algorithm is experimentally compared to competitors on YABBOB, a BBOB comparable testbed, and on some variants of it, and then validated on several real world testbeds.

cs.AI↗

arXiv2020

Variance Reduction for Better Sampling in Continuous Domains

Design of experiments, random search, initialization of population-based methods, or sampling inside an epoch of an evolutionary algorithm use a sample drawn according to some probability distribution for approximating the location of an optimum. Recent papers have shown that the optimal search distribution, used for the sampling, might be more peaked around the center of the distribution than the prior distribution modelling our uncertainty about the location of the optimum. We confirm this statement, provide explicit values for this reshaping of the search distribution depending on the population size $λ$ and the dimension $d$, and validate our results experimentally.

cs.NE↗

arXiv2020

Polygames: Improved Zero Learning

Since DeepMind's AlphaZero, Zero learning quickly became the state-of-the-art method for many board games. It can be improved using a fully convolutional structure (no fully connected layer). Using such an architecture plus global pooling, we can create bots independent of the board size. The training can be made more robust by keeping track of the best checkpoints during the training and by training against them. Using these features, we release Polygames, our framework for Zero learning, with its library of games and its checkpoints. We won against strong humans at the game of Hex in 19x19, which was often said to be untractable for zero learning; and in Havannah. We also won several first places at the TAAI competitions.

cs.LG↗

arXiv2019

Yet another but more efficient black-box adversarial attack: tiling and evolution strategies

We introduce a new black-box attack achieving state of the art performances. Our approach is based on a new objective function, borrowing ideas from $\ell_\infty$-white box attacks, and particularly designed to fit derivative-free optimization requirements. It only requires to have access to the logits of the classifier without any other information which is a more realistic scenario. Not only we introduce a new objective function, we extend previous works on black box adversarial attacks to a larger spectrum of evolution strategies and other derivative-free optimization methods. We also highlight a new intriguing property that deep neural networks are not robust to single shot tiled attacks. Our models achieve, with a budget limited to $10,000$ queries, results up to $99.2\%$ of success rate against InceptionV3 classifier with $630$ queries to the network on average in the untargeted attacks setting, which is an improvement by $90$ queries of the current state of the art. In the targeted setting, we are able to reach, with a limited budget of $100,000$, $100\%$ of success rate with a budget of $6,662$ queries on average, i.e. we need $800$ queries less than the current state of the art.

cs.LG↗

arXiv2019

Ultra thin anti-reflective coatings designed using Differential Evolution

We use a state-of-the-art optimization algorithm combined with a careful methodology to find optimal anti-reflective coatings. Our results show that ultra thin structures (less than $300 \,nm$ thick) outperform much thicker gradual patterns as well as traditional interferential anti-reflective coatings. These optimal designs actually combine a gradual increase of the refractive index with patterns meant to leverage interferential effects. Contrarily to gradual patterns, they do not require extremely low refractive index materials, so that they can actually be fabricated. Using a cheap and easily deployable vapor deposition technique, we fabricated a 4-layer anti-reflective coating, which proved very efficient over the whole visible spectrum despite a total thickness of only 200 nm.

physics.optics↗

arXiv2018

Scenario-based decision-making for power systems investment planning

The optimization of power systems involves complex uncertainties, such as technological progress, political context, geopolitical constraints. Negotiations at COP21 are complicated by the huge number of scenarios that various people want to consider; these scenarios correspond to many uncertainties. These uncertainties are difficult to modelize as probabilities, due to the lack of data for future technologies and due to partially adversarial geopolitical decision makers. Tools for such difficult decision making problems include Wald and Savage criteria, possibilistic reasoning and Nash equilibria. We investigate the rationale behind the use of a two-player Nash equilibrium approach in such a difficult context; we show that the computational cost is indeed smaller than for simpler criteria. Moreover, it naturally provides a selection of decisions and scenarios, and it has a natural interpretation in the sense that Nature does not make decisions taking into account our own decisions. The algorithm naturally provides a matrix of results, namely the matrix of outcomes in the most interesting decisions and for the most critical scenarios. These decisions and scenarios are also equipped with a ranking.

cs.GT↗

arXiv2018

Evolutionary algorithms converge towards evolved biological photonic structures

Nature features a plethora of extraordinary photonic architectures that have been optimized through natural evolution. While numerical optimization is increasingly and successfully used in photonics, it has yet to replicate any of these complex naturally occurring structures. Using evolutionary algorithms directly inspired by natural evolution, we have retrieved emblematic natural photonic structures, indicating how such regular structures might have spontaneously emerged in nature and to which precise optical or fabrication constraints they respond. Comparisons between algorithms show that recombination between individuals inspired by sexual reproduction confers a clear advantage in this context of modular problems and suggest further ways to improve the algorithms. Such an in silico evolution can also suggest original and elegant solutions to practical problems, as illustrated by the design of counter-intuitive anti-reflective coating for solar cells.

physics.optics↗

arXiv2018

Surprising strategies obtained by stochastic optimization in partially observable games

This paper studies the optimization of strategies in the context of possibly randomized two players zero-sum games with incomplete information. We compare 5 algorithms for tuning the parameters of strategies over a benchmark of 12 games. A first evolutionary approach consists in designing a highly randomized opponent (called naive opponent) and optimizing the parametric strategy against it; a second one is optimizing iteratively the strategy, i.e. constructing a sequence of strategies starting from the naive one. 2 versions of coevolutions, real and approximate, are also tested as well as a seed method. The coevolution methods were performing well, but results were not stable from one game to another. In spite of its simplicity, the seed method, which can be seen as an extremal version of coevolution, works even when nothing else works. Incidentally, these methods brought out some unexpected strategies for some games, such as Batawaf or the game of War, which seem, at first view, purely random games without any structured actions possible for the players or Guess Who, where a dichotomy between the characters seems to be the most reasonable strategy. All source codes of games are written in Matlab/Octave and are freely available for download.

cs.GT↗

arXiv2018

Exact Distributed Training: Random Forest with Billions of Examples

We introduce an exact distributed algorithm to train Random Forest models as well as other decision forest models without relying on approximating best split search. We explain the proposed algorithm and compare it to related approaches for various complexity measures (time, ram, disk, and network complexity analysis). We report its running performances on artificial and real-world datasets of up to 18 billions examples. This figure is several orders of magnitude larger than datasets tackled in the existing literature. Finally, we empirically show that Random Forest benefits from being trained on more data, even in the case of already gigantic datasets. Given a dataset with 17.3B examples with 82 features (3 numerical, other categorical with high arity), our implementation trains a tree in 22h.

cs.LG↗

arXiv2018

PSO-based Fuzzy Markup Language for Student Learning Performance Evaluation and Educational Application

This paper proposes an agent with particle swarm optimization (PSO) based on a Fuzzy Markup Language (FML) for students learning performance evaluation and educational applications, and the proposed agent is according to the response data from a conventional test and an item response theory. First, we apply a GS-based parameter estimation mechanism to estimate the items parameters according to the response data, and then to compare its results with those of an IRT-based Bayesian parameter estimation mechanism. In addition, we propose a static-IRT test assembly mechanism to assemble a form for the conventional test. The presented FML-based dynamic assessment mechanism infers the probability of making a correct response to the item for a student with various abilities. Moreover, this paper also proposes a novel PFML learning mechanism for optimizing the parameters between items and students. Finally, we adopt a K-fold cross validation mechanism to evaluate the performance of the proposed agent. Experimental results show that the novel PFML learning mechanism for the parameter estimation and learning optimization performs favorably. We believe the proposed PFML will be a reference for education research and pedagogy and an important co-learning mechanism for future human-machine educational applications.

cs.AI↗