arXiv subjects

Alain Zemkoho

Publications and source records attributed to Alain Zemkoho.

23 records · Page 2Linked to original sources

arXiv2022

Extension of the value function reformulation to multiobjective bilevel optimization

We consider a multiobjective bilevel optimization problem with vector-valued upper- and lower-level objective functions. Such problems have attracted a lot of interest in recent years. However, so far, scalarization has appeared to be the main approach used to deal with the lower-level problem. Here, we utilize the concept of frontier map that extends the notion of optimal value function to our parametric multiobjective lower-level problem. Based on this, we build a tractable constraint qualification that we use to derive necessary optimality conditions for the problem. Subsequently, we show that our resulting necessary optimality conditions represent a natural extension from standard optimistic bilevel programs with scalar objective functions.

math.OC↗

arXiv2021

Bilevel hyperparameter optimization for support vector classification: theoretical analysis and a solution method

Support vector classification (SVC) is a classical and well-performed learning method for classification problems. A regularization parameter, which significantly affects the classification performance, has to be chosen and this is usually done by the cross-validation procedure. In this paper, we reformulate the hyperparameter selection problem for support vector classification as a bilevel optimization problem in which the upper-level problem minimizes the average number of misclassified data points over all the cross-validation folds, and the lower-level problems are the l1-loss SVC problems, with each one for each fold in T-fold cross-validation. The resulting bilevel optimization model is then converted to a mathematical program with equilibrium constraints (MPEC). To solve this MPEC, we propose a global relaxation cross-validation algorithm (GR-CV) based on the well-know Sholtes-type global relaxation method (GRM). It is proven to converge to a C-stationary point. Moreover, we prove that the MPEC-tailored version of the Mangasarian-Fromovitz constraint qualification (MFCQ), which is a key property to guarantee the convergence of the GRM, automatically holds at each feasible point of this MPEC. Extensive numerical results verify the efficiency of the proposed approach. In particular, compared with other methods, our algorithm enjoys superior generalization performance over almost all the data sets used in this paper.

math.OC↗

arXiv2020

Theoretical and numerical comparison of the Karush-Kuhn-Tucker and value function reformulations in bilevel optimization

The Karush-Kuhn-Tucker and value function (lower-level value function, to be precise) reformulations are the most common single-level transformations of the bilevel optimization problem. So far, these reformulations have either been studied independently or as a joint optimization problem in an attempt to take advantage of the best properties from each model. To the best of our knowledge, these reformulations have not yet been compared in the existing literature. This paper is a first attempt towards establishing whether one of these reformulations is best at solving a given class of the optimistic bilevel optimization problem. We design a comparison framework, which seems fair, considering the theoretical properties of these reformulations. This work reveals that although none of the models seems to particularly dominate the other from the theoretical point of view, the value function reformulation seems to numerically outperform the Karush-Kuhn-Tucker reformulation on a Newton-type algorithm. The computational experiments here are mostly based on test problems from the Bilevel Optimization LIBrary (BOLIB).

math.OC↗

arXiv2020

Gauss-Newton-type methods for bilevel optimization

This article studies Gauss-Newton-type methods for over-determined systems to find solutions to bilevel programming problems. To proceed, we use the lower-level value function reformulation of bilevel programs and consider necessary optimality conditions under appropriate assumptions. First under strict complementarity for upper- and lower-level feasibility constraints, we prove the convergence of a Gauss-Newton-type method in computing points satisfying these optimality conditions under additional tractable qualification conditions. Potential approaches to address the shortcomings of the method are then proposed, leading to alternatives such as the pseudo or smoothing Gauss-Newton-type methods for bilevel optimization. Our numerical experiments conducted on 124 examples from the recently released Bilevel Optimization LIBrary (BOLIB) compare the performance of our method under different scenarios and show that it is a tractable approach to solve bilevel optimization problems with continuous variables.

math.OC↗

arXiv2018

An inertial extrapolation method for convex simple bilevel optimization

We consider a scalar objective minimization problem over the solution set of another optimization problem. This problem is known as simple bilevel optimization problem and has drawn a significant attention in the last few years. Our inner problem consists of minimizing the sum of smooth and nonsmooth functions while the outer one is the minimization of a smooth convex function. We propose and establish the convergence of a fixed-point iterative method with inertial extrapolation to solve the problem. Our numerical experiments show that the method proposed in this paper outperforms the currently best known algorithm to solve the class of problem considered.

math.OC↗