arXiv subjects

Patrick Mehlitz

Publications and source records attributed to Patrick Mehlitz.

At least 37 records · Page 2Linked to original sources

arXiv2023

Convergence Analysis of the Proximal Gradient Method in the Presence of the Kurdyka-Łojasiewicz Property without Global Lipschitz Assumptions

We consider a composite optimization problem where the sum of a continuously differentiable and a merely lower semicontinuous function has to be minimized. The proximal gradient algorithm is the classical method for solving such a problem numerically. The corresponding global convergence and local rate-of-convergence theory typically assumes, besides some technical conditions, that the smooth function has a globally Lipschitz continuous gradient and that the objective function satisfies the Kurdyka-Łojasiewicz property. Though this global Lipschitz assumption is satisfied in several applications where the objective function is, e.g., quadratic, this requirement is very restrictive in the non-quadratic case. Some recent contributions therefore try to overcome this global Lipschitz condition by replacing it with a local one, but, to the best of our knowledge, they still require some extra condition in order to obtain the desired global and rate-of-convergence results. The aim of this paper is to show that the local Lipschitz assumption together with the Kurdyka-Łojasiewicz property is sufficient to recover these convergence results.

math.OC↗

arXiv2023

On the directional asymptotic approach in optimization theory Part A: approximate, M-, and mixed-order stationarity

We show that, for a fixed order $γ\geq 1$, each local minimizer of a rather general nonsmooth optimization problem in Euclidean spaces is either M-stationary in the classical sense (corresponding to stationarity of order $1$), satisfies stationarity conditions in terms of a coderivative construction of order $γ$, or is approximately stationary with respect to a critical direction as well as $γ$ in a certain sense. By ruling out the latter case with a constraint qualification not stronger than directional metric subregularity, we end up with new necessary optimality conditions comprising a mixture of limiting variational tools of order $1$ and $γ$. These abstract findings are carved out for the broad class of geometric constraints. As a byproduct, we obtain new constraint qualifications ensuring M-stationarity of local minimizers. The paper closes by illustrating these results in the context of standard nonlinear, complementarity-constrained, and nonlinear semidefinite programming.

math.OC↗

arXiv2023

On the directional asymptotic approach in optimization theory Part B: constraint qualifications

During the last years, asymptotic (or sequential) constraint qualifications, which postulate upper semicontinuity of certain set-valued mappings and provide a natural companion of asymptotic stationarity conditions, have been shown to be comparatively mild, on the one hand, while possessing inherent practical relevance from the viewpoint of numerical solution methods, on the other one. Based on recent developments, the theory in this paper enriches asymptotic constraint qualifications for very general nonsmooth optimization problems over inverse images of set-valued mappings by incorporating directional data. We compare these new directional asymptotic regularity conditions with standard constraint qualifications from nonsmooth optimization. Further, we introduce directional concepts of pseudo- and quasi-normality which apply to set-valued mappings. It is shown that these properties provide sufficient conditions for the validity of directional asymptotic regularity. Finally, a novel coderivative-like variational tool is introduced which allows to study the presence of directional asymptotic regularity. For geometric constraints, it is illustrated that all appearing objects can be calculated in terms of initial problem data.

math.OC↗

arXiv2023

A simple proof of second-order sufficient optimality conditions in nonlinear semidefinite optimization

In this note, we present an elementary proof for a well-known second-order sufficient optimality condition in nonlinear semidefinite optimization which does not rely on the enhanced theory of second-order tangents. Our approach builds on an explicit elementary computation of the so-called second subderivative of the indicator function associated with the semidefinite cone which recovers the best curvature term known in the literature.

math.OC↗

arXiv2023

Constrained composite optimization and augmented Lagrangian methods

We investigate finite-dimensional constrained structured optimization problems, featuring composite objective functions and set-membership constraints. Offering an expressive yet simple language, this problem class provides a modeling framework for a variety of applications. We study stationarity and regularity concepts, and propose a flexible augmented Lagrangian scheme. We provide a theoretical characterization of the algorithm and its asymptotic properties, deriving convergence results for fully nonconvex problems. It is demonstrated how the inner subproblems can be solved by off-the-shelf proximal methods, notwithstanding the possibility to adopt any solvers, insofar as they return approximate stationary points. Finally, we describe our matrix-free implementation of the proposed algorithm and test it numerically. Illustrative examples show the versatility of constrained composite programs as a modeling tool and expose difficulties arising in this vast problem class.

math.OC↗

arXiv2023

Why second-order sufficient conditions are, in a way, easy -- or -- revisiting calculus for second subderivatives

In this paper, we readdress the classical topic of second-order sufficient optimality conditions for optimization problems with nonsmooth structure. Based on the so-called second subderivative of the objective function and of the indicator function associated with the feasible set, one easily obtains second-order sufficient optimality conditions of abstract form. In order to exploit further structure of the problem, e.g., composite terms in the objective function or feasible sets given as (images of) pre-images of closed sets under smooth transformations, to make these conditions fully explicit, we study calculus rules for the second subderivative under mild conditions. To be precise, we investigate a chain rule and a marginal function rule, which then also give a pre-image and image rule, respectively. As it turns out, the chain rule and the pre-image rule yield lower estimates desirable in order to obtain sufficient optimality conditions for free. Similar estimates for the marginal function and the image rule are valid under a comparatively mild inner calmness* assumption. Our findings are illustrated by several examples including problems from composite, disjunctive, and nonlinear second-order cone programming.

math.OC↗

arXiv2022

Convergence Properties of Monotone and Nonmonotone Proximal Gradient Methods Revisited

Composite optimization problems, where the sum of a smooth and a merely lower semicontinuous function has to be minimized, are often tackled numerically by means of proximal gradient methods as soon as the lower semicontinuous part of the objective function is of simple enough structure. The available convergence theory associated with these methods (mostly) requires the derivative of the smooth part of the objective function to be (globally) Lipschitz continuous, and this might be a restrictive assumption in some practically relevant scenarios. In this paper, we readdress this classical topic and provide convergence results for the classical (monotone) proximal gradient method and one of its nonmonotone extensions which are applicable in the absence of (strong) Lipschitz assumptions. This is possible since, for the price of forgoing convergence rates, we omit the use of descent-type lemmas in our analysis.

math.OC↗

arXiv2022

An Augmented Lagrangian Method for Optimization Problems with Structured Geometric Constraints

This paper is devoted to the theoretical and numerical investigation of an augmented Lagrangian method for the solution of optimization problems with geometric constraints. Specifically, we study situations where parts of the constraints are nonconvex and possibly complicated, but allow for a fast computation of projections onto this nonconvex set. Typical problem classes which satisfy this requirement are optimization problems with disjunctive constraints (like complementarity or cardinality constraints) as well as optimization problems over sets of matrices which have to satisfy additional rank constraints. The key idea behind our method is to keep these complicated constraints explicitly in the constraints and to penalize only the remaining constraints by an augmented Lagrangian function. The resulting subproblems are then solved with the aid of a problem-tailored nonmonotone projected gradient method. The corresponding convergence theory allows for an inexact solution of these subproblems. Nevertheless, the overall algorithm computes so-called Mordukhovich-stationary points of the original problem under a mild asymptotic regularity condition, which is generally weaker than most of the respective available problem-tailored constraint qualifications. Extensive numerical experiments addressing complementarity- and cardinality-constrained optimization problems as well as a semidefinite reformulation of MAXCUT problems visualize the power of our approach.

math.OC↗

arXiv2021

Optimality conditions, approximate stationarity, and applications -- a story beyond Lipschitzness

Approximate necessary optimality conditions in terms of Fréchet subgradients and normals for a rather general optimization problem with a potentially non-Lipschitzian objective function are established with the aid of Ekeland's variational principle, the fuzzy Fréchet subdifferential sum rule, and a novel notion of lower semicontinuity relative to a set-valued mapping or set. Feasible points satisfying these optimality conditions are referred to as approximately stationary. As applications, we derive a new general version of the extremal principle. Furthermore, we study approximate stationarity conditions for an optimization problem with a composite objective function and geometric constraints, a qualification condition guaranteeing that approximately stationary points of such a problem are M-stationary, and a multiplier-penalty-method which naturally computes approximately stationary points of the underlying problem. Finally, necessary optimality conditions for an optimal control problem with a non-Lipschitzian sparsity-promoting term in the objective function are established.

math.OC↗

arXiv2021

Asymptotic regularity for Lipschitzian nonlinear optimization problems with applications to complementarity-constrained and bilevel programming

Asymptotic stationarity and regularity conditions turned out to be quite useful to study the qualitative properties of numerical solution methods for standard nonlinear and complementarity-constrained programs. In this paper, we first extend these notions to nonlinear optimization problems with nonsmooth but Lipschitzian data functions in order to find reasonable notions of asymptotic stationarity and regularity in terms of Clarke's and Mordukhovich's subdifferential construction. Particularly, we compare the associated novel asymptotic constraint qualifications with already existing ones. The second part of the paper presents two applications of the obtained theory. On the one hand, we specify our findings for complementarity-constrained optimization problems and recover recent results from the literature which demonstrates the power of the approach. Furthermore, we hint at potential extensions to or- and vanishing-constrained optimization. On the other hand, we demonstrate the usefulness of asymptotic regularity in the context of bilevel optimization. More precisely, we justify a well-known stationarity system for affinely constrained bilevel optimization problems in a novel way. Afterwards, we suggest a solution algorithm for this class of bilevel optimization problems which combines a penalty method with ideas from DC-programming. After a brief convergence analysis, we present results of some numerical experiments.

math.OC↗

arXiv2021

On implicit variables in optimization theory

Implicit variables of a mathematical program are variables which do not need to be optimized but are used to model feasibility conditions. They frequently appear in several different problem classes of optimization theory comprising bilevel programming, evaluated multiobjective optimization, or nonlinear optimization problems with slack variables. In order to deal with implicit variables, they are often interpreted as explicit ones. Here, we first point out that this is a light-headed approach which induces artificial locally optimal solutions. Afterwards, we derive various Mordukhovich-stationarity-type necessary optimality conditions which correspond to treating the implicit variables as explicit ones on the one hand, or using them only implicitly to model the constraints on the other. A detailed comparison of the obtained stationarity conditions as well as the associated underlying constraint qualifications will be provided. Overall, we proceed in a fairly general setting relying on modern tools of variational analysis. Finally, we apply our findings to different well-known problem classes of mathematical optimization in order to visualize the obtained theory.

math.OC↗

arXiv2021

Subdifferentiation of nonconvex sparsity-promoting functionals on Lebesgue spaces

Sparsity-promoting terms are incorporated into the objective functions of optimal control problems in order to ensure that optimal controls vanish on large parts of the underlying domain. Typical candidates for those terms are integral functions on Lebesgue spaces based on the $\ell_p$-metric for $p\in[0,1)$ which are nonconvex as well as non-Lipschitz and, thus, variationally challenging. In this paper, we derive exact formulas for the Fréchet, limiting, and singular subdifferential of these functionals. These generalized derivatives can be used for the derivation of necessary optimality conditions for optimal control problems comprising such sparsity-promoting terms.

math.OC↗

arXiv2021

Calmness and Calculus: Two Basic Patterns

We establish two types of estimates for generalized derivatives of set-valued mappings which carry the essence of two basic patterns observed troughout the pile of calculus rules. These estimates also illustrate the role of the essential assumptions that accompany these two patters, namely calmness on the one hand and (fuzzy) inner calmness* on the other. Afterwards, we study the relationship between and sufficient conditions for the various notions of (inner) calmness. The aforementioned estimates are applied in order to recover several prominent calculus rules for tangents and normals as well as generalized derivatives of marginal functions and compositions as well as Cartesian products of set-valued mappings under mild conditions. We believe that our enhanced approach puts the overall generalized calculus into some other light. Some applications of our findings are presented which exemplary address necessary optimality conditions for minimax optimization problems as well as the calculus related to the recently introduced semismoothness* property.

math.OC↗

arXiv2020

R-regularity of set-valued mappings under the relaxed constant positive linear dependence constraint qualification with applications to parametric and bilevel optimization

The presence of Lipschitzian properties for solution mappings associated with nonlinear parametric optimization problems is desirable in the context of stability analysis or bilevel optimization. An example of such a Lipschitzian property for set-valued mappings, whose graph is the solution set of a system of nonlinear inequalities and equations, is R-regularity. Based on the so-called relaxed constant positive linear dependence constraint qualification, we provide a criterion ensuring the presence of the R-regularity property. In this regard, our analysis generalizes earlier results of that type which exploited the stronger Mangasarian-Fromovitz or constant rank constraint qualification. Afterwards, we apply our findings in order to derive new sufficient conditions which guarantee the presence of R-regularity for solution mappings in parametric optimization. Finally, our results are used to derive an existence criterion for solutions in pessimistic bilevel optimization and a sufficient condition for the presence of the so-called partial calmness property in optimistic bilevel optimization.

math.OC↗

arXiv2020

Asymptotic stationarity and regularity for nonsmooth optimization problems

Based on the tools of limiting variational analysis, we derive a sequential necessary optimality condition for nonsmooth mathematical programs which holds without any additional assumptions. In order to ensure that stationary points in this new sense are already Mordukhovich-stationary, the presence of a constraint qualification which we call AM-regularity is necessary. We investigate the relationship between AM-regularity and other constraint qualifications from nonsmooth optimization like metric (sub-)regularity of the underlying feasibility mapping. Our findings are applied to optimization problems with geometric and, particularly, disjunctive constraints. This way, it is shown that AM-regularity recovers recently introduced cone-continuity-type constraint qualifications, sometimes referred to as AKKT-regularity, from standard nonlinear and complementarity-constrained optimization. Finally, we discuss some consequences of AM-regularity for the limiting variational calculus.

math.OC↗

arXiv2020

Reformulation of the M-stationarity conditions as a system of discontinuous equations and its solution by a semismooth Newton method

We show that the Mordukhovich-stationarity system associated with a mathematical program with complementarity constraints (MPCC) can be equivalently written as a system of discontinuous equations which can be tackled with a semismooth Newton method. We show that the resulting algorithm can be interpreted as an active set strategy for MPCCs. Local fast convergence of the method is guaranteed under validity of an MPCC-tailored version of LICQ and a suitable second-order condition. In case of linear-quadratic MPCCs, the LICQ-type constraint qualification can be replaced by a weaker condition which depends on the underlying multipliers. We discuss a suitable globalization strategy for our method. Some numerical results are presented in order to illustrate our theoretical findings.

math.OC↗

arXiv2020

A note on partial calmness for bilevel optimization problems with linearly structured lower level

Partial calmness is a celebrated but restrictive property of bilevel optimization problems whose presence opens a way to the derivation of Karush--Kuhn--Tucker-type necessary optimality conditions in order to characterize local minimizers. In the past, sufficient conditions for the validity of partial calmness have been investigated. In this regard, the presence of a linearly structured lower level problem has turned out to be beneficial. However, the associated literature suffers from inaccurate results. In this note, we clarify some regarding erroneous statements and visualize the underlying issues with the aid of illustrative counterexamples.

math.OC↗

arXiv2020

New Constraint Qualifications for Optimization Problems in Banach Spaces based on Asymptotic KKT Conditions

Optimization theory in Banach spaces suffers from the lack of available constraint qualifications. Despite the fact that there exist only a very few constraint qualifications, they are, in addition, often violated even in simple applications. This is very much in contrast to finite-dimensional nonlinear programs, where a large number of constraint qualifications is known. Since these constraint qualifications are usually defined using the set of active inequality constraints, it is difficult to extend them to the infinite-dimensional setting. One exception is a recently introduced sequential constraint qualification based on asymptotic KKT conditions. This paper shows that this so-called asymptotic KKT regularity allows suitable extensions to the Banach space setting in order to obtain new constraint qualifications. The relation of these new constraint qualifications to existing ones is discussed in detail. Their usefulness is also shown by several examples as well as an algorithmic application to the class of augmented Lagrangian methods.

math.OC↗