arXiv subjects

Ernö Robert Csetnek

Publications and source records attributed to Ernö Robert Csetnek.

At least 37 records · Page 2Linked to original sources

arXiv2017

Approaching nonsmooth nonconvex minimization through second order proximal-gradient dynamical systems

We investigate the asymptotic properties of the trajectories generated by a second-order dynamical system of proximal-gradient type stated in connection with the minimization of the sum of a nonsmooth convex and a (possibly nonconvex) smooth function. The convergence of the generated trajectory to a critical point of the objective is ensured provided a regularization of the objective function satisfies the Kurdyka-Łojasiewicz property. We also provide convergence rates for the trajectory formulated in terms of the Łojasiewicz exponent.

math.OC↗

arXiv2017

ADMM for monotone operators: convergence analysis and rates

We propose in this paper a unifying scheme for several algorithms from the literature dedicated to the solving of monotone inclusion problems involving compositions with linear continuous operators in infinite dimensional Hilbert spaces. We show that a number of primal-dual algorithms for monotone inclusions and also the classical ADMM numerical scheme for convex optimization problems, along with some of its variants, can be embedded in this unifying scheme. While in the first part of the paper convergence results for the iterates are reported, the second part is devoted to the derivation of convergence rates obtained by combining variable metric techniques with strategies based on suitable choice of dynamical step sizes.

math.OC↗

arXiv2017

Newton-like dynamics associated to nonconvex optimization problems

We consider the dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partialϕ(x(t))\\ λ\dot x(t) + \dot v(t) + v(t) + \nabla ψ(x(t))=0, \end{array}\right.\end{equation*} where $ϕ:\R^n\to\R\cup\{+\infty\}$ is a proper, convex and lower semicontinuous function, $ψ:\R^n\to\R$ is a (possibly nonconvex) smooth function and $λ>0$ is a parameter which controls the velocity. We show that the set of limit points of the trajectory $x$ is contained in the set of critical points of the objective function $ϕ+ψ$, which is here seen as the set of the zeros of its limiting subdifferential. If the objective function satisfies the Kurdyka-Łojasiewicz property, then we can prove convergence of the whole trajectory $x$ to a critical point. Furthermore, convergence rates for the orbits are obtained in terms of the Łojasiewicz exponent of the objective function, provided the latter satisfies the Łojasiewicz property.

math.OC↗

arXiv2017

Second order dynamical systems with penalty terms associated to monotone inclusions

In this paper we investigate in a Hilbert space setting a second order dynamical system of the form $$\ddot{x}(t)+\g(t)\dot{x}(t)+x(t)-J_{λ(t) A}\big(x(t)-λ(t) D(x(t))-λ(t)β(t)B(x(t))\big)=0,$$ where $A:{\mathcal H}\toto{\mathcal H}$ is a maximal monotone operator, $J_{λ(t) A}:{\mathcal H}\To{\mathcal H}$ is the resolvent operator of $λ(t)A$ and $D,B: {\mathcal H}\rightarrow{\mathcal H}$ are cocoercive operators, and $λ,β:[0,+\infty)\rightarrow (0,+\infty)$, and $γ:[0,+\infty)\rightarrow (0,+\infty)$ are step size, penalization and, respectively, damping functions, all depending on time. We show the existence and uniqueness of strong global solutions in the framework of the Cauchy-Lipschitz-Picard Theorem and prove ergodic asymptotic convergence for the generated trajectories to a zero of the operator $A+D+{N}_C,$ where $C=\zer B$ and $N_C$ denotes the normal cone operator of $C$. To this end we use Lyapunov analysis combined with the celebrated Opial Lemma in its ergodic continuous version. Furthermore, we show strong convergence for trajectories to the unique zero of $A+D+{N}_C$, provided that $A$ is a strongly monotone operator.

math.DS↗

arXiv2016

Approaching nonsmooth nonconvex optimization problems through first order dynamical systems with hidden acceleration and Hessian driven damping terms

In this paper we carry out an asymptotic analysis of the proximal-gradient dynamical system \begin{equation*}\left\{ \begin{array}{ll} \dot x(t) +x(t) = \prox_{γf}\big[x(t)-γ\nablaΦ(x(t))-ax(t)-by(t)\big],\\ \dot y(t)+ax(t)+by(t)=0 \end{array}\right.\end{equation*} where $f$ is a proper, convex and lower semicontinuous function, $Φ$ a possibly nonconvex smooth function and $γ, a$ and $b$ are positive real numbers. We show that the generated trajectories approach the set of critical points of $f+Φ$, here understood as zeros of its limiting subdifferential, under the premise that a regularization of this sum function satisfies the Kurdyka-Łojasiewicz property. We also establish convergence rates for the trajectories, formulated in terms of the Łojasiewicz exponent of the considered regularization function.

math.OC↗

arXiv2016

A second order dynamical system with Hessian-driven damping and penalty term associated to variational inequalities

We consider the minimization of a convex objective function subject to the set of minima of another convex function, under the assumption that both functions are twice continuously differentiable. We approach this optimization problem from a continuous perspective by means of a second order dynamical system with Hessian-driven damping and a penalty term corresponding to the constrained function. By constructing appropriate energy functionals, we prove weak convergence of the trajectories generated by this differential equation to a minimizer of the optimization problem as well as convergence for the objective function values along the trajectories. The performed investigations rely on Lyapunov analysis in combination with the continuous version of the Opial Lemma. In case the objective function is strongly convex, we can even show strong convergence of the trajectories.

math.OC↗

arXiv2016

Levenberg-Marquardt dynamics associated to variational inequalities

In connection with the optimization problem $$\inf_{x\in argmin Ψ}\{Φ(x)+Θ(x)\},$$ where $Φ$ is a proper, convex and lower semicontinuous function and $Θ$ and $Ψ$ are convex and smooth functions defined on a real Hilbert space, we investigate the asymptotic behavior of the trajectories of the nonautonomous Levenberg-Marquardt dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partialΦ(x(t))\\ λ(t)\dot x(t) + \dot v(t) + v(t) + \nabla Θ(x(t))+β(t)\nabla Ψ(x(t))=0, \end{array}\right.\end{equation*} where $λ$ and $β$ are functions of time controlling the velocity and the penalty term, respectively. We show weak convergence of the generated trajectory to an optimal solution as well as convergence of the objective function values along the trajectories, provided $λ$ is monotonically decreasing, $β$ satisfies a growth condition and a relation expressed via the Fenchel conjugate of $Ψ$ is fulfilled. When the objective function is assumed to be strongly convex, we can even show strong convergence of the trajectories.

math.OC↗

arXiv2016

Second order forward-backward dynamical systems for monotone inclusion problems

We begin by considering second order dynamical systems of the from $\ddot x(t) + γ(t)\dot x(t) + λ(t)B(x(t))=0$, where $B: {\cal H}\rightarrow{\cal H}$ is a cocoercive operator defined on a real Hilbert space ${\cal H}$, $λ:[0,+\infty)\rightarrow [0,+\infty)$ is a relaxation function and $γ:[0,+\infty)\rightarrow [0,+\infty)$ a damping function, both depending on time. For the generated trajectories, we show existence and uniqueness of the generated trajectories as well as their weak asymptotic convergence to a zero of the operator $B$. The framework allows to address from similar perspectives second order dynamical systems associated with the problem of finding zeros of the sum of a maximally monotone operator and a cocoercive one. This captures as particular case the minimization of the sum of a nonsmooth convex function with a smooth convex one. Furthermore, we prove that when $B$ is the gradient of a smooth convex function the value of the latter converges along the ergodic trajectory to its minimal value with a rate of ${\cal O}(1/t)$.

math.OC↗

arXiv2016

Second order dynamical systems associated to variational inequalities

We investigate the asymptotic convergence of the trajectories generated by the second order dynamical system $\ddot x(t) + γ\dot x(t) + \nabla ϕ(x(t))+β(t)\nabla ψ(x(t))=0$, where $ϕ,ψ:{\cal H}\rightarrow \R$ are convex and smooth functions defined on a real Hilbert space ${\cal H}$, $γ>0$ and $β$ is a function of time which controls the penalty term. We show weak convergence of the trajectories to a minimizer of the function $ϕ$ over the (nonempty) set of minima of $ψ$ as well as convergence for the objective function values along the trajectories, provided a condition expressed via the Fenchel conjugate of $ψ$ is fulfilled. When the function $ϕ$ is assumed to be strongly convex, we can even show strong convergence of the trajectories. The results can be seen as the second order counterparts of the ones given by Attouch and Czarnecki (Journal of Differential Equations 248(6), 1315--1344, 2010) for first order dynamical systems associated to constrained variational inequalities. At the same time we give a positive answer to an open problem posed in \cite{att-cza-16} by the same authors.

math.OC↗

arXiv2016

Proximal-gradient algorithms for fractional programming

In this paper we propose two proximal gradient algorithms for fractional programming problems in real Hilbert spaces, where the numerator is a proper, convex and lower semicontinuous function and the denominator is a smooth function, either concave or convex. In the iterative schemes, we perform a proximal step with respect to the nonsmooth numerator and a gradient step with respect to the smooth denominator. The algorithm in case of a concave denominator has the particularity that it generates sequences which approach both the (global) optimal solutions set and the optimal objective value of the underlying fractional programming problem. In case of a convex denominator the numerical scheme approaches the set of critical points of the objective function, provided the latter satisfies the Kurdyka-Łojasiewicz property.

math.OC↗

arXiv2015

Penalty schemes with inertial effects for monotone inclusion problems

We introduce a penalty term-based splitting algorithm with inertial effects designed for solving monotone inclusion problems involving the sum of maximally monotone operators and the convex normal cone to the (nonempty) set of zeros of a monotone and Lipschitz continuous operator. We show weak ergodic convergence of the generated sequence of iterates to a solution of the monotone inclusion problem, provided a condition expressed via the Fitzpatrick function of the operator describing the underlying set of the normal cone is verified. Under strong monotonicity assumptions we can even show strong nonergodic convergence of the iterates. This approach constitutes the starting point for investigating from a similar perspective monotone inclusion problems involving linear compositions of parallel-sum operators and, further, for the minimization of a complexly structured convex objective function subject to the set of minima of another convex and differentiable function.

math.OC↗

arXiv2015

A forward-backward dynamical approach to the minimization of the sum of a nonsmooth convex with a smooth nonconvex function

We address the minimization of the sum of a proper, convex and lower semicontinuous with a (possibly nonconvex) smooth function from the perspective of an implicit dynamical system of forward-backward type. The latter is formulated by means of the gradient of the smooth function and of the proximal point operator of the nonsmooth one. The trajectory generated by the dynamical system is proved to asymptotically converge to a critical point of the objective, provided a regularization of the latter satisfies the Kurdyka-Łojasiewicz property. Convergence rates for the trajectory in terms of the Łojasiewicz exponent of the regularized objective function are also provided.

math.OC↗

arXiv2015

Convergence rates for forward-backward dynamical systems associated with strongly monotone inclusions

We investigate the convergence rates of the trajectories generated by implicit first and second order dynamical systems associated to the determination of the zeros of the sum of a maximally monotone operator and a monotone and Lipschitz continuous one in a real Hilbert space. We show that these trajectories strongly converge with exponential rate to a zero of the sum, provided the latter is strongly monotone. We derive from here convergence rates for the trajectories generated by dynamical systems associated to the minimization of the sum of a proper, convex and lower semicontinuous function with a smooth convex one provided the objective function fulfills a strong convexity assumption. In the particular case of minimizing a smooth and strongly convex function, we prove that its values converge along the trajectory to its minimum value with exponential rate, too.

math.OC↗

arXiv2015

Approaching the solving of constrained variational inequalities via penalty term-based dynamical systems

We investigate the existence and uniqueness of (locally) absolutely continuous trajectories of a penalty term-based dynamical system associated to a constrained variational inequality expressed as a monotone inclusion problem. Relying on Lyapunov analysis and on the ergodic continuous version of the celebrated Opial Lemma we prove weak ergodic convergence of the orbits to a solution of the constrained variational inequality under investigation. If one of the operators involved satisfies stronger monotonicity properties, then strong convergence of the trajectories can be shown.

math.OC↗

arXiv2014

A dynamical system associated with the fixed points set of a nonexpansive operator

We study the existence and uniqueness of (locally) absolutely continuous trajectories of a dynamical system governed by a nonexpansive operator. The weak convergence of the orbits to a fixed point of the operator is investigated by relying on Lyapunov analysis. We show also an order of convergence of $o(\frac{1}{\sqrt{t}})$ for the fixed point residual of the trajectory of the dynamical system. We apply the results to dynamical systems associated with the problem of finding the zeros of the sum of a maximally monotone operator and a cocoercive one. Several dynamical systems from the literature turn out to be particular instances of this general approach.

math.DS↗

arXiv2014

An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions

We propose a forward-backward proximal-type algorithm with inertial/memory effects for minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting. The sequence of iterates generated by the algorithm converges to a critical point of the objective function provided an appropriate regularization of the objective satisfies the Kurdyka-Łojasiewicz inequality, which is for instance fulfilled for semi-algebraic functions. We illustrate the theoretical results by considering two numerical experiments: the first one concerns the ability of recovering the local optimal solutions of nonconvex optimization problems, while the second one refers to the restoration of a noisy blurred image.

math.OC↗

arXiv2014

A hybrid proximal-extragradient algorithm with inertial effects

We incorporate inertial terms in the hybrid proximal-extragradient algorithm and investigate the convergence properties of the resulting iterative scheme designed for finding the zeros of a maximally monotone operator in real Hilbert spaces. The convergence analysis relies on extended Fejér monotonicity techniques combined with the celebrated Opial Lemma. We also show that the classical hybrid proximal-extragradient algorithm and the inertial versions of the proximal point, the forward-backward and the forward-backward-forward algorithms can be embedded in the framework of the proposed iterative scheme.

math.FA↗

arXiv2014

An inertial Tseng's type proximal algorithm for nonsmooth and nonconvex optimization problems

We investigate the convergence of a forward-backward-forward proximal-type algorithm with inertial and memory effects when minimizing the sum of a nonsmooth function with a smooth one in the absence of convexity. The convergence is obtained provided an appropriate regularization of the objective satisfies the Kurdyka-Łojasiewicz inequality, which is for instance fulfilled for semi-algebraic functions.

math.OC↗