arXiv subjects

Radu Ioan Bot

Publications and source records attributed to Radu Ioan Bot.

At least 55 records · Page 3Linked to original sources

arXiv2018

The Proximal Alternating Minimization Algorithm for two-block separable convex optimization problems with linear constraints

The Alternating Minimization Algorithm (AMA) has been proposed by Tseng to solve convex programming problems with two-block separable linear constraints and objectives, whereby (at least) one of the components of the latter is assumed to be strongly convex. The fact that one of the subproblems to be solved within the iteration process of AMA does not usually correspond to the calculation of a proximal operator through a closed formula, affects the implementability of the algorithm. In this paper we allow in each block of the objective a further smooth convex function and propose a proximal version of AMA, called Proximal AMA, which is achieved by equipping the algorithm with proximal terms induced by variable metrics. For suitable choices of the latter, the solving of the two subproblems in the iterative scheme can be reduced to the computation of proximal operators. We investigate the convergence of the proposed algorithm in a real Hilbert space setting and illustrate its numerical performances on two applications in image processing and machine learning.

math.OC↗

arXiv2017

Inducing strong convergence into the asymptotic behaviour of proximal splitting algorithms in Hilbert spaces

Proximal splitting algorithms for monotone inclusions (and convex optimization problems) in Hilbert spaces share the common feature to guarantee for the generated sequences in general weak convergence to a solution. In order to achieve strong convergence, one usually needs to impose more restrictive properties for the involved operators, like strong monotonicity (respectively, strong convexity for optimization problems). In this paper, we propose a modified Krasnosel'ski\uı--Mann algorithm in connection with the determination of a fixed point of a nonexpansive mapping and show strong convergence of the iteratively generated sequence to the minimal norm solution of the problem. Relying on this, we derive a forward-backward and a Douglas-Rachford algorithm, both endowed with Tikhonov regularization terms, which generate iterates that strongly converge to the minimal norm solution of the set of zeros of the sum of two maximally monotone operators. Furthermore, we formulate strong convergent primal-dual algorithms of forward-backward and Douglas-Rachford-type for highly structured monotone inclusion problems involving parallel-sums and compositions with linear operators. The resulting iterative schemes are particularized to the solving of convex minimization problems.

math.OC↗

arXiv2017

Approaching nonsmooth nonconvex minimization through second order proximal-gradient dynamical systems

We investigate the asymptotic properties of the trajectories generated by a second-order dynamical system of proximal-gradient type stated in connection with the minimization of the sum of a nonsmooth convex and a (possibly nonconvex) smooth function. The convergence of the generated trajectory to a critical point of the objective is ensured provided a regularization of the objective function satisfies the Kurdyka-Łojasiewicz property. We also provide convergence rates for the trajectory formulated in terms of the Łojasiewicz exponent.

math.OC↗

arXiv2017

ADMM for monotone operators: convergence analysis and rates

We propose in this paper a unifying scheme for several algorithms from the literature dedicated to the solving of monotone inclusion problems involving compositions with linear continuous operators in infinite dimensional Hilbert spaces. We show that a number of primal-dual algorithms for monotone inclusions and also the classical ADMM numerical scheme for convex optimization problems, along with some of its variants, can be embedded in this unifying scheme. While in the first part of the paper convergence results for the iterates are reported, the second part is devoted to the derivation of convergence rates obtained by combining variable metric techniques with strategies based on suitable choice of dynamical step sizes.

math.OC↗

arXiv2017

Newton-like dynamics associated to nonconvex optimization problems

We consider the dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partialϕ(x(t))\\ λ\dot x(t) + \dot v(t) + v(t) + \nabla ψ(x(t))=0, \end{array}\right.\end{equation*} where $ϕ:\R^n\to\R\cup\{+\infty\}$ is a proper, convex and lower semicontinuous function, $ψ:\R^n\to\R$ is a (possibly nonconvex) smooth function and $λ>0$ is a parameter which controls the velocity. We show that the set of limit points of the trajectory $x$ is contained in the set of critical points of the objective function $ϕ+ψ$, which is here seen as the set of the zeros of its limiting subdifferential. If the objective function satisfies the Kurdyka-Łojasiewicz property, then we can prove convergence of the whole trajectory $x$ to a critical point. Furthermore, convergence rates for the orbits are obtained in terms of the Łojasiewicz exponent of the objective function, provided the latter satisfies the Łojasiewicz property.

math.OC↗

arXiv2017

Second order dynamical systems with penalty terms associated to monotone inclusions

In this paper we investigate in a Hilbert space setting a second order dynamical system of the form $$\ddot{x}(t)+\g(t)\dot{x}(t)+x(t)-J_{λ(t) A}\big(x(t)-λ(t) D(x(t))-λ(t)β(t)B(x(t))\big)=0,$$ where $A:{\mathcal H}\toto{\mathcal H}$ is a maximal monotone operator, $J_{λ(t) A}:{\mathcal H}\To{\mathcal H}$ is the resolvent operator of $λ(t)A$ and $D,B: {\mathcal H}\rightarrow{\mathcal H}$ are cocoercive operators, and $λ,β:[0,+\infty)\rightarrow (0,+\infty)$, and $γ:[0,+\infty)\rightarrow (0,+\infty)$ are step size, penalization and, respectively, damping functions, all depending on time. We show the existence and uniqueness of strong global solutions in the framework of the Cauchy-Lipschitz-Picard Theorem and prove ergodic asymptotic convergence for the generated trajectories to a zero of the operator $A+D+{N}_C,$ where $C=\zer B$ and $N_C$ denotes the normal cone operator of $C$. To this end we use Lyapunov analysis combined with the celebrated Opial Lemma in its ergodic continuous version. Furthermore, we show strong convergence for trajectories to the unique zero of $A+D+{N}_C$, provided that $A$ is a strongly monotone operator.

math.DS↗

arXiv2016

A general double-proximal gradient algorithm for d.c. programming

The possibilities of exploiting the special structure of d.c. programs, which consist of optimizing the difference of convex functions, are currently more or less limited to variants of the DCA proposed by Pham Dinh Tao and Le Thi Hoai An in 1997. These assume that either the convex or the concave part, or both, are evaluated by one of their subgradients. In this paper we propose an algorithm which allows the evaluation of both the concave and the convex part by their proximal points. Additionally, we allow a smooth part, which is evaluated via its gradient. In the spirit of primal-dual splitting algorithms, the concave part might be the composition of a concave function with a linear operator, which are, however, evaluated separately. For this algorithm we show that every cluster point is a solution of the optimization problem. Furthermore, we show the connection to the Toland dual problem and prove a descent property for the objective function values of a primal-dual formulation of the problem. Convergence of the iterates is shown if this objective function satisfies the Kurdyka--Łojasiewicz property. In the last part, we apply the algorithm to an image processing model.

math.OC↗

arXiv2016

Approaching nonsmooth nonconvex optimization problems through first order dynamical systems with hidden acceleration and Hessian driven damping terms

In this paper we carry out an asymptotic analysis of the proximal-gradient dynamical system \begin{equation*}\left\{ \begin{array}{ll} \dot x(t) +x(t) = \prox_{γf}\big[x(t)-γ\nablaΦ(x(t))-ax(t)-by(t)\big],\\ \dot y(t)+ax(t)+by(t)=0 \end{array}\right.\end{equation*} where $f$ is a proper, convex and lower semicontinuous function, $Φ$ a possibly nonconvex smooth function and $γ, a$ and $b$ are positive real numbers. We show that the generated trajectories approach the set of critical points of $f+Φ$, here understood as zeros of its limiting subdifferential, under the premise that a regularization of this sum function satisfies the Kurdyka-Łojasiewicz property. We also establish convergence rates for the trajectories, formulated in terms of the Łojasiewicz exponent of the considered regularization function.

math.OC↗

arXiv2016

A second order dynamical system with Hessian-driven damping and penalty term associated to variational inequalities

We consider the minimization of a convex objective function subject to the set of minima of another convex function, under the assumption that both functions are twice continuously differentiable. We approach this optimization problem from a continuous perspective by means of a second order dynamical system with Hessian-driven damping and a penalty term corresponding to the constrained function. By constructing appropriate energy functionals, we prove weak convergence of the trajectories generated by this differential equation to a minimizer of the optimization problem as well as convergence for the objective function values along the trajectories. The performed investigations rely on Lyapunov analysis in combination with the continuous version of the Opial Lemma. In case the objective function is strongly convex, we can even show strong convergence of the trajectories.

math.OC↗

arXiv2016

Conditional stability versus ill-posedness for operator equations with monotone operators in Hilbert space

In the literature on singular perturbation (Lavrentiev regularization) for the stable approximate solution of operator equations with monotone operators in the Hilbert space the phenomena of conditional stability and local well-posedness and ill-posedness are rarely investigated. Our goal is to present some studies which try to bridge this gap. So we discuss the impact of conditional stability on error estimates and convergence rates for the Lavrentiev regularization and distinguish for linear problems well-posedness and ill-posedness in a specific manner motivated by a saturation result. The role of the regularization error in the noise-free case, called bias, is a crucial point in the paper for nonlinear and linear problems. In particular, for linear operator equations general convergence rates, including logarithmic rates, are derived by means of the method of approximate source conditions. This allows us to extend well-known convergence rates results for the Lavrentiev regularization that were based on general source conditions to the case of non-selfadjoint linear monotone forward operators for which general source conditions fail. Examples presenting the self-adjoint multiplication operator as well as the non-selfadjoint fractional integral operator and Cesàro operator illustrate the theoretical results. Extensions to the nonlinear case under specific conditions on the nonlinearity structure complete the paper.

math.NA↗

arXiv2016

Levenberg-Marquardt dynamics associated to variational inequalities

In connection with the optimization problem $$\inf_{x\in argmin Ψ}\{Φ(x)+Θ(x)\},$$ where $Φ$ is a proper, convex and lower semicontinuous function and $Θ$ and $Ψ$ are convex and smooth functions defined on a real Hilbert space, we investigate the asymptotic behavior of the trajectories of the nonautonomous Levenberg-Marquardt dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partialΦ(x(t))\\ λ(t)\dot x(t) + \dot v(t) + v(t) + \nabla Θ(x(t))+β(t)\nabla Ψ(x(t))=0, \end{array}\right.\end{equation*} where $λ$ and $β$ are functions of time controlling the velocity and the penalty term, respectively. We show weak convergence of the generated trajectory to an optimal solution as well as convergence of the objective function values along the trajectories, provided $λ$ is monotonically decreasing, $β$ satisfies a growth condition and a relation expressed via the Fenchel conjugate of $Ψ$ is fulfilled. When the objective function is assumed to be strongly convex, we can even show strong convergence of the trajectories.

math.OC↗

arXiv2016

Second order forward-backward dynamical systems for monotone inclusion problems

We begin by considering second order dynamical systems of the from $\ddot x(t) + γ(t)\dot x(t) + λ(t)B(x(t))=0$, where $B: {\cal H}\rightarrow{\cal H}$ is a cocoercive operator defined on a real Hilbert space ${\cal H}$, $λ:[0,+\infty)\rightarrow [0,+\infty)$ is a relaxation function and $γ:[0,+\infty)\rightarrow [0,+\infty)$ a damping function, both depending on time. For the generated trajectories, we show existence and uniqueness of the generated trajectories as well as their weak asymptotic convergence to a zero of the operator $B$. The framework allows to address from similar perspectives second order dynamical systems associated with the problem of finding zeros of the sum of a maximally monotone operator and a cocoercive one. This captures as particular case the minimization of the sum of a nonsmooth convex function with a smooth convex one. Furthermore, we prove that when $B$ is the gradient of a smooth convex function the value of the latter converges along the ergodic trajectory to its minimal value with a rate of ${\cal O}(1/t)$.

math.OC↗

arXiv2016

Second order dynamical systems associated to variational inequalities

We investigate the asymptotic convergence of the trajectories generated by the second order dynamical system $\ddot x(t) + γ\dot x(t) + \nabla ϕ(x(t))+β(t)\nabla ψ(x(t))=0$, where $ϕ,ψ:{\cal H}\rightarrow \R$ are convex and smooth functions defined on a real Hilbert space ${\cal H}$, $γ>0$ and $β$ is a function of time which controls the penalty term. We show weak convergence of the trajectories to a minimizer of the function $ϕ$ over the (nonempty) set of minima of $ψ$ as well as convergence for the objective function values along the trajectories, provided a condition expressed via the Fenchel conjugate of $ψ$ is fulfilled. When the function $ϕ$ is assumed to be strongly convex, we can even show strong convergence of the trajectories. The results can be seen as the second order counterparts of the ones given by Attouch and Czarnecki (Journal of Differential Equations 248(6), 1315--1344, 2010) for first order dynamical systems associated to constrained variational inequalities. At the same time we give a positive answer to an open problem posed in \cite{att-cza-16} by the same authors.

math.OC↗

arXiv2016

Proximal-gradient algorithms for fractional programming

In this paper we propose two proximal gradient algorithms for fractional programming problems in real Hilbert spaces, where the numerator is a proper, convex and lower semicontinuous function and the denominator is a smooth function, either concave or convex. In the iterative schemes, we perform a proximal step with respect to the nonsmooth numerator and a gradient step with respect to the smooth denominator. The algorithm in case of a concave denominator has the particularity that it generates sequences which approach both the (global) optimal solutions set and the optimal objective value of the underlying fractional programming problem. In case of a convex denominator the numerical scheme approaches the set of critical points of the objective function, provided the latter satisfies the Kurdyka-Łojasiewicz property.

math.OC↗

arXiv2015

Penalty schemes with inertial effects for monotone inclusion problems

We introduce a penalty term-based splitting algorithm with inertial effects designed for solving monotone inclusion problems involving the sum of maximally monotone operators and the convex normal cone to the (nonempty) set of zeros of a monotone and Lipschitz continuous operator. We show weak ergodic convergence of the generated sequence of iterates to a solution of the monotone inclusion problem, provided a condition expressed via the Fitzpatrick function of the operator describing the underlying set of the normal cone is verified. Under strong monotonicity assumptions we can even show strong nonergodic convergence of the iterates. This approach constitutes the starting point for investigating from a similar perspective monotone inclusion problems involving linear compositions of parallel-sum operators and, further, for the minimization of a complexly structured convex objective function subject to the set of minima of another convex and differentiable function.

math.OC↗

arXiv2015

A forward-backward dynamical approach to the minimization of the sum of a nonsmooth convex with a smooth nonconvex function

We address the minimization of the sum of a proper, convex and lower semicontinuous with a (possibly nonconvex) smooth function from the perspective of an implicit dynamical system of forward-backward type. The latter is formulated by means of the gradient of the smooth function and of the proximal point operator of the nonsmooth one. The trajectory generated by the dynamical system is proved to asymptotically converge to a critical point of the objective, provided a regularization of the latter satisfies the Kurdyka-Łojasiewicz property. Convergence rates for the trajectory in terms of the Łojasiewicz exponent of the regularized objective function are also provided.

math.OC↗

arXiv2015

A forward-backward-forward differential equation and its asymptotic properties

In this paper, we approach the problem of finding the zeros of the sum of a maximally monotone operator and a monotone and Lipschitz continuous one in a real Hilbert space via an implicit forward-backward-forward dynamical system with nonconstant relaxation parameters and stepsizes of the resolvents. Besides proving existence and uniqueness of strong global solutions for the differential equation under consideration, we show weak convergence of the generated trajectories and, under strong monotonicity assumptions, strong convergence with exponential rate. In the particular setting of minimizing the sum of a proper, convex and lower semicontinuous function with a smooth convex one, we provide a rate for the convergence of the objective function along the ergodic trajectory to its minimum value.

math.OC↗

arXiv2015

Convergence rates for forward-backward dynamical systems associated with strongly monotone inclusions

We investigate the convergence rates of the trajectories generated by implicit first and second order dynamical systems associated to the determination of the zeros of the sum of a maximally monotone operator and a monotone and Lipschitz continuous one in a real Hilbert space. We show that these trajectories strongly converge with exponential rate to a zero of the sum, provided the latter is strongly monotone. We derive from here convergence rates for the trajectories generated by dynamical systems associated to the minimization of the sum of a proper, convex and lower semicontinuous function with a smooth convex one provided the objective function fulfills a strong convexity assumption. In the particular case of minimizing a smooth and strongly convex function, we prove that its values converge along the trajectory to its minimum value with exponential rate, too.

math.OC↗