arXiv subjects

Hung M. Phan

Publications and source records attributed to Hung M. Phan.

At least 19 recordsLinked to original sources

arXiv2026

Primal-dual splitting for structured composite monotone inclusions with or without cocoercivity

In this paper, we propose a primal-dual splitting algorithm for a broad class of structured composite monotone inclusions that involve finitely many set-valued operators, compositions of set-valued operators with bounded linear operators, and single-valued operators possibly without cocoercivity. The proposed algorithm is not only a unification for several contemporary algorithms but also a blueprint to generate new algorithms with graph-based structures using a single transparent convergence analysis. Our approach reduces dimensionality compared with the standard product space technique, which typically reformulates the original problem as the sum of two maximally monotone operators in order to apply splitting methods. It accommodates different cocoercive or Lipschitz constants as well as different resolvent parameters, and yields a larger allowable stepsize range than recent methods. We demonstrate the practicality of the approach by a numerical experiment on cancer detection using the decentralized fused LASSO problem.

math.OC↗

arXiv2025

Projected proximal gradient trust-region algorithm for nonsmooth optimization

We consider trust-region methods for solving optimization problems where the objective is the sum of a smooth, nonconvex function and a nonsmooth, convex regularizer. We extend the global convergence theory of such methods to include worst-case complexity bounds in the case of unbounded model Hessian growth, and introduce a new, simple nonsmooth trust-region subproblem solver based on combining several iterations of proximal gradient descent with a single projection into the trust region, which meets the sufficient descent requirements for algorithm convergence and has promising numerical results.

math.OC↗

arXiv2023

Stochastic Variance-Reduced Majorization-Minimization Algorithms

We study a class of nonconvex nonsmooth optimization problems in which the objective is a sum of two functions: One function is the average of a large number of differentiable functions, while the other function is proper, lower semicontinuous and has a surrogate function that satisfies standard assumptions. Such problems arise in machine learning and regularized empirical risk minimization applications. However, nonconvexity and the large-sum structure are challenging for the design of new algorithms. Consequently, effective algorithms for such scenarios are scarce. We introduce and study three stochastic variance-reduced majorization-minimization (MM) algorithms, combining the general MM principle with new variance-reduced techniques. We provide almost surely subsequential convergence of the generated sequence to a stationary point. We further show that our algorithms possess the best-known complexity bounds in terms of gradient evaluations. We demonstrate the effectiveness of our algorithms on sparse binary classification problems, sparse multi-class logistic regressions, and neural networks by employing several widely-used and publicly available data sets.

math.OC↗

arXiv2022

An Adaptive Alternating Direction Method of Multipliers

The alternating direction method of multipliers (ADMM) is a powerful splitting algorithm for linearly constrained convex optimization problems. In view of its popularity and applicability, a growing attention is drawn towards the ADMM in nonconvex settings. Recent studies of minimization problems for noncovex functions include various combinations of assumptions on the objective function including, in particular, a Lipschitz gradient assumption. We consider the case where the objective is the sum of a strongly convex function and a weakly convex function. To this end we present and study an adaptive version of the ADMM which incorporates generalized notions of convexity and varying penalty parameters adapted to the convexity constants of the functions. We prove convergence of the scheme under natural assumptions. To this end we employ the recent adaptive Douglas--Rachford algorithm by revisiting the well known duality relation between the classical ADMM and the Douglas--Rachford splitting algorithm, generalizing this connection to our setting. We illustrate our approach by relating and comparing to alternatives, and by numerical experiments on a signal denoising problem.

math.OC↗

arXiv2021

An adaptive splitting algorithm for the sum of three operators

Splitting algorithms for finding a zero of sum of operators often involve multiple steps which are referred to as forward or backward steps. Forward steps are the explicit use of the operators and backward steps involve the operators implicitly via their resolvents. In this paper, we study an adaptive splitting algorithm for finding a zero of the sum of three operators. We assume that two of the operators are generalized monotone and their resolvents are computable, while the other operator is cocoercive but its resolvent is missing or costly to compute. Our splitting algorithm adapts new parameters to the generalized monotonicity of the operators and, at the same time, combines appropriate forward and backward steps to guarantee convergence to a solution of the problem.

math.OC↗

arXiv2020

Conical averagedness and convergence analysis of fixed point algorithms

We study a conical extension of averaged nonexpansive operators and the role it plays in convergence analysis of fixed point algorithms. Various properties of conically averaged operators are systematically investigated, in particular, the stability under relaxations, convex combinations and compositions. We derive conical averagedness properties of resolvents of generalized monotone operators. These properties are then utilized in order to analyze the convergence of the proximal point algorithm, the forward-backward algorithm, and the adaptive Douglas-Rachford algorithm. Our study unifies, improves and casts new light on recent studies of these topics.

math.OC↗

arXiv2020

Demiclosedness principles for generalized nonexpansive mappings

Demiclosedness principles are powerful tools in the study of convergence of iterative methods. For instance, a multi-operator demiclosedness principle for firmly nonexpansive mappings is useful in obtaining simple and transparent arguments for the weak convergence of the shadow sequence generated by the Douglas-Rachford algorithm. We provide extensions of this principle which are compatible with the framework of more general families of mappings such as cocoercive and conically averaged mappings. As an application, we derive the weak convergence of the shadow sequence generated by the adaptive Douglas-Rachford algorithm.

math.OC↗

arXiv2019

Adaptive Douglas-Rachford splitting algorithm for the sum of two operators

The Douglas-Rachford algorithm is a classical and powerful splitting method for minimizing the sum of two convex functions and, more generally, finding a zero of the sum of two maximally monotone operators. Although this algorithm is well understood when the involved operators are monotone or strongly monotone, the convergence theory for weakly monotone settings is far from being complete. In this paper, we propose an adaptive Douglas-Rachford splitting algorithm for the sum of two operators, one of which is strongly monotone while the other one is weakly monotone. With appropriately chosen parameters, the algorithm converges globally to a fixed point from which we derive a solution of the problem. When one operator is Lipschitz continuous, we prove global linear convergence, which sharpens recent known results.

math.OC↗

arXiv2019

Multi-marginal maximal monotonicity and convex analysis

Monotonicity and convex analysis arise naturally in the framework of multi-marginal optimal transport theory. However, a comprehensive multi-marginal monotonicity and convex analysis theory is still missing. To this end we study extensions of classical monotone operator theory and convex analysis into the multi-marginal setting. We characterize multi-marginal c-monotonicity in terms of classical monotonicity and firmly nonexpansive mappings. We provide Minty type, continuity and conjugacy criteria for multi-marginal maximal monotonicity. We extend the partition of the identity into a sum of firmly nonexpansive mappings and Moreau's decomposition of the quadratic function into envelopes and proximal mappings into the multi-marginal settings. We illustrate our discussion with examples and provide applications for the determination of multi-marginal maximal monotonicity and multi-marginal conjugacy. We also point out several open questions.

math.FA↗

arXiv2019

Optimization of triangular networks with spatial constraints

A common representation of a three dimensional object in computer applications, such as graphics and design, is in the form of a triangular mesh. In many instances, individual or groups of triangles in such representation need to satisfy spatial constraints that are imposed either by observation from the real world, or by concrete design specifications of the object. As these problems tend to be of large scale, choosing a mathematical optimization approach can be particularly challenging. In this paper, we model various geometric constraints as convex sets in Euclidean spaces, and find the corresponding projections in closed forms. We also present an interesting idea to successfully maneuver around some important nonconvex constraints while still preserving the intrinsic nature of the original design problem. We then use these constructions in modern first-order splitting methods to find optimal solutions.

math.OC↗

arXiv2019

Conditional extragradient algorithms for solving variational inequalities

In this paper, we generalize the classical extragradient algorithm for solving variational inequality problems by utilizing nonzero normal vectors of the feasible set. In particular, conceptual algorithms are proposed with two different linesearchs. We then establish convergence results for these algorithms under mild assumptions. Our study suggests that nonzero normal vectors may significantly improve convergence if chosen appropriately.

math.OC↗

arXiv2018

Computing the resolvent of the sum of operators with application to best approximation problems

We propose a flexible approach for computing the resolvent of the sum of weakly monotone operators in real Hilbert spaces. This relies on splitting methods where strong convergence is guaranteed. We also prove linear convergence under Lipschitz continuity assumption. The approach is then applied to computing the proximity operator of the sum of weakly convex functions, and particularly to finding the best approximation to the intersection of convex sets.

math.OC↗

arXiv2018

Linear convergence of the generalized Douglas-Rachford algorithm for feasibility problems

In this paper, we study the generalized Douglas-Rachford algorithm and its cyclic variants which include many projection-type methods such as the classical Douglas-Rachford algorithm and the alternating projection algorithm. Specifically, we establish several local linear convergence results for the algorithm in solving feasibility problems with finitely many closed possibly nonconvex sets under different assumptions. Our findings not only relax some regularity conditions but also improve linear convergence rates in the literature. In the presence of convexity, the linear convergence is global.

math.OC↗

arXiv2017

Linear Convergence of Projection Algorithms

Projection algorithms are well known for their simplicity and flexibility in solving feasibility problems. They are particularly important in practice due to minimal requirements for software implementation and maintenance. In this work, we study linear convergence of several projection algorithms for systems of finitely many closed sets. The results complement contemporary research on the same topic.

math.OC↗

arXiv2015

On Slater's condition and finite convergence of the Douglas-Rachford algorithm

The Douglas-Rachford algorithm is a classical and very successful method for solving optimization and feasibility problems. In this paper, we provide novel conditions sufficient for finite convergence in the context of convex feasibility problems. Our analysis builds upon, and considerably extends, pioneering work by Spingarn. Specifically, we obtain finite convergence in the presence of Slater's condition in the affine-polyhedral and in a hyperplanar-epigraphical case. Various examples illustrate our results. Numerical experiments demonstrate the competitiveness of the Douglas-Rachford algorithm for solving linear equations with a positivity constraint when compared to the method of alternating projections and the method of reflection-projection.

math.OC↗

arXiv2015

Linear Convergence of the Douglas-Rachford Method for Two Closed Sets

In this paper, we investigate the Douglas-Rachford method for two closed (possibly nonconvex) sets in Euclidean spaces. We show that under certain regularity conditions, the Douglas-Rachford method converges locally with R-linear rate. In convex settings, we prove that the linear convergence is global. Our study recovers recent results on the same topic.

math.OC↗

arXiv2015

Proximal point algorithm, Douglas-Rachford algorithm and alternating projections: a case study

Many iterative methods for solving optimization or feasibility problems have been invented, and often convergence of the iterates to some solution is proven. Under favourable conditions, one might have additional bounds on the distance of the iterate to the solution leading thus to worst case estimates, i.e., how fast the algorithm must converge. Exact convergence estimates are typically hard to come by. In this paper, we consider the complementary problem of finding best case estimates, i.e., how slow the algorithm has to converge, and we also study exact asymptotic rates of convergence. Our investigation focuses on convex feasibility in the Euclidean plane, where one set is the real axis while the other is the epigraph of a convex function. This case study allows us to obtain various convergence rate results. We focus on the popular method of alternating projections and the Douglas-Rachford algorithm. These methods are connected to the proximal point algorithm which is also discussed. Our findings suggest that the Douglas-Rachford algorithm outperforms the method of alternating projections in the absence of constraint qualifications. Various examples illustrate the theory.

math.OC↗

arXiv2014

Stadium norm and Douglas-Rachford splitting: a new approach to road design optimization

The basic optimization problem of road design is quite challenging due to a objective function that is the sum of nonsmooth functions and the presence of set constraints. In this paper, we model and solve this problem by employing the Douglas-Rachford splitting algorithm. This requires a careful study of new proximity operators related to minimizing area and to the stadium norm. We compare our algorithm to a state-of-the-art projection algorithm. Our numerical results illustrate the potential of this algorithm to significantly reduce cost in road design.

math.OC↗