arXiv subjects

Khanh-Hung Giang-Tran

Publications and source records attributed to Khanh-Hung Giang-Tran.

7 recordsLinked to original sources

arXiv2026

Projection-Free Algorithms for Nonsmooth Stochastic Convex-Concave Saddle-Point Problems

We study nonsmooth convex-concave saddle-point problems over compact convex sets, assuming access to stochastic subgradients of the payoff function. We develop single-loop projection-free algorithms that use linear minimization oracles over the primal and dual domains. Unlike prior projection-free approaches that rely on smoothing, our methods are purely subgradient-based and handle nonsmoothness directly. This design makes the framework modular: when a linear minimization oracle is not used over a specific domain, the corresponding update can be replaced by a standard projection oracle without changing the single-loop structure. Thus, our framework covers both fully projection-free and hybrid oracle configurations. We prove anytime strong saddle gap guarantees under standard unbiased stochastic oracle assumptions with bounded variance. Our algorithms achieve an $O(ε^{-2})$ iteration complexity, matching projected stochastic subgradient methods. We also provide matching lower bounds for the corresponding oracle models, establishing minimax optimality. Our results show that, for nonsmooth stochastic minimax problems, the computational advantages of linear minimization oracles need not come at the expense of statistical or oracle efficiency.

math.OC↗

arXiv2026

Auto-Conditioned Frank-Wolfe Algorithms

Frank-Wolfe methods are projection-free algorithms for constrained optimization whose practical performance often depends critically on the choice of step size. Classical closed-loop step-size rules typically require prior knowledge of a global smoothness constant, while line-search variants avoid this requirement at the cost of additional function evaluations and implementation overhead. In this paper, we develop a fully auto-conditioned framework for Frank-Wolfe-type methods. The framework replaces the global Lipschitz constant in closed-loop step sizes with a local Lipschitz estimator computed from first-order information along the iterates. We show that this abstraction captures several important projection-free subroutines, including standard Frank-Wolfe, Matching Pursuit, pairwise Frank-Wolfe, and away-step Frank-Wolfe. For the resulting general class of methods, we establish convergence to stationary points in the nonconvex setting and recover the standard sublinear convergence guarantees in the convex setting, without requiring prior knowledge of a global smoothness constant. We further show that, when specialized to particular Frank-Wolfe variants and combined with additional structural assumptions, the same auto-conditioned framework yields accelerated convergence rates. Numerical experiments demonstrate that the proposed methods provide substantial practical improvements over line-search-based alternatives, highlighting the benefits of adapting to local curvature while retaining the simplicity of closed-loop step-size rules.

math.OC↗

arXiv2026

Projection-Free Algorithms for Minimax Problems

This paper addresses constrained smooth saddle-point problems in settings where projection onto the feasible sets is computationally expensive. We bridge the gap between projection-based and projection-free optimization by introducing a unified dual dynamic smoothing framework that enables the design of efficient single-loop algorithms. Within this framework, we establish convergence results for nonconvex-concave and nonconvex-strongly concave settings. Furthermore, we show that this framework is naturally applicable to convex-concave problems. We propose and analyze three algorithmic variants based on the application of a linear minimization oracle over the minimization variable, the maximization variable, or both. Notably, our analysis yields anytime convergence guarantees without requiring a pre-specified iteration horizon. These results significantly narrow the performance gap between projection-free and projection-based methods for minimax optimization.

math.OC↗

arXiv2025

Conditional Gradient Methods with Standard LMO for Stochastic Simple Bilevel Optimization

We propose efficient methods for solving stochastic simple bilevel optimization problems with convex inner levels, where the goal is to minimize an outer stochastic objective function subject to the solution set of an inner stochastic optimization problem. Existing methods often rely on costly projection or linear optimization oracles over complex sets, limiting their scalability. To overcome this, we propose an iteratively regularized conditional gradient approach that leverages linear optimization oracles exclusively over the base feasible set. Our proposed methods employ a vanishing regularization sequence that progressively emphasizes the inner problem while biasing towards desirable minimal outer objective solutions. In the one-sample stochastic setting and under standard convexity assumptions, we establish non-asymptotic convergence rates of $O(t^{-1/4})$ for both the outer and inner objectives. In the finite-sum setting with a mini-batch scheme, the corresponding rates become $O(t^{-1/2})$. When the outer objective is nonconvex, we prove non-asymptotic convergence rates of $O(t^{-1/7})$ for both the outer and inner objectives in the one-sample stochastic setting, and $O(t^{-1/4})$ in the finite-sum setting. Experimental results on over-parametrized regression and dictionary learning tasks demonstrate the practical advantages of our approach over existing methods, confirming our theoretical findings.

math.OC↗

arXiv2025

Convergence, Duality and Well-Posedness in Convex Bilevel Optimization

We consider the convex bilevel optimization problem, also known as simple bilevel programming. There are two challenges in solving convex bilevel optimization problems. Firstly, strong duality is not guaranteed due to the lack of Slater constraint qualification. Secondly, we demonstrate through an example that convergence of algorithms is not guaranteed even when usual subotimality gap bounds are present, due to the possibility of encountering super-optimal solutions. We show that strong duality (but not necessarily dual solvability) is exactly equivalent to ensuring correct asymptotic convergence of both inner and outer function values, and provide a simple condition that guarantees strong duality. Unfortunately, we also show that this simple condition is not sufficient to guarantee convergence to the optimal solution set. We draw connections to Levitin-Polyak well-posedness, and leverage this together with our strong duality equivalence to provide another condition that ensures convergence to the optimal solution set. We also discuss how our conditions have been implicitly present in existing algorithmic work.

math.OC↗

arXiv2024

A Projection-Free Method for Solving Convex Bilevel Optimization Problems

When faced with multiple minima of an "inner-level" convex optimization problem, the convex bilevel optimization problem selects an optimal solution which also minimizes an auxiliary "outer-level" convex objective of interest. Bilevel optimization requires a different approach compared to single-level optimization problems since the set of minimizers for the inner-level objective is not given explicitly. In this paper, we propose a new projection-free method for convex bilevel optimization which require only a linear optimization oracle over the base domain. We establish $O(t^{-1/2})$ convergence rate guarantees for our method in terms of both inner- and outer-level objectives, and demonstrate how additional assumptions such as quadratic growth and strong convexity result in accelerated rates of up to $O(t^{-1})$ and $O(t^{-2/3})$ for inner- and outer-levels respectively. Lastly, we conduct a numerical study to demonstrate the performance of our method.

math.OC↗

arXiv2024

Mistake, Manipulation and Margin Guarantees in Online Strategic Classification

We consider an online strategic classification problem where each arriving agent can manipulate their true feature vector to obtain a positive predicted label, while incurring a cost that depends on the amount of manipulation. The learner seeks to predict the agent's true label given access to only the manipulated features. After the learner releases their prediction, the agent's true label is revealed. Previous algorithms such as the strategic perceptron guarantee finitely many mistakes under a margin assumption on agents' true feature vectors. However, these are not guaranteed to encourage agents to be truthful. Promoting truthfulness is intimately linked to obtaining adequate margin on the predictions, thus we provide two new algorithms aimed at recovering the maximum margin classifier in the presence of strategic agent behavior. We prove convergence, finite mistake and finite manipulation guarantees for a variety of agent cost structures. We also provide generalized versions of the strategic perceptron with mistake guarantees for different costs. Our numerical study on real and synthetic data demonstrates that the new algorithms outperform previous ones in terms of margin, number of manipulation and number of mistakes.

cs.LG↗