arXiv subjects

Nam Ho-Nguyen

Publications and source records attributed to Nam Ho-Nguyen.

At least 19 recordsLinked to original sources

arXiv2026

Auto-Conditioned Frank-Wolfe Algorithms

Frank-Wolfe methods are projection-free algorithms for constrained optimization whose practical performance often depends critically on the choice of step size. Classical closed-loop step-size rules typically require prior knowledge of a global smoothness constant, while line-search variants avoid this requirement at the cost of additional function evaluations and implementation overhead. In this paper, we develop a fully auto-conditioned framework for Frank-Wolfe-type methods. The framework replaces the global Lipschitz constant in closed-loop step sizes with a local Lipschitz estimator computed from first-order information along the iterates. We show that this abstraction captures several important projection-free subroutines, including standard Frank-Wolfe, Matching Pursuit, pairwise Frank-Wolfe, and away-step Frank-Wolfe. For the resulting general class of methods, we establish convergence to stationary points in the nonconvex setting and recover the standard sublinear convergence guarantees in the convex setting, without requiring prior knowledge of a global smoothness constant. We further show that, when specialized to particular Frank-Wolfe variants and combined with additional structural assumptions, the same auto-conditioned framework yields accelerated convergence rates. Numerical experiments demonstrate that the proposed methods provide substantial practical improvements over line-search-based alternatives, highlighting the benefits of adapting to local curvature while retaining the simplicity of closed-loop step-size rules.

math.OC↗

arXiv2026

Chebyshev Center-Based Direction Selection for Multi-Objective Optimization and Training PINNs

Physics-informed neural networks (PINNs) are a promising approach for solving partial differential equations (PDEs). Their training, however, is often difficult because multiple loss terms induced by PDE residuals and boundary or initial conditions must be optimized simultaneously. To address this difficulty, existing approaches often construct update directions by explicitly enforcing particular desirable properties, such as scale robustness and simultaneous descent. While effective in many cases, such property-by-property designs can make it unclear which conditions are essential, what geometric principle determines the selected update direction, and how different methods are structurally related. In this work, we formulate update-direction selection for PINN training as a Chebyshev-center problem in the dual cone. The proposed formulation selects a normalized direction that maximizes the minimum distance to the cone facets. The resulting formulation admits an efficient dual problem in a much lower-dimensional space and yields a convergence guarantee in the nonconvex setting. It also recovers the key desirable properties targeted by existing approaches without imposing them separately; rather, they follow from the single geometric criterion underlying the formulation. This makes the selected direction interpretable through a single geometric rule and provides a unified basis for systematically comparing related direction-selection methods. Experiments on several PINN benchmarks further demonstrate strong empirical performance of the proposed method.

cs.LG↗

arXiv2026

Projection-Free Algorithms for Minimax Problems

This paper addresses constrained smooth saddle-point problems in settings where projection onto the feasible sets is computationally expensive. We bridge the gap between projection-based and projection-free optimization by introducing a unified dual dynamic smoothing framework that enables the design of efficient single-loop algorithms. Within this framework, we establish convergence results for nonconvex-concave and nonconvex-strongly concave settings. Furthermore, we show that this framework is naturally applicable to convex-concave problems. We propose and analyze three algorithmic variants based on the application of a linear minimization oracle over the minimization variable, the maximization variable, or both. Notably, our analysis yields anytime convergence guarantees without requiring a pre-specified iteration horizon. These results significantly narrow the performance gap between projection-free and projection-based methods for minimax optimization.

math.OC↗

arXiv2025

Conditional Gradient Methods with Standard LMO for Stochastic Simple Bilevel Optimization

We propose efficient methods for solving stochastic simple bilevel optimization problems with convex inner levels, where the goal is to minimize an outer stochastic objective function subject to the solution set of an inner stochastic optimization problem. Existing methods often rely on costly projection or linear optimization oracles over complex sets, limiting their scalability. To overcome this, we propose an iteratively regularized conditional gradient approach that leverages linear optimization oracles exclusively over the base feasible set. Our proposed methods employ a vanishing regularization sequence that progressively emphasizes the inner problem while biasing towards desirable minimal outer objective solutions. In the one-sample stochastic setting and under standard convexity assumptions, we establish non-asymptotic convergence rates of $O(t^{-1/4})$ for both the outer and inner objectives. In the finite-sum setting with a mini-batch scheme, the corresponding rates become $O(t^{-1/2})$. When the outer objective is nonconvex, we prove non-asymptotic convergence rates of $O(t^{-1/7})$ for both the outer and inner objectives in the one-sample stochastic setting, and $O(t^{-1/4})$ in the finite-sum setting. Experimental results on over-parametrized regression and dictionary learning tasks demonstrate the practical advantages of our approach over existing methods, confirming our theoretical findings.

math.OC↗

arXiv2025

Convergence, Duality and Well-Posedness in Convex Bilevel Optimization

We consider the convex bilevel optimization problem, also known as simple bilevel programming. There are two challenges in solving convex bilevel optimization problems. Firstly, strong duality is not guaranteed due to the lack of Slater constraint qualification. Secondly, we demonstrate through an example that convergence of algorithms is not guaranteed even when usual subotimality gap bounds are present, due to the possibility of encountering super-optimal solutions. We show that strong duality (but not necessarily dual solvability) is exactly equivalent to ensuring correct asymptotic convergence of both inner and outer function values, and provide a simple condition that guarantees strong duality. Unfortunately, we also show that this simple condition is not sufficient to guarantee convergence to the optimal solution set. We draw connections to Levitin-Polyak well-posedness, and leverage this together with our strong duality equivalence to provide another condition that ensures convergence to the optimal solution set. We also discuss how our conditions have been implicitly present in existing algorithmic work.

math.OC↗

arXiv2025

Efficient Online Large-Margin Classification via Dual Certificates

Online classification is a central problem in optimization, statistical learning and data science. Classical algorithms such as the perceptron offer efficient updates and finite mistake guarantees on linearly separable data, but they do not exploit the underlying geometric structure of the classification problem. We study the offline maximum margin problem through its dual formulation and use the resulting geometric insights to design a principled and efficient algorithm for the online setting. A key feature of our method is its translation invariance, inherited from the offline formulation, which plays a central role in its performance analysis. Our theoretical analysis yields improved mistake and margin bounds that depend only on translation-invariant quantities, offering stronger guarantees than existing algorithms under the same assumptions in favorable settings. In particular, we identify a parameter regime where our algorithm makes at most two mistakes per sequence, whereas the perceptron can be forced to make arbitrarily many mistakes. Our numerical study on real data further demonstrates that our method matches the computational efficiency of existing online algorithms, while significantly outperforming them in accuracy.

math.OC↗

arXiv2025

Poisson Inventory Models with Many Items: An Empirical Bayes Approach

We consider inventory decisions with many items, each of which has Poisson demand. The rate of demand for individual items is estimated on the basis of observations of past demand. The problem is to determine the items to hold in stock and the amount of each one. Our setting provides a natural framework for the application of the empirical Bayes methodology. We show how to do this in practice and demonstrate the importance of making posterior estimates of different demand levels, rather than just estimating the Poisson rate. We also address the question of when it is beneficial to separately analyse a group of items which are distinguished in some way. An example occurs when looking at inventory for a book retailer, who may find it advantageous to look separately at certain types of book (e.g. biographies). The empirical Bayes methodology is valuable when dealing with items having Poisson demand, and can be effective even with relatively small numbers of distinct items (e.g. 100). We discuss the best way to apply an empirical Bayes methodology in this context, and also show that doing this in the wrong way will reduce or eliminate the potential benefits.

stat.ME↗

arXiv2024

A Projection-Free Method for Solving Convex Bilevel Optimization Problems

When faced with multiple minima of an "inner-level" convex optimization problem, the convex bilevel optimization problem selects an optimal solution which also minimizes an auxiliary "outer-level" convex objective of interest. Bilevel optimization requires a different approach compared to single-level optimization problems since the set of minimizers for the inner-level objective is not given explicitly. In this paper, we propose a new projection-free method for convex bilevel optimization which require only a linear optimization oracle over the base domain. We establish $O(t^{-1/2})$ convergence rate guarantees for our method in terms of both inner- and outer-level objectives, and demonstrate how additional assumptions such as quadratic growth and strong convexity result in accelerated rates of up to $O(t^{-1})$ and $O(t^{-2/3})$ for inner- and outer-levels respectively. Lastly, we conduct a numerical study to demonstrate the performance of our method.

math.OC↗

arXiv2024

Mistake, Manipulation and Margin Guarantees in Online Strategic Classification

We consider an online strategic classification problem where each arriving agent can manipulate their true feature vector to obtain a positive predicted label, while incurring a cost that depends on the amount of manipulation. The learner seeks to predict the agent's true label given access to only the manipulated features. After the learner releases their prediction, the agent's true label is revealed. Previous algorithms such as the strategic perceptron guarantee finitely many mistakes under a margin assumption on agents' true feature vectors. However, these are not guaranteed to encourage agents to be truthful. Promoting truthfulness is intimately linked to obtaining adequate margin on the predictions, thus we provide two new algorithms aimed at recovering the maximum margin classifier in the presence of strategic agent behavior. We prove convergence, finite mistake and finite manipulation guarantees for a variety of agent cost structures. We also provide generalized versions of the strategic perceptron with mistake guarantees for different costs. Our numerical study on real and synthetic data demonstrates that the new algorithms outperform previous ones in terms of margin, number of manipulation and number of mistakes.

cs.LG↗

arXiv2023

Non-Smooth, Hölder-Smooth, and Robust Submodular Maximization

We study the problem of maximizing a continuous DR-submodular function that is not necessarily smooth. We prove that the continuous greedy algorithm achieves an $[(1-1/e)\OPT-ε]$ guarantee when the function is monotone and Hölder-smooth, meaning that it admits a Hölder-continuous gradient. For functions that are non-differentiable or non-smooth, we propose a variant of the mirror-prox algorithm that attains an $[(1/2)\OPT-ε]$ guarantee. We apply our algorithmic frameworks to robust submodular maximization and distributionally robust submodular maximization under Wasserstein ambiguity. In particular, the mirror-prox method applies to robust submodular maximization to obtain a single feasible solution whose value is at least $(1/2)\OPT-ε$. For distributionally robust maximization under Wasserstein ambiguity, we deduce and work over a submodular-convex maximin reformulation whose objective function is Hölder-smooth, for which we may apply both the continuous greedy and the mirror-prox algorithms.

math.OC↗

arXiv2023

Projection-Free Online Convex Optimization with Stochastic Constraints

This paper develops projection-free algorithms for online convex optimization with stochastic constraints. We design an online primal-dual projection-free framework that can take any projection-free algorithms developed for online convex optimization with no long-term constraint. With this general template, we deduce sublinear regret and constraint violation bounds for various settings. Moreover, for the case where the loss and constraint functions are smooth, we develop a primal-dual conditional gradient method that achieves $O(\sqrt{T})$ regret and $O(T^{3/4})$ constraint violations. Furthermore, for the setting where the loss and constraint functions are stochastic and strong duality holds for the associated offline stochastic optimization problem, we prove that the constraint violation can be reduced to have the same asymptotic growth as the regret.

math.OC↗

arXiv2022

Political districting without geography

Geographical considerations such as contiguity and compactness are necessary elements of political districting in practice. Yet an analysis of the problem without such constraints yields mathematical insights that can inform real-world model construction. In particular, it clarifies the sharp contrast between proportionality and competitiveness and how it might be overcome in a properly formulated objective function. It also reveals serious weaknesses of the much-discussed efficiency gap as a criterion for gerrymandering.

physics.soc-ph↗

arXiv2021

Adversarial Classification via Distributional Robustness with Wasserstein Ambiguity

We study a model for adversarial classification based on distributionally robust chance constraints. We show that under Wasserstein ambiguity, the model aims to minimize the conditional value-at-risk of the distance to misclassification, and we explore links to adversarial classification models proposed earlier and to maximum-margin classifiers. We also provide a reformulation of the distributionally robust model for linear classification, and show it is equivalent to minimizing a regularized ramp loss objective. Numerical experiments show that, despite the nonconvexity of this formulation, standard descent methods appear to converge to the global minimizer for this problem. Inspired by this observation, we show that, for a certain class of distributions, the only stationary point of the regularized ramp loss minimization problem is the global minimizer.

cs.LG↗

arXiv2021

Strong Formulations for Distributionally Robust Chance-Constrained Programs with Left-Hand Side Uncertainty under Wasserstein Ambiguity

Distributionally robust chance-constrained programs (DR-CCP) over Wasserstein ambiguity sets exhibit attractive out-of-sample performance and admit big-$M$-based mixed-integer programming (MIP) reformulations with conic constraints. However, the resulting formulations often suffer from scalability issues as sample size increases. To address this shortcoming, we derive stronger formulations that scale well with respect to the sample size. Our focus is on ambiguity sets under the so-called left-hand side (LHS) uncertainty, where the uncertain parameters affect the coefficients of the decision variables in the linear inequalities defining the safety sets. The interaction between the uncertain parameters and the variable coefficients in the safety set definition causes challenges in strengthening the original big-$M$ formulations. By exploiting the connection between nominal chance-constrained programs and DR-CCP, we obtain strong formulations with significant enhancements. In particular, through this connection, we derive a linear number of valid inequalities, which can be immediately added to the formulations to obtain improved formulations in the original space of variables. In addition, we suggest a quantile-based strengthening procedure that allows us to reduce the big-$M$ coefficients drastically. Furthermore, based on this procedure, we propose an exponential class of inequalities that can be separated efficiently within a branch-and-cut framework. The quantile-based strengthening procedure can be expensive. Therefore, for the special case of covering and packing type problems, we identify an efficient scheme to carry out this procedure. We demonstrate the computational efficacy of our proposed formulations on two classes of problems, namely stochastic portfolio optimization and resource planning.

math.OC↗

arXiv2020

Risk Guarantees for End-to-End Prediction and Optimization Processes

Prediction models are often employed in estimating parameters of optimization models. Despite the fact that in an end-to-end view, the real goal is to achieve good optimization performance, the prediction performance is measured on its own. While it is usually believed that good prediction performance in estimating the parameters will result in good subsequent optimization performance, formal theoretical guarantees on this are notably lacking. In this paper, we explore conditions that allow us to explicitly describe how the prediction performance governs the optimization performance. Our weaker condition allows for an asymptotic convergence result, while our stronger condition allows for exact quantification of the optimization performance in terms of the prediction performance. In general, verification of these conditions is a non-trivial task. Nevertheless, we show that our weaker condition is equivalent to the well-known Fisher consistency concept from the learning theory literature. This then allows us to easily check our weaker condition for several loss functions. We also establish that the squared error loss function satisfies our stronger condition. Consequently, we derive the exact theoretical relationship between prediction performance measured with the squared loss, as well as a class of symmetric loss functions, and the subsequent optimization performance. In a computational study on portfolio optimization, fractional knapsack and multiclass classification problems, we compare the optimization performance of using of several prediction loss functions (some that are Fisher consistent and some that are not) and demonstrate that lack of consistency of the loss function can indeed have a detrimental effect on performance.

math.OC↗

arXiv2020

Distributionally Robust Chance-Constrained Programs with Right-Hand Side Uncertainty under Wasserstein Ambiguity

We consider exact deterministic mixed-integer programming (MIP) reformulations of distributionally robust chance-constrained programs (DR-CCP) with random right-hand sides over Wasserstein ambiguity sets. The existing MIP formulations are known to have weak continuous relaxation bounds, and, consequently, for hard instances with small radius, or with large problem sizes, the branch-and-bound based solution processes suffer from large optimality gaps even after hours of computation time. This significantly hinders the practical application of the DR-CCP paradigm. Motivated by these challenges, we conduct a polyhedral study to strengthen these formulations. We reveal several hidden connections between DR-CCP and its nominal counterpart (the sample average approximation), mixing sets, and robust 0-1 programming. By exploiting these connections in combination, we provide an improved formulation and two classes of valid inequalities for DR-CCP. We test the impact of our results on a stochastic transportation problem numerically. Our experiments demonstrate the effectiveness of our approach; in particular our improved formulation and proposed valid inequalities reduce the overall solution times remarkably. Moreover, this allows us to significantly scale up the problem sizes that can be handled in such DR-CCP formulations by reducing the solution times from hours to seconds.

math.OC↗

arXiv2020

Dynamic Data-Driven Estimation of Non-Parametric Choice Models

We study non-parametric estimation of choice models, which were introduced to alleviate unreasonable assumptions in traditional parametric models, and are prevalent in several application areas. Existing literature focuses only on the static observational setting where all of the observations are given upfront, they are not equipped with explicit convergence rate guarantees, and consequently they cannot provide an a priori analysis for the model accuracy vs sparsity trade-off on the actual estimated model returned by their algorithms. As opposed to this, we focus on estimating a non-parametric choice model from observational data in a \emph{dynamic} setting, where observations are obtained over time. We show that choice model estimation can be cast as a convex-concave saddle-point (SP) joint estimation and optimization (JEO) problem, and we provide a primal-dual framework for deriving algorithms to solve this based on online convex optimization. By tailoring our framework carefully to the choice model estimation problem, we obtain tractable algorithms with provable convergence guarantees and explicit bounds on the sparsity of the estimated model. Our numerical experiments confirm the effectiveness of the algorithms derived from our framework.

math.OC↗

arXiv2020

Coordinate Descent Without Coordinates: Tangent Subspace Descent on Riemannian Manifolds

We extend coordinate descent to manifold domains, and provide convergence analyses for geodesically convex and non-convex smooth objective functions. Our key insight is to draw an analogy between coordinate blocks in Euclidean space and tangent subspaces of a manifold. Hence, our method is called tangent subspace descent (TSD). The core principle behind ensuring convergence of TSD is the appropriate choice of subspace at each iteration. To this end, we propose two novel conditions, the gap ensuring and $C$-randomized norm conditions on deterministic and randomized modes of subspace selection respectively, that promise convergence for smooth functions and that are satisfied in practical contexts. We propose two subspace selection rules of particular practical interest that satisfy these conditions: a deterministic one for the manifold of square orthogonal matrices, and a randomized one for the Stiefel manifold. Our proof-of-concept numerical experiments on the orthogonal Procrustes problem demonstrate TSD's efficacy.

math.OC↗