arXiv subjects

Karthik Prakhya

Publications and source records attributed to Karthik Prakhya.

5 recordsLinked to original sources

arXiv2026

Inexact PDHG for Large-Scale Semidefinite Programming

First-order methods, such as the primal-dual hybrid gradient (PDHG) method, have recently been explored for large-scale semidefinite programming (SDP). However, exact projections can be computationally prohibitive, motivating inexact projections using conditional gradient methods. Although computationally cheap and projection-free, these methods typically converge sublinearly. Recent variants, such as Spectrahedron Frank-Wolfe (Garber, 2026), achieve linear convergence in expectation near the optimal face of the spectrahedron. We develop an inexact PDHG framework with tractable spectrahedral projections using two techniques: (i) adaptive error tolerances that avoid unnecessary inner iterations while guaranteeing convergence, and (ii) a heuristic spectral warm start based on the Lanczos algorithm that accelerates Spectrahedron Frank-Wolfe by initializing near the optimal face. We establish an O(1/t) convergence rate for PDHG in the presence of projection errors and estimate the total number of inner iterations required for convergence. Experiments on large-scale SDP instances demonstrate that combining spectral warm starts with Spectrahedron Frank-Wolfe outperforms Lanczos-based inexact projections.

math.OC↗

arXiv2026

Primal-Dual Adaptive Subgradient Methods

We propose adaptive primal-dual subgradient methods for constrained convex optimization. Our first method applies an AdaGrad-style step-sizes to the dual problem and recovers primal solutions by averaging the associated Fenchel-type oracle outputs. It achieves $\widetilde{\mathcal{O}}(T^{-1/2})$ convergence for the dual objective residual, primal objective residual, and feasibility gap in the nonsmooth regime, while automatically improving to $\mathcal{O}(T^{-1})$ when the dual objective is smooth. We further develop an accelerated variant, which retains the $\widetilde{\mathcal{O}}(T^{-1/2})$ nonsmooth rate while achieving $\mathcal{O}(T^{-2})$ convergence in the smooth regime under mild assumptions. These guarantees require no prior knowledge of the underlying regularity. We demonstrate the empirical performance of the proposed methods on optimal transport and semidefinite relaxation of the Max-Cut problem.

math.OC↗

arXiv2026

Exact Convex Reformulations of Linear Neural Networks via Completely Positive Lifting

We show that the training problem of a deep linear neural network under the squared loss admits an exact convex reformulation in a lifted space over a generalized completely positive cone. The reformulation has the same optimal value as the original nonconvex problem and is linear in the lifted variables, with all nonconvexity encoded in the cone constraint. Its ambient lifted dimension depends only on the input and output dimensions, independent of the network depth and the number of data points, and the bottleneck width enters only through scalar constraints. The construction proceeds by reducing the multilayer parameterization to a bilinear factorization, lifting it to a rank-constrained semidefinite program, expressing the rank constraint via a complementarity condition, and applying a completely positive lifting. The resulting formulation gives a conic representation of the nonconvexity induced by linear factorization and connects linear neural network training with copositive programming.

cs.LG↗

arXiv2025

Convex Formulations for Training Two-Layer ReLU Neural Networks

Solving non-convex, NP-hard optimization problems is crucial for training machine learning models, including neural networks. However, non-convexity often leads to black-box machine learning models with unclear inner workings. While convex formulations have been used for verifying neural network robustness, their application to training neural networks remains less explored. In response to this challenge, we reformulate the problem of training infinite-width two-layer ReLU networks as a convex completely positive program in a finite-dimensional (lifted) space. Despite the convexity, solving this problem remains NP-hard due to the complete positivity constraint. To overcome this challenge, we introduce a semidefinite relaxation that can be solved in polynomial time. We then experimentally evaluate the tightness of this relaxation, demonstrating its competitive performance in test accuracy across a range of classification tasks.

cs.LG↗

arXiv2024

Federated Frank-Wolfe Algorithm

Federated learning (FL) has gained a lot of attention in recent years for building privacy-preserving collaborative learning systems. However, FL algorithms for constrained machine learning problems are still limited, particularly when the projection step is costly. To this end, we propose a Federated Frank-Wolfe Algorithm (FedFW). FedFW features data privacy, low per-iteration cost, and communication of sparse signals. In the deterministic setting, FedFW achieves an $\varepsilon$-suboptimal solution within $O(\varepsilon^{-2})$ iterations for smooth and convex objectives, and $O(\varepsilon^{-3})$ iterations for smooth but non-convex objectives. Furthermore, we present a stochastic variant of FedFW and show that it finds a solution within $O(\varepsilon^{-3})$ iterations in the convex setting. We demonstrate the empirical performance of FedFW on several machine learning tasks.

cs.LG↗