arXiv subjects

Parth Nobel

Publications and source records attributed to Parth Nobel.

9 recordsLinked to original sources

arXiv2026

GPU-Enabled Large-Scale Optimization Using Randomized Linear Algebra

This paper introduces rlaopt, a PyTorch-based package for large-scale optimization and scientific computing using randomized numerical linear algebra (RandNLA). Despite substantial progress in RandNLA-based algorithms, few implementations combine GPU acceleration with a simple interface for specifying optimization problems. rlaopt addresses this gap by providing GPU-enabled solvers for positive-definite linear systems and convex empirical risk minimization with constraints and regularizers. These solvers use RandNLA to accelerate conjugate gradient (NystromPCG), operator splitting (NysADMM), and stochastic gradient methods (SAPPHIRE). Moreover, rlaopt includes a modeling language that lets users specify problems using natural mathematical syntax. rlaopt automatically checks compatibility with the selected solver and performs the required problem decomposition. The solvers also support differentiation through their iterations, enabling applications such as hyperparameter tuning. Experiments on ridge regression, bounded multinomial logistic regression, and bounded elastic net identify when randomized preconditioning improves performance and demonstrate substantial speedups from GPU execution. The package is open-source under an Apache license, with source code at https://github.com/udellgroup/rlaopt and version 0.1.0 available on PyPI.

cs.LG↗

arXiv2026

CVXPY 1.9: Recent Advances in Optimization Modeling Software

CVXPY is a Python-embedded domain-specific language for convex optimization that lets users express problems in mathematical notation while the system verifies convexity and reduces valid programs to solver-ready form. This paper reports on the major advances from versions 1.1 through 1.9. These include a unified conic quadratic program (CQP) standard form for canonicalization; a stacked-slices backend that accelerates parameterized programs; first-class support for N-dimensional expressions; explicit sparsity for variables; support for multiple variable attributes; cones/atoms relevant to quantum information theory; and the introduction of disciplined nonlinear programming (DNLP). We outline the design, algorithms, and modeling consequences of these features.

math.OC↗

arXiv2026

Disciplined Nonlinear Programming

We introduce disciplined nonlinear programming (DNLP), a syntax for specifying nonlinear programming problems. DNLP is inspired by disciplined convex programming (DCP) and allows smooth functions to be freely mixed with nonsmooth convex and concave functions, with rules governing how the nonsmooth functions can be used. Problems expressed in DNLP form can be automatically canonicalized to a standard nonlinear programming (NLP) form and passed to a suitable NLP solver. As in DCP, the canonicalization relaxes nonsmooth convex and concave functions in a lossless way, allowing them to be handled by NLP solvers that require smooth functions. In addition to extending NLP to include useful nondifferentiable convex and concave functions, transforming the original problem to an equivalent NLP form offers several advantages, including simpler problem initialization. We describe the language and our open-source implementation of DNLP as an extension of CVXPY, a parser for DCP.

math.OC↗

arXiv2025

Differentiating Through a Quadratic Cone Program

Quadratic cone programs are rapidly becoming the standard canonical form for convex optimization problems. In this paper we address the question of differentiating the solution map for such problems, generalizing previous work for linear cone programs. We follow a similar path, using the implicit function theorem applied to the optimality conditions for a homogenous primal-dual embedding. Along with our proof of differentiability, we present methods for efficiently evaluating the derivative operator and its adjoint at a vector. Additionally, we present an open-source implementation of these methods, named \texttt{diffqcp}, that can execute on CPUs and GPUs. GPU-compatibility is already of consequence as it enables convex optimization solvers to be integrated into neural networks with reduced data movement, but we go a step further demonstrating that \texttt{diffqcp}'s performance on GPUs surpasses the performance of its CPU-based counterpart for larger quadratic cone programs.

math.OC↗

arXiv2024

CuClarabel: GPU Acceleration for a Conic Optimization Solver

We present the GPU implementation of the general-purpose interior-point solver Clarabel for convex optimization problems with conic constraints. We introduce a mixed parallel computing strategy that processes linear constraints first, then handles other conic constraints in parallel. The GPU solver currently supports linear equality and inequality constraints, second-order cones, exponential cones, power cones and positive semidefinite cones of the same dimensionality. We demonstrate that integrating a mixed parallel computing strategy with GPU-based direct linear system solvers enhances the performance of GPU-based conic solvers, surpassing their CPU-based counterparts across a wide range of conic optimization problems. We also show that employing mixed-precision linear system solvers can potentially achieve additional acceleration without compromising solution accuracy.

math.OC↗

arXiv2024

RandALO: Out-of-sample risk estimation in no time flat

Estimating out-of-sample risk for models trained on large high-dimensional datasets is an expensive but essential part of the machine learning process, enabling practitioners to optimally tune hyperparameters. Cross-validation (CV) serves as the de facto standard for risk estimation but poorly trades off high bias ($K$-fold CV) for computational cost (leave-one-out CV). We propose a randomized approximate leave-one-out (RandALO) risk estimator that is not only a consistent estimator of risk in high dimensions but also less computationally expensive than $K$-fold CV. We support our claims with extensive simulations on synthetic and real data and provide a user-friendly Python package implementing RandALO available on PyPI as randalo and at https://github.com/cvxgrp/randalo.

math.ST↗

arXiv2023

Fast Path Planning Through Large Collections of Safe Boxes

We present a fast algorithm for the design of smooth paths (or trajectories) that are constrained to lie in a collection of axis-aligned boxes. We consider the case where the number of these safe boxes is large, and basic preprocessing of them (such as finding their intersections) can be done offline. At runtime we quickly generate a smooth path between given initial and terminal positions. Our algorithm designs trajectories that are guaranteed to be safe at all times, and detects infeasibility whenever such a trajectory does not exist. Our algorithm is based on two subproblems that we can solve very efficiently: finding a shortest path in a weighted graph, and solving (multiple) convex optimal-control problems. We demonstrate the proposed path planner on large-scale numerical examples, and we provide an efficient open-source software implementation, fastpathplanning.

cs.RO↗

arXiv2022

Tractable Evaluation of Stein's Unbiased Risk Estimate with Convex Regularizers

Stein's unbiased risk estimate (SURE) gives an unbiased estimate of the $\ell_2$ risk of any estimator of the mean of a Gaussian random vector. We focus here on the case when the estimator minimizes a quadratic loss term plus a convex regularizer. For these estimators SURE can be evaluated analytically for a few special cases, and generically using recently developed general purpose methods for differentiating through convex optimization problems; these generic methods however do not scale to large problems. In this paper we describe methods for evaluating SURE that handle a wide class of estimators, and also scale to large problem sizes.

math.ST↗

arXiv2021

Computing Tighter Bounds on the $n$-Queens Constant via Newton's Method

In recent work Simkin shows that bounds on an exponent occurring in the famous $n$-queens problem can be evaluated by solving convex optimization problems, allowing him to find bounds far tighter than previously known. In this note we use Simkin's formulation, a sharper bound developed by Knuth, and a Newton method that scales to large problem instances, to find even sharper bounds.

math.OC↗