arXiv subjects

Panchajanya Sanyal

Publications and source records attributed to Panchajanya Sanyal.

2 recordsLinked to original sources

arXiv2026

Stochastic Sequential Quadratic Programming for Optimization with Functional Constraints

Stochastic convex optimization problems with nonlinear functional constraints are ubiquitous in signal processing applications including constrained least-squares, set-membership adaptive filtering, and trajectory optimization under uncertain fields. The presence of non-linear functional constraints renders the traditional projected stochastic gradient descent and related projection-based methods inefficient, and motivates the use of first-order methods. However, existing first-order methods, including primal and primal--dual algorithms, typically rely on a bounded (sub-)gradient assumption, which may be too restrictive in high-dimensional settings. We propose a stochastic sequential quadratic programming (SSQP) algorithm that works entirely in the primal domain, avoids projecting onto the feasible region, obviates the need for bounded gradients, and achieves state-of-the-art oracle complexity under standard smoothness and convexity assumptions. A faster version, namely SSQP-Skip, is also proposed where the quadratic subproblems can be skipped in most iterations. Finally, we develop an accelerated variance-reduced version of SSQP (VARAS), whose oracle complexity bounds match those for solving unconstrained finite-sum convex optimization problems. The superior performance of the proposed algorithms is demonstrated via numerical experiments on real datasets.

math.OC↗

arXiv2020

Nonparametric Compositional Stochastic Optimization for Risk-Sensitive Kernel Learning

In this work, we address optimization problems where the objective function is a nonlinear function of an expected value, i.e., compositional stochastic {strongly convex programs}. We consider the case where the decision variable is not vector-valued but instead belongs to a reproducing Kernel Hilbert Space (RKHS), motivated by risk-aware formulations of supervised learning and Markov Decision Processes defined over continuous spaces. We develop the first memory-efficient stochastic algorithm for this setting, which we call Compositional Online Learning with Kernels (COLK). COLK, at its core a two-time-scale stochastic approximation method, addresses the fact that (i) compositions of expected value problems cannot be addressed by classical stochastic gradient due to the presence of the inner expectation; and (ii) the RKHS-induced parameterization has complexity which is proportional to the iteration index which is mitigated through greedily constructed subspace projections. We establish almost sure convergence of COLK with attenuating step-sizes, and linear convergence in mean to a neighborhood with constant step-sizes, as well as the fact that its complexity is at-worst finite. The experiments with robust formulations of supervised learning demonstrate that COLK reliably converges, attains consistent performance across training runs, and thus overcomes overfitting.

math.OC↗