arXiv subjects

Ketan Rajawat

Publications and source records attributed to Ketan Rajawat.

At least 19 recordsLinked to original sources

arXiv2026

Hinge-Proximal Stochastic Gradient Methods for Convex Optimization with Functional Constraints

This paper considers stochastic convex optimization problems with smooth functional constraints arising in constrained estimation and robust signal recovery. We operate in the high-dimensional and highly-constrained setting, where oracle access is restricted to one or a few objective and constraint gradients per-iteration, as in streaming or online estimation. Existing approaches to solve such problems are based on either the stochastic primal-dual or stochastic subgradient methods, and require globally Lipschitz continuous constraint functions. In this work, we develop a hinge-proximal framework that utilizes an exact penalty reformulation to yield updates involving only one linearized constraint (and hence accessing one constraint gradient) per-iteration. The updates also admit a novel hinge-proximal three-point inequality relying on smoothness rather than global Lipschitz continuity of the constraint functions. The framework leads to three algorithms: a baseline hinge-proximal SGD (HPS), a variance-reduced HPS version for finite-sum settings, and a nested HPS version whose performance depends on a geometric regularity constant of the constraint region rather than explicitly on the number of constraints, while achieving near-SGD sample complexity. The superior empirical performance of the proposed algorithms is demonstrated on a robust regression problem with noisy features, representative of errors-in-variables estimation.

math.OC↗

arXiv2026

UNet: A Generic and Reliable Multi-UAV Communication and Networking Architecture for Heterogeneous Applications

The rapid growth of UAV applications necessitates a robust communication and networking system architecture capable of addressing the diverse requirements of various applications concurrently, rather than relying on application-specific solutions. This paper proposes a generic and reliable multi-UAV communication and networking system architecture designed to support the varying demands of heterogeneous applications, including short-range and long-range communication, star and mesh topologies, different data rates, and multiple wireless standards. Our architecture is designed for both ad hoc and infrastructure networks, ensuring seamless connectivity throughout the network. Additionally, we present the design of a multi-protocol UAV gateway that enables interoperability among various communication protocols to enhance connectivity. Furthermore, we introduce a data processing and service layer framework with a graphical user interface of a ground control station that facilitates remote control and monitoring from any location at any time. We practically implemented the proposed architecture and evaluated its performance using different metrics, demonstrating its effectiveness.

cs.NI↗

arXiv2026

Stochastic Sequential Quadratic Programming for Optimization with Functional Constraints

Stochastic convex optimization problems with nonlinear functional constraints are ubiquitous in signal processing applications including constrained least-squares, set-membership adaptive filtering, and trajectory optimization under uncertain fields. The presence of non-linear functional constraints renders the traditional projected stochastic gradient descent and related projection-based methods inefficient, and motivates the use of first-order methods. However, existing first-order methods, including primal and primal--dual algorithms, typically rely on a bounded (sub-)gradient assumption, which may be too restrictive in high-dimensional settings. We propose a stochastic sequential quadratic programming (SSQP) algorithm that works entirely in the primal domain, avoids projecting onto the feasible region, obviates the need for bounded gradients, and achieves state-of-the-art oracle complexity under standard smoothness and convexity assumptions. A faster version, namely SSQP-Skip, is also proposed where the quadratic subproblems can be skipped in most iterations. Finally, we develop an accelerated variance-reduced version of SSQP (VARAS), whose oracle complexity bounds match those for solving unconstrained finite-sum convex optimization problems. The superior performance of the proposed algorithms is demonstrated via numerical experiments on real datasets.

math.OC↗

arXiv2026

Communication-Constrained Energy-Optimal Trajectory Generation for Quadrotor UAVs in Urban Environments

Communication-aware trajectory generation for unmanned aerial vehicles (UAVs) operating in urban environments requires simultaneous consideration of vehicle dynamics, wireless communication quality, obstacle avoidance, and onboard energy limitations. In such missions, UAVs must navigate through obstacle-rich environments while ensuring reliable relay of mission-critical sensory information to ground infrastructure. This results in a highly nonlinear and nonconvex optimal control problem involving coupled communication and flight-dynamics constraints. This paper presents a communication-constrained energy-optimal trajectory generation framework for quadrotor UAVs operating in urban environments. The proposed formulation incorporates full rigid-body quadrotor dynamics, urban wireless communication models, cumulative data throughput constraints, and obstacle avoidance requirements within a unified free-final-time optimal control framework. Unlike conventional approaches based on simplified kinematic or point-mass models, the proposed framework generates dynamically feasible trajectories suitable for practical aerial platforms. The resulting nonconvex optimal control problem is solved iteratively using sequential convex programming (SCP). Numerical simulations for multiple urban mission scenarios demonstrate the ability of the proposed framework to generate energy-efficient and communication-aware trajectories while adapting mission duration according to data relay requirements. The proposed methodology provides a practical framework for autonomous UAV operations requiring reliable communication in dense urban environments.

eess.SY↗

arXiv2026

Decentralized Stochastic Constrained Optimization via Prox-Linearization

This paper studies consensus-based decentralized stochastic optimization for minimizing possibly non-convex expected objectives with convex non-smooth regularizers and nonlinear functional inequality constraints. We reformulate the constrained problem using the exact-penalty model and develop two algorithms that require only local stochastic gradients and first-order constraint information. The first method, Decentralized Stochastic Momentum-based Prox-Linear Algorithm (D-SMPL), combines constraint linearization with a prox-linear step, resulting in a linearly constrained quadratic subproblem per iteration. Building on this approach, we propose a successive convex approximation (SCA) variant, Decentralized SCA Momentum-based Prox-Linear (D-SCAMPL), which handles additional objective structure through strongly convex surrogate subproblems while still allowing infeasible initialization. Both methods incorporate recursive momentum-based gradient estimators and a consensus mechanism requiring only two communication rounds per iteration. Under standard smoothness and regularity assumptions, both algorithms achieve an oracle complexity of $\mathcal{O}(ε^{-3/2})$, matching the optimal rate known for unconstrained centralized stochastic non-convex optimization. Numerical experiments on energy-optimal ocean trajectory planning corroborate the theory and demonstrate improved performance over existing decentralized baselines.

math.OC↗

arXiv2026

Rank-one Riemannian Subspace Descent for Nonlinear Matrix Equations

We propose a rank-one Riemannian subspace descent algorithm for computing symmetric positive definite (SPD) solutions to nonlinear matrix equations arising in control theory, dynamic programming, and stochastic filtering. For solution matrices of size $n\times n$, standard approaches for dense matrix equations typically incur $\mathcal{O}(n^3)$ cost per-iteration, while the efficient $\mathcal{O}(n^2)$ methods either rely on sparsity or low-rank solutions, or have iteration counts that scale poorly. The proposed method entails updating along the dominant eigen-component of a transformed Riemannian gradient, identified using at most $\mathcal{O}(\log(n))$ power iterations. The update structure also enables exact step-size selection in many cases at minimal additional cost. For objectives defined as compositions of standard matrix operations, each iteration can be implemented using only matrix--vector products, yielding $\mathcal{O}(n^2)$ arithmetic cost. We prove an $\mathcal{O}(n)$ iteration bound under standard smoothness assumptions, with improved bounds under geodesic strong convexity. Numerical experiments on large-scale CARE, DARE, and other nonlinear matrix equations show that the proposed algorithm solves instances (up to $n=10{,}000$ in our tests) for which the compared solvers, including MATLAB's \texttt{icare}, structure-preserving doubling, and subspace-descent baselines fail to return a solution. These results demonstrate that rank-one manifold updates provide a practical approach for high-dimensional and dense SPD-constrained matrix equations. MATLAB code implementation is publicly available on GitHub : \href{https://github.com/yogeshd-iitk/nonlinear_matrix_equation_R1RSD}{\textcolor{blue}{https://github.com/yogeshd-iitk/nonlinear\_matrix \_equation\_R1RSD}}

math.NA↗

arXiv2025

Constrained Stochastic Recursive Momentum Successive Convex Approximation

We consider stochastic optimization problems with non-convex functional constraints, such as those arising in trajectory generation, sparse approximation, and robust classification. To this end, we put forth a recursive momentum-based accelerated successive convex approximation (SCA) algorithm. At each iteration, the proposed algorithm entails constructing convex surrogates of the stochastic objective and the constraint functions, and solving the resulting convex optimization problem. A recursive update rule is employed to track the gradient of the stochastic objective function, which contributes to variance reduction and hence accelerates the algorithm convergence. A key ingredient of the proof is a new parameterized version of the standard Mangasarian-Fromowitz Constraints Qualification, that allows us to bound the dual variables and hence obtain problem-dependent bounds on the rate at which the iterates approach an $ε$-stationary point. Remarkably, the proposed algorithm achieves near-optimal stochastic first-order (SFO) complexity with adaptive step sizes closely matching that achieved by state-of-the-art stochastic optimization algorithms for solving unconstrained problems. As an example, we detail an obstacle-avoiding trajectory optimization problem that can be solved using the proposed algorithm and show that its performance is superior to that of the existing algorithms used for trajectory optimization. The performance of the proposed algorithm is also shown to be comparable to that of a specialized sparse classification algorithm applied to a binary classification problem.

math.OC↗

arXiv2025

Communication and Energy-Aware Multi-UAV Coverage Path Planning for Networked Operations

This paper presents a communication and energy-aware multi-UAV Coverage Path Planning (mCPP) method for scenarios requiring continuous inter-UAV communication, such as cooperative search and rescue and surveillance missions. Unlike existing mCPP solutions that focus on energy, time, or coverage efficiency, the proposed method generates coverage paths that minimize a specified combination of energy and inter-UAV connectivity radius. Key features of the proposed algorithm include a simplified and validated energy consumption model, an efficient connectivity radius estimator, and an optimization framework that enables us to search for the optimal paths over irregular and obstacle-rich regions. The effectiveness and utility of the proposed algorithm is validated through simulations on various test regions with and without no-fly-zones. Real-world experiments on a three-UAV system demonstrate the remarkably high 99% match between the estimated and actual communication range requirement.

cs.RO↗

arXiv2024

Analysis of Decentralized Stochastic Successive Convex Approximation for composite non-convex problems

This work considers the decentralized successive convex approximation (SCA) method for minimizing stochastic non-convex objectives subject to convex constraints, along with possibly non-smooth convex regularizers. Although SCA has been widely applied in decentralized settings, its stochastic first order (SFO) complexity is unknown, and it is thought to be slower than the centralized momentum-enhanced SCA variants. In this work, we advance the state-of-the-art for SCA methods by proposing an accelerated variant, namely the \textbf{D}ecentralized \textbf{M}omentum-based \textbf{S}tochastic \textbf{SCA} (\textbf{D-MSSCA}) and analyze its SFO complexity. The proposed algorithm entails creating a stochastic surrogate of the objective at every iteration, which is minimized at each node separately. Remarkably, the D-MSSCA achieves an SFO complexity of $\mathcal{O}(ε^{-3/2})$ to reach an $ε$-stationary point, which is at par with the SFO complexity lower bound for unconstrained stochastic non-convex optimization in centralized setting.

math.OC↗

arXiv2024

Sharpened Lazy Incremental Quasi-Newton Method

The problem of minimizing the sum of $n$ functions in $d$ dimensions is ubiquitous in machine learning and statistics. In many applications where the number of observations $n$ is large, it is necessary to use incremental or stochastic methods, as their per-iteration cost is independent of $n$. Of these, Quasi-Newton (QN) methods strike a balance between the per-iteration cost and the convergence rate. Specifically, they exhibit a superlinear rate with $O(d^2)$ cost in contrast to the linear rate of first-order methods with $O(d)$ cost and the quadratic rate of second-order methods with $O(d^3)$ cost. However, existing incremental methods have notable shortcomings: Incremental Quasi-Newton (IQN) only exhibits asymptotic superlinear convergence. In contrast, Incremental Greedy BFGS (IGS) offers explicit superlinear convergence but suffers from poor empirical performance and has a per-iteration cost of $O(d^3)$. To address these issues, we introduce the Sharpened Lazy Incremental Quasi-Newton Method (SLIQN) that achieves the best of both worlds: an explicit superlinear convergence rate, and superior empirical performance at a per-iteration $O(d^2)$ cost. SLIQN features two key changes: first, it incorporates a hybrid strategy of using both classic and greedy BFGS updates, allowing it to empirically outperform both IQN and IGS. Second, it employs a clever constant multiplicative factor along with a lazy propagation strategy, which enables it to have a cost of $O(d^2)$. Additionally, our experiments demonstrate the superiority of SLIQN over other incremental and stochastic Quasi-Newton variants and establish its competitiveness with second-order incremental methods.

math.OC↗

arXiv2024

Optimized Gradient Tracking for Decentralized Online Learning

This work considers the problem of decentralized online learning, where the goal is to track the optimum of the sum of time-varying functions, distributed across several nodes in a network. The local availability of the functions and their gradients necessitates coordination and consensus among the nodes. We put forth the Generalized Gradient Tracking (GGT) framework that unifies a number of existing approaches, including the state-of-the-art ones. The performance of the proposed GGT algorithm is theoretically analyzed using a novel semidefinite programming-based analysis that yields the desired regret bounds under very general conditions and without requiring the gradient boundedness assumption. The results are applicable to the special cases of GGT, which include various state-of-the-art algorithms as well as new dynamic versions of various classical decentralized algorithms. To further minimize the regret, we consider a condensed version of GGT with only four free parameters. A procedure for offline tuning of these parameters using only the problem parameters is also detailed. The resulting optimized GGT (oGGT) algorithm not only achieves improved dynamic regret bounds, but also outperforms all state-of-the-art algorithms on both synthetic and real-world datasets.

cs.LG↗

arXiv2023

Low-complexity subspace-descent over symmetric positive definite manifold

This work puts forth low-complexity Riemannian subspace descent algorithms for the minimization of functions over the symmetric positive definite (SPD) manifold. Different from the existing Riemannian gradient descent variants, the proposed approach utilizes carefully chosen subspaces that allow the update to be written as a product of the Cholesky factor of the iterate and a sparse matrix. The resulting updates avoid the costly matrix operations like matrix exponentiation and dense matrix multiplication, which are generally required in almost all other Riemannian optimization algorithms on SPD manifold. We further identify a broad class of functions, arising in diverse applications, such as kernel matrix learning, covariance estimation of Gaussian distributions, maximum likelihood parameter estimation of elliptically contoured distributions, and parameter estimation in Gaussian mixture model problems, over which the Riemannian gradients can be calculated efficiently. The proposed uni-directional and multi-directional Riemannian subspace descent variants incur per-iteration complexities of $O(n)$ and $O(n^2)$ respectively, as compared to the $O(n^3)$ or higher complexity incurred by all existing Riemannian gradient descent variants. The superior runtime and low per-iteration complexity of the proposed algorithms is also demonstrated via numerical tests on large-scale covariance estimation and matrix square root problems. MATLAB code implementation is publicly available on GitHub : https://github.com/yogeshd-iitk/subspace_descent_over_SPD_manifold

stat.ML↗

arXiv2023

Proximal Algorithms for Smoothed Online Convex Optimization with Predictions

We consider a smoothed online convex optimization (SOCO) problem with predictions, where the learner has access to a finite lookahead window of time-varying stage costs, but suffers a switching cost for changing its actions at each stage. Based on the Alternating Proximal Gradient Descent (APGD) framework, we develop Receding Horizon Alternating Proximal Descent (RHAPD) for proximable, non-smooth and strongly convex stage costs, and RHAPD-Smooth (RHAPD-S) for non-proximable, smooth and strongly convex stage costs. In addition to outperforming gradient descent-based algorithms, while maintaining a comparable runtime complexity, our proposed algorithms also allow us to solve a wider range of problems. We provide theoretical upper bounds on the dynamic regret achieved by the proposed algorithms, which decay exponentially with the length of the lookahead window. The performance of the presented algorithms is empirically demonstrated via numerical experiments on non-smooth regression, dynamic trajectory tracking, and {economic power dispatch} problems.

math.OC↗

arXiv2022

Stochastic Compositional Gradient Descent under Compositional Constraints

This work studies constrained stochastic optimization problems where the objective and constraint functions are convex and expressed as compositions of stochastic functions. The problem arises in the context of fair classification, fair regression, and the design of queuing systems. Of particular interest is the large-scale setting where an oracle provides the stochastic gradients of the constituent functions, and the goal is to solve the problem with a minimal number of calls to the oracle. Owing to the compositional form, the stochastic gradients provided by the oracle do not yield unbiased estimates of the objective or constraint gradients. Instead, we construct approximate gradients by tracking the inner function evaluations, resulting in a quasi-gradient saddle point algorithm. We prove that the proposed algorithm is guaranteed to find the optimal and feasible solution almost surely. We further establish that the proposed algorithm requires $\mathcal{O}(1/ε^4)$ data samples in order to obtain an $ε$-approximate optimal point while also ensuring zero constraint violation. The result matches the sample complexity of the stochastic compositional gradient descent method for unconstrained problems and improves upon the best-known sample complexity results for the constrained settings. The efficacy of the proposed algorithm is tested on both fair classification and fair regression problems. The numerical results show that the proposed algorithm outperforms the state-of-the-art algorithms in terms of the convergence rate.

cs.LG↗

arXiv2022

FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learning

Newton-type methods are popular in federated learning due to their fast convergence. Still, they suffer from two main issues, namely: low communication efficiency and low privacy due to the requirement of sending Hessian information from clients to parameter server (PS). In this work, we introduced a novel framework called FedNew in which there is no need to transmit Hessian information from clients to PS, hence resolving the bottleneck to improve communication efficiency. In addition, FedNew hides the gradient information and results in a privacy-preserving approach compared to the existing state-of-the-art. The core novel idea in FedNew is to introduce a two level framework, and alternate between updating the inverse Hessian-gradient product using only one alternating direction method of multipliers (ADMM) step and then performing the global model update using Newton's method. Though only one ADMM pass is used to approximate the inverse Hessian-gradient product at each iteration, we develop a novel theoretical approach to show the converging behavior of FedNew for convex problems. Additionally, a significant reduction in communication overhead is achieved by utilizing stochastic quantization. Numerical results using real datasets show the superiority of FedNew compared to existing methods in terms of communication costs.

cs.LG↗

arXiv2022

Sharpened Quasi-Newton Methods: Faster Superlinear Rate and Larger Local Convergence Neighborhood

Non-asymptotic analysis of quasi-Newton methods have gained traction recently. In particular, several works have established a non-asymptotic superlinear rate of $\mathcal{O}((1/\sqrt{t})^t)$ for the (classic) BFGS method by exploiting the fact that its error of Newton direction approximation approaches zero. Moreover, a greedy variant of BFGS was recently proposed which accelerates its convergence by directly approximating the Hessian, instead of the Newton direction, and achieves a fast local quadratic convergence rate. Alas, the local quadratic convergence of Greedy-BFGS requires way more updates compared to the number of iterations that BFGS requires for a local superlinear rate. This is due to the fact that in Greedy-BFGS the Hessian is directly approximated and the Newton direction approximation may not be as accurate as the one for BFGS. In this paper, we close this gap and present a novel BFGS method that has the best of both worlds in that it leverages the approximation ideas of both BFGS and Greedy-BFGS to properly approximate the Newton direction and the Hessian matrix simultaneously. Our theoretical results show that our method out-performs both BFGS and Greedy-BFGS in terms of convergence rate, while it reaches its quadratic convergence rate with fewer steps compared to Greedy-BFGS. Numerical experiments on various datasets also confirm our theoretical findings.

math.OC↗

arXiv2022

Sparse Representations of Positive Functions via First and Second-Order Pseudo-Mirror Descent

We consider expected risk minimization problems when the range of the estimator is required to be nonnegative, motivated by the settings of maximum likelihood estimation (MLE) and trajectory optimization. To facilitate nonlinear interpolation, we hypothesize that the search space is a Reproducing Kernel Hilbert Space (RKHS). We develop first and second-order variants of stochastic mirror descent employing (i) \emph{pseudo-gradients} and (ii) complexity-reducing projections. Compressive projection in the first-order scheme is executed via kernel orthogonal matching pursuit (KOMP), which overcomes the fact that the vanilla RKHS parameterization grows unbounded with the iteration index in the stochastic setting. Moreover, pseudo-gradients are needed when gradient estimates for cost are only computable up to some numerical error, which arise in, e.g., integral approximations. Under constant step-size and compression budget, we establish tradeoffs between the radius of convergence of the expected sub-optimality and the projection budget parameter, as well as non-asymptotic bounds on the model complexity. To refine the solution's precision, we develop a second-order extension which employs recursively averaged pseudo-gradient outer-products to approximate the Hessian inverse, whose convergence in mean is established under an additional eigenvalue decay condition on the Hessian of the optimal RKHS element, which is unique to this work. Experiments demonstrate favorable performance on inhomogeneous Poisson Process intensity estimation in practice.

stat.ML↗

arXiv2022

Projection-Free Algorithm for Stochastic Bi-level Optimization

This work presents the first projection-free algorithm to solve stochastic bi-level optimization problems, where the objective function depends on the solution of another stochastic optimization problem. The proposed $\textbf{S}$tochastic $\textbf{Bi}$-level $\textbf{F}$rank-$\textbf{W}$olfe ($\textbf{SBFW}$) algorithm can be applied to streaming settings and does not make use of large batches or checkpoints. The sample complexity of SBFW is shown to be $\mathcal{O}(ε^{-3})$ for convex objectives and $\mathcal{O}(ε^{-4})$ for non-convex objectives. Improved rates are derived for the stochastic compositional problem, which is a special case of the bi-level problem, and entails minimizing the composition of two expected-value functions. The proposed $\textbf{S}$tochastic $\textbf{C}$ompositional $\textbf{F}$rank-$\textbf{W}$olfe ($\textbf{SCFW}$) is shown to achieve a sample complexity of $\mathcal{O}(ε^{-2})$ for convex objectives and $\mathcal{O}(ε^{-3})$ for non-convex objectives, at par with the state-of-the-art sample complexities for projection-free algorithms solving single-level problems. We demonstrate the advantage of the proposed methods by solving the problem of matrix completion with denoising and the problem of policy value evaluation in reinforcement learning.

math.OC↗