arXiv subjects

Basil M. Idrees

Publications and source records attributed to Basil M. Idrees.

4 recordsLinked to original sources

arXiv2026

Decentralized Stochastic Constrained Optimization via Prox-Linearization

This paper studies consensus-based decentralized stochastic optimization for minimizing possibly non-convex expected objectives with convex non-smooth regularizers and nonlinear functional inequality constraints. We reformulate the constrained problem using the exact-penalty model and develop two algorithms that require only local stochastic gradients and first-order constraint information. The first method, Decentralized Stochastic Momentum-based Prox-Linear Algorithm (D-SMPL), combines constraint linearization with a prox-linear step, resulting in a linearly constrained quadratic subproblem per iteration. Building on this approach, we propose a successive convex approximation (SCA) variant, Decentralized SCA Momentum-based Prox-Linear (D-SCAMPL), which handles additional objective structure through strongly convex surrogate subproblems while still allowing infeasible initialization. Both methods incorporate recursive momentum-based gradient estimators and a consensus mechanism requiring only two communication rounds per iteration. Under standard smoothness and regularity assumptions, both algorithms achieve an oracle complexity of $\mathcal{O}(\epsilon^{-3/2})$, matching the optimal rate known for unconstrained centralized stochastic non-convex optimization. Numerical experiments on energy-optimal ocean trajectory planning corroborate the theory and demonstrate improved performance over existing decentralized baselines.

math.OC↗

arXiv2024

Analysis of Decentralized Stochastic Successive Convex Approximation for composite non-convex problems

This work considers the decentralized successive convex approximation (SCA) method for minimizing stochastic non-convex objectives subject to convex constraints, along with possibly non-smooth convex regularizers. Although SCA has been widely applied in decentralized settings, its stochastic first order (SFO) complexity is unknown, and it is thought to be slower than the centralized momentum-enhanced SCA variants. In this work, we advance the state-of-the-art for SCA methods by proposing an accelerated variant, namely the \textbf{D}ecentralized \textbf{M}omentum-based \textbf{S}tochastic \textbf{SCA} (\textbf{D-MSSCA}) and analyze its SFO complexity. The proposed algorithm entails creating a stochastic surrogate of the objective at every iteration, which is minimized at each node separately. Remarkably, the D-MSSCA achieves an SFO complexity of $\mathcal{O}(\epsilon^{-3/2})$ to reach an $\epsilon$-stationary point, which is at par with the SFO complexity lower bound for unconstrained stochastic non-convex optimization in centralized setting.

math.OC↗

arXiv2024

Constrained Stochastic Recursive Momentum Successive Convex Approximation

We consider stochastic optimization problems with non-convex functional constraints, such as those arising in trajectory generation, sparse approximation, and robust classification. To this end, we put forth a recursive momentum-based accelerated successive convex approximation (SCA) algorithm. At each iteration, the proposed algorithm entails constructing convex surrogates of the stochastic objective and the constraint functions, and solving the resulting convex optimization problem. A recursive update rule is employed to track the gradient of the stochastic objective function, which contributes to variance reduction and hence accelerates the algorithm convergence. A key ingredient of the proof is a new parameterized version of the standard Mangasarian-Fromowitz Constraints Qualification, that allows us to bound the dual variables and hence obtain problem-dependent bounds on the rate at which the iterates approach an $\epsilon$-stationary point. Remarkably, the proposed algorithm achieves near-optimal stochastic first-order (SFO) complexity with adaptive step sizes closely matching that achieved by state-of-the-art stochastic optimization algorithms for solving unconstrained problems. As an example, we detail an obstacle-avoiding trajectory optimization problem that can be solved using the proposed algorithm and show that its performance is superior to that of the existing algorithms used for trajectory optimization. The performance of the proposed algorithm is also shown to be comparable to that of a specialized sparse classification algorithm applied to a binary classification problem.

math.OC↗

arXiv2020

Practical Precoding via Asynchronous Stochastic Successive Convex Approximation

We consider stochastic optimization of a smooth non-convex loss function with a convex non-smooth regularizer. In the online setting, where a single sample of the stochastic gradient of the loss is available at every iteration, the problem can be solved using the proximal stochastic gradient descent (SGD) algorithm and its variants. However in many problems, especially those arising in communications and signal processing, information beyond the stochastic gradient may be available thanks to the structure of the loss function. Such extra-gradient information is not used by SGD, but has been shown to be useful, for instance in the context of stochastic expectation-maximization, stochastic majorization-minimization, and stochastic successive convex approximation (SCA) approaches. By constructing a stochastic strongly convex surrogates of the loss function at every iteration, the stochastic SCA algorithms can exploit the structural properties of the loss function and achieve superior empirical performance as compared to the SGD. In this work, we take a closer look at the stochastic SCA algorithm and develop its asynchronous variant which can be used for resource allocation in wireless networks. While the stochastic SCA algorithm is known to converge asymptotically, its iteration complexity has not been well-studied, and is the focus of the current work. The insights obtained from the non-asymptotic analysis allow us to develop a more practical asynchronous variant of the stochastic SCA algorithm which allows the use of surrogates calculated in earlier iterations. We characterize precise bound on the maximum delay the algorithm can tolerate, while still achieving the same convergence rate. We apply the algorithm to the problem of linear precoding in wireless sensor networks, where it can be implemented at low complexity but is shown to perform well in practice.

math.OC↗