Searcharxiv⌕ Search

arXiv subjects

M. Vidyasagar

Publications and source records attributed to M. Vidyasagar.

8 recordsLinked to original sources
arXiv2024

Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications

In this paper, we study the convergence properties of the Stochastic Gradient Descent (SGD) method for finding a stationary point of a given objective function $J(\cdot)$. The objective function is not required to be convex. Rather, our results apply to a class of ``invex'' functions, which have the property that every stationary point is also a global minimizer. First, it is assumed that $J(\cdot)$ satisfies a property that is slightly weaker than the Kurdyka-Lojasiewicz (KL) condition, denoted here as (KL'). It is shown that the iterations $J(\boldsymbolθ_t)$ converge almost surely to the global minimum of $J(\cdot)$. Next, the hypothesis on $J(\cdot)$ is strengthened from (KL') to the Polyak-Lojasiewicz (PL) condition. With this stronger hypothesis, we derive estimates on the rate of convergence of $J(\boldsymbolθ_t)$ to its limit. Using these results, we show that for functions satisfying the PL property, the convergence rate of both the objective function and the norm of the gradient with SGD is the same as the best-possible rate for convex functions. While some results along these lines have been published in the past, our contributions contain two distinct improvements. First, the assumptions on the stochastic gradient are more general than elsewhere, and second, our convergence is almost sure, and not in expectation. We also study SGD when only function evaluations are permitted. In this setting, we determine the ``optimal'' increments or the size of the perturbations. Using the same set of ideas, we establish the global convergence of the Stochastic Approximation (SA) algorithm under more general assumptions on the measurement error, compared to the existing literature. We also derive bounds on the rate of convergence of the SA algorithm under appropriate assumptions.

stat.ML↗
arXiv2023

Convergence of Batch Updating Methods with Approximate Gradients and/or Noisy Measurements: Theory and Computational Results

In this paper, we present a unified and general framework for analyzing the batch updating approach to nonlinear, high-dimensional optimization. The framework encompasses all the currently used batch updating approaches, and is applicable to nonconvex as well as convex functions. Moreover, the framework permits the use of noise-corrupted gradients, as well as first-order approximations to the gradient (sometimes referred to as "gradient-free" approaches). By viewing the analysis of the iterations as a problem in the convergence of stochastic processes, we are able to establish a very general theorem, which includes most known convergence results for zeroth-order and first-order methods. The analysis of "second-order" or momentum-based methods is not a part of this paper, and will be studied elsewhere. However, numerical experiments indicate that momentum-based methods can fail if the true gradient is replaced by its first-order approximation. This requires further theoretical analysis.

math.OC↗
arXiv2023

Convergence of Stochastic Approximation via Martingale and Converse Lyapunov Methods

In this paper, we study the almost sure boundedness and the convergence of the stochastic approximation (SA) algorithm. At present, most available convergence proofs are based on the ODE method, and the almost sure boundedness of the iterations is an assumption and not a conclusion. In Borkar-Meyn (2000), it is shown that if the ODE has only one globally attractive equilibrium, then under additional assumptions, the iterations are bounded almost surely, and the SA algorithm converges to the desired solution. Our objective in the present paper is to provide an alternate proof of the above, based on martingale methods, which are simpler and less technical than those based on the ODE method. As a prelude, we prove a new sufficient condition for the global asymptotic stability of an ODE. Next we prove a "converse" Lyapunov theorem on the existence of a suitable Lyapunov function with a globally bounded Hessian, for a globally exponentially stable system. Both theorems are of independent interest to researchers in stability theory. Then, using these results, we provide sufficient conditions for the almost sure boundedness and the convergence of the SA algorithm. We show through examples that our theory covers some situations that are not covered by currently known results, specifically Borkar-Meyn (2000).

stat.ML↗
arXiv2021

Convergence of Batch Asynchronous Stochastic Approximation With Applications to Reinforcement Learning

We begin by briefly surveying some results on the convergence of the Stochastic Gradient Descent (SGD) Method, proved in a companion paper by the present authors. These results are based on viewing SGD as a version of Stochastic Approximation (SA). Ever since its introduction in the classic paper of Robbins and Monro in 1951, SA has become a standard tool for finding a solution of an equation of the form $f(\theta) = 0$, when only noisy measurements of $f(\cdot)$ are available. In most situations, \textit{every component} of the putative solution $\theta_t$ is updated at each step $t$. In some applications in Reinforcement Learning (RL), \textit{only one component} of $\theta_t$ is updated at each $t$. This is known as \textbf{asynchronous} SA. In this paper, we study \textbf{Block Asynchronous SA (BASA)}, in which, at each step $t$, \textit{some but not necessarily all} components of $\theta_t$ are updated. The theory presented here embraces both conventional (synchronous) SA as well as asynchronous SA, and all in-between possibilities. We provide sufficient conditions for the convergence of BASA, and also prove bounds on the \textit{rate} of convergence of $\theta_t$ to the solution. For the case of conventional SGD, these results reduce to those proved in our companion paper. Then we apply these results to the problem of finding a fixed point of a map with only noisy measurements. This problem arises frequently in RL. We prove sufficient conditions for convergence as well as estimates for the rate of convergence.

stat.ML↗
arXiv2020

Estimating Hidden Asymptomatics, Herd Immunity Threshold and Lockdown Effects using a COVID-19 Specific Model

A quantitative COVID-19 model that incorporates hidden asymptomatic patients is developed, and an analytic solution in parametric form is given. The model incorporates the impact of lockdown and resulting spatial migration of population due to announcement of lockdown. A method is presented for estimating the model parameters from real-world data. It is shown that increase of infections slows down and herd immunity is achieved when symptomatic patients are 4-6\% of the population for the European countries we studied, when the total infected fraction is between 50-56 \%. Finally, a method for estimating the number of asymptomatic patients, who have been the key hidden link in the spread of the infections, is presented.

q-bio.PE↗
arXiv2016

Reverse Engineering Gene Interaction Networks Using the Phi-Mixing Coefficient

Constructing gene interaction networks (GINs) from high-throughput gene expression data is an important and challenging problem in systems biology. Existing algorithms produce networks that either have undirected and unweighted edges, or else are constrained to contain no cycles, both of which are biologically unrealistic. In the present paper we propose a new algorithm, based on a concept from probability theory known as the phi-mixing coefficient, that produces networks whose edges are weighted and directed, and are permitted to contain cycles. Because there is no "ground truth" for genome-wide networks on a human scale, we analyzed the outcomes of several experiments on lung cancer, and matched the predictions from the inferred networks with experimental results. Specifically, we inferred three networks (NSCLC, Neuro-endocrine NSCLC plus SCLC, and normal) from the gene expression measurements of 157 lung cancer and 59 normal cell lines, compared with the outcomes of siRNA screening of 19,000+ genes on 11 NSCLC cell lines, and analyzed data from a ChIP-Seq experiment to determine putative downstream targets of the lineage specific oncogenic transcription factor ASCL1. The inferred networks displayed a scale-free or power law behavior between the degree of a node and the number of nodes with that degree. There was a strong correlation between the degree of a gene in the inferred NSCLC network and its essentiality for the survival of the cells. The inferred downstream neighborhood genes of ASCL1 in the SCLC network were significantly enriched by ChIP-Seq determined putative target genes, while no such enrichment was found in the inferred NSCLC network.

q-bio.GN↗
arXiv2015

Error Bounds for Compressed Sensing Algorithms With Group Sparsity: A Unified Approach

In compressed sensing, in order to recover a sparse or nearly sparse vector from possibly noisy measurements, the most popular approach is $\ell_1$-norm minimization. Upper bounds for the $\ell_2$- norm of the error between the true and estimated vectors are given in [1] and reviewed in [2], while bounds for the $\ell_1$-norm are given in [3]. When the unknown vector is not conventionally sparse but is "group sparse" instead, a variety of alternatives to the $\ell_1$-norm have been proposed in the literature, including the group LASSO, sparse group LASSO, and group LASSO with tree structured overlapping groups. However, no error bounds are available for any of these modified objective functions. In the present paper, a unified approach is presented for deriving upper bounds on the error between the true vector and its approximation, based on the notion of decomposable and $γ$-decomposable norms. The bounds presented cover all of the norms mentioned above, and also provide a guideline for choosing norms in future to accommodate alternate forms of sparsity.

stat.ML↗