Searcharxiv⌕ Search

arXiv subjects

Vivek S Borkar

Publications and source records attributed to Vivek S Borkar.

4 recordsLinked to original sources
arXiv2026

Exploding and vanishing gradients in deep neural networks: the effect of residual connections

The well known phenomenon of exploding and vanishing gradients in deep neural networks is analyzed using multiplicative ergodic theory. The effect of adding a residual connection is explained in this context. Specifically, a characterization of Liapunov exponents due to Furstenberg and Kifer is exploited in order to make a precise statement about the Liapunov spectrum and the effect of residual connections on it.

math.OC↗
arXiv2025

Small noise limits of Markov chains and the PageRank

We recall the classical formulation of PageRank as the stationary distribution of a singularly perturbed irreducible Markov chain that is not irreducible when the perturbation parameter goes to zero. Specifically, we use the Markov chain tree theorem to derive explicit expressions for the PageRank. This analysis leads to some surprising results. These results are then extended to a much more general class of perturbations that subsume personalized PageRank. We also give examples where even simpler formulas for PageRank are possible.

math.PR↗
arXiv2024

Reinforcement Learning in Non-Markovian Environments

Motivated by the novel paradigm developed by Van Roy and coauthors for reinforcement learning in arbitrary non-Markovian environments, we propose a related formulation and explicitly pin down the error caused by non-Markovianity of observations when the Q-learning algorithm is applied on this formulation. Based on this observation, we propose that the criterion for agent design should be to seek good approximations for certain conditional laws. Inspired by classical stochastic control, we show that our problem reduces to that of recursive computation of approximate sufficient statistics. This leads to an autoencoder-based scheme for agent design which is then numerically tested on partially observed reinforcement learning environments.

eess.SY↗