arXiv subjects

Debangshu Banerjee

Publications and source records attributed to Debangshu Banerjee.

22 records · Page 2Linked to original sources

arXiv2023

Markov Chain Concentration with an Application in Reinforcement Learning

Given $X_1,\cdot ,X_N$ random variables whose joint distribution is given as $μ$ we will use the Martingale Method to show any Lipshitz Function $f$ over these random variables is subgaussian. The Variance parameter however can have a simple expression under certain conditions. For example under the assumption that the random variables follow a Markov Chain and that the function is Lipschitz under a Weighted Hamming Metric. We shall conclude with certain well known techniques from concentration of suprema of random processes with applications in Reinforcement Learning

cs.LG↗

arXiv2022

Critic Algorithms using Cooperative Networks

An algorithm is proposed for policy evaluation in Markov Decision Processes which gives good empirical results with respect to convergence rates. The algorithm tracks the Projected Bellman Error and is implemented as a true gradient based algorithm. In this respect this algorithm differs from TD($λ$) class of algorithms. This algorithm tracks the Projected Bellman Algorithm and is therefore different from the class of residual algorithms. Further the convergence of this algorithm is empirically much faster than GTD2 class of algorithms which aim at tracking the Projected Bellman Error. We implemented proposed algorithm in DQN and DDPG framework and found that our algorithm achieves comparable results in both of these experiments

cs.AI↗

arXiv2021

Vertex Guarding for Dynamic Orthogonal Art Galleries

We devise an algorithm for surveying a dynamic orthogonal polygonal domain by placing one guard at each vertex in a subset of its vertices, i.e., whenever an orthogonal polygonal domain {\cal P'} is modified to result in another orthogonal polygonal domain {\cal P}, our algorithm updates the set of vertex guards surveying {\cal P'} so that the updated guard set surveys {\cal P}. Our algorithm modifies the guard placement in O(k \lg{(n+n')}) amortized time while ensuring the updated orthogonal polygonal domain with h holes and n vertices is guarded using at most \lfloor (n+2h)/4 \rfloor vertex guards. For the special case of the initial orthogonal polygon being hole-free and each update resulting in a hole-free orthogonal polygon, our guard update algorithm takes O(k\lg{(n+n')}) worst-case time. Here, n' and n are the number of vertices of the orthogonal polygon before and after the update, respectively; and, k is the sum of |n - n'| and the number of updates to a few structures maintained by our algorithm. Further, by giving a construction, we show it suffices for the algorithm to consider only the case in which the parity of the number of reflex vertices of both {\cal P'} and {\cal P} are equal.

cs.CG↗

arXiv2020

HEX and Neurodynamic Programming

Hex is a complex game with a high branching factor. For the first time Hex is being attempted to be solved without the use of game tree structures and associated methods of pruning. We also are abstaining from any heuristic information about Virtual Connections or Semi Virtual Connections which were previously used in all previous known computer versions of the game. The H-search algorithm which was the basis of finding such connections and had been used with success in previous Hex playing agents has been forgone. Instead what we use is reinforcement learning through self play and approximations through neural networks to by pass the problem of high branching factor and maintaining large tables for state-action evaluations. Our code is based primarily on NeuroHex. The inspiration is drawn from the recent success of AlphaGo Zero.

cs.LG↗