arXiv subjects

Nishanth Anand

Publications and source records attributed to Nishanth Anand.

5 recordsLinked to original sources

arXiv2026

Discovering Failure Modes in Vision-Language Models using RL

Vision-language Models (VLMs), despite achieving strong performance on multimodal benchmarks, often misinterpret straightforward visual concepts that humans identify effortlessly, such as counting, spatial reasoning, and viewpoint understanding. Previous studies manually identified these weaknesses and found that they often stem from deficits in specific skills. However, such manual efforts are costly, unscalable, and subject to human bias, which often overlooks subtle details in favour of salient objects, resulting in an incomplete understanding of a model's vulnerabilities. To address these limitations, we propose a Reinforcement Learning (RL)-based framework to automatically discover the failure modes or blind spots of any ``candidate VLM'' on a given data distribution without human intervention. Our framework trains a questioner agent that adaptively generates queries based on the candidate VLM's responses to elicit incorrect answers. Our approach increases question complexity by focusing on fine-grained visual details and distinct skill compositions as training progresses, consequently identifying novel failure modes in which VLMs struggle. We demonstrate the broad applicability of our framework by showcasing its generalizability across various model combinations.

cs.CV↗

arXiv2024

Optical and spin properties of nitrogen vacancy centers formed along the tracks of high energy heavy ions

Exposure of matter to high energy, heavy ions induces defects along the trajectories of the ions through electronic and nuclear energy loss processes. Defects, including color centers, can recombine or form along latent damage tracks in many materials, such as insulators and semiconductors. Latent tracks in diamond were only recently observed. Here, we report on color center formation in diamond along the latent tracks of 1 GeV gold and uranium ions. Using depth-resolved photoluminescence, we observe direct formation of single vacancy related color centers (GR1-centers) along the ion tracks. Mobile vacancies can form NV-centers with native nitrogen atoms during thermal annealing. Molecular dynamics simulations show that isolated vacancies and vacancy clusters form through electronic stopping processes, leading to color center formation along ion trajectories from the sample surface to a depth of about 25 microns. We further report on the creation of individually isolated quasi-1D chains of NV-centers by using 1 GeV Au ions with a dilute fluence. The individual 1D NV-chains appear as isolated bright luminescence strings and present competitive electron spin properties compared to a background of NV-centers. Such spin textures can be explored as building blocks for applications in quantum information processing and quantum sensing.

quant-ph↗

arXiv2023

Prediction and Control in Continual Reinforcement Learning

Temporal difference (TD) learning is often used to update the estimate of the value function which is used by RL agents to extract useful policies. In this paper, we focus on value function estimation in continual reinforcement learning. We propose to decompose the value function into two components which update at different timescales: a permanent value function, which holds general knowledge that persists over time, and a transient value function, which allows quick adaptation to new situations. We establish theoretical results showing that our approach is well suited for continual learning and draw connections to the complementary learning systems (CLS) theory from neuroscience. Empirically, this approach improves performance significantly on both prediction and control problems.

cs.LG↗

arXiv2021

Preferential Temporal Difference Learning

Temporal-Difference (TD) learning is a general and very useful tool for estimating the value function of a given policy, which in turn is required to find good policies. Generally speaking, TD learning updates states whenever they are visited. When the agent lands in a state, its value can be used to compute the TD-error, which is then propagated to other states. However, it may be interesting, when computing updates, to take into account other information than whether a state is visited or not. For example, some states might be more important than others (such as states which are frequently seen in a successful trajectory). Or, some states might have unreliable value estimates (for example, due to partial observability or lack of data), making their values less desirable as targets. We propose an approach to re-weighting states used in TD updates, both when they are the input and when they provide the target for the update. We prove that our approach converges with linear function approximation and illustrate its desirable empirical behaviour compared to other TD-style methods.

cs.LG↗

arXiv2019

Recurrent Value Functions

Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurrent Value Functions (RVFs) as an alternative to estimate the value function of a state. We propose to estimate the value function of the current state using the value function of past states visited along the trajectory. Due to the nature of their formulation, RVFs have a natural way of learning an emphasis function that selectively emphasizes important states. First, we establish RVF's asymptotic convergence properties in tabular settings. We then demonstrate their robustness on a partially observable domain and continuous control tasks. Finally, we provide a qualitative interpretation of the learned emphasis function.

cs.LG↗