arXiv subjects

Saurav Sthapit

Publications and source records attributed to Saurav Sthapit.

6 recordsLinked to original sources

arXiv2026

Fair and Efficient Scheduling for Sensor Networks via Online Whittle Index Policy

Wake-Up Radio (WUR) enables resource-constrained, battery-powered sensor nodes to remain in a low-power deep sleep state while continuously listening for a Wake-Up Signal (WUS). Sensor nodes only wake and transmit data after receiving the WUS, significantly reducing energy consumption. However, polling nodes whose transmitted data provides little or no meaningful update to the remote monitor can still result in unnecessary energy usage and increased storage overhead. To address this issue, this paper uses the Age of Incorrect Information (AoII) metric to prioritise the polling of nodes that provide informative updates to the remote monitor. Determining the optimal set of nodes to poll based on AoII can be formulated as a Restless Multi-Armed Bandit (RMAB) problem, which traditionally requires prior knowledge of the monitored process transition dynamics. Since such dynamics are often unknown in practical deployments, we propose an online learning framework based on state estimation to derive Whittle Index AoII (WAoII) and Fair Whittle Index AoII (FWAoII) policies without assuming known transition probabilities. The proposed policies efficiently schedule node polling while adapting to unknown process behaviour. Experimental evaluation using both real-world and synthetic datasets demonstrates that the proposed online WAoII policy can reduce packet transmissions by up to 70\% compared to the widely used Round Robin (RR) polling strategy, while maintaining Root Mean Squared Error (RMSE) values within acceptable application error tolerances. These results demonstrate the effectiveness of WAoII and FWAoII as energy-efficient polling techniques for low-power WUR sensor networks.

eess.SP↗

arXiv2026

Adaptive Scheduling: A Reinforcement Learning Whittle Index Approach for Wireless Sensor Networks

We propose a reinforcement learning based scheduling framework for Restless Multi-Armed Bandit (RMAB) problems, centred on a Whittle Index Q-Learning policy with Upper Confidence Bound (UCB) exploration, referred to as WIQL-UCB. Unlike existing approaches that rely on fixed or adaptive epsilon-greedy strategies and require careful hyperparameter tuning, the proposed method removes problem-specific tuning and is therefore more generalisable across diverse RMAB settings. We evaluate WIQL-UCB on standard RMAB benchmarks and on a practical sensor scheduling application based on the Age of Incorrect Information (AoII), using an edge-based state estimation scheme that requires no prior knowledge of system dynamics. Experimental results show that WIQL-UCB achieves near-optimal performance while significantly improving computational and memory efficiency. For a representative problem size of N = 15 and M = 3, the proposed method requires only around 600 bytes of memory, compared with several kilobytes for tabular Q-learning and hundreds of kilobytes to megabytes for deep reinforcement learning baselines. In addition, WIQL-UCB achieves sub-millisecond per-decision runtimes and is several times faster than deep reinforcement learning approaches, while maintaining competitive performance. Overall, these results demonstrate that WIQL-UCB consistently outperforms both non-Whittle-based and Whittle-index learning baselines across a wide range of RMAB settings.

eess.SY↗

arXiv2025

An agentic system with reinforcement-learned subsystem improvements for parsing form-like documents

Extracting alphanumeric data from form-like documents such as invoices, purchase orders, bills, and financial documents is often performed via vision (OCR) and learning algorithms or monolithic pipelines with limited potential for systemic improvements. We propose an agentic AI system that leverages Large Language Model (LLM) agents and a reinforcement learning (RL) driver agent to automate consistent, self-improving extraction under LLM inference uncertainty. Our work highlights the limitations of monolithic LLM-based extraction and introduces a modular, multi-agent framework with task-specific prompts and an RL policy of rewards and penalties to guide a meta-prompting agent to learn from past errors and improve prompt-based actor agents. This self-corrective adaptive system handles diverse documents, file formats, layouts, and LLMs, aiming to automate accurate information extraction without the need for human intervention. Results as reported on two benchmark datasets of SOIRE, and CORD, are promising for the agentic AI framework.

cs.IR↗

arXiv2022

Data-Driven Detection and Identification of IoT-Enabled Load-Altering Attacks in Power Grids

Advances in edge computing are powering the development and deployment of Internet of Things (IoT) systems to provide advanced services and resource efficiency. However, large-scale IoT-based load-altering attacks (LAAs) can seriously impact power grid operations, such as destabilising the grid's control loops. Timely detection and identification of any compromised nodes are essential to minimise the adverse effects of these attacks on power grid operations. In this work, two data-driven algorithms are proposed to detect and identify compromised nodes and the attack parameters of the LAAs. The first method, based on the Sparse Identification of Nonlinear Dynamics (SINDy) approach, adopts a sparse regression framework to identify attack parameters that best describe the observed dynamics. The second method, based on physics-informed neural networks (PINN), employs neural networks to infer the attack parameters from the measurements. Both algorithms are presented utilising edge computing for deployment over decentralised architectures. Extensive simulations are performed on IEEE 6-,14- and 39-bus systems to verify the effectiveness of the proposed methods. Numerical results confirm that the proposed algorithms outperform existing approaches, such as those based on unscented Kalman filter, support vector machines (SVM), and neural networks (NN), and effectively detect and identify locations of attack in a timely manner.

eess.SY↗

arXiv2021

Reinforcement Learning for Security-Aware Computation Offloading in Satellite Networks

The rise of NewSpace provides a platform for small and medium businesses to commercially launch and operate satellites in space. In contrast to traditional satellites, NewSpace provides the opportunity for delivering computing platforms in space. However, computational resources within space are usually expensive and satellites may not be able to compute all computational tasks locally. Computation Offloading (CO), a popular practice in Edge/Fog computing, could prove effective in saving energy and time in this resource-limited space ecosystem. However, CO alters the threat and risk profile of the system. In this paper, we analyse security issues in space systems and propose a security-aware algorithm for CO. Our method is based on the reinforcement learning technique, Deep Deterministic Policy Gradient (DDPG). We show, using Monte-Carlo simulations, that our algorithm is effective under a variety of environment and network conditions and provide novel insights into the challenge of optimised location of computation.

cs.CR↗

arXiv2021

A Comparison of Data-Driven Techniques for Power Grid Parameter Estimation

Power grid parameter estimation involves the estimation of unknown parameters, such as inertia and damping coefficients, using observed dynamics. In this work, we present a comparison of data-driven algorithms for the power grid parameter estimation problem. First, we propose a new algorithm to solve the parameter estimation problem based on the Sparse Identification of Nonlinear Dynamics (SINDy) approach, which uses linear regression to infer the parameters that best describe the observed data. We then compare its performance against two benchmark algorithms, namely, the unscented Kalman filter (UKF) approach and the physics-informed neural networks (PINN) approach. We perform extensive simulations on IEEE bus systems to examine the performance of the aforementioned algorithms. Our results show that the SINDy algorithm outperforms the PINN and UKF algorithms in being able to accurately estimate the power grid parameters over a wide range of system parameters (including high and low inertia systems). Moreover, it is extremely efficient computationally and so takes significantly less time than the PINN algorithm, thus making it suitable for real-time parameter estimation.

eess.SY↗