arXiv subjects

Ankita Samaddar

Publications and source records attributed to Ankita Samaddar.

7 recordsLinked to original sources

arXiv2026

Learning Red Agent Policy from Observations for Neurosymbolic Autonomous Cyber Agents

With sophisticated cyber-attacks becoming increasingly prevalent, modern networks require intelligent autonomous cyber-defense agents trained via Reinforcement Learning (RL). These agents employ neurosymbolic approaches such as behavior trees with learning-enabled components (LECs) to learn, reason, adapt, and implement security rules while maintaining critical operations. However, these autonomous networks are partially observable systems, i.e., the cyber-attacker's (red agent's) actions are not observable, making it difficult for the defender to predict red actions, learn red policies, or assess the attacker's intrusion levels. To address this, we propose a Policy Learning Technique using imitation learning to learn policies for partially observable RL agents with discrete states and discrete actions. We apply this technique in an autonomous cyber environment to predict red agent's actions from network observations and defender actions. Integrated with a neurosymbolic cyber-defense agent, our method effectively handles different red policies and achieves high prediction accuracy across diverse simulated scenarios.

cs.CR↗

arXiv2026

Reward Shaping and Action Masking for Compositional Tasks using Behavior Trees and LLMs

Decomposing complex tasks into a sequence of simpler subtasks can improve learning efficiency for an autonomous agent. Reinforcement learning (RL) can be used to optimize agent policies to complete subtasks, but requires well-defined subtask rewards and benefits from action masking. Recent work uses large language models (LLMs) to automate reward shaping and action masking, however none of them fully address reactivity to subtask failure and modularity to varying objects for compositional tasks. To overcome these challenges, we develop masking reward behavior tree (MRBT), a symbolic structure used as a reactive and modular reward and action mask function. We design an MRBT template and derive logical specifications to construct and verify MRBTs for a sequence of object-interaction subtasks. Further, we develop an automated pipeline that uses an LLM to generate MRBTs robust to varying task objects, an SMT-solver to verify correctness of specifications, and a neurosymbolic RL loop to train agents on compositional tasks. Experiments demonstrate successful generation and refinement of five MRBTs, consistently improving training efficiency and task success rates over baselines and MRBTs without action masking. We further highlight three advantages of MRBTs: transferability, modularity, and verifiability.

cs.LG↗

arXiv2026

Building Better Environments for Autonomous Cyber Defence

In November 2025, the authors ran a workshop on the topic of what makes a good reinforcement learning (RL) environment for autonomous cyber defence (ACD). This paper details the knowledge shared by participants both during the workshop and shortly afterwards by contributing herein. The workshop participants come from academia, industry, and government, and have extensive hands-on experience designing and working with RL and cyber environments. While there is now a sizeable body of literature describing work in RL for ACD, there is nevertheless a great deal of tradecraft, domain knowledge, and common hazards which are not detailed comprehensively in a single resource. With a specific focus on building better environments to train and evaluate autonomous RL agents in network defence scenarios, including government and critical infrastructure networks, the contributions of this work are twofold: (1) a framework for decomposing the interface between RL cyber environments and real systems, and (2) guidelines on current best practice for RL-based ACD environment development and agent evaluation, based on the key findings from our workshop.

cs.CR↗

arXiv2024

Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agents

Autonomous agents for cyber applications take advantage of modern defense techniques by adopting intelligent agents with conventional and learning-enabled components. These intelligent agents are trained via reinforcement learning (RL) algorithms, and can learn, adapt to, reason about and deploy security rules to defend networked computer systems while maintaining critical operational workflows. However, the knowledge available during training about the state of the operational network and its environment may be limited. The agents should be trustworthy so that they can reliably detect situations they cannot handle, and hand them over to cyber experts. In this work, we develop an out-of-distribution (OOD) Monitoring algorithm that uses a Probabilistic Neural Network (PNN) to detect anomalous or OOD situations of RL-based agents with discrete states and discrete actions. To demonstrate the effectiveness of the proposed approach, we integrate the OOD monitoring algorithm with a neurosymbolic autonomous cyber agent that uses behavior trees with learning-enabled components. We evaluate the proposed approach in a simulated cyber environment under different adversarial strategies. Experimental results over a large number of episodes illustrate the overall efficiency of our proposed approach.

cs.LG↗

arXiv2024

Designing Robust Cyber-Defense Agents with Evolving Behavior Trees

Modern network defense can benefit from the use of autonomous systems, offloading tedious and time-consuming work to agents with standard and learning-enabled components. These agents, operating on critical network infrastructure, need to be robust and trustworthy to ensure defense against adaptive cyber-attackers and, simultaneously, provide explanations for their actions and network activity. However, learning-enabled components typically use models, such as deep neural networks, that are not transparent in their high-level decision-making leading to assurance challenges. Additionally, cyber-defense agents must execute complex long-term defense tasks in a reactive manner that involve coordination of multiple interdependent subtasks. Behavior trees are known to be successful in modelling interpretable, reactive, and modular agent policies with learning-enabled components. In this paper, we develop an approach to design autonomous cyber defense agents using behavior trees with learning-enabled components, which we refer to as Evolving Behavior Trees (EBTs). We learn the structure of an EBT with a novel abstract cyber environment and optimize learning-enabled components for deployment. The learning-enabled components are optimized for adapting to various cyber-attacks and deploying security mechanisms. The learned EBT structure is evaluated in a simulated cyber environment, where it effectively mitigates threats and enhances network visibility. For deployment, we develop a software architecture for evaluating EBT-based agents in computer network defense scenarios. Our results demonstrate that the EBT-based agent is robust to adaptive cyber-attacks and provides high-level explanations for interpreting its decisions and actions.

cs.AI↗

arXiv2024

A Location Validation Technique to Mitigate GPS Spoofing Attacks in IEEE 802.11p based Fleet Operator's Network of Electric Vehicles

Most vehicular applications in electric vehicles use IEEE 802.11p protocol for vehicular communications. Vehicle rebalancing application is one such application that has been used by many car rental service providers to overcome the disparity between vehicle demand and vehicle supply at different charging stations. Vehicle rebalancing application uses the GPS location data of the vehicles periodically to determine the vehicle(s) to be moved to a different charging station for rebalancing. However, a malicious attacker residing in the network can spoof the GPS location data packets of the target vehicle(s) resulting in misinterpretation of the location of the vehicle(s). This can result in wrong rebalancing decision leading to unmet demands of the customers and under utilization of the system. To detect and prevent this attack, we propose a location tracking technique that can validate the current location of a vehicle based on its previous location and roadmaps. We used OpenStreetMap and SUMO simulator to generate the roadmap data from the roadmaps of Singapore. Extensive experiments on the generated datasets show the efficacy of our proposed technique.

cs.CR↗

arXiv2019

SlotSwapper: A Schedule Randomization protocol for Real-Time WirelessHART Networks

Industrial process control systems are time-critical systems where reliable communications between sensors and actuators need to be guaranteed within strict deadlines to maintain safe operation of all the components of the system. WirelessHART is the most widely adopted standard which serve as the medium of communication in industrial setups due to its support for Time Division Multiple Access (TDMA)based communication, multiple channels, channel hopping, centralized architecture, redundant routes and avoidance of spatial re-use of channels. However, the communication schedule in WirelessHART network is decided by a centralized network manager at the time of network initialization and the same communication schedule repeats every hyper-period. Due to predictability in the time slots of the communication schedule, these systems are vulnerable to timing attacks which eventually can disrupt the safety of the system. In this work, we present a moving target defense mechanism, the SlotSwapper, which uses schedule randomization techniques to randomize the time slots over a hyper-period schedule, while still preserving all the feasibility constraints of a real-time WirelessHART network and makes the schedule uncertain every hyper-period. We tested the feasibility of the generated schedules on random topologies with 100 simulated motes in Cooja simulator. We use schedule entropy to measure the confidentiality of our algorithm in terms of randomness in the time slots of the generated schedules.

cs.NI↗