Digital-Twin-Referenced Belief-Guided Q-Learning for Jammer Identification and Beam-Power Control in a mmWave Model

Abstract

Digital Twin (DT) combined with reinforcement learning (RL) can support adaptive resource allocation and security-aware decision-making by comparing expected and observed network behavior. This paper presents a simulation-based DT reference and a belief-guided Q-learning (BGQL) for joint jammer identification and beam-power control in a site-specific 28-GHz network. For each action, the same channel simulator is evaluated under an attack-free and an attacked configuration. Receiver-level residuals are combined with spatial jamming footprints and accumulated to form a normalized suspicion vector. BGQL incorporates this vector into an aggregated state, a belief-concentration reward, and exploratory beam-selection rule. The BGQL scheme is compared with SARSA, Greedy, and Random under matched attacker sets. Across 30 paired Monte Carlo runs in four configurations, with the number of active jammers supplied to every method, BGQL achieves mean aggregate full-band link-rate scores of approximately 437.4--564.2~Mbit/s and rate-to-device-power scores of 60.37--76.79~Mbit/J. Its detection accuracy is 100\% in one-jammer configurations and approximately 78\%--81\% in two-jammer configurations. BGQL requires approximately 80--130~s per run, compared with 5--14~s for SARSA, reflecting evaluation of 625 beam pairs during exploratory steps. These findings provide a matched-model simulation proof of concept for DT-referenced belief-guided learning for security-aware beam-power control in 6G networks.