MEO: Mining Reliable Expert Signals for Offline Reinforcement Learning in Wireless Networks

Abstract

The operation of next-generation wireless networks increasingly hinges on Deep Reinforcement Learning (DRL) to optimize critical network decisions. Offline RL trains agents using accessible datasets and thus eliminates costly online interactions, rendering it particularly advantageous in dynamic wireless environments that demand low latency and high safety. However, its capability remains underexplored in wireless settings, where offline data are often limited and subject to partial observability. As a result, the learned policy fails to fully capture the informative decision patterns hidden in the dataset. To bridge this gap, we propose MEO, a framework that mines reliable expert signals from wireless datasets for offline RL deployment. MEO first identifies reliable expert samples from imperfect offline data, and then learns a structured latent representation to better distinguish expert and non-expert behaviors. Based on the representation, MEO further derives a compensable reward signal that helps offline RL algorithms more effectively exploit valuable expert knowledge. We evaluate MEO in three practical wireless network scenarios using datasets collected from real-world deployments. The results show that MEO consistently outperforms existing methods across all three scenarios. Notably, MEO can be seamlessly integrated with any offline RL algorithms, highlighting its generalizability and potential for wide adoption in wireless networks. Code is available at https://github.com/lpzu/MEO.