arXiv subjects

Yuanzhe Geng

Publications and source records attributed to Yuanzhe Geng.

8 recordsLinked to original sources

arXiv2026

Cooperative MARL-Based Energy-Efficient Power Control for Two-Hop Relaying Networks

In this paper, we study a cooperative game in the cooperative communication network, where each relay makes decisions autonomously and aims to achieve the same optimization objective of maximizing energy efficiency. We consider the non-ideal situation where instantaneous channel state information (CSI) is difficult to obtain and only partially observable outdated CSI is available. To solve this game problem, we define a delayed reward-based state-action value function and propose a multi-agent deep Q network learning framework. Then, we prove analytically that utilities obtained by game-theoretic approaches with the instantaneous CSI serve as upper bounds for those of the proposed method. Simulation results reveal that our approach considerably outperforms its potential alternatives and is only about 5.2% away from the optimal solution.

eess.SY↗

arXiv2025

LAPA-based Dynamic Privacy Optimization for Wireless Federated Learning in Heterogeneous Environments

Federated Learning (FL) is a distributed machine learning paradigm based on protecting data privacy of devices, which however, can still be broken by gradient leakage attack via parameter inversion techniques. Differential privacy (DP) technology reduces the risk of private data leakage by adding artificial noise to the gradients, but detrimental to the FL utility at the same time, especially in the scenario where the data is Non-Independent Identically Distributed (Non-IID). Based on the impact of heterogeneous data on aggregation performance, this paper proposes a Lightweight Adaptive Privacy Allocation (LAPA) strategy, which assigns personalized privacy budgets to devices in each aggregation round without transmitting any additional information beyond gradients, ensuring both privacy protection and aggregation efficiency. Furthermore, the Deep Deterministic Policy Gradient (DDPG) algorithm is employed to optimize the transmission power, in order to determine the optimal timing at which the adaptively attenuated artificial noise aligns with the communication noise, enabling an effective balance between DP and system utility. Finally, a reliable aggregation strategy is designed by integrating communication quality and data distribution characteristics, which improves aggregation performance while preserving privacy. Experimental results demonstrate that the personalized noise allocation and dynamic optimization strategy based on LAPA proposed in this paper enhances convergence performance while satisfying the privacy requirements of FL.

cs.LG↗

arXiv2024

Balancing Performance and Cost for Two-Hop Cooperative Communications: Stackelberg Game and Distributed Multi-Agent Reinforcement Learning

This paper aims to balance performance and cost in a two-hop wireless cooperative communication network where the source and relays have contradictory optimization goals and make decisions in a distributed manner. This differs from most existing works that have typically assumed that source and relay nodes follow a schedule created implicitly by a central controller. We propose that the relays form an alliance in an attempt to maximize the benefit of relaying while the source aims to increase the channel capacity cost-effectively. To this end, we establish the trade problem as a Stackelberg game, and prove the existence of its equilibrium. Another important aspect is that we use multi-agent reinforcement learning (MARL) to approach the equilibrium in a situation where the instantaneous channel state information (CSI) is unavailable, and the source and relays do not have knowledge of each other's goal. A multi-agent deep deterministic policy gradient-based framework is designed, where the relay alliance and the source act as agents. Experiments demonstrate that the proposed method can obtain an acceptable performance that is close to the game-theoretic equilibrium for all players under time-invariant environments, which considerably outperforms its potential alternatives and is only about 2.9% away from the optimal solution.

eess.SY↗

arXiv2022

Reinforcement Learning Based Robust Policy Design for Relay and Power Optimization in DF Relaying Networks

In this paper, we study the outage minimization problem in a decode-and-forward cooperative network with relay uncertainty. To reduce the outage probability and improve the quality of service, existing researches usually rely on the assumption of both exact instantaneous channel state information (CSI) and environmental uncertainty. However, it is difficult to obtain perfect instantaneous CSI immediately under practical situations where channel states change rapidly, and the uncertainty in communication environments may not be observed, which makes traditional methods not applicable. Therefore, we turn to reinforcement learning (RL) methods for solutions, which do not need any prior knowledge of underlying channel or assumptions of environmental uncertainty. RL method is to learn from the interaction with communication environment, optimize its action policy, and then propose relay selection and power allocation schemes. We first analyse the robustness of RL action policy by giving the lower bound of the worst-case performance, when RL methods are applied to communication scenarios with environment uncertainty. Then, we propose a robust algorithm for outage probability minimization based on RL. Simulation results reveal that compared with traditional RL methods, our approach has better generalization ability and can improve the worst-case performance by about 6% when evaluated in unseen environments.

cs.IT↗

arXiv2021

Deep Deterministic Policy Gradient for Relay Selection and Power Allocation in Cooperative Communication Network

Perfect channel state information (CSI) is usually required when considering relay selection and power allocation in cooperative communication. However, it is difficult to get an accurate CSI in practical situations. In this letter, we study the outage probability minimizing problem based on optimizing relay selection and transmission power. We propose a prioritized experience replay aided deep deterministic policy gradient learning framework, which can find an optimal solution by dealing with continuous action space, without any prior knowledge of CSI. Simulation results reveal that our approach outperforms reinforcement learning based methods in existing literatures, and improves the communication success rate by about 4%.

cs.IT↗

arXiv2021

Hierarchical Reinforcement Learning for Relay Selection and Power Optimization in Two-Hop Cooperative Relay Network

Cooperative communication is an effective approach to improve spectrum utilization. In order to reduce outage probability of communication system, most studies propose various schemes for relay selection and power allocation, which are based on the assumption of channel state information (CSI). However, it is difficult to get an accurate CSI in practice. In this paper, we study the outage probability minimizing problem subjected to a total transmission power constraint in a two-hop cooperative relay network. We use reinforcement learning (RL) methods to learn strategies for relay selection and power allocation, which do not need any prior knowledge of CSI but simply rely on the interaction with communication environment. It is noted that conventional RL methods, including most deep reinforcement learning (DRL) methods, cannot perform well when the search space is too large. Therefore, we first propose a DRL framework with an outage-based reward function, which is then used as a baseline. Then, we further propose a hierarchical reinforcement learning (HRL) framework and training algorithm. A key difference from other RL-based methods in existing literatures is that, our proposed HRL approach decomposes relay selection and power allocation into two hierarchical optimization objectives, which are trained in different levels. With the simplification of search space, the HRL approach can solve the problem of sparse reward, while the conventional RL method fails. Simulation results reveal that compared with traditional DRL method, the HRL training algorithm can reach convergence 30 training iterations earlier and reduce the outage probability by 5% in two-hop relay network with the same outage threshold.

eess.SY↗

arXiv2020

Deep Reinforcement Learning Based Dynamic Route Planning for Minimizing Travel Time

Route planning is important in transportation. Existing works focus on finding the shortest path solution or using metrics such as safety and energy consumption to determine the planning. It is noted that most of these studies rely on prior knowledge of road network, which may be not available in certain situations. In this paper, we design a route planning algorithm based on deep reinforcement learning (DRL) for pedestrians. We use travel time consumption as the metric, and plan the route by predicting pedestrian flow in the road network. We put an agent, which is an intelligent robot, on a virtual map. Different from previous studies, our approach assumes that the agent does not need any prior information about road network, but simply relies on the interaction with the environment. We propose a dynamically adjustable route planning (DARP) algorithm, where the agent learns strategies through a dueling deep Q network to avoid congested roads. Simulation results show that the DARP algorithm saves 52% of the time under congestion condition when compared with traditional shortest path planning algorithms.

cs.LG↗

arXiv2020

Channel Estimation and Power Scaling Law of Large Reflecting Surface with Non-Ideal Hardware

Large reflecting surface (LRS) has emerged as a new solution to improve the energy and spectrum efficiency of wireless communication system. Most existing studies were conducted with an assumption of ideal hardware, and the impact of hardware impairments receives little attention. However, the non-negligible hardware impairments should be taken into consideration when we evaluate the system performance. In this paper, we consider an LRS assisted communication system with hardware impairments, and focus on the channel estimation study and the power scaling law analysis. First, with linear minimum mean square error estimation, we theoretically characterize the relationship between channel estimation performance and impairment level, number of reflecting elements, and pilot power. After that, we analyze the power scaling law and reveal that if the base station (BS) has perfect channel state information, the transmit power of user can be made inversely proportional to the number of BS antennas and the square of the number of reflecting elements with no reduction in performance; If the BS has imperfectly estimated channel state information, to achieve the same performance, the transmit power of user can be made inversely proportional to the square-root of the number of BS antennas and the square of the number of reflecting elements.

cs.IT↗