arXiv subjects

Kexuan Wang

Publications and source records attributed to Kexuan Wang.

8 recordsLinked to original sources

arXiv2026

Fused Constrained Policy Reuse Optimization for Wireless Resource Allocation

Deep reinforcement learning (DRL) has been widely adopted for wireless resource allocation due to its model-free adaptability. However, online exploration is costly, as randomly initialized policies may violate long-term constraints before sufficient data are collected. Future wireless systems must cope with increasingly dynamic traffic, fluctuating channel conditions, and stringent energy efficiency requirements, demanding algorithms that can learn quickly with minimal environment interactions to reduce both energy consumption and signaling overhead. We develop Fused-CPRO, a knowledge-fused constrained policy reuse optimization method addressing these challenges. Fused-CPRO constructs the allocation policy as a mixture of a learnable target policy, source policies from related scenarios, and domain-knowledge (DK) policies from expert rules, jointly optimizing the target policy and reuse probabilities under a constrained Markov decision process (CMDP). This fusion of heterogeneous priors accelerates convergence and enhances robustness. Constrained stochastic successive convex approximation (CSSCA) handles non-convex objectives and constraints, while a critic trained from mixed offline-online data improves sample efficiency by reusing pre-collected experience. We prove almost-sure convergence to a Karush-Kuhn-Tucker (KKT) point. Simulations on delay-constrained multi-user multiple-input multiple-output (MU-MIMO) power control and Cramer-Rao bound (CRB)-constrained multiple-input multiple-output integrated sensing and communication (MIMO-ISAC) beamforming demonstrate that Fused-CPRO improves empirical performance and converges substantially faster than representative baselines.

eess.SP↗

arXiv2026

All-Optically Controlled Memristive Reservoir Computing Capable of Bipolar and Parallel Coding

Physical reservoir computing (RC) utilizes the intrinsic dynamical evolution of physical systems for efficient data processing. Emerging optoelectronic RC platforms,such as light-driven memristors, merge the benefits of electronic and photonic computation. However, conventional designs are often limited by the unipolar photoresponse of optoelectronic devices, which restricts reservoir state diversity and reduces computational accuracy. To overcome these limitations, we introduce an all-optically controlled RC system employing an oxide memristor array that demonstrates exceptional uniformity and stability. The memristive devices exhibit wavelength-dependent bipolar photoresponse, originating from light-induced dynamic evolution of oxygen vacancies. Tuning the power density and irradiation mode of dual-wavelength light pulses enables dynamic control of photocurrent relaxation and nonlinearity. By leveraging these unique device properties, we develop bipolar and parallel coding strategies to significantly enrich reservoir dynamics and enhance nonlinear mapping capability. In word recognition and time-series prediction tasks, the bipolar coding demonstrates markedly improved accuracy compared to unipolar coding. The parallel coding supports multi-source signal fusion within a single reservoir, maintaining high computational accuracy while significantly reducing hardware consumption. This work provides a high-performance approach to physical RC, paving the way for intelligent edge computing.

physics.app-ph↗

arXiv2025

A Lightweight RL-Driven Deep Unfolding Network for Robust WMMSE Precoding in Massive MU-MIMO-OFDM Systems

Weighted Minimum Mean Square Error (WMMSE) precoding is widely recognized for its near-optimal weighted sum rate performance. However, its practical deployment in massive multi-user (MU) multiple-input multiple-output (MIMO) orthogonal frequency-division multiplexing (OFDM) systems is hindered by the assumption of perfect channel state information (CSI) and high computational complexity. To address these issues, we first develop a wideband stochastic WMMSE (SWMMSE) algorithm that iteratively maximizes the ergodic weighted sum-rate (EWSR) under imperfect CSI. Building on this, we propose a lightweight reinforcement learning (RL)-driven deep unfolding (DU) network (RLDDU-Net), where each SWMMSE iteration is mapped to a network layer. Specifically, its DU module integrates approximation techniques and leverages beam-domain sparsity as well as frequency-domain subcarrier correlation, significantly accelerating convergence and reducing computational overhead. Furthermore, the RL module adaptively adjusts the network depth and generates compensation matrices to mitigate approximation errors. Simulation results under imperfect CSI demonstrate that RLDDU-Net outperforms existing baselines in EWSR performance while offering superior computational and convergence efficiency.

cs.LG↗

arXiv2025

Joint User Priority and Power Scheduling for QoS-Aware WMMSE Precoding: A Constrained-Actor Attentive-Critic Approach

6G wireless networks are expected to support diverse quality-of-service (QoS) demands while maintaining high energy efficiency. Weighted Minimum Mean Square Error (WMMSE) precoding with fixed user priorities and transmit power is widely recognized for enhancing overall system performance but lacks flexibility to adapt to user-specific QoS requirements and time-varying channel conditions. To address this, we propose a novel constrained reinforcement learning (CRL) algorithm, Constrained-Actor Attentive-Critic (CAAC), which uses a policy network to dynamically allocate user priorities and power for WMMSE precoding. Specifically, CAAC integrates a Constrained Stochastic Successive Convex Approximation (CSSCA) method to optimize the policy, enabling more effective handling of energy efficiency goals and satisfaction of stochastic non-convex QoS constraints compared to traditional and existing CRL methods. Moreover, CAAC employs lightweight attention-enhanced Q-networks to evaluate policy updates without prior environment model knowledge. The network architecture not only enhances representational capacity but also boosts learning efficiency. Simulation results show that CAAC outperforms baselines in both energy efficiency and QoS satisfaction.

cs.LG↗

arXiv2025

A Hybrid Reinforcement Learning Framework for Hard Latency Constrained Resource Scheduling

In the forthcoming 6G era, extend reality (XR) has been regarded as an emerging application for ultra-reliable and low latency communications (URLLC) with new traffic characteristics and more stringent requirements. In addition to the quasi-periodical traffic in XR, burst traffic with both large frame size and random arrivals in some real world low latency communication scenarios has become the leading cause of network congestion or even collapse, and there still lacks an efficient algorithm for the resource scheduling problem under burst traffic with hard latency constraints. We propose a novel hybrid reinforcement learning framework for resource scheduling with hard latency constraints (HRL-RSHLC), which reuses polices from both old policies learned under other similar environments and domain-knowledge-based (DK) policies constructed using expert knowledge to improve the performance. The joint optimization of the policy reuse probabilities and new policy is formulated as an Markov Decision Problem (MDP), which maximizes the hard-latency constrained effective throughput (HLC-ET) of users. We prove that the proposed HRL-RSHLC can converge to KKT points with an arbitrary initial point. Simulations show that HRL-RSHLC can achieve superior performance with faster convergence speed compared to baseline algorithms.

cs.NI↗

arXiv2025

Context-aware Constrained Reinforcement Learning Based Energy-Efficient Power Scheduling for Non-stationary XR Data Traffic

In XR downlink transmission, energy-efficient power scheduling (EEPS) is essential for conserving power resource while delivering large data packets within hard-latency constraints. Traditional constrained reinforcement learning (CRL) algorithms show promise in EEPS but still struggle with non-convex stochastic constraints, non-stationary data traffic, and sparse delayed packet dropout feedback (rewards) in XR. To overcome these challenges, this paper models the EEPS in XR as a dynamic parameter-constrained Markov decision process (DP-CMDP) with a varying transition function linked to the non-stationary data traffic and solves it by a proposed context-aware constrained reinforcement learning (CACRL) algorithm, which consists of a context inference (CI) module and a CRL module. The CI module trains an encoder and multiple potential networks to characterize the current transition function and reshape the packet dropout rewards according to the context, transforming the original DP-CMDP into a general CMDP with immediate dense rewards. The CRL module employs a policy network to make EEPS decisions under this CMDP and optimizes the policy using a constrained stochastic successive convex approximation (CSSCA) method, which is better suited for non-convex stochastic constraints. Finally, theoretical analyses provide deep insights into the CADAC algorithm, while extensive simulations demonstrate that it outperforms advanced baselines in both power conservation and satisfying packet dropout constraints.

eess.SY↗

arXiv2024

A Single-Loop Deep Actor-Critic Algorithm for Constrained Reinforcement Learning with Provable Convergence

Deep Actor-Critic algorithms, which combine Actor-Critic with deep neural network (DNN), have been among the most prevalent reinforcement learning algorithms for decision-making problems in simulated environments. However, the existing deep Actor-Critic algorithms are still not mature to solve realistic problems with non-convex stochastic constraints and high cost to interact with the environment. In this paper, we propose a single-loop deep Actor-Critic (SLDAC) algorithmic framework for general constrained reinforcement learning (CRL) problems. In the actor step, the constrained stochastic successive convex approximation (CSSCA) method is applied to handle the non-convex stochastic objective and constraints. In the critic step, the critic DNNs are only updated once or a few finite times for each iteration, which simplifies the algorithm to a single-loop framework (the existing works require a sufficient number of updates for the critic step to ensure a good enough convergence of the inner loop for each iteration). Moreover, the variance of the policy gradient estimation is reduced by reusing observations from the old policy. The single-loop design and the observation reuse effectively reduce the agent-environment interaction cost and computational complexity. In spite of the biased policy gradient estimation incurred by the single-loop design and observation reuse, we prove that the SLDAC with a feasible initial point can converge to a Karush-Kuhn-Tuker (KKT) point of the original problem almost surely. Simulations show that the SLDAC algorithm can achieve superior performance with much lower interaction cost.

cs.LG↗

arXiv2022

Joint Pilot Optimization, Target Detection and Channel Estimation for Integrated Sensing and Communication Systems

Radar sensing will be integrated into the 6G communication system to support various applications. In this integrated sensing and communication system, a radar target may also be a communication channel scatterer. In this case, the radar and communication channels exhibit certain joint burst sparsity. We propose a two-stage joint pilot optimization, target detection and channel estimation scheme to exploit such joint burst sparsity and pilot beamforming gain to enhance detection/estimation performance. In Stage 1, the base station (BS) sends downlink pilots (DP) for initial target search, and the user sends uplink pilots (UP) for channel estimation. Then the BS performs joint target detection and channel estimation based on the reflected DP and received UP signals. In Stage 2, the BS exploits the prior information obtained in Stage 1 to optimize the DP signal to achieve beamforming gain and further refine the performance. A Turbo Sparse Bayesian inference algorithm is proposed for joint target detection and channel estimation in both stages. The pilot optimization problem in Stage 2 is a semi-definite programming with rank-1 constraints. By replacing the rank-1 constraint with a tight and smooth approximation, we propose an efficient pilot optimization algorithm based on the majorization-minimization method. Simulations verify the advantages of the proposed scheme.

cs.IT↗