arXiv subjects

Hangguan Shan

Publications and source records attributed to Hangguan Shan.

16 recordsLinked to original sources

arXiv2026

SHAP-Guided Kernel Actor-Critic for Explainable Reinforcement Learning

Actor-critic (AC) methods are a cornerstone of reinforcement learning (RL) but offer limited interpretability. Current explainable RL methods seldom use state attributions to assist training. Rather, they treat all state features equally, thereby neglecting the heterogeneous impacts of individual state dimensions on the reward. We propose RKHS-SHAP-based Advanced Actor-Critic (RSA2C), an attribution-aware, kernelized, two-timescale AC algorithm, including Actor, Value Critic, and Advantage Critic. The Actor is instantiated in a vector-valued reproducing kernel Hilbert space (RKHS) with a Mahalanobis-weighted operator-valued kernel, while the Value Critic and Advantage Critic reside in scalar RKHSs. These RKHS-enhanced components use sparsified dictionaries: the Value Critic maintains its own dictionary, while the Actor and Advantage Critic share one. State attributions, computed from the Value Critic via RKHS-SHAP (kernel mean embedding for on-manifold and conditional mean embedding for off-manifold expectations), are converted into Mahalanobis-gated weights that modulate Actor gradients and Advantage Critic targets. We derive a global, non-asymptotic convergence bound under state perturbations, showing stability through the perturbation-error term and efficiency through the convergence-error term. Empirical results on three continuous-control environments show that RSA2C achieves efficiency, stability, and interpretability. Our code is available at https://github.com/Na-Li66/RSA2C.

cs.LG↗

arXiv2025

Multivariate Time Series Forecasting with Hybrid Euclidean-SPD Manifold Graph Neural Networks

Multivariate Time Series (MTS) forecasting plays a vital role in various real-world applications, such as traffic management and predictive maintenance. Existing approaches typically model MTS data in either Euclidean or Riemannian space, limiting their ability to capture the diverse geometric structures and complex spatio-temporal dependencies inherent in real-world data. To overcome this limitation, we propose the Hybrid Symmetric Positive-Definite Manifold Graph Neural Network (HSMGNN), a novel graph neural network-based model that captures data geometry within a hybrid Euclidean-Riemannian framework. To the best of our knowledge, this is the first work to leverage hybrid geometric representations for MTS forecasting, enabling expressive and comprehensive modeling of geometric properties. Specifically, we introduce a Submanifold-Cross-Segment (SCS) embedding to project input MTS into both Euclidean and Riemannian spaces, thereby capturing spatio-temporal variations across distinct geometric domains. To alleviate the high computational cost of Riemannian distance, we further design an Adaptive-Distance-Bank (ADB) layer with a trainable memory mechanism. Finally, a Fusion Graph Convolutional Network (FGCN) is devised to integrate features from the dual spaces via a learnable fusion operator for accurate prediction. Experiments on three benchmark datasets demonstrate that HSMGNN achieves up to a 13.8 percent improvement over state-of-the-art baselines in forecasting accuracy.

cs.LG↗

arXiv2025

Provable Memory Efficient Self-Play Algorithm for Model-free Reinforcement Learning

The thriving field of multi-agent reinforcement learning (MARL) studies how a group of interacting agents make decisions autonomously in a shared dynamic environment. Existing theoretical studies in this area suffer from at least two of the following obstacles: memory inefficiency, the heavy dependence of sample complexity on the long horizon and the large state space, the high computational complexity, non-Markov policy, non-Nash policy, and high burn-in cost. In this work, we take a step towards settling this problem by designing a model-free self-play algorithm \emph{Memory-Efficient Nash Q-Learning (ME-Nash-QL)} for two-player zero-sum Markov games, which is a specific setting of MARL. ME-Nash-QL is proven to enjoy the following merits. First, it can output an $\varepsilon$-approximate Nash policy with space complexity $O(SABH)$ and sample complexity $\widetilde{O}(H^4SAB/\varepsilon^2)$, where $S$ is the number of states, $\{A, B\}$ is the number of actions for two players, and $H$ is the horizon length. It outperforms existing algorithms in terms of space complexity for tabular cases, and in terms of sample complexity for long horizons, i.e., when $\min\{A, B\}\ll H^2$. Second, ME-Nash-QL achieves the lowest computational complexity $O(T\mathrm{poly}(AB))$ while preserving Markov policies, where $T$ is the number of samples. Third, ME-Nash-QL also achieves the best burn-in cost $O(SAB\,\mathrm{poly}(H))$, whereas previous algorithms have a burn-in cost of at least $O(S^3 AB\,\mathrm{poly}(H))$ to attain the same level of sample complexity with ours.

cs.LG↗

arXiv2025

Sample-Efficient Tabular Self-Play for Offline Robust Reinforcement Learning

Multi-agent reinforcement learning (MARL), as a thriving field, explores how multiple agents independently make decisions in a shared dynamic environment. Due to environmental uncertainties, policies in MARL must remain robust to tackle the sim-to-real gap. We focus on robust two-player zero-sum Markov games (TZMGs) in offline settings, specifically on tabular robust TZMGs (RTZMGs). We propose a model-based algorithm (\textit{RTZ-VI-LCB}) for offline RTZMGs, which is optimistic robust value iteration combined with a data-driven Bernstein-style penalty term for robust value estimation. By accounting for distribution shifts in the historical dataset, the proposed algorithm establishes near-optimal sample complexity guarantees under partial coverage and environmental uncertainty. An information-theoretic lower bound is developed to confirm the tightness of our algorithm's sample complexity, which is optimal regarding both state and action spaces. To the best of our knowledge, RTZ-VI-LCB is the first to attain this optimality, sets a new benchmark for offline RTZMGs, and is validated experimentally.

cs.LG↗

arXiv2025

Learning Causal States Under Partial Observability and Perturbation

A critical challenge for reinforcement learning (RL) is making decisions based on incomplete and noisy observations, especially in perturbed and partially observable Markov decision processes (P$^2$OMDPs). Existing methods fail to mitigate perturbations while addressing partial observability. We propose \textit{Causal State Representation under Asynchronous Diffusion Model (CaDiff)}, a framework that enhances any RL algorithm by uncovering the underlying causal structure of P$^2$OMDPs. This is achieved by incorporating a novel asynchronous diffusion model (ADM) and a new bisimulation metric. ADM enables forward and reverse processes with different numbers of steps, thus interpreting the perturbation of P$^2$OMDP as part of the noise suppressed through diffusion. The bisimulation metric quantifies the similarity between partially observable environments and their causal counterparts. Moreover, we establish the theoretical guarantee of CaDiff by deriving an upper bound for the value function approximation errors between perturbed observations and denoised causal states, reflecting a principled trade-off between approximation errors of reward and transition-model. Experiments on Roboschool tasks show that CaDiff enhances returns by at least 14.18\% compared to baselines. CaDiff is the first framework that approximates causal states using diffusion models with both theoretical rigor and practicality.

cs.LG↗

arXiv2025

RayFusion: Ray Fusion Enhanced Collaborative Visual Perception

Collaborative visual perception methods have gained widespread attention in the autonomous driving community in recent years due to their ability to address sensor limitation problems. However, the absence of explicit depth information often makes it difficult for camera-based perception systems, e.g., 3D object detection, to generate accurate predictions. To alleviate the ambiguity in depth estimation, we propose RayFusion, a ray-based fusion method for collaborative visual perception. Using ray occupancy information from collaborators, RayFusion reduces redundancy and false positive predictions along camera rays, enhancing the detection performance of purely camera-based collaborative perception systems. Comprehensive experiments show that our method consistently outperforms existing state-of-the-art models, substantially advancing the performance of collaborative visual perception. The code is available at https://github.com/wangsh0111/RayFusion.

cs.CV↗

arXiv2025

Federated Learning Resilient to Byzantine Attacks and Data Heterogeneity

This paper addresses federated learning (FL) in the context of malicious Byzantine attacks and data heterogeneity. We introduce a novel Robust Average Gradient Algorithm (RAGA), which uses the geometric median for aggregation and {allows flexible round number for local updates.} Unlike most existing resilient approaches, which base their convergence analysis on strongly-convex loss functions or homogeneously distributed datasets, this work conducts convergence analysis for both strongly-convex and non-convex loss functions over heterogeneous datasets. The theoretical analysis indicates that as long as the fraction of the {data} from malicious users is less than half, RAGA can achieve convergence at a rate of $\mathcal{O}({1}/{T^{2/3- δ}})$ for non-convex loss functions, where $T$ is the iteration number and $δ\in (0, 2/3)$. For strongly-convex loss functions, the convergence rate is linear. Furthermore, the stationary point or global optimal solution is shown to be attainable as data heterogeneity diminishes. Experimental results validate the robustness of RAGA against Byzantine attacks and demonstrate its superior convergence performance compared to baselines under varying intensities of Byzantine attacks on heterogeneous datasets.

cs.LG↗

arXiv2024

IFTR: An Instance-Level Fusion Transformer for Visual Collaborative Perception

Multi-agent collaborative perception has emerged as a widely recognized technology in the field of autonomous driving in recent years. However, current collaborative perception predominantly relies on LiDAR point clouds, with significantly less attention given to methods using camera images. This severely impedes the development of budget-constrained collaborative systems and the exploitation of the advantages offered by the camera modality. This work proposes an instance-level fusion transformer for visual collaborative perception (IFTR), which enhances the detection performance of camera-only collaborative perception systems through the communication and sharing of visual features. To capture the visual information from multiple agents, we design an instance feature aggregation that interacts with the visual features of individual agents using predefined grid-shaped bird eye view (BEV) queries, generating more comprehensive and accurate BEV features. Additionally, we devise a cross-domain query adaptation as a heuristic to fuse 2D priors, implicitly encoding the candidate positions of targets. Furthermore, IFTR optimizes communication efficiency by sending instance-level features, achieving an optimal performance-bandwidth trade-off. We evaluate the proposed IFTR on a real dataset, DAIR-V2X, and two simulated datasets, OPV2V and V2XSet, achieving performance improvements of 57.96%, 9.23% and 12.99% in AP@70 metrics compared to the previous SOTAs, respectively. Extensive experiments demonstrate the superiority of IFTR and the effectiveness of its key components. The code is available at https://github.com/wangsh0111/IFTR.

cs.CV↗

arXiv2023

Adaptive Base-class Suppression and Prior Guidance Network for One-Shot Object Detection

One-shot object detection (OSOD) aims to detect all object instances towards the given category specified by a query image. Most existing studies in OSOD endeavor to explore effective cross-image correlation and alleviate the semantic feature misalignment, however, ignoring the phenomenon of the model bias towards the base classes and the generalization degradation on the novel classes. Observing this, we propose a novel framework, namely Base-class Suppression and Prior Guidance (BSPG) network to overcome the problem. Specifically, the objects of base categories can be explicitly detected by a base-class predictor and adaptively eliminated by our base-class suppression module. Moreover, a prior guidance module is designed to calculate the correlation of high-level features in a non-parametric manner, producing a class-agnostic prior map to provide the target features with rich semantic cues and guide the subsequent detection process. Equipped with the proposed two modules, we endow the model with a strong discriminative ability to distinguish the target objects from distractors belonging to the base classes. Extensive experiments show that our method outperforms the previous techniques by a large margin and achieves new state-of-the-art performance under various evaluation settings.

cs.CV↗

arXiv2020

Throughput Analysis of Small Cell Networks under D-TDD and FFR

Dynamic time-division duplex (D-TDD) has emerged as an effective solution to accommodate the unaligned downlink and uplink traffic in small cell networks. However, the flexibility of traffic configuration also introduces additional inter-cell interference. In this letter, we study the effectiveness of applying fractional frequency reuse (FFR) as an interference coordination technique for D-TDD small cell networks. We derive the analytical expressions of downlink and uplink mean packet throughput (MPT), then study a network parameter optimization problem to maximize MPT while guaranteeing each user's throughput. Numerical results corroborate the benefits of the proposed FFR-based D-TDD in terms of improving throughput.

cs.IT↗

arXiv2019

Three Dimensional Convolutional Neural Network Pruning with Regularization-Based Method

Despite enjoying extensive applications in video analysis, three-dimensional convolutional neural networks (3D CNNs)are restricted by their massive computation and storage consumption. To solve this problem, we propose a threedimensional regularization-based neural network pruning method to assign different regularization parameters to different weight groups based on their importance to the network. Further we analyze the redundancy and computation cost for each layer to determine the different pruning ratios. Experiments show that pruning based on our method can lead to 2x theoretical speedup with only 0.41% accuracy loss for 3DResNet18 and 3.28% accuracy loss for C3D. The proposed method performs favorably against other popular methods for model compression and acceleration.

cs.LG↗

arXiv2018

Energy Efficient Massive MIMO through Distributed Precoder Design

This paper presents an energy-efficient downlink precoding scheme with the objective of maximizing system energy efficiency in a multi-cell massive MIMO system. The proposed precoding design jointly considers the issues of power control, interference management, antenna switching and user throughput in a cluster of base stations (BS). We demonstrate that the precoding design can be formulated into a general sparsity-inducing non-convex problem, which is NP-hard. We thus apply a smooth approximation of zero-norm in the antenna power management to enable the application of the gradient-based algorithms. The non-convexity of the problem may also cause slow convergence or even divergence if some classical gradient algorithms are directly applied. We thus develop an efficient alternative algorithm combining features from augmented multiplier (AM) and quadratic programming (QP) to guarantee the convergence. We theoretically prove the convergence conditions for our algorithm both locally and globally under realistic assumptions. Our proposed algorithms further facilitate a distributed implementation that reduces backhaul overhead by offloading data-intensive steps to local computation at each BS. Numerical results confirm that our methods indeed achieve higher energy efficiency with superior convergence rate, compared to some well-known existing methods.

cs.NI↗

arXiv2018

Throughput Analysis and Energy Efficiency Optimization for Standalone LTE-U Networks with Randomly Delayed CSI

To coexist with Wi-Fi friendly, a standalone long-term evolution network over unlicensed spectrum (LTE-U) under listen-before-talk (LBT) mechanism can only access channel in a random and intermittent way, which results in random and time-variant delay in both data and signaling transmissions. In this work, we explore the impact of randomly delayed channel state information (CSI) on the performance of a standalone LTE-U network by analyzing its downlink throughput and users' energy efficiency (EE) under different CSI feedback schemes and frequency-domain schedulers. Further, aiming at maximizing users' EE of a standalone LTE-U network while guaranteeing fair coexistence with Wi-Fi, joint optimization on the medium access control (MAC) protocol and CSI feedback scheme is studied. Due to the non-convex and non-concave characteristic of the formulated optimization problems, decomposition-based low-complexity yet efficient algorithms are proposed. Simulation results not only verify the analysis and the effectiveness of the proposed algorithms but also show diverse impacts of CSI feedback scheme, frequency-domain scheduler, and traffic load of Wi-Fi on the standalone LTE-U network performance.

cs.NI↗

arXiv2016

Simulation Results of User Behavior-Aware Scheduling Based on Time-Frequency Resource Conversion

Integrating time-frequency resource conversion (TFRC), a new network resource allocation strategy, with call admission control can not only increase the cell capacity but also reduce network congestion effectively. However, the optimal setting of TFRC-oriented call admission control suffers from the curse of dimensionality, due to Markov chain-based optimization in a high-dimensional space. To address the scalability issue of TFRC, in [1] we extend the study of TFRC into the area of scheduling. Specifically, we study downlink scheduling based on TFRC for an LTE-type cellular network, to maximize service delivery. The service scheduling of interest is formulated as a joint request, channel and slot allocation problem which is NP-hard. An offline deflation and sequential fixing based algorithm (named DSFRB) with only polynomial-time complexity is proposed to solve the problem. For practical online implementation, two TFRC-enabled low-complexity algorithms, modified Smith ratio algorithm (named MSR) and modified exponential capacity algorithm (named MEC), are proposed as well. In this report, we present detailed numerical results of the proposed offline and online algorithms, which not only show the effectiveness of the proposed algorithms but also corroborate the advantages of the proposed TFRC-based schedule techniques in terms of quality-of-service (QoS) provisioning for each user and revenue improvement for a service operator.

cs.NI↗

arXiv2013

State Transition Analysis of Time-Frequency Resource Conversion-based Call Admission Control for LTE-Type Cellular Network

To address network congestion stemmed from traffic generated by advanced user equipments, in [1] we propose a novel network resource allocation strategy, time-frequency resource conversion (TFRC), via exploiting user behavior, a specific kind of context information. Considering an LTE-type cellular network, a call admission control policy called double-threshold guard channel policy is proposed there to facilitate the implementation of TFRC. In this report, we present state transition analysis of this TFRC-based call admission control policy for an LTE-type cellular network. Overall, there are five categories of events that can trigger a transition of the system state: 1) a new call arrival; 2) a handoff user arrival; 3) a handoff user departure; 4) a call termination; and 5) a periodic time-frequency resource conversion. We analyze them case by case in this report and the validation of the analysis has been provided in [1].

cs.NI↗

arXiv2012

Moving Window Network Coding in Cooperative Multicast (v1)

Cooperative multicast is an effective solution to address the bottleneck problem of single-hop broadcast in wireless networks. By incorporating with the random linear network coding technique, the existing schemes can reduce the retransmission overhead significantly. However, the receivers may incur large decoding delay and complexity due to the batch decoding scheme. In addition, the dependency on the explicit feedback leads to scalability problem in larger networks. In this paper, a cooperative multicast protocol named MWNCast is proposed based on a novel moving window network coding technique. We prove three properties of the proposed scheme. Firstly, without explicit feedback, MWNCast can approach the cooperative capacity with the packet loss probability dropping almost exponentially with the increase of window size. Secondly, the average decoding delay of a receiver is on the order of $O(\frac{1}{(1-ρ)^2})$ with respect to its traffic intensity $ρ$. Thirdly, MWNCast can achieve the linear decoding complexity of $O(W)$ with respect to the window size $W$. Simulation results show that MWNCast outperforms the existing schemes by achieving better tradeoff between the throughput and decoding delay, meanwhile keeping the packet loss probability and decoding complexity at a very low levelwithout explicit feedback.

cs.NI↗