arXiv subjects

Songnam Hong

Publications and source records attributed to Songnam Hong.

At least 19 recordsLinked to original sources

arXiv2026

Multi-step Proximal Policy Improvement in Offline Reinforcement Learning

Offline reinforcement learning (RL) must reconcile two competing requirements: policy updates should stay near dataset-supported actions to keep value estimates reliable, yet meaningful gains often require moving beyond the behavior distribution. We develop a geometric view of offline actor updates by modeling policies as a probability manifold endowed with a chosen metric geometry. Under this lens, a broad class of offline actor objectives can be interpreted as a single proximal policy improvement step (SPI), i.e., an implicit discretization of a manifold gradient flow induced by a critic-defined energy. Building on this insight, we propose multi-step proximal policy improvement (MPI), a plug-in refinement mechanism that composes sequential re-centered proximal steps. MPI enables controlled policy improvement beyond dataset support while retaining proximal control at each refinement. The framework accommodates multiple policy geometries and admits practical instantiations for deterministic and diagonal-Gaussian policies. Experiments on D4RL benchmarks show that small numbers of MPI refinements improve strong offline baselines, including TD3+BC, ReBRAC, and IQL, on many tasks. Focused diagnostics further distinguish re-centered refinement from fixed-objective update scheduling and characterize limitations under critic error.

cs.LG↗

arXiv2026

PathBridger: Subgoal Bridges for Offline Goal-Conditioned Reinforcement Learning

Offline goal-conditioned reinforcement learning (GCRL) aims to learn policies for reaching diverse goals entirely from fixed trajectory data. Long-horizon offline GCRL remains challenging because sparse goal-reaching signals must be propagated over many steps, while execution errors cannot be corrected through additional environment interaction. Existing methods address these challenges by improving long-range value estimation or reducing the effective decision horizon through subgoals, options, and action chunks. In several hierarchical methods, however, a selected subgoal specifies where to go, while the intervening state-space path remains implicit in an endpoint-conditioned low-level policy. To address this interface, we propose PathBridger, a hierarchical offline GCRL method that explicitly connects subgoal selection to short-horizon execution. PathBridger constructs a state-space bridge toward the selected intermediate endpoint and decodes it into a short executable action chunk using an inverse dynamics model. Experiments across the evaluated OGBench tasks demonstrate strong aggregate performance, with particularly large gains on the multi-object Cube manipulation tasks. Code: https://github.com/SChoish/PathBridger

cs.LG↗

arXiv2026

Heterogeneous Multi-Agent Reinforcement Learning for Radio Resource Management under Coupled Finite-Horizon Constraints

Maximizing throughput under proportional fairness in dense wireless networks requires jointly managing user association, scheduling, base station (BS) activation, and handover control under hard finite-horizon energy and handover budgets, which induces a fundamental tension between BS-side energy management and user-side handover regulation. While multi-agent reinforcement learning (MARL) is a natural framework for such distributed sequential control, its application here faces two difficulties: finite-horizon budget constraints cannot be evaluated at each time slot, and the nonlinear proportional fairness utility admits no principled per-slot decomposition. We propose HeLyMARL, a Lyapunov-embedded heterogeneous MARL framework that resolves both via drift-plus-penalty decomposition with virtual queues. The energy and handover constraint pressures are internalized directly into a unified per-slot reward, converting the constrained finite-horizon problem into an unconstrained MARL problem. Comparison against two Lagrangian-based alternatives reveals a timescale separation: Lagrangian relaxation regulates constraints only across training episodes, whereas the virtual queues of HeLyMARL bound cumulative budget consumption at every partial horizon within an episode, a pacing guarantee beyond the reach of greedy Lyapunov-based control. Simulations show that HeLyMARL is the only method that sustains the throughput-fairness balance together with uninterrupted service throughout the horizon, outperforming conventional MARL, Lyapunov-based, and constrained MARL benchmarks without premature budget exhaustion.

cs.LG↗

arXiv2026

Information-Theoretically Secure Aggregation for Lightweight Federated Learning: Resilient to Dropouts and Adversaries

On-device federated learning (FL) enables privacy-preserving and personalized model training on resource-constrained devices such as smartphones and IoT nodes. To reduce communication cost, sign-based methods (e.g., signSGD) transmit one-bit gradients. However, exposing gradient signs makes them vulnerable to inference attacks, while existing secure aggregation schemes are often incompatible with such methods or incur significant computational and communication overhead. We propose a lightweight and information-theoretically secure aggregation framework tailored for sign-based FL. The framework securely computes the majority vote (MV) polynomial through single-round secure multiplication, ensuring end-to-end information-theoretic security under the honest-majority assumption while revealing only the final aggregated sign to the server. To enhance efficiency and scalability, we introduce two key techniques. First, inverse-form exponent reduction halves the effective MV polynomial degree, reducing both communication and computation costs. Second, we propose single-round secure multiplication, achieving linear offline complexity and storage with only a single online communication. Together, these techniques reduce online communication by up to 99.5% and latency by up to 85.7% compared to conventional approaches. Also, by leveraging inherent MDS-code-based decoding, the framework achieves robustness against both dropouts and adversarial behaviors, yielding accuracy gains of up to 20.65% and 10.74%, respectively. Overall, the proposed framework establishes a practical foundation for large-scale, low-latency, and information-theoretically secure aggregation in sign-based FL.

cs.LG↗

arXiv2026

Scalable Rate-Splitting Precoding via Recurrent Structure-Preserving Graph Neural Networks

Graph neural network (GNN)-based precoding has demonstrated strong potential for scalable multi-user beamforming in multi-user multiple-input single-output (MU-MISO) systems under space division multiple access (SDMA). However, direct extension to rate-splitting multiple access (RSMA) is non-trivial due to the coupled common/private-stream structure inherent to RSMA, which requires a fundamentally different graph representation and permutation equivariance structure. Motivated by this, we propose a recurrent structure-preserving graph neural network (RS-GNN) for scalable RSMA precoding. RS-GNN constructs precoder-dependent graph features at every refinement layer, enabling closed-loop interference-aware message passing, and recovers the common and private precoders through an analytically grounded structure-based reconstruction via a differentiable linear solver. This design decouples the learnable parameters from fixed system dimensions, enabling generalization to unseen system sizes without retraining. We formally prove that RS-GNN satisfies mixed permutation equivariance with respect to both user and antenna orderings, and show that RS-GNN reduces to conventional SDMA precoding as a special case by deactivating the common-stream branch. Simulation results demonstrate that RS-GNN achieves near-WMMSE sum-rate performance with significantly lower online inference time, while generalizing robustly to unseen system sizes; its SDMA special case consistently outperforms existing GNN-based precoders across unseen antenna and user configurations, SNR regimes, and channel distributions.

eess.SP↗

arXiv2026

Channel Estimation for Reconfigurable Intelligent Surface Assisted Upper Mid-Band MIMO Systems

The upper mid-band (UMB) spectrum is a key enabler for 6G systems, yet reconfigurable intelligent surface (RIS)-assisted UMB communications face severe channel estimation challenges due to near-field propagation and transitional scattering, which induce strong spatial correlation and ill-conditioned least-squares (LS) formulations. To overcome this limitation, we propose a conditioning-aware channel estimation framework that transforms the inherently ill-conditioned high-dimensional problem into multiple well-conditioned subproblems via greedy column grouping. By systematically separating highly correlated RIS elements into distinct sub-blocks via piecewise RIS phase design, the proposed method directly improves Gram matrix conditioning and stabilizes piecewise LS reconstruction without relying on sparsity assumptions. Simulation results demonstrate that the proposed method significantly outperforms conventional LS and OMP-based estimators in pilot-limited and transitional UMB regimes, achieving robust performance with low computational complexity.

eess.SP↗

arXiv2025

Hi-SAFE: Hierarchical Secure Aggregation for Lightweight Federated Learning

Federated learning (FL) faces challenges in ensuring both privacy and communication efficiency, particularly in resource-constrained environments such as Internet of Things (IoT) and edge networks. While sign-based methods, such as sign stochastic gradient descent with majority voting (SIGNSGD-MV), offer substantial bandwidth savings, they remain vulnerable to inference attacks due to exposure of gradient signs. Existing secure aggregation techniques are either incompatible with sign-based methods or incur prohibitive overhead. To address these limitations, we propose Hi-SAFE, a lightweight and cryptographically secure aggregation framework for sign-based FL. Our core contribution is the construction of efficient majority vote polynomials for SIGNSGD-MV, derived from Fermat's Little Theorem. This formulation represents the majority vote as a low-degree polynomial over a finite field, enabling secure evaluation that hides intermediate values and reveals only the final result. We further introduce a hierarchical subgrouping strategy that ensures constant multiplicative depth and bounded per-user complexity, independent of the number of users n.

cs.LG↗

arXiv2025

CSIT-Free Beamforming for Multi-Group Multicast in Overloaded mmWave Systems

We study downlink multi-group multicast (MGM) transmission in overloaded millimeter-wave (mmWave) systems, where the number of users exceeds the number of transmit antennas. We first show that, under realistic line-of-sight (LoS)-dominant user geometries, the conventional single-slot MGM scheme suffers from a fundamental collapse of the max-min fairness degrees of freedom (MMF-DoF), regardless of beamforming optimization. Although this collapse can in principle be avoided via aggressive time-division scheduling, it requires excessive time sharing and results in severe throughput loss in overloaded regimes. To address this limitation, we propose a CSIT-free multi-group multicast framework (CF-MGM) that does not rely on instantaneous channel state information at the transmitter (CSIT) and is based on a deterministic multi-slot transmission structure. By exploiting structured precoding and receiver-side combining across multiple slots, the proposed framework eliminates inter-group interference by construction. We show that CF-MGM guarantees a strictly positive MMF-DoF in overloaded LoS mmWave systems, in sharp contrast to the DoF collapse of conventional single-slot MGM. Simulation results demonstrate that CF-MGM significantly outperforms state-of-the-art CSIT-based MGM schemes while substantially reducing signaling overhead.

eess.SP↗

arXiv2025

CSIT-Free Downlink Transmission for mmWave MU-MISO Systems in High-Mobility Scenario

This paper investigates the downlink (DL) transmission in millimeter-wave (mmWave) multi-user multiple-input single-output (MU-MISO) systems especially focusing on a high speed mobile scenario. To complete the DL transmission within an extremely short channel coherence time, we propose a novel DL transmission framework that eliminates the need for channel state information at the transmitter (CSIT), of which acquisition process requires a substantial overhead, instead fully exploiting the given channel coherence time. Harnessing the characteristic of mmWave channel and uniquely designed CSIT-free unitary precoding, we propose a symbol detection method along with the simultaneous CSI at the receiver (CSIR) and Doppler shift estimation method to completely cancel the interferences while achieving a full combining gain. Via simulations, we demonstrate the effectiveness of the proposed method comparing with the existing baselines.

eess.SP↗

arXiv2025

Federated Reinforcement Learning in Heterogeneous Environments

We investigate a Federated Reinforcement Learning with Environment Heterogeneity (FRL-EH) framework, where local environments exhibit statistical heterogeneity. Within this framework, agents collaboratively learn a global policy by aggregating their collective experiences while preserving the privacy of their local trajectories. To better reflect real-world scenarios, we introduce a robust FRL-EH framework by presenting a novel global objective function. This function is specifically designed to optimize a global policy that ensures robust performance across heterogeneous local environments and their plausible perturbations. We propose a tabular FRL algorithm named FedRQ and theoretically prove its asymptotic convergence to an optimal policy for the global objective function. Furthermore, we extend FedRQ to environments with continuous state space through the use of expectile loss, addressing the key challenge of minimizing a value function over a continuous subset of the state space. This advancement facilitates the seamless integration of the principles of FedRQ with various Deep Neural Network (DNN)-based RL algorithms. Extensive empirical evaluations validate the effectiveness and robustness of our FRL algorithms across diverse heterogeneous environments, consistently achieving superior performance over the existing state-of-the-art FRL algorithms.

cs.LG↗

arXiv2025

Moderate Actor-Critic Methods: Controlling Overestimation Bias via Expectile Loss

Overestimation is a fundamental characteristic of model-free reinforcement learning (MF-RL), arising from the principles of temporal difference learning and the approximation of the Q-function. To address this challenge, we propose a novel moderate target in the Q-function update, formulated as a convex optimization of an overestimated Q-function and its lower bound. Our primary contribution lies in the efficient estimation of this lower bound through the lower expectile of the Q-value distribution conditioned on a state. Notably, our moderate target integrates seamlessly into state-of-the-art (SOTA) MF-RL algorithms, including Deep Deterministic Policy Gradient (DDPG) and Soft Actor Critic (SAC). Experimental results validate the effectiveness of our moderate target in mitigating overestimation bias in DDPG, SAC, and distributional RL algorithms.

cs.LG↗

arXiv2025

Blind Massive MIMO for Dense IoT Networks

In this paper, we investigate the downlink communication challenges in heavy-load Internet-of-Things (IoT) networks supported by frequency-division-duplexing (FDD) millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) systems. The excessive overhead required for obtaining channel state information at the transmitter (CSIT) is essential to achieve high spectral efficiency through conventional massive MIMO techniques; however, it hinders the deployment of ultra-reliable low-latency communications (URLLC) and leads to significant energy expenditure, particularly in dense IoT networks. To address this challenge, we propose an innovative CSIT-Free MIMO precoding method, referred to as CIRculant information Classification via Linear Estimation (CIRCLE). Our major contribution is the design of a CSIT-independent (or deterministic) precoding, which is constructed by leveraging the circulant permutation of the discrete Fourier transform (DFT) matrix. This design enables interference-free signal combining at the IoT devices. Through theoretical analysis and simulations, we verify the effectiveness of the proposed CIRCLE method.

eess.SP↗

arXiv2025

Piecewise Beam Training and Channel Estimation for RIS-Aided Near-Field Communications

In this paper, we investigate the channel estimation challenge in reconfigurable intelligent surface (RIS)-aided near-field communication systems. Current channel estimation techniques require substantial pilot overhead and computational complexity, especially when the number of RIS elements is extremely large. To address this issue, we introduce a two-timescale channel estimation strategy that leverages the asymmetric coherence times of both the RIS-base station (BS) channel and the User-RIS channel. We derive a time-scaling property indicating that, for any two effective channels within the longer coherence time, one effective channel can be represented as the product of a vector, termed the small-timescale effective channel, and the other effective channel. By integrating the estimated effective channel from the initial time block with observations from our piecewise beam training, we present an efficient method for estimating subsequent small-timescale effective channels. We theoretically verify the efficacy of the proposed RIS design and demonstrate, through simulations, that our channel estimation method outperforms existing methods in pilot overhead and computational complexity across various realistic channel models.

eess.SP↗

arXiv2024

Near-Field LoS/NLoS Channel Estimation for RIS-Aided MU-MIMO Systems: Piece-Wise Low-Rank Approximation Approach

We study the channel estimation problem for a reconfigurable intelligent surface (RIS)-assisted millimeter-wave (mmWave) multi-user multiple-input multiple-output (MU-MIMO) system. In particular, it is assumed that the channel between a RIS and a base station (BS) exhibits a near-field line-of-sight (LoS) channel, which is a dominant signal path in mmWave communication systems. Due to the high-rankness and non-sparsity of the RIS-BS channel matrix in our system, the state-of-the-art (SOTA) methods, which are constructed based on far-field or near-field non-LoS (NLoS) channel, cannot provide attractive estimation performances. We for the first time propose an efficient near-field LoS/NLoS channel estimation method for RIS-assisted MU-MIMO systems by means of a piece-wise low-rank approximation. Specifically, an effective channel (to be estimated) is partitioned into piece-wise effective channels containing low-rank structures and then, they are estimated via collaborative low-rank approximation. The proposed method is named PW-CLRA. Via simulations, we verify the effectiveness of the proposed PW-CLRA.

eess.SP↗

arXiv2024

Asymptotically Near-Optimal Hybrid Beamforming for mmWave IRS-Aided MIMO Systems

Hybrid beamforming is an emerging technology for massive multiple-input multiple-output (MIMO) systems due to the advantages of lower complexity, cost, and power consumption. Recently, intelligent reflection surface (IRS) has been proposed as the cost-effective technique for robust millimeter-wave (mmWave) MIMO systems. Thus, it is required to jointly optimize a reflection vector and hybrid beamforming matrices for IRS-aided mmWave MIMO systems. Due to the lack of RF chain in the IRS, it is unavailable to acquire the TX-IRS and IRS-RX channels separately. Instead, there are efficient methods to estimate the so-called effective (or cascaded) channel in literature. We for the first time derive the near-optimal solution of the aforementioned joint optimization only using the effective channel. Based on our theoretical analysis, we develop the practical reflection vector and hybrid beamforming matrices by projecting the asymptotic solution into the modulus constraint. Via simulations, it is demonstrated that the proposed construction can outperform the state-of-the-art (SOTA) method, where the latter even requires the knowledge of the TX-IRS and IRS-RX channels separately. Furthermore, our construction can provide robustness for channel estimation errors, which is inevitable for practical massive MIMO systems.

eess.SP↗

arXiv2024

Near-Field Channel Estimation for XL-RIS Assisted Multi-User XL-MIMO Systems: Hybrid Beamforming Architectures

Channel estimation is one of the key challenges for the deployment of extremely large-scale reconfigurable intelligent surface (XL-RIS) assisted multiple-input multiple-output (MIMO) systems. In this paper, we study the channel estimation problem for XL-RIS assisted multi-user XL-MIMO systems with hybrid beamforming structures. For this system, we propose an {\em unified} channel estimation method that yields a notable estimation accuracy in the near-field BS-RIS and near-field RIS-User channels (in short, near-near field channels), far-near field channels, and far-far field channels. Our key idea is that the effective (or cascaded) channels to be estimated can be each factorized as the product of low-rank matrices (i.e., the product of the common (or user-independent) matrix and the user-specific coefficient matrix). The common matrix whose columns are the basis of the column space of the BS-RIS channel matrix is efficiently estimated via a {\em collaborative} low-rank approximation (CLRA). Leveraging the hybrid beamforming structures, we develop an efficient iterative algorithm that jointly optimizes the user-specific coefficient matrices. Via experiments and complexity analysis, we verify the effectiveness of the proposed channel estimation method (named CLRA-JO) in the aforementioned three classes of wireless channels.

eess.SP↗

arXiv2023

On Practical Robust Reinforcement Learning: Practical Uncertainty Set and Double-Agent Algorithm

Robust reinforcement learning (RRL) aims at seeking a robust policy to optimize the worst case performance over an uncertainty set of Markov decision processes (MDPs). This set contains some perturbed MDPs from a nominal MDP (N-MDP) that generate samples for training, which reflects some potential mismatches between training (i.e., N-MDP) and true environments. In this paper we present an elaborated uncertainty set by excluding some implausible MDPs from the existing sets. Under this uncertainty set, we develop a sample-based RRL algorithm (named ARQ-Learning) for tabular setting and characterize its finite-time error bound. Also, it is proved that ARQ-Learning converges as fast as the standard Q-Learning and robust Q-Learning while ensuring better robustness. We introduce an additional pessimistic agent which can tackle the major bottleneck for the extension of ARQ-Learning into the cases with larger or continuous state spaces. Incorporating this idea into RL algorithms, we propose double-agent algorithms for model-free RRL. Via experiments, we demonstrate the effectiveness of the proposed algorithms.

cs.LG↗

arXiv2023

Tighter Regret Analysis and Optimization of Online Federated Learning

In federated learning (FL), it is commonly assumed that all data are placed at clients in the beginning of machine learning (ML) optimization (i.e., offline learning). However, in many real-world applications, it is expected to proceed in an online fashion. To this end, online FL (OFL) has been introduced, which aims at learning a sequence of global models from decentralized streaming data such that the so-called cumulative regret is minimized. Combining online gradient descent and model averaging, in this framework, FedOGD is constructed as the counterpart of FedSGD in FL. While it can enjoy an optimal sublinear regret, FedOGD suffers from heavy communication costs. In this paper, we present a communication-efficient method (named OFedIQ) by means of intermittent transmission (enabled by client subsampling and periodic transmission) and quantization. For the first time, we derive the regret bound that captures the impact of data-heterogeneity and the communication-efficient techniques. Through this, we efficiently optimize the parameters of OFedIQ such as sampling rate, transmission period, and quantization levels. Also, it is proved that the optimized OFedIQ can asymptotically achieve the performance of FedOGD while reducing the communication costs by 99%. Via experiments with real datasets, we demonstrate the effectiveness of the optimized OFedIQ.

cs.LG↗