Searcharxiv⌕ Search

arXiv subjects

Cong Shen

Publications and source records attributed to Cong Shen.

At least 19 recordsLinked to original sources

SWAN-Rx: Efficient and Bandwidth-Scalable Neural Reception for MIMO-OFDM

We present a Scalable Windowed Axial-attention Neural Receiver (SWAN-Rx) for multiple-input multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) systems. The receiver improves reception accuracy and reduces forward-pass computation while supporting frozen-weight reuse across bandwidth allocations and channel conditions. It combines a common resource-element representation of received observations and pilot context with alternating time-axis and local frequency-axis attention. Shared parameters and relative positional information permit soft detection across allocation lengths without inference-time parameter updates. At fixed subcarrier spacing, model configuration, and symbol count, bounded frequency neighborhoods give linear growth of forward-pass computation with bandwidth. Evaluations compatible with the 5G New Radio physical layer reuse weights trained on four physical resource blocks (PRBs) at up to 273 PRBs and on unseen channels. At the training bandwidth, SWAN-Rx achieves lower pre- and post-decoding bit error rates than the implemented three-step neural receiver (NRX) and DeepRx-MIMO baselines, with 16.9% and 39.7% fewer parameters and 56.3% and 35.7% fewer forward-pass operations, respectively. The same weights achieve lower pre-decoding error rates than these baselines throughout the evaluated bandwidth-extrapolation and 16-PRB channel-transfer tests, with an estimated 0.4-0.9 dB signal-to-noise ratio reduction relative to NRX at a pre-decoding error rate of $10^{-2}$ in the 16-PRB tests.

eess.SP↗

A Note on Kinematic Flow and Differential Equations for Two-Site One-Loop Graph in FRW Spacetime

In this work, we systematically study the differential systems governing loop-level wavefunction coefficients of conformally-coupled scalar field theory within a general power-law FRW cosmology. By utilizing the twisted cohomology, hyperplane arrangements, and IBP techniques, we derive the canonical differential equations for two-site one-loop bubble and tadpole systems, revealing distinct structural differences. We present new insights into the one-loop tadpole system, uncovering that its integral family can include multiple parent functions due to distinct pairs of relative hyperplanes associated with each function, unlike the single parent function appearing in the one-loop bubble case. Moreover, we demonstrate that the tadpole correlator selectively probes only a subset of the cohomology space, despite the hyperplane arrangement suggesting a higher-dimensional structure. Another novel contribution of this work is the extension of kinematic flow framework to the loop-level scenarios for the first time. Using a graphical approach based on family trees generated by marked tubing graphs, which encode singularity structures, we efficiently construct the differential equations and uncover the hierarchical relationships among the associated master integrals. Additionally, we provide a preliminary discussion on generalization to two-site higher-loop configurations. We propose a general decomposition formula for the canonical form of a two-site diagram with arbitrary loops, breaking it into unshifted and shifted components associated with the fundamental tree-level and bubble-like structures, and establish a block-wise decomposition rule for the matrix $A$ in the corresponding differential system. These advancements provide a unified framework for two-site loop-level correlators and lay the groundwork for future study of more complex multi-site loop systems.

hep-th↗

From Trees to All Loops: The Combinatorial Geometry of Two-Site Cosmological Wavefunctions

We prove, at arbitrary loop order and including arbitrary tadpole decorations, the shifted-tree decomposition of two-site cosmological wavefunction coefficients proposed in our previous work, and uncover the combinatorial and geometric structure underlying it. For conformally coupled scalars with polynomial interactions, this decomposition expresses each loop coefficient as a finite signed sum of tree coefficients evaluated at shifted energies. For bubble-like graphs, the alternating sum over a Boolean lattice can be written in terms of finite differences or maximal chains. We prove the maximal-chain identity algebraically by induction on loop order and geometrically by decomposing a cubical Laplace integral into simplices. The maximal chains correspond to permutohedron vertices, whose incident facets encode the chain-dependent divisors. For tadpole-decorated graphs, the finite-difference Leibniz rule gives an explicit expansion indexed by ordered partitions and local maximal chains, which glue into global maximal chains and organize the coefficient as a combinatorial "path integral". A natural ordering of the divisors yields a term-by-term match with graph tubings and completes the all-loop proof, while other orderings give alternative tubing representations of the same coefficient. These identities hold before spatial loop-momentum integration and extend to power-law FRW backgrounds by linearity of the twisted integral. Two-site loop coefficients can therefore be obtained directly from known tree coefficients without solving new loop-level differential systems.

hep-th↗

SpecMind: Enabling Spectrum Intelligence via Multi-Agent Hybrid Retrieval-Augmented Generation

The exponential growth of wireless devices is driving unprecedented spectrum demand, pushing spectrum management toward more fine-grained decisions across space, time, and device constraints. As a result, spectrum policymakers and engineers must process large volumes of data that come from diverse sources and take many different forms, such as text and tables. These data sources are often disaggregated and require significant time and effort to integrate, search, and interpret. Furthermore, most of this information is formatted for human understanding and is not readily accessible to automated systems. To address this challenge, we propose SpecMind, a novel Multi-Agent Retrieval-Augmented Generation (RAG) system for spectrum intelligence that performs reasoning over heterogeneous data sources. This system enables autonomous agents to coordinate specialized sub-agents that retrieve and synthesize knowledge across policy proceedings, legal regulations, and license databases. We develop SpecBench, a question and answer (Q&A) dataset based on real-world license records and policy proceedings, addressing the lack of evaluation resources for RAG systems in the spectrum domain. Experimental results demonstrate that SpecMind outperforms traditional, general-purpose RAG systems across spectrum-related tasks, achieving over 80% win rate against strong baselines. The agent-based design enables more accurate retrieval, better contextual reasoning, and improved task completion across diverse query types.

cs.AI↗

Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management

This paper develops a hybrid offline-online multi-agent reinforcement learning framework based on decision transformers. The policy is first pretrained offline via supervised sequence modeling of trajectories generated by existing policies, providing a safe and sample-efficient initialization. It is then fine-tuned online using a hybrid objective that incorporates critic-guided gradients, enabling performance improvements beyond the offline policy. To facilitate stable offline-to-online transfer and effective multi-agent coordination, the framework incorporates return-weighted sampling, a critic conditioned on neighbors' actions, and neighborhood-correlated exploration. The approach is fully distributed: both training and execution rely only on local observations and limited information exchange among neighboring agents. Evaluations with dynamic traffic arrivals in two settings: (i) joint scheduling and power allocation and (ii) coordinated beamforming, show that the proposed method achieves quality-of-service (QoS) performance comparable to centralized methods. Moreover, when pretrained on lower-quality datasets, online fine-tuning is also observed to surpass the initial offline policy. These results demonstrate a promising learning-based alternative for wireless resource management.

eess.SY↗

SheafIQ: Sheaf-Theoretic Information Quantification of Vector Fields on Geometric Graphs

Vector fields on graph structures naturally arise in diverse biological and engineered systems, where vector-valued states are defined on the nodes and evolve through the network interactions. Existing methods primarily characterize either the graph topology or individual signals, but generally do not quantify how local interactions among node-associated vectors are organized across the graph. To address this limitation, a sheaf-theoretic framework, termed SheafIQ, is proposed to represent neighboring vectors in a common edge-associated coordinate system, map local incompatibilities to a residual energy distribution, and quantify its global organization through entropy. Across proteins, functional brain networks, urban traffic systems, and power grids, SheafIQ consistently reveals complementary organizational information beyond conventional graph- and signal-based descriptors. More broadly, it establishes a unified information-theoretic framework for quantifying the organization of vector-valued states on geometric graphs, extending network analysis beyond graph topology alone.

cs.IT↗

On the Convergence Analysis of Muon

The majority of parameters in neural networks are naturally represented as matrices. However, most commonly used optimizers treat these matrix parameters as flattened vectors during optimization, potentially overlooking their inherent structural properties. Recently, an optimizer called Muon has been proposed, specifically designed to optimize matrix-structured parameters. Extensive empirical evidence shows that Muon can significantly outperform traditional optimizers when training neural networks. Nonetheless, the theoretical understanding of Muon's convergence behavior and the reasons behind its superior performance remain limited. In this work, we present a comprehensive convergence rate analysis of Muon and its comparison with Gradient Descent (GD). We characterize the conditions under which Muon can outperform GD. Our theoretical results reveal that Muon can benefit from the low-rank structure of Hessian matrices, a phenomenon widely observed in practical neural network training. Our experimental results support and corroborate the theoretical findings.

stat.ML↗

The Past Is Prologue: A Plug-in Controller for Selective Updates in Sequentially Evolving LLM Memory

Sequentially evolving LLM memory enables agents to reuse past experience, but existing systems usually deploy each locally generated memory update without checking whether it improves future behavior. As a result, updates that help the current task may overwrite useful knowledge, introduce over-specific rules, or bias the final memory toward recent examples. We propose Janus, a plug-in memory controller that decides whether to accept a candidate memory update or retain the previous memory. To make this decision efficient, Janus uses a Memory Momentum Trigger to identify suspicious deviations in the memory-update trajectory, and compares old and new memories on a compact hybrid evaluation set of coverage, boundary, and fresh tasks instead of replaying the full history. Janus is method-agnostic and wraps existing updaters without changing their update rules. Across six datasets, two backbone LLMs, and two memory updaters, Janus improves average accuracy by +2.7 to +4.6 points over the corresponding base updaters.

cs.AI↗

Channel-Wise Mixed-Precision Quantization for Large Language Models

Large Language Models (LLMs) have demonstrated remarkable success across a wide range of language tasks, but their deployment on edge devices remains challenging due to the substantial memory requirements imposed by their large parameter sizes. Weight-only quantization presents a promising solution to reduce the memory footprint of LLMs. However, existing approaches primarily focus on integer-bit quantization, limiting their adaptability to fractional-bit quantization tasks and preventing the full utilization of available storage space on devices. In this paper, we introduce Channel-Wise Mixed-Precision Quantization (CMPQ), a novel mixed-precision quantization method that allocates quantization precision in a channel-wise pattern based on activation distributions. By assigning different precision levels to different weight channels, CMPQ supports arbitrary average bit-widths in the low-bit regime (e.g., between 2 and 4 bits). CMPQ employs a non-uniform quantization strategy and incorporates two outlier extraction techniques that collaboratively preserve the critical information, thereby minimizing the quantization loss. Experiments on nine different LLMs demonstrate that CMPQ not only enhances performance in integer-bit quantization tasks but also achieves significant performance gains with a modest increase in memory usage by performing in a mixed-precision way. CMPQ represents an adaptive and effective approach to LLM quantization, offering substantial benefits across diverse device capabilities.

cs.CL↗

Efficient Multi-objective Prompt Optimization via Pure-exploration Bandits

Prompt engineering has become central to eliciting the capabilities of large language models (LLMs). At its core lies prompt selection -- efficiently identifying the most effective prompts. However, most prior investigations overlook a key challenge: the inherently multi-faceted nature of prompt performance, which cannot be captured by a single metric. To fill this gap, we study the multi-objective prompt selection problem under two practical settings: Pareto prompt set recovery and best feasible prompt identification. Casting the problem into the pure-exploration bandits framework, we adapt provably efficient algorithms from multi-objective bandits and further introduce a novel design for best feasible arm identification in structured bandits, with theoretical guarantees on the identification error in the linear case. Extensive experiments across multiple LLMs show that the bandit-based approaches yield significant improvements over baselines, establishing a principled and efficient framework for multi-objective prompt optimization.

cs.LG↗

Is One Score Enough? Rethinking the Evaluation of Sequentially Evolving LLM Memory

Memory plays a central role in enabling large language models (LLMs) to operate over sequential tasks by accumulating and reusing experience over time. However, existing evaluations of LLM memory mostly rely on aggregate metrics such as final hold-out accuracy or cumulative online performance, which can obscure critical failure modes such as forgetting and negative transfer. In this paper, we introduce SeqMem-Eval, a diagnostic evaluation framework for sequentially evolving LLM memory. Drawing inspiration from continual learning, it targets a test-time setting in which memory is external, prompt-mediated, and updated without modifying model parameters. Rather than focusing only on final performance, SeqMem-Eval evaluates how memory states evolve, generalize, consolidate experience, and retain useful information during sequential inference. Specifically, it measures online utility, hold-out generalization, backward transfer, and forgetting, providing a finer-grained view of memory quality. Through extensive experiments across diverse tasks and memory methods, we show that higher final or cumulative accuracy does not necessarily imply better memory quality: many methods exhibit strong performance gains while suffering from substantial forgetting or negative transfer. Moreover, different memory designs exhibit distinct trade-offs between adaptability and stability that remain invisible under standard evaluation metrics.

cs.LG↗

$f$-Divergence Regularized RLHF: Two Tales of Sampling and Unified Analyses

Reinforcement Learning from Human Feedback (RLHF) has become a cornerstone technique for post-training large language models. While most existing approaches rely on the reverse KL-regularization, recent empirical studies have begun exploring alternative divergences (e.g., forward KL, chi-squared) as regularizers in RLHF. However, a unified theoretical understanding of general $f$-divergence regularization remains under-explored. To fill this gap, this work develops a comprehensive theoretical framework for online RLHF with a general $f$-divergence regularized objective. Rather than treating each possible divergence function individually, we adopt a holistic perspective across the entire function class and propose two algorithms based on distinct sampling principles. The first extends the classical optimism principle with a carefully designed exploration bonus, while the second introduces a new method that exploits the sensitivity of the optimal policy to reward perturbations under $f$-divergence regularization. Theoretical analysis shows that $O(\log T)$ regret and $O(1/T)$ sub-optimality gap are achievable, establishing provable efficiency of both algorithms and, to the best of our knowledge, the first performance bounds for online RLHF under general $f$-divergence regularization.

cs.LG↗

MLorc: Momentum Low-rank Compression for Memory Efficient Large Language Model Adaptation

With increasing size of large language models (LLMs), full-parameter fine-tuning imposes substantial memory demands. To alleviate this, we propose a novel memory-efficient training paradigm called Momentum Low-rank compression (MLorc). The key idea of MLorc is to compress and reconstruct the momentum of matrix parameters during training to reduce memory consumption. Compared to LoRA, MLorc avoids enforcing a fixed-rank constraint on weight update matrices and thus enables full-parameter learning. Compared to GaLore, MLorc directly compress the momentum rather than gradients, thereby better preserving the training dynamics of full-parameter fine-tuning. We provide a theoretical guarantee for its convergence under mild assumptions. Empirically, MLorc consistently outperforms other memory-efficient training methods, matches or even exceeds the performance of full fine-tuning at small ranks (e.g., $r=4$), and generalizes well across different optimizers, all while not compromising time or memory efficiency.

cs.LG↗

A Survey of Scaling in Large Language Model Reasoning

The rapid advancements in large Language models (LLMs) have significantly enhanced their reasoning capabilities, driven by various strategies such as multi-agent collaboration. However, unlike the well-established performance improvements achieved through scaling data and model size, the scaling of reasoning in LLMs is more complex and can even negatively impact reasoning performance, introducing new challenges in model alignment and robustness. In this survey, we provide a comprehensive examination of scaling in LLM reasoning, categorizing it into multiple dimensions and analyzing how and to what extent different scaling strategies contribute to improving reasoning capabilities. We begin by exploring scaling in input size, which enables LLMs to process and utilize a more extensive context for improved reasoning. Next, we analyze scaling in reasoning steps that improve multi-step inference and logical consistency. We then examine scaling in reasoning rounds, where iterative interactions refine reasoning outcomes. Furthermore, we discuss scaling in training-enabled reasoning, focusing on optimization through iterative model improvement. Finally, we outline future directions for further advancing LLM reasoning. By synthesizing these diverse perspectives, this survey aims to provide insights into how scaling strategies fundamentally enhance the reasoning capabilities of LLMs and further guide the development of next-generation AI systems.

cs.AI↗

Verification of Machine Unlearning is Fragile

As privacy concerns escalate in the realm of machine learning, data owners now have the option to utilize machine unlearning to remove their data from machine learning models, following recent legislation. To enhance transparency in machine unlearning and avoid potential dishonesty by model providers, various verification strategies have been proposed. These strategies enable data owners to ascertain whether their target data has been effectively unlearned from the model. However, our understanding of the safety issues of machine unlearning verification remains nascent. In this paper, we explore the novel research question of whether model providers can circumvent verification strategies while retaining the information of data supposedly unlearned. Our investigation leads to a pessimistic answer: \textit{the verification of machine unlearning is fragile}. Specifically, we categorize the current verification strategies regarding potential dishonesty among model providers into two types. Subsequently, we introduce two novel adversarial unlearning processes capable of circumventing both types. We validate the efficacy of our methods through theoretical analysis and empirical experiments using real-world datasets. This study highlights the vulnerabilities and limitations in machine unlearning verification, paving the way for further research into the safety of machine unlearning.

cs.LG↗

HeteroFedSyn: Differentially Private Tabular Data Synthesis for Heterogeneous Federated Settings

Traditional Differential Privacy (DP) mechanisms are typically tailored to specific analysis tasks, which limits the reusability of protected data. DP tabular data synthesis overcomes this by generating synthetic datasets that can be shared for arbitrary downstream tasks. However, existing synthesis methods predominantly assume centralized or local settings and overlook the more practical horizontal federated scenario. Naively synthesizing data locally or perturbing individual records either produces biased mixtures or introduces excessive noise, especially under heterogeneous data distributions across participants. We propose HeteroFedSyn, the first DP tabular data synthesis framework designed specifically for the horizontal federated setting. Built upon the PrivSyn paradigm of 2-way marginal-based synthesis, HeteroFedSyn introduces three key innovations for distributed marginal selection: (i) an L2-based dependency metric with random projection for noise-efficient correlation measurement, (ii) an unbiased estimator to correct multiplicative noise, and (iii) an adaptive selection strategy that dynamically updates dependency scores to avoid redundancy. Extensive experiments on range queries, Wasserstein fidelity, and machine learning tasks show that, despite the increased noise inherent to federated execution, HeteroFedSyn achieves utility comparable to centralized synthesis. Our code is open-sourced via the link.

cs.CR↗

Following the Diagnostic Trace: Visual Cognition-guided Cooperative Network for Chest X-Ray Diagnosis

Computer-aided diagnosis (CAD) has significantly advanced automated chest X-ray diagnosis but remains isolated from clinical workflows and lacks reliable decision support and interpretability. Human-AI collaboration seeks to enhance the reliability of diagnostic models by integrating the behaviors of controllable radiologists. However, the absence of interactive tools seamlessly embedded within diagnostic routines impedes collaboration, while the semantic gap between radiologists' decision-making patterns and model representations further limits clinical adoption. To overcome these limitations, we propose a visual cognition-guided collaborative network (VCC-Net) to achieve the cooperative diagnostic paradigm. VCC-Net centers on visual cognition (VC) and employs clinically compatible interfaces, such as eye-tracking or the mouse, to capture radiologists' visual search traces and attention patterns during diagnosis. VCC-Net employs VC as a spatial cognition guide, learning hierarchical visual search strategies to localize diagnostically key regions. A cognition-graph co-editing module subsequently integrates radiologist VC with model inference to construct a disease-aware graph. The module captures dependencies among anatomical regions and aligns model representations with VC-driven features, mitigating radiologist bias and facilitating complementary, transparent decision-making. Experiments on the public datasets SIIM-ACR, EGD-CXR, and self-constructed TB-Mouse dataset achieved classification accuracies of 88.40%, 85.05%, and 92.41%, respectively. The attention maps produced by VCC-Net exhibit strong concordance with radiologists' gaze distributions, demonstrating a mutual reinforcement of radiologist and model inference. The code is available at https://github.com/IPMI-NWU/VCC-Net.

cs.CV↗

A Single-Loop First-Order Algorithm for Linearly Constrained Bilevel Optimization

We study bilevel optimization problems where the lower-level problems are strongly convex and have coupled linear constraints. To overcome the potential non-smoothness of the hyper-objective and the computational challenges associated with the Hessian matrix, we utilize penalty and augmented Lagrangian methods to reformulate the original problem as a single-level one. Especially, we establish a strong theoretical connection between the reformulated function and the original hyper-objective by characterizing the closeness of their values and derivatives. Based on this reformulation, we propose a single-loop, first-order algorithm for linearly constrained bilevel optimization (SFLCB). We provide rigorous analyses of its non-asymptotic convergence rates, showing an improvement over prior double-loop algorithms -- form $O(ε^{-3}\log(ε^{-1}))$ to $O(ε^{-3})$. The experiments corroborate our theoretical findings and demonstrate the practical efficiency of the proposed SFLCB algorithm. Simulation code is provided at https://github.com/ShenGroup/SFLCB.

math.OC↗