arXiv subjects

Wei Guo

Publications and source records attributed to Wei Guo.

At least 145 records · Page 8Linked to original sources

arXiv2025

Pangu Pro MoE: Mixture of Grouped Experts for Efficient Sparsity

The surgence of Mixture of Experts (MoE) in Large Language Models promises a small price of execution cost for a much larger model parameter count and learning capacity, because only a small fraction of parameters are activated for each input token. However, it is commonly observed that some experts are activated far more often than others, leading to system inefficiency when running the experts on different devices in parallel. Therefore, we introduce Mixture of Grouped Experts (MoGE), which groups the experts during selection and balances the expert workload better than MoE in nature. It constrains tokens to activate an equal number of experts within each predefined expert group. When a model execution is distributed on multiple devices, this architectural design ensures a balanced computational load across devices, significantly enhancing throughput, particularly for the inference phase. Further, we build Pangu Pro MoE on Ascend NPUs, a sparse model based on MoGE with 72 billion total parameters, 16 billion of which are activated for each token. The configuration of Pangu Pro MoE is optimized for Ascend 300I Duo and 800I A2 through extensive system simulation studies. Our experiments indicate that MoGE indeed leads to better expert load balancing and more efficient execution for both model training and inference on Ascend NPUs. The inference performance of Pangu Pro MoE achieves 1148 tokens/s per card and can be further improved to 1528 tokens/s per card by speculative acceleration, outperforming comparable 32B and 72B Dense models. Furthermore, we achieve an excellent cost-to-performance ratio for model inference on Ascend 300I Duo. Our studies show that Ascend NPUs are capable of training Pangu Pro MoE with massive parallelization to make it a leading model within the sub-100B total parameter class, outperforming prominent open-source models like GLM-Z1-32B and Qwen3-32B.

cs.CL↗

arXiv2025

DLF: Enhancing Explicit-Implicit Interaction via Dynamic Low-Order-Aware Fusion for CTR Prediction

Click-through rate (CTR) prediction is a critical task in online advertising and recommender systems, relying on effective modeling of feature interactions. Explicit interactions capture predefined relationships, such as inner products, but often suffer from data sparsity, while implicit interactions excel at learning complex patterns through non-linear transformations but lack inductive biases for efficient low-order modeling. Existing two-stream architectures integrate these paradigms but face challenges such as limited information sharing, gradient imbalance, and difficulty preserving low-order signals in sparse CTR data. We propose a novel framework, Dynamic Low-Order-Aware Fusion (DLF), which addresses these limitations through two key components: a Residual-Aware Low-Order Interaction Network (RLI) and a Network-Aware Attention Fusion Module (NAF). RLI explicitly preserves low-order signals while mitigating redundancy from residual connections, and NAF dynamically integrates explicit and implicit representations at each layer, enhancing information sharing and alleviating gradient imbalance. Together, these innovations balance low-order and high-order interactions, improving model expressiveness. Extensive experiments on public datasets demonstrate that DLF achieves state-of-the-art performance in CTR prediction, addressing key limitations of existing models. The implementation is publicly available at https://github.com/USTC-StarTeam/DLF.

cs.IR↗

arXiv2025

Fluid Antenna-Assisted MU-MIMO Systems with Decentralized Baseband Processing

The fluid antenna system (FAS) has emerged as a disruptive technology, offering unprecedented degrees of freedom (DoF) for wireless communication systems. However, optimizing fluid antenna (FA) positions entails significant computational costs, especially when the number of FAs is large. To address this challenge, we introduce a decentralized baseband processing (DBP) architecture to FAS, which partitions the FA array into clusters and enables parallel processing. Based on the DBP architecture, we formulate a weighted sum rate (WSR) maximization problem through joint beamforming and FA position design for FA-assisted multiuser multiple-input multiple-output (MU-MIMO) systems. To solve the WSR maximization problem, we propose a novel decentralized block coordinate ascent (BCA)-based algorithm that leverages matrix fractional programming (FP) and majorization-minimization (MM) methods. The proposed decentralized algorithm achieves low computational, communication, and storage costs, thus unleashing the potential of the DBP architecture. Simulation results show that our proposed algorithm under the DBP architecture reduces computational time by over 70% compared to centralized architectures with negligible WSR performance loss.

cs.IT↗

arXiv2025

Accurate and Fast Channel Estimation for Fluid Antenna Systems with Diffusion Models

Fluid antenna systems (FAS) offer enhanced spatial diversity for next-generation wireless systems. However, acquiring accurate channel state information (CSI) remains challenging due to the large number of reconfigurable ports and the limited availability of radio-frequency (RF) chains -- particularly in high-dimensional FAS scenarios. To address this challenge, we propose an efficient posterior sampling-based channel estimator that leverages a diffusion model (DM) with a simplified U-Net architecture to capture the spatial correlation structure of two-dimensional FAS channels. The DM is initially trained offline in an unsupervised way and then applied online as a learned implicit prior to reconstruct CSI from partial observations via posterior sampling through a denoising diffusion restoration model (DDRM). To accelerate the online inference, we introduce a skipped sampling strategy that updates only a subset of latent variables during the sampling process, thereby reducing the computational cost with minimal accuracy degradation. Simulation results demonstrate that the proposed approach achieves significantly higher estimation accuracy and over 20x speedup compared to state-of-the-art compressed sensing-based methods, highlighting its potential for practical deployment in high-dimensional FAS.

cs.IT↗

arXiv2025

Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs

Sparse large language models (LLMs) with Mixture of Experts (MoE) and close to a trillion parameters are dominating the realm of most capable language models. However, the massive model scale poses significant challenges for the underlying software and hardware systems. In this paper, we aim to uncover a recipe to harness such scale on Ascend NPUs. The key goals are better usage of the computing resources under the dynamic sparse model structures and materializing the expected performance gain on the actual hardware. To select model configurations suitable for Ascend NPUs without repeatedly running the expensive experiments, we leverage simulation to compare the trade-off of various model hyperparameters. This study led to Pangu Ultra MoE, a sparse LLM with 718 billion parameters, and we conducted experiments on the model to verify the simulation results. On the system side, we dig into Expert Parallelism to optimize the communication between NPU devices to reduce the synchronization overhead. We also optimize the memory efficiency within the devices to further reduce the parameter and activation management overhead. In the end, we achieve an MFU of 30.0% when training Pangu Ultra MoE, with performance comparable to that of DeepSeek R1, on 6K Ascend NPUs, and demonstrate that the Ascend system is capable of harnessing all the training stages of the state-of-the-art language models. Extensive experiments indicate that our recipe can lead to efficient training of large-scale sparse language models with MoE. We also study the behaviors of such models for future reference.

cs.CL↗

arXiv2025

WirelessAgent: Large Language Model Agents for Intelligent Wireless Networks

The rapid evolution of wireless networks presents unprecedented challenges in managing complex and dynamic systems. Existing methods are increasingly facing fundamental limitations in addressing these challenges. In this paper, we introduce WirelessAgent, a novel framework that harnesses large language models (LLMs) to create autonomous AI agents for diverse wireless network tasks. This framework integrates four core modules that mirror human cognitive processes: perception, memory, planning, and action. To implement it, we provide a basic usage based on agentic workflows and the LangGraph architecture. We demonstrate the effectiveness of WirelessAgent through a comprehensive case study on network slicing. The numerical results show that WirelessAgent achieves $44.4\%$ higher bandwidth utilization than the \emph{Prompt-based} method, while performing only $4.3\%$ below the \emph{Rule-based optimality}. Notably, WirelessAgent delivers near-optimal network throughput across diverse network scenarios. These underscore the framework's potential for intelligent and autonomous resource management in future wireless networks. The code is available at \url{https://github.com/jwentong/WirelessAgent_R1}.

eess.SP↗

arXiv2025

A Study on Group Decision Making Problem Based on Fuzzy Reasoning and Bayesian Networks

Aiming at the group decision - making problem with multi - objective attributes, this study proposes a group decision - making system that integrates fuzzy inference and Bayesian network. A fuzzy rule base is constructed by combining threshold values, membership functions, expert experience, and domain knowledge to address quantitative challenges such as scale differences and expert linguistic variables. A hierarchical Bayesian network is designed, featuring a directed acyclic graph with nodes selected by experts, and maximum likelihood estimation is used to dynamically optimize the conditional probability table, modeling the nonlinear correlations among multidimensional indices for posterior probability aggregation. In a comprehensive student evaluation case, this method is compared with the traditional weighted scoring approach. The results indicate that the proposed method demonstrates effectiveness in both rule criterion construction and ranking consistency, with a classification accuracy of 86.0% and an F1 value improvement of 53.4% over the traditional method. Additionally, computational experiments on real - world datasets across various group decision scenarios assess the method's performance and robustness, providing evidence of its reliability in diverse contexts.

cs.AI↗

arXiv2025

Killing Two Birds with One Stone: Unifying Retrieval and Ranking with a Single Generative Recommendation Model

In recommendation systems, the traditional multi-stage paradigm, which includes retrieval and ranking, often suffers from information loss between stages and diminishes performance. Recent advances in generative models, inspired by natural language processing, suggest the potential for unifying these stages to mitigate such loss. This paper presents the Unified Generative Recommendation Framework (UniGRF), a novel approach that integrates retrieval and ranking into a single generative model. By treating both stages as sequence generation tasks, UniGRF enables sufficient information sharing without additional computational costs, while remaining model-agnostic. To enhance inter-stage collaboration, UniGRF introduces a ranking-driven enhancer module that leverages the precision of the ranking stage to refine retrieval processes, creating an enhancement loop. Besides, a gradient-guided adaptive weighter is incorporated to dynamically balance the optimization of retrieval and ranking, ensuring synchronized performance improvements. Extensive experiments demonstrate that UniGRF significantly outperforms existing models on benchmark datasets, confirming its effectiveness in facilitating information transfer. Ablation studies and further experiments reveal that UniGRF not only promotes efficient collaboration between stages but also achieves synchronized optimization. UniGRF provides an effective, scalable, and compatible framework for generative recommendation systems.

cs.IR↗

arXiv2025

Background subtraction method is not only much simpler, but also as applicable as covariant counterterm method

As the criterion for the applicability of the background subtraction method, not only the finiteness condition of the resulting Hamiltonian but also the condition for the validity of the first law of black hole thermodynamics can be reduced to the form amenable to much simpler analysis at infinity by using the covariant phase space formalism. With this, we further establish that the background subtraction method is as applicable as the covariant counterterm method not only to Einstein's gravity, but also to its higher derivative corrections for black hole thermodynamics in both asymptotically flat and AdS spacetimes. In addition, our framework also provides us with the first derivation of the universal expression of the Gibbs free energy in terms of the Euclidean on-shell action beyond Einstein's gravity. Among others, our findings have a far reaching impact on the shift in methodology for the Euclidean approach to black hole thermodynamics, where the well justified background subtraction method by our wieldy criterion is supposed to be the favored choice compared to the covariant counterterm method.

hep-th↗

arXiv2025

Pangu Ultra: Pushing the Limits of Dense Large Language Models on Ascend NPUs

We present Pangu Ultra, a Large Language Model (LLM) with 135 billion parameters and dense Transformer modules trained on Ascend Neural Processing Units (NPUs). Although the field of LLM has been witnessing unprecedented advances in pushing the scale and capability of LLM in recent years, training such a large-scale model still involves significant optimization and system challenges. To stabilize the training process, we propose depth-scaled sandwich normalization, which effectively eliminates loss spikes during the training process of deep models. We pre-train our model on 13.2 trillion diverse and high-quality tokens and further enhance its reasoning capabilities during post-training. To perform such large-scale training efficiently, we utilize 8,192 Ascend NPUs with a series of system optimizations. Evaluations on multiple diverse benchmarks indicate that Pangu Ultra significantly advances the state-of-the-art capabilities of dense LLMs such as Llama 405B and Mistral Large 2, and even achieves competitive results with DeepSeek-R1, whose sparse model structure contains much more parameters. Our exploration demonstrates that Ascend NPUs are capable of efficiently and effectively training dense models with more than 100 billion parameters. Our model and system will be available for our commercial customers.

cs.CL↗

arXiv2025

Low Energy Backgrounds and Excess Noise in a Two-Channel Low-Threshold Calorimeter

We describe observations of low energy excess (LEE) events, background events observed in all light dark matter direct detection calorimeters, and noise in a Transition Edge Sensor based two-channel silicon athermal phonon detector with 375 meV baseline energy resolution. We measure two distinct LEE populations: ``shared'' multichannel events with a pulse shape consistent with substrate athermal phonon events, and sub-eV events that couple nearly exclusively to a single channel with a significantly faster pulse shape. These ``singles'' are consistent with events occurring within the aluminum athermal phonon collection fins. Similarly, our measured detector noise is higher than the theoretical expectation. Measured noise can be split into an uncorrelated component, consistent with shot noise from small energy depositions within the athermal phonon sensor itself, and a correlated component, consistent with shot noise from energy depositions within the silicon substrate's phonon system.

physics.ins-det↗

arXiv2025

Global Rice Multi-Class Segmentation Dataset (RiceSEG): A Comprehensive and Diverse High-Resolution RGB-Annotated Images for the Development and Benchmarking of Rice Segmentation Algorithms

Developing computer vision-based rice phenotyping techniques is crucial for precision field management and accelerating breeding, thereby continuously advancing rice production. Among phenotyping tasks, distinguishing image components is a key prerequisite for characterizing plant growth and development at the organ scale, enabling deeper insights into eco-physiological processes. However, due to the fine structure of rice organs and complex illumination within the canopy, this task remains highly challenging, underscoring the need for a high-quality training dataset. Such datasets are scarce, both due to a lack of large, representative collections of rice field images and the time-intensive nature of annotation. To address this gap, we established the first comprehensive multi-class rice semantic segmentation dataset, RiceSEG. We gathered nearly 50,000 high-resolution, ground-based images from five major rice-growing countries (China, Japan, India, the Philippines, and Tanzania), encompassing over 6,000 genotypes across all growth stages. From these original images, 3,078 representative samples were selected and annotated with six classes (background, green vegetation, senescent vegetation, panicle, weeds, and duckweed) to form the RiceSEG dataset. Notably, the sub-dataset from China spans all major genotypes and rice-growing environments from the northeast to the south. Both state-of-the-art convolutional neural networks and transformer-based semantic segmentation models were used as baselines. While these models perform reasonably well in segmenting background and green vegetation, they face difficulties during the reproductive stage, when canopy structures are more complex and multiple classes are involved. These findings highlight the importance of our dataset for developing specialized segmentation models for rice and other crops.

eess.IV↗

arXiv2025

Turbulent diffusion and dispersion in a superfluid

Single-body diffusion and two-body dispersion are fundamental processes in classical turbulence, governing particle mixing and transport. However, their behaviors in superfluid turbulence remain largely unexplored. In this study, we numerically investigate the diffusion and relative dispersion of quantized vortices and superfluid parcels in the 0 K limit in two distinct turbulence regimes: ultra-quantum turbulence, characterized by a randomized vortex tangle, and quasiclassical turbulence, in which locally polarized vortices create large-scale flows resembling classical turbulence. Our results reveal that while vortex segments exhibit similar superdiffusion behavior at short times in both regimes, superfluid parcels behave differently: following the same superdiffusion scaling in ultra-quantum turbulence but deviating significantly in quasiclassical turbulence. This contrast provides a key clue to the origin of short-time superdiffusion, a puzzle since its recent discovery. Additionally, we show that two-body dispersion of both vortex segments and superfluid parcels exhibits distinct scaling behaviors in ultra-quantum and quasiclassical turbulence, highlighting fundamental differences in these two turbulence regimes. Our findings bridge a critical gap in superfluid turbulence research, offering new insights into turbulent transport in inviscid quantum fluids.

cond-mat.quant-gas↗

arXiv2025

From Chaos to Order: The Atomic Reasoner Framework for Fine-grained Reasoning in Large Language Models

Recent advances in large language models (LLMs) have shown remarkable progress, yet their capacity for logical ``slow-thinking'' reasoning persists as a critical research frontier. Current inference scaling paradigms suffer from two fundamental constraints: fragmented thought flows compromising logical coherence, and intensively computational complexity that escalates with search space dimensions. To overcome these limitations, we present \textbf{Atomic Reasoner} (\textbf{AR}), a cognitive inference strategy that enables fine-grained reasoning through systematic atomic-level operations. AR decomposes the reasoning process into atomic cognitive units, employing a cognitive routing mechanism to dynamically construct reasoning representations and orchestrate inference pathways. This systematic methodology implements stepwise, structured cognition, which ensures logical coherence while significantly reducing cognitive load, effectively simulating the cognitive patterns observed in human deep thinking processes. Extensive experimental results demonstrate AR's superior reasoning capabilities without the computational burden of exhaustive solution searches, particularly excelling in linguistic logic puzzles. These findings substantiate AR's effectiveness in enhancing LLMs' capacity for robust, long-sequence logical reasoning and deliberation.

cs.CL↗

arXiv2025

Non-oscillatory entropy stable DG schemes for hyperbolic conservation law

In this paper, we propose a class of non-oscillatory, entropy-stable discontinuous Galerkin (NOES-DG) schemes for solving hyperbolic conservation laws. By incorporating a specific form of artificial viscosity, our new scheme directly controls entropy production and suppresses spurious oscillations. To address the stiffness introduced by the artificial terms, which can restrict severely time step sizes, we employ the integration factor strong stability-preserving Runge-Kutta method for time discretization. Furthermore, our method remains compatible with positivity-preserving limiters under suitable CFL conditions in extreme cases. Various numerical examples demonstrate the efficiency of the proposed scheme, showing that it maintains high-order accuracy in smooth regions and avoids spurious oscillations near discontinuities.

math.NA↗

arXiv2025

THz Beam Squint Mitigation via 3D Rotatable Antennas

Analog beamforming holds great potential for future terahertz (THz) communications due to its ability to generate high-gain directional beams with low-cost phase shifters.However, conventional analog beamforming may suffer substantial performance degradation in wideband systems due to the beam-squint effects. Instead of relying on high-cost true time delayers, we propose in this paper an efficient three-dimensional (3D) rotatable antenna technology to mitigate the beam-squint effects, motivated by the fact that beam squint disappears along the boresight direction. In particular, we focus on a wideband wide-beam coverage problem in this paper, aiming to maximize the minimum beamforming gain within a given angle and frequency range by jointly optimizing the analog beamforming vector and the 3D rotation angles of the antenna array. However, this problem is non-convex and difficult to be optimally solved due to the coupling of the spatial and frequency domains and that of the antenna weights and rotation. To tackle this issue, we first reformulate the problem into an equivalent form by merging the spatial and frequency domains into a single composite domain. Next, we combine alternating optimization (AO) and successive convex approximation (SCA) algorithms to optimize the analog beamforming and rotation angles within this composite domain. Simulation results demonstrate that the proposed scheme can significantly outperform conventional schemes without antenna rotation, thus offering a cost-effective solution for wideband transmission over THz bands.

eess.SP↗

arXiv2025

Structure-preserving nodal DG method for the Euler equations with gravity: well-balanced, entropy stable, and positivity preserving

We propose an entropy stable and positivity preserving discontinuous Galerkin (DG) scheme for the Euler equations with gravity, which is also well-balanced for hydrostatic equilibrium states. To achieve these properties, we utilize the nodal DG framework and carefully design the source term discretization using entropy conservative fluxes. Furthermore, we demonstrate that the proposed methodology is compatible with a positivity preserving scaling limiter, ensuring positivity of density and pressure under an appropriate CFL condition. To the best of our knowledge, this is the first DG scheme to simultaneously achieve these three properties with theoretical justification. Numerical examples further demonstrate its robustness and efficiency.

math.NA↗

arXiv2025

A Universal Framework for Compressing Embeddings in CTR Prediction

Accurate click-through rate (CTR) prediction is vital for online advertising and recommendation systems. Recent deep learning advancements have improved the ability to capture feature interactions and understand user interests. However, optimizing the embedding layer often remains overlooked. Embedding tables, which represent categorical and sequential features, can become excessively large, surpassing GPU memory limits and necessitating storage in CPU memory. This results in high memory consumption and increased latency due to frequent GPU-CPU data transfers. To tackle these challenges, we introduce a Model-agnostic Embedding Compression (MEC) framework that compresses embedding tables by quantizing pre-trained embeddings, without sacrificing recommendation quality. Our approach consists of two stages: first, we apply popularity-weighted regularization to balance code distribution between high- and low-frequency features. Then, we integrate a contrastive learning mechanism to ensure a uniform distribution of quantized codes, enhancing the distinctiveness of embeddings. Experiments on three datasets reveal that our method reduces memory usage by over 50x while maintaining or improving recommendation performance compared to existing models. The implementation code is accessible in our project repository https://github.com/USTC-StarTeam/MEC.

cs.IR↗