arXiv subjects

Xiaodong Wang

Publications and source records attributed to Xiaodong Wang.

At least 73 records · Page 4Linked to original sources

arXiv2025

Scaling Generative Recommendations with Context Parallelism on Hierarchical Sequential Transducers

Large-scale recommendation systems are pivotal to process an immense volume of daily user interactions, requiring the effective modeling of high cardinality and heterogeneous features to ensure accurate predictions. In prior work, we introduced Hierarchical Sequential Transducers (HSTU), an attention-based architecture for modeling high cardinality, non-stationary streaming recommendation data, providing good scaling law in the generative recommender framework (GR). Recent studies and experiments demonstrate that attending to longer user history sequences yields significant metric improvements. However, scaling sequence length is activation-heavy, necessitating parallelism solutions to effectively shard activation memory. In transformer-based LLMs, context parallelism (CP) is a commonly used technique that distributes computation along the sequence-length dimension across multiple GPUs, effectively reducing memory usage from attention activations. In contrast, production ranking models typically utilize jagged input tensors to represent user interaction features, introducing unique CP implementation challenges. In this work, we introduce context parallelism with jagged tensor support for HSTU attention, establishing foundational capabilities for scaling up sequence dimensions. Our approach enables a 5.3x increase in supported user interaction sequence length, while achieving a 1.55x scaling factor when combined with Distributed Data Parallelism (DDP).

cs.IR↗

arXiv2025

Efficient Speculative Decoding for Llama at Scale: Challenges and Solutions

Speculative decoding is a standard method for accelerating the inference speed of large language models. However, scaling it for production environments poses several engineering challenges, including efficiently implementing different operations (e.g., tree attention and multi-round speculative decoding) on GPU. In this paper, we detail the training and inference optimization techniques that we have implemented to enable EAGLE-based speculative decoding at a production scale for Llama models. With these changes, we achieve a new state-of-the-art inference latency for Llama models. For example, Llama4 Maverick decodes at a speed of about 4 ms per token (with a batch size of one) on 8 NVIDIA H100 GPUs, which is 10% faster than the previously best known method. Furthermore, for EAGLE-based speculative decoding, our optimizations enable us to achieve a speed-up for large batch sizes between 1.4x and 2.0x at production scale.

cs.CL↗

arXiv2025

Align-for-Fusion: Harmonizing Triple Preferences via Dual-oriented Diffusion for Cross-domain Sequential Recommendation

Personalized sequential recommendation aims to predict appropriate items for users based on their behavioral sequences. To alleviate data sparsity and interest drift issues, conventional approaches typically incorporate auxiliary behaviors from other domains via cross-domain transition. However, existing cross-domain sequential recommendation (CDSR) methods often follow an align-then-fusion paradigm that performs representation-level alignment across multiple domains and combines them mechanically for recommendation, overlooking the fine-grained fusion of domain-specific preferences. Inspired by recent advances in diffusion models (DMs) for distribution matching, we propose an align-for-fusion framework for CDSR to harmonize triple preferences via dual-oriented DMs, termed HorizonRec. Specifically, we investigate the uncertainty injection of DMs and identify stochastic noise as a key source of instability in existing DM-based recommenders. To address this, we introduce a mixed-conditioned distribution retrieval strategy that leverages distributions retrieved from users' authentic behavioral logic as semantic bridges across domains, enabling consistent multi-domain preference modeling. Furthermore, we propose a dual-oriented preference diffusion method to suppress potential noise and emphasize target-relevant interests during multi-domain user representation fusion. Extensive experiments on four CDSR datasets from two distinct platforms demonstrate the effectiveness and robustness of HorizonRec in fine-grained triple-domain preference fusion.

cs.IR↗

arXiv2025

Stimulated Brillouin Amplification with Flying Focus

Material damage thresholds pose a fundamental limit to chirped pulse amplification (CPA) in high-power laser systems. Plasma-based amplification via stimulated Brillouin scattering (SBS) offers a damage-free alternative, yet its effectiveness has been hindered by instabilities that constrain interaction length. In this study, we report the first experimental demonstration of SBS amplification driven by a flying focus in a 3-mm plasma channel. The flying focus is generated using chromatic aberration from spherical lenses, with its velocity precisely measured by an interferometric ionization method achieving 6.6 fs timing resolution. At a focus velocity near -c, SBS amplification is realized at pump and seed intensities more than two orders of magnitude lower than in conventional setups, yielding a conversion efficiency of 14.5%. These results validate flying focus as a powerful tool for extending interaction lengths and enabling efficient plasma-based laser amplification at reduced intensities.

physics.plasm-ph↗

arXiv2025

FedVLA: Federated Vision-Language-Action Learning with Dual Gating Mixture-of-Experts for Robotic Manipulation

Vision-language-action (VLA) models have significantly advanced robotic manipulation by enabling robots to interpret language instructions for task execution. However, training these models often relies on large-scale user-specific data, raising concerns about privacy and security, which in turn limits their broader adoption. To address this, we propose FedVLA, the first federated VLA learning framework, enabling distributed model training that preserves data privacy without compromising performance. Our framework integrates task-aware representation learning, adaptive expert selection, and expert-driven federated aggregation, enabling efficient and privacy-preserving training of VLA models. Specifically, we introduce an Instruction Oriented Scene-Parsing mechanism, which decomposes and enhances object-level features based on task instructions, improving contextual understanding. To effectively learn diverse task patterns, we design a Dual Gating Mixture-of-Experts (DGMoE) mechanism, where not only input tokens but also self-aware experts adaptively decide their activation. Finally, we propose an Expert-Driven Aggregation strategy at the federated server, where model aggregation is guided by activated experts, ensuring effective cross-client knowledge transfer.Extensive simulations and real-world robotic experiments demonstrate the effectiveness of our proposals. Notably, DGMoE significantly improves computational efficiency compared to its vanilla counterpart, while FedVLA achieves task success rates comparable to centralized training, effectively preserving data privacy.

cs.RO↗

arXiv2025

Fast and Simplex: 2-Simplicial Attention in Triton

Recent work has shown that training loss scales as a power law with both model size and the number of tokens, and that achieving compute-optimal models requires scaling model size and token count together. However, these scaling laws assume an infinite supply of data and apply primarily in compute-bound settings. As modern large language models increasingly rely on massive internet-scale datasets, the assumption that they are compute-bound is becoming less valid. This shift highlights the need for architectures that prioritize token efficiency. In this work, we investigate the use of the 2-simplicial Transformer, an architecture that generalizes standard dot-product attention to trilinear functions through an efficient Triton kernel implementation. We demonstrate that the 2-simplicial Transformer achieves better token efficiency than standard Transformers: for a fixed token budget, similarly sized models outperform their dot-product counterparts on tasks involving mathematics, coding, reasoning, and logic. We quantify these gains by demonstrating that $2$-simplicial attention changes the exponent in the scaling laws for knowledge and reasoning tasks compared to dot product attention.

cs.LG↗

arXiv2025

Supporting renewable energy planning and operation with data-driven high-resolution ensemble weather forecast

The planning and operation of renewable energy, especially wind power, depend crucially on accurate, timely, and high-resolution weather information. Coarse-grid global numerical weather forecasts are typically downscaled to meet these requirements, introducing challenges of scale inconsistency, process representation error, computation cost, and entanglement of distinct uncertainty sources from chaoticity, model bias, and large-scale forcing. We address these challenges by learning the climatological distribution of a target wind farm using its high-resolution numerical weather simulations. An optimal combination of this learned high-resolution climatological prior with coarse-grid large scale forecasts yields highly accurate, fine-grained, full-variable, large ensemble of weather pattern forecasts. Using observed meteorological records and wind turbine power outputs as references, the proposed methodology verifies advantageously compared to existing numerical/statistical forecasting-downscaling pipelines, regarding either deterministic/probabilistic skills or economic gains. Moreover, a 100-member, 10-day forecast with spatial resolution of 1 km and output frequency of 15 min takes < 1 hour on a moderate-end GPU, as contrast to $\mathcal{O}(10^3)$ CPU hours for conventional numerical simulation. By drastically reducing computational costs while maintaining accuracy, our method paves the way for more efficient and reliable renewable energy planning and operation.

cs.LG↗

arXiv2025

A GENERIC-guided active learning SPH method for viscoelastic fluids using Gaussian process regression

When applying machine learning methods to learn viscoelastic constitutive relations, the polymer history dependence in viscoelastic fluids and the generalization ability of machine learning models are challenging. In this paper, guided by the general equation for nonequilibrium reversible-irreversible coupling (GENERIC) framework, a novel GENERIC-guided active learning smoothed particle hydrodynamics (${\rm{G^2ALSPH}}$) method is proposed to obtain effective constitutive relations for reliable simulations of viscoelastic flows. By utilizing the GENERIC framework, the target viscoelastic constitutive relation is reduced to a simple functional relation between the eigenvalues of the conformation tensor and the eigenvalues of its thermodynamically conjugated tensorial variable, which incorporates the flow-history-dependent memory effect. Based on data and Gaussian process regression (GPR), a new active learning strategy is developed to obtain the simplified constitutive relation, in which the generalization ability is ensured by actively acquiring more data points when needed. Moreover, a novel relative uncertainty is devised to establish an accuracy evaluation tool for the GPR prediction results, which reduces the number of required training data points while maintaining accuracy. Furthermore, the SPH method combined with the latest techniques serves as an effective macroscopic numerical method. Eventually, the Poiseuille flows and the flows around a periodic array of cylinders at different Weissenberg numbers are simulated to validate the effectiveness and accuracy of the ${\rm{G^2ALSPH}}$ method. The Oldroyd-B model is used as the ground truth constitutive relation to provide data for GPR, bringing analytical solutions for comparison. The excellent performance demonstrates that the ${\rm{G^2ALSPH}}$ method has promising applications in data-driven simulations of viscoelastic fluids.

physics.flu-dyn↗

arXiv2025

PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization

Large Language Models (LLMs) have demonstrated remarkable performance across a wide range of applications, e.g., medical question-answering, mathematical sciences, and code generation. However, they also exhibit inherent limitations, such as outdated knowledge and susceptibility to hallucinations. Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm to address these issues, but it also introduces new vulnerabilities. Recent efforts have focused on the security of RAG-based LLMs, yet existing attack methods face three critical challenges: (1) their effectiveness declines sharply when only a limited number of poisoned texts can be injected into the knowledge database, (2) they lack sufficient stealth, as the attacks are often detectable by anomaly detection systems, which compromises their effectiveness, and (3) they rely on heuristic approaches to generate poisoned texts, lacking formal optimization frameworks and theoretic guarantees, which limits their effectiveness and applicability. To address these issues, we propose coordinated Prompt-RAG attack (PR-attack), a novel optimization-driven attack that introduces a small number of poisoned texts into the knowledge database while embedding a backdoor trigger within the prompt. When activated, the trigger causes the LLM to generate pre-designed responses to targeted queries, while maintaining normal behavior in other contexts. This ensures both high effectiveness and stealth. We formulate the attack generation process as a bilevel optimization problem leveraging a principled optimization framework to develop optimal poisoned texts and triggers. Extensive experiments across diverse LLMs and datasets demonstrate the effectiveness of PR-Attack, achieving a high attack success rate even with a limited number of poisoned texts and significantly improved stealth compared to existing methods.

cs.CR↗

arXiv2025

LeanPO: Lean Preference Optimization for Likelihood Alignment in Video-LLMs

Most Video Large Language Models (Video-LLMs) adopt preference alignment techniques, e.g., DPO~\citep{rafailov2024dpo}, to optimize the reward margin between a winning response ($y_w$) and a losing response ($y_l$). However, the likelihood displacement observed in DPO indicates that both $\log π_θ(y_w\mid x)$ and $\log π_θ(y_l\mid x) $ often decrease during training, inadvertently boosting the probabilities of non-target responses. In this paper, we systematically revisit this phenomenon from LLMs to Video-LLMs, showing that it intensifies when dealing with the redundant complexity of video content. To alleviate the impact of this phenomenon, we propose \emph{Lean Preference Optimization} (LeanPO), a reference-free approach that reformulates the implicit reward as the average likelihood of the response with respect to the policy model. A key component of LeanPO is the reward-trustworthiness correlated self-generated preference data pipeline, which carefully infuses relevant prior knowledge into the model while continuously refining the preference data via self-reflection. This allows the policy model to obtain high-quality paired data and accurately estimate the newly defined reward, thus mitigating the unintended drop. In addition, we introduce a dynamic label smoothing strategy that mitigates the impact of noise in responses from diverse video content, preventing the model from overfitting to spurious details. Extensive experiments demonstrate that LeanPO significantly enhances the performance of state-of-the-art Video-LLMs, consistently boosting baselines of varying capacities with minimal additional training overhead. Moreover, LeanPO offers a simple yet effective solution for aligning Video-LLM preferences with human trustworthiness, paving the way toward the reliable and efficient Video-LLMs.

cs.CV↗

arXiv2025

LongDWM: Cross-Granularity Distillation for Building a Long-Term Driving World Model

Driving world models are used to simulate futures by video generation based on the condition of the current state and actions. However, current models often suffer serious error accumulations when predicting the long-term future, which limits the practical application. Recent studies utilize the Diffusion Transformer (DiT) as the backbone of driving world models to improve learning flexibility. However, these models are always trained on short video clips (high fps and short duration), and multiple roll-out generations struggle to produce consistent and reasonable long videos due to the training-inference gap. To this end, we propose several solutions to build a simple yet effective long-term driving world model. First, we hierarchically decouple world model learning into large motion learning and bidirectional continuous motion learning. Then, considering the continuity of driving scenes, we propose a simple distillation method where fine-grained video flows are self-supervised signals for coarse-grained flows. The distillation is designed to improve the coherence of infinite video generation. The coarse-grained and fine-grained modules are coordinated to generate long-term and temporally coherent videos. In the public benchmark NuScenes, compared with the state-of-the-art front-view model, our model improves FVD by $27\%$ and reduces inference time by $85\%$ for the video task of generating 110+ frames. More videos (including 90s duration) are available at https://Wang-Xiaodong1899.github.io/longdwm/.

cs.CV↗

arXiv2025

Proximal Algorithm Unrolling: Flexible and Efficient Reconstruction Networks for Single-Pixel Imaging

Deep-unrolling and plug-and-play (PnP) approaches have become the de-facto standard solvers for single-pixel imaging (SPI) inverse problem. PnP approaches, a class of iterative algorithms where regularization is implicitly performed by an off-the-shelf deep denoiser, are flexible for varying compression ratios (CRs) but are limited in reconstruction accuracy and speed. Conversely, unrolling approaches, a class of multi-stage neural networks where a truncated iterative optimization process is transformed into an end-to-end trainable network, typically achieve better accuracy with faster inference but require fine-tuning or even retraining when CR changes. In this paper, we address the challenge of integrating the strengths of both classes of solvers. To this end, we design an efficient deep image restorer (DIR) for the unrolling of HQS (half quadratic splitting) and ADMM (alternating direction method of multipliers). More importantly, a general proximal trajectory (PT) loss function is proposed to train HQS/ADMM-unrolling networks such that learned DIR approximates the proximal operator of an ideal explicit restoration regularizer. Extensive experiments demonstrate that, the resulting proximal unrolling networks can not only flexibly handle varying CRs with a single model like PnP algorithms, but also outperform previous CR-specific unrolling networks in both reconstruction accuracy and speed. Source codes and models are available at https://github.com/pwangcs/ProxUnroll.

eess.IV↗

arXiv2025

ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos

Real-world driving requires people to observe the current environment, anticipate the future, and make appropriate driving decisions. This requirement is aligned well with the capabilities of world models, which understand the environment and predict the future. However, recent world models in autonomous driving are built explicitly, where they could predict the future by controllable driving video generation. We argue that driving world models should have two additional abilities: action control and action prediction. Following this line, previous methods are limited because they predict the video requires given actions of the same length as the video and ignore the dynamical action laws. To address these issues, we propose ProphetDWM, a novel end-to-end driving world model that jointly predicts future videos and actions. Our world model has an action module to learn latent action from the present to the future period by giving the action sequence and observations. And a diffusion-model-based transition module to learn the state distribution. The model is jointly trained by learning latent actions given finite states and predicting action and video. The joint learning connects the action dynamics and states and enables long-term future prediction. We evaluate our method in video generation and action prediction tasks on the Nuscenes dataset. Compared to the state-of-the-art methods, our method achieves the best video consistency and best action prediction accuracy, while also enabling high-quality long-term video and action generation.

cs.CV↗

arXiv2025

Spatiotemporal plasma hologram

We present the first experimental realization of a four-dimensional (4D) plasma hologram capable of recording and reconstructing the full spatiotemporal information of intense laser pulses. The holographic encoding is achieved through the interference of a long object pulse and a counter-propagating short reference pulse, generating an ionized plasma grating that captures both spatial and temporal characteristics of the laser field. A first-order diffractive probe enables the retrieval of encoded information, successfully reconstructing the spatiotemporal profiles of Gaussian and Laguerre-Gaussian beams. The experiment demonstrates the ability to encode artificial information into the laser pulse via spectral modulation and retrieve it through plasma grating diffraction, high-lighting potential applications in ultraintense optical data processing. Key innovations include a single-shot, background-free method for direct far-field spatiotemporal measurement and the obser-vation of laser focus propagation dynamics in plasma. The plasma grating exhibits a stable lifetime of 30-40 ps and supports high repetition rates, suggesting usage for high-speed optical switches and plasmatic analog memory. These advancements establish plasma holography as a robust platform for ultrafast laser manipulation, with implications for secure optical communication, analog computing,and precision spatiotemporal control of high-intensity lasers.

physics.plasm-ph↗

arXiv2025

Tensor Generalized Approximate Message Passing

We propose a tensor generalized approximate message passing (TeG-AMP) algorithm for low-rank tensor inference, which can be used to solve tensor completion and decomposition problems. We derive TeG-AMP algorithm as an approximation of the sum-product belief propagation algorithm in high dimensions where the central limit theorem and Taylor series approximations are applicable. As TeG-AMP is developed based on a general TR decomposition model, it can be directly applied to many low-rank tensor types. Moreover, our TeG-AMP can be simplified based on the CP decomposition model and a tensor simplified AMP is proposed for low CP-rank tensor inference problems. Experimental results demonstrate that the proposed methods significantly improve recovery performances since it takes full advantage of tensor structures.

cs.LG↗

arXiv2025

Scalable manufacturing of polarization-insensitive metalenses with high-uniform focal arrays in the visible

Multi-foci metalenses with uniform focal arrays attract special attention as they enable a single incident beam to focus on the same focal plane and share the identical numerical apertures. In this work, we demonstrate the scalable manufacturing of polarization-insensitive metalenses with high-uniform focal arrays in the visible. To overcome the limitations inherent in conventional imprint resins characterized by a low refractive index, hybrid meta-atoms are formed using the imprint resin itself, and a thin layer of titanium dioxide (TiO2) is deposited on it via atomic layer deposition (ALD) to increase the effective refractive index, enabling full phase coverage. We propose the gradient-descent-optimization strategy for phase retrieval of multi-foci metalens, which renders our design free from cross-talk and makes it feasible to achieve uniform focal arrays with flexible geometries. Based on this inverse-design scheme, the capability of nanoimprinted metalenses are explored by directly producing various focal arrays, including complex geometries such as square, rhombic lattices, as well as intact and defective rings. We envision this work may pave the way for various fields, including optical trapping, materials science, and quantum optics.

physics.optics↗

arXiv2025

Covert Communication via Action-Dependent States

This paper studies covert communication over channels with ADSI when the state is available either non-causally or causally at the transmitter. Covert communication refers to reliable communication between a transmitter and a receiver while ensuring a low probability of detection by an adversary, which we refer to as `warden'. It is well known that in a point-to-point DMC, it is possible to communicate on the order of $\sqrt{N}$ bits reliably and covertly over $N$ channel uses while the transmitter and the receiver are required to share a secret key on the order of $\sqrt{N}$ bits. This paper studies achieving reliable and covert communication of positive rate, i.e., reliable and covert communication on the order of N bits in N channel uses, over a channel with ADSI while the transmitter has non-causal or causal access to the ADSI, and the transmitter and the receiver share a secret key of negligible rate. We derive achievable rates for both the non-causal and causal scenarios by using block-Markov encoding and secret key generation from the ADSI, which subsumes the best achievable rates for channels with random states. We also derive upper bounds, for both non-causal and causal scenarios, that meet our achievable rates for some special cases. As an application of our problem setup, we study covert communication over channels with rewrite options, which are closely related to recording covert information on memory, and show that a positive covert rate can be achieved in such channels. As a special case of our problem, we study the AWGN channels and provide lower and upper bounds on the covert capacity that meet when the transmitter and the receiver share a secret key of sufficient rate and when the warden's channel is noisier than the legitimate receiver channel. As another application of our problem setup, we show that cooperation can lead to a positive covert rate in Gaussian channels.

cs.IT↗

arXiv2025

A High-throughput and Secure Coded Blockchain for IoT

We propose a new coded blockchain scheme suitable for the Internet-of-Things (IoT) network. In contrast to existing works for coded blockchains, especially blockchain-of-things, the proposed scheme is more realistic, practical, and secure while achieving high throughput. This is accomplished by: 1) modeling the variety of transactions using a reward model, based on which an optimization problem is solved to select transactions that are more accessible and cheaper computational-wise to be processed together; 2) a transaction-based and lightweight consensus algorithm that emphasizes on using the minimum possible number of miners for processing the transactions; and 3) employing the raptor codes with linear-time encoding and decoding which results in requiring lower storage to maintain the blockchain and having a higher throughput. We provide detailed analysis and simulation results on the proposed scheme and compare it with the state-of-the-art coded IoT blockchain schemes including Polyshard and LCB, to show the advantages of our proposed scheme in terms of security, storage, decentralization, and throughput.

cs.DC↗