arXiv subjects

Shijia Wang

Publications and source records attributed to Shijia Wang.

16 recordsLinked to original sources

arXiv2026

Melo: A Production LLM-Powered Music Recommendation Agent

We describe Melo, an LLM-powered music recommendation agent deployed on NetEase Cloud Music. Melo is structured as a deterministic five-node state graph over heterogeneous tools, with a prompt- and state-machine-driven orchestration policy rather than a fine-tuned controller. At industrial scale, the bottleneck is not how smart the brain is but how the system detects and recovers from the mistakes that brain makes. Two production failure modes drove the design: entity hallucination, where the agent commits to interpretations unsupported by the live catalog or user-behavior index, and long-tail degradation, where over-constrained requests collapse to generic popular fallbacks. We address them with two complementary mechanisms. Inference-time entity grounding repurposes the production search index as a verification primitive that gates entity decisions before they propagate downstream. Reflective retry verbalizes failure reasons from a broken tool chain and feeds them into the next planning step, so the system can relax or revise constraints rather than fall back blindly. A one-month online A/B test across NetEase Cloud Music's playlist surfaces reports an over 2 pp lift in a primary playlist retention metric and a lift of over one minute in a core playlist engagement metric. Offline ablation isolates a 7.8 pp reduction in entity misidentification from the three-layer grounding stack on our evaluation set, and a triggered-session analysis on our evaluation set shows reflective retry firing on 5.8% of sessions with 59% process-level recovery. Our deployment experience suggests that progress on LLM-powered music recommendation at this scale depends as much on the named, ablatable runtime machinery that catches and corrects the brain's mistakes as on the brain itself: a hypothesis we offer for the community to test.

cs.IR↗

arXiv2026

Accelerating Bayesian Phylogenetic Inference via Delayed Acceptance Sequential Monte Carlo with Random Forest Surrogates

In Bayesian phylogenetics, our goal is to estimate the posterior distribution over phylogenetic trees. Markov chain Monte Carlo methods are widely used to approximate the phylogenetic posterior distributions. For large-scale sequence data, repeated evaluation of the likelihood function incurs a high computational cost. In this article, we propose a machine-learning algorithm with over 35 topological and branch-length features to predict the changes in the likelihood function caused by tree moves (\eg,~eSPR, stNNI) used in standard MCMC approaches. This algorithm is then used to design a delayed acceptance MCMC kernel, which utilized the predicted surrogate function for preliminary rejection, to accelerate tree space searches. Furthermore, we integrate our proposed MCMC kernel into the sequential Monte Carlo sampler framework. We validate the proposed delayed-acceptance sequential Monte Carlo approach (DA-SMC) on simulation and real data sets. Our delayed acceptance kernel can maintain robust estimation while reduces the number of likelihood evaluations significantly, yielding substantial computational time savings. We develop a Python package that is available at https://github.com/wentYu/DAphyloSMC.

stat.ME↗

arXiv2026

Climber-Pilot: A Non-Myopic Generative Recommendation Model Towards Better Instruction-Following

Generative retrieval has emerged as a promising paradigm in recommender systems, offering superior sequence modeling capabilities over traditional dual-tower architectures. However, in large-scale industrial scenarios, such models often suffer from inherent myopia: due to single-step inference and strict latency constraints, they tend to collapse diverse user intents into locally optimal predictions, failing to capture long-horizon and multi-item consumption patterns. Moreover, real-world retrieval systems must follow explicit retrieval instructions, such as category-level control and policy constraints. Incorporating such instruction-following behavior into generative retrieval remains challenging, as existing conditioning or post-hoc filtering approaches often compromise relevance or efficiency. In this work, we present Climber-Pilot, a unified generative retrieval framework to address both limitations. First, we introduce Time-Aware Multi-Item Prediction (TAMIP), a novel training paradigm designed to mitigate inherent myopia in generative retrieval. By distilling long-horizon, multi-item foresight into model parameters through time-aware masking, TAMIP alleviates locally optimal predictions while preserving efficient single-step inference. Second, to support flexible instruction-following retrieval, we propose Condition-Guided Sparse Attention (CGSA), which incorporates business constraints directly into the generative process via sparse attention, without introducing additional inference steps. Extensive offline experiments and online A/B testing at NetEase Cloud Music, one of the largest music streaming platforms, demonstrate that Climber-Pilot significantly outperforms state-of-the-art baselines, achieving a 4.24\% lift of the core business metric.

cs.IR↗

arXiv2026

A multifidelity approximate Bayesian computation with pre-filtering

Approximate Bayesian Computation (ABC) methods often require extensive simulations, resulting in high computational costs. This paper focuses on multifidelity simulation models and proposes a pre-filtering hierarchical importance sampling algorithm. Under mild assumptions, we theoretically prove that the proposed algorithm satisfies posterior concentration properties, characterize the error upper bound and the relationship between algorithmic efficiency and pre-filtering criteria. Additionally, we provide a practical strategy to assess the suitability of multifidelity models for the proposed method. Finally, we develop a multifidelity ABC sequential Monte Carlo with adaptive pre-filtering strategy. Numerical experiments are used to demonstrate the effectiveness of the proposed approach. We develop an R package that is available at https://github.com/caofff/MAPS

stat.CO↗

arXiv2026

Poisson Hyperplane Processes with Rectified Linear Units

Neural networks have shown state-of-the-art performances in various classification and regression tasks. Rectified linear units (ReLU) are often used as activation functions for the hidden layers in a neural network model. In this article, we establish the connection between the Poisson hyperplane processes (PHP) and two-layer ReLU neural networks. We show that the PHP with a Gaussian prior is an alternative probabilistic representation to a two-layer ReLU neural network. In addition, we show that a two-layer neural network constructed by PHP is scalable to large-scale problems via the decomposition propositions. Finally, we propose an annealed sequential Monte Carlo algorithm for Bayesian inference. Our numerical experiments demonstrate that our proposed method outperforms the classic two-layer ReLU neural network. The implementation of our proposed model is available at https://github.com/ShufeiGe/Pois_Relu.git.

cs.LG↗

arXiv2025

FLAME: A Serving System Optimized for Large-Scale Generative Recommendation with Efficiency

Generative recommendation (GR) models possess greater scaling power compared to traditional deep learning recommendation models (DLRMs), yet they also impose a tremendous increase in computational burden. Measured in FLOPs, a typical GR model's workload sits in $10^9 \sim 10^{11}$ range, roughly four orders of magnitude higher than traditional DLRMs. Delivering accurate results in a few tens of milliseconds while processing billions of such requests per day puts extreme demands on the performance of the online serving system. Therefore, for industry practitioners, the alluring gains of GR models are tempered by the formidable challenge of online deployment at scale in production services. In this work, we introduce a comprehensive solution of online serving system tailored For Large-scale GenerAtive RecoMmendation with Efficiency (FLAME). Specifically, we leveraging CPU-GPU heterogeneous hardware to decouple feature pre-processing and model computation. We encapsulated several memory optimization features as the Proximal Data Accelerator (PDA) module to make full use of limited bandwidth and storage resources, which achieves a 1.9x throughput gain and a 1.7x latency reduction. We implement the Fused Kernel Engine (FKE) module based on the functionality and interface of NVIDIA TensorRT to boost model computation, delivering a speedup ratio of 4.6x-6.1x, throughput gain ratio of 4.7x-6.3x one step further. In addition, we design the Dynamic Stream Orchestrator (DSO) module to coordinate concurrent requests, enhancing the system throughput performance with 1.3x improvement in throughput and 2.3x speed-up under non-uniform distribution of upstream candidates. Comprehensive evaluations demonstrate that our FLAME effectively supports large-scale online deployment of GR models and achieves remarkable improvements in system performance.

cs.DC↗

arXiv2025

Climber: Toward Efficient Scaling Laws for Large Recommendation Models

Transformer-based generative models have achieved remarkable success across domains with various scaling law manifestations. However, our extensive experiments reveal persistent challenges when applying Transformer to recommendation systems: (1) Transformer scaling is not ideal with increased computational resources, due to structural incompatibilities with recommendation-specific features such as multi-source data heterogeneity; (2) critical online inference latency constraints (tens of milliseconds) that intensify with longer user behavior sequences and growing computational demands. We propose Climber, an efficient recommendation framework comprising two synergistic components: the model architecture for efficient scaling and the co-designed acceleration techniques. Our proposed model adopts two core innovations: (1) multi-scale sequence extraction that achieves a time complexity reduction by a constant factor, enabling more efficient scaling with sequence length; (2) dynamic temperature modulation adapting attention distributions to the multi-scenario and multi-behavior patterns. Complemented by acceleration techniques, Climber achieves a 5.15$\times$ throughput gain without performance degradation by adopting a "single user, multiple item" batched processing and memory-efficient Key-Value caching. Comprehensive offline experiments on multiple datasets validate that Climber exhibits a more ideal scaling curve. To our knowledge, this is the first publicly documented framework where controlled model scaling drives continuous online metric growth (12.19\% overall lift) without prohibitive resource costs. Climber has been successfully deployed on Netease Cloud Music, one of China's largest music streaming platforms, serving tens of millions of users daily.

cs.IR↗

arXiv2025

Progressive Semantic Residual Quantization for Multimodal-Joint Interest Modeling in Music Recommendation

In music recommendation systems, multimodal interest learning is pivotal, which allows the model to capture nuanced preferences, including textual elements such as lyrics and various musical attributes such as different instruments and melodies. Recently, methods that incorporate multimodal content features through semantic IDs have achieved promising results. However, existing methods suffer from two critical limitations: 1) intra-modal semantic degradation, where residual-based quantization processes gradually decouple discrete IDs from original content semantics, leading to semantic drift; and 2) inter-modal modeling gaps, where traditional fusion strategies either overlook modal-specific details or fail to capture cross-modal correlations, hindering comprehensive user interest modeling. To address these challenges, we propose a novel multimodal recommendation framework with two stages. In the first stage, our Progressive Semantic Residual Quantization (PSRQ) method generates modal-specific and modal-joint semantic IDs by explicitly preserving the prefix semantic feature. In the second stage, to model multimodal interest of users, a Multi-Codebook Cross-Attention (MCCA) network is designed to enable the model to simultaneously capture modal-specific interests and perceive cross-modal correlations. Extensive experiments on multiple real-world datasets demonstrate that our framework outperforms state-of-the-art baselines. This framework has been deployed on one of China's largest music streaming platforms, and online A/B tests confirm significant improvements in commercial metrics, underscoring its practical value for industrial-scale recommendation systems.

cs.IR↗

arXiv2025

A near-exact linear mixed model for genome-wide association studies

Linear mixed models (LMM) are widely adopted in genome-wide association studies (GWAS) to account for population stratification and cryptic relatedness. However, the parameter estimation of LMMs imposes substantial computational burdens due to large-scale operations on genetic similarity matrices (GSM). We introduced the near-exact linear mixed model (NExt-LMM), a novel LMM framework that overcomes critical computational bottlenecks in GWAS through the following key innovations. Firstly, we exploit the inherent low-rank structure of the GSM iteratively with the Hierarchical Off-Diagonal Low-Rank (HODLR) format, which is much faster than traditional decomposition methods. Secondly, we leverage the HODLR-approximated GSM to dramatically accelerate the further maximum likelihood estimation with the shared heritability ratios. Moreover, we establish rigorous error bounds for the NExt-LMM estimator, proving that Kullback-Leibler divergence between the approximated and exact estimators can be arbitrarily small. Consequently, our proposed dual approach accelerates inference of LMMs while guaranteeing low approximation errors. We use numerical experiments to demonstrate that the NExt-LMM significantly improves inference efficiency compared to existing methods. We develop a Python package that is available at https://github.com/ZhibinPU/NExt-LMM.

stat.CO↗

arXiv2024

An adaptive approximate Bayesian computation MCMC with Global-Local proposals

In this paper, we address the challenge of Markov Chain Monte Carlo (MCMC) algorithms within the approximate Bayesian Computation (ABC) framework, which often get trapped in local optima due to their inherent local exploration mechanism. We propose a novel Global-Local ABC-MCMC algorithm that combines the ``exploration" capabilities of global proposals with the ``exploitation" finesse of local proposals. We integrate iterated importance resampling into the likelihood-free framework to establish an effective global proposal distribution. For high-dimensional parameter spaces, we optimize the efficiency of the local sampler by leveraging Langevin dynamics and common random numbers. Furthermore, we introduce two adaptive schemes to enhance the algorithmic performance. The first scheme divides the update target of the importance proposal into a sequence of intermediate target distributions that progressively approximate the ABC posterior, thereby gradually updating the importance proposal distribution during the iterations. The second adaptive scheme automatically selects the optimal mixture of global and local moves through sequential optimization, based on a relative version of the expected squared jumping distance (ESJD). We theoretically and numerically demonstrate that our method is able to improve sampling efficiency and achieve more reliable convergence for complex posteriors. We develop a software package that is available at https://github.com/caofff/GL-ABC-MCMC.

stat.CO↗

arXiv2024

Using early rejection Markov chain Monte Carlo and Gaussian processes to accelerate ABC methods

Approximate Bayesian computation (ABC) is a class of Bayesian inference algorithms that targets for problems with intractable or {unavailable} likelihood function. It uses synthetic data drawn from the simulation model to approximate the posterior distribution. However, ABC is computationally intensive for complex models in which simulating synthetic data is very expensive. In this article, we propose an early rejection Markov chain Monte Carlo (ejMCMC) sampler based on Gaussian processes to accelerate inference speed. We early reject samples in the first stage of the kernel using a discrepancy model, in which the discrepancy between the simulated and observed data is modeled by Gaussian process (GP). Hence, the synthetic data is generated only if the parameter space is worth exploring. We demonstrate from theory, simulation experiments, and real data analysis that the new algorithm significantly improves inference efficiency compared to existing early-rejection MCMC algorithms. In addition, we employ our proposed method within an ABC sequential Monte Carlo (SMC) sampler. In our numerical experiments, we use examples of ordinary differential equations, stochastic differential equations, and delay differential equations to demonstrate the effectiveness of the proposed algorithm. We develop an R package that is available at https://github.com/caofff/ejMCMC.

stat.CO↗

arXiv2022

Shape Modeling with Spline Partitions

Shape modelling (with methods that output shapes) is a new and important task in Bayesian nonparametrics and bioinformatics. In this work, we focus on Bayesian nonparametric methods for capturing shapes by partitioning a space using curves. In related work, the classical Mondrian process is used to partition spaces recursively with axis-aligned cuts, and is widely applied in multi-dimensional and relational data. The Mondrian process outputs hyper-rectangles. Recently, the random tessellation process was introduced as a generalization of the Mondrian process, partitioning a domain with non-axis aligned cuts in an arbitrary dimensional space, and outputting polytopes. Motivated by these processes, in this work, we propose a novel parallelized Bayesian nonparametric approach to partition a domain with curves, enabling complex data-shapes to be acquired. We apply our method to HIV-1-infected human macrophage image dataset, and also simulated datasets sets to illustrate our approach. We compare to support vector machines, random forests and state-of-the-art computer vision methods such as simple linear iterative clustering super pixel image segmentation. We develop an R package that is available at \url{https://github.com/ShufeiGe/Shape-Modeling-with-Spline-Partitions}.

stat.ML↗

arXiv2021

Adaptive semiparametric Bayesian differential equations via sequential Monte Carlo

Nonlinear differential equations (DEs) are used in a wide range of scientific problems to model complex dynamic systems. The differential equations often contain unknown parameters that are of scientific interest, which have to be estimated from noisy measurements of the dynamic system. Generally, there is no closed-form solution for nonlinear DEs, and the likelihood surface for the parameter of interest is multi-modal and very sensitive to different parameter values. We propose a Bayesian framework for nonlinear DE systems. A flexible nonparametric function is used to represent the dynamic process such that expensive numerical solvers can be avoided. A sequential Monte Carlo algorithm in the annealing framework is proposed to conduct Bayesian inference for parameters in DEs. In our numerical experiments, we use examples of ordinary differential equations and delay differential equations to demonstrate the effectiveness of the proposed algorithm. We developed an R package that is available at \url{https://github.com/shijiaw/smcDE}.

stat.CO↗

arXiv2020

Particle Gibbs Sampling for Bayesian Phylogenetic inference

The combinatorial sequential Monte Carlo (CSMC) has been demonstrated to be an efficient complementary method to the standard Markov chain Monte Carlo (MCMC) for Bayesian phylogenetic tree inference using biological sequences. It is appealing to combine the CSMC and MCMC in the framework of the particle Gibbs (PG) sampler to jointly estimate the phylogenetic trees and evolutionary parameters. However, the Markov chain of the particle Gibbs may mix poorly if the underlying SMC suffers from the path degeneracy issue. Some remedies, including the particle Gibbs with ancestor sampling and the interacting particle MCMC, have been proposed to improve the PG. But they either cannot be applied to or remain inefficient for the combinatorial tree space. We introduce a novel CSMC method by proposing a more efficient proposal distribution. It also can be combined into the particle Gibbs sampler framework to infer parameters in the evolutionary model. The new algorithm can be easily parallelized by allocating samples over different computing cores. We validate that the developed CSMC can sample trees more efficiently in various particle Gibbs samplers via numerical experiments. Our implementation is available at https://github.com/liangliangwangsfu/phyloPMCMC

stat.CO↗

arXiv2019

Random Tessellation Forests

Space partitioning methods such as random forests and the Mondrian process are powerful machine learning methods for multi-dimensional and relational data, and are based on recursively cutting a domain. The flexibility of these methods is often limited by the requirement that the cuts be axis aligned. The Ostomachion process and the self-consistent binary space partitioning-tree process were recently introduced as generalizations of the Mondrian process for space partitioning with non-axis aligned cuts in the two dimensional plane. Motivated by the need for a multi-dimensional partitioning tree with non-axis aligned cuts, we propose the Random Tessellation Process (RTP), a framework that includes the Mondrian process and the binary space partitioning-tree process as special cases. We derive a sequential Monte Carlo algorithm for inference, and provide random forest methods. Our process is self-consistent and can relax axis-aligned constraints, allowing complex inter-dimensional dependence to be captured. We present a simulation study, and analyse gene expression data of brain tissue, showing improved accuracies over other methods.

stat.ML↗

arXiv2019

An Annealed Sequential Monte Carlo Method for Bayesian Phylogenetics

We describe an "embarrassingly parallel" method for Bayesian phylogenetic inference, annealed Sequential Monte Carlo, based on recent advances in the Sequential Monte Carlo literature such as adaptive determination of annealing parameters. The algorithm provides an approximate posterior distribution over trees and evolutionary parameters as well as an unbiased estimator for the marginal likelihood. This unbiasedness property can be used for the purpose of testing the correctness of posterior simulation software. We evaluate the performance of phylogenetic annealed Sequential Monte Carlo by reviewing and comparing with other computational Bayesian phylogenetic methods, in particular, different marginal likelihood estimation methods. Unlike previous Sequential Monte Carlo methods in phylogenetics, our annealed method can utilize standard Markov chain Monte Carlo tree moves and hence benefit from the large inventory of such moves available in the literature. Consequently, the annealed Sequential Monte Carlo method should be relatively easy to incorporate into existing phylogenetic software packages based on Markov chain Monte Carlo algorithms. We illustrate our method using simulation studies and real data analysis.

q-bio.PE↗