arXiv subjects

Yinbin Ma

Publications and source records attributed to Yinbin Ma.

15 recordsLinked to original sources

arXiv2026

ROCS: Request-Oriented Compute Sharing for Efficient Large-Scale Recommendation

Modern recommendation models gain prediction quality by scaling feature-interaction and sequence modules, but production cost constraints cap how far systems can scale. In this work, we propose Request-Oriented Compute Sharing (ROCS), a modeling and inference paradigm that exploits a unique property of recommendation inference: each user request is evaluated against many candidates, while request-side features are shared across candidates. ROCS defers request-candidate interactions as late as possible, isolates candidate-dependent representations, and evaluates substantial portions of the model once per request rather than once per candidate, significantly improving inference efficiency while maintaining or improving prediction quality. To realize this paradigm, we develop Generalized Layer Masking (GLM) to enforce candidate isolation in feature-interaction architectures, and Deep Cross Attention (DCA) to extend request-oriented sharing to sequence architectures. To support efficient GPU deployment, we co-design In-Kernel Broadcast Optimization (IKBO) that significantly accelerates ROCS model execution. Experiments on public benchmarks show that ROCS consistently improves the quality-efficiency tradeoff across recommendation backbones. On production-scale workloads, ROCS achieves up to a 3x QPS improvement on retrieval models without quality degradation and a 0.5% relative LogLoss improvement with a 50% QPS gain on a short-form video ranking model. ROCS has been deployed across large-scale recommendation systems spanning ads and organic surfaces, retrieval and ranking stages, and more than two orders of magnitude in inference complexity, delivering significant online gains at reduced infrastructure cost.

cs.LG↗

arXiv2026

LoKA: Low-precision Kernel Applications for Recommendation Models At Scale

Recent GPU generations deliver significantly higher FLOPs using lower-precision arithmetic, such as FP8. While successfully applied to large language models (LLMs), its adoption in large recommendation models (LRMs) has been limited. This is because LRMs are numerically sensitive, dominated by small matrix multiplications (GEMMs) followed by normalization, and trained in communication-intensive environments. Applying FP8 directly to LRMs often degrades model quality and prolongs training time. These challenges are inherent to LRM workloads and cannot be resolved merely by introducing better FP8 kernels. Instead, a system-model co-design approach is needed to successfully integrate FP8. We present LoKA (Low-precision Kernel Applications), a framework that makes FP8 practical for LRMs through three principles: profile under realistic distributions to know where low precision is safe, co-design model components with hardware to expand where it is safe, and orchestrate across kernel libraries to maximize the gains. Concretely, LoKA Probe is a statistically grounded, online benchmarking method that learns activation and weight statistics, and quantifies per-layer errors. This process pinpoints safe and unsafe, fast and slow sites for FP8 adoption. LoKA Mods is a set of reusable model adaptations that improve both numerical stability and execution efficiency with FP8. LoKA Dispatch is a runtime that leverages the statistical insights from LoKA Probe to select the fastest FP8 kernel that satisfies the accuracy requirements.

cs.LG↗

arXiv2026

On Coded Caching Systems with Decentralized Linear Coding Placement

cs.IT↗

arXiv2025

Computer-aided Characterization of Fundamental Limits of Coded Caching with Linear Coding

Inspired by prior work by Tian and by Cao and Xu, this paper presents an efficient computer-aided framework to characterize the fundamental limits of coded caching systems under the constraint of linear coding. The proposed framework considers non-Shannon-type inequalities which are valid for representable polymatroids (and hence for linear codes), and leverages symmetric structure and problem-specific constraints of coded caching to reduce the complexity of the linear program. The derived converse bounds are tighter compared to previous known analytic methods, and prove the optimality of some achievable memory-load tradeoff points under the constraint of linear coding placement and delivery. These results seem to indicate that small, structured demand subsets combined with minimal common information constructions may be sufficient to characterize optimal tradeoffs under linear coding.

cs.IT↗

arXiv2025

External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation

Ads recommendation is a prominent service of online advertising systems and has been actively studied. Recent studies indicate that scaling-up and advanced design of the recommendation model can bring significant performance improvement. However, with a larger model scale, such prior studies have a significantly increasing gap from industry as they often neglect two fundamental challenges in industrial-scale applications. First, training and inference budgets are restricted for the model to be served, exceeding which may incur latency and impair user experience. Second, large-volume data arrive in a streaming mode with data distributions dynamically shifting, as new users/ads join and existing users/ads leave the system. We propose the External Large Foundation Model (ExFM) framework to address the overlooked challenges. Specifically, we develop external distillation and a data augmentation system (DAS) to control the computational cost of training/inference while maintaining high performance. We design the teacher in a way like a foundation model (FM) that can serve multiple students as vertical models (VMs) to amortize its building cost. We propose Auxiliary Head and Student Adapter to mitigate the data distribution gap between FM and VMs caused by the streaming data issue. Comprehensive experiments on internal industrial-scale applications and public datasets demonstrate significant performance gain by ExFM.

cs.IR↗

arXiv2025

An Achievable Scheme for the K-user Linear Computation Broadcast Channel

This paper presents a new achievable scheme for the K-user Linear Computation Broadcast Channel (K-LCBC). A K-LCBC comprises data stored on a server and K users, each aiming to retrieve a desired linear function of the data by leveraging their prior locally available side information in the form of another linear function of the data. The proposed scheme is based on a subspace decomposition derived from representable polymatroid spaces. This decomposition enables the server to effectively design multicast messages that simultaneously benefit multiple users and allow users to eliminate interference using their available side information. This work extends existing results for the 3-LCBC by introducing a linear programming framework to optimize multicast opportunities across an arbitrary number of users. The proposed approach can be used to derive achievable scheme for the K-user coded caching problem with linear coded placement and scalar linear function retrieval, which was our original motivation to investigate the K-LCBC.

cs.IT↗

arXiv2024

A Novel Scheme for Coded Caching with Coded Placement in Small Memory Regime

This paper presents a novel achievable scheme for coded caching systems with $N$ files and $K$ users, specifically when $N \leq K$. This new scheme employs linear coding both during the placement phase - where cache contents are linear combinations of files from the library - and the delivery phase. The multi-step delivery phase enables users to decode the cached coded content and eliminate interference effectively. In the small memory regime, the proposed scheme outperforms existing methods, particularly when $K$ and $N$ values are similar, it maintains manageable sub-packetization levels, and operates over a finite field of size $3$ regardless of the system parameters.

cs.IT↗

arXiv2024

Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation

We study a mismatch between the deep learning recommendation models' flat architecture, common distributed training paradigm and hierarchical data center topology. To address the associated inefficiencies, we propose Disaggregated Multi-Tower (DMT), a modeling technique that consists of (1) Semantic-preserving Tower Transform (SPTT), a novel training paradigm that decomposes the monolithic global embedding lookup process into disjoint towers to exploit data center locality; (2) Tower Module (TM), a synergistic dense component attached to each tower to reduce model complexity and communication volume through hierarchical feature interaction; and (3) Tower Partitioner (TP), a feature partitioner to systematically create towers with meaningful feature interactions and load balanced assignments to preserve model quality and training throughput via learned embeddings. We show that DMT can achieve up to 1.9x speedup compared to the state-of-the-art baselines without losing accuracy across multiple generations of hardware at large data center scales.

cs.LG↗

arXiv2024

On Coded Caching Systems with Offline Users, with and without Demand Privacy against Colluding Users

Coded caching is a technique that leverages locally cached contents at the end users to reduce the network's peak-time communication load. Coded caching has been shown to achieve significant performance gains compared to uncoded schemes and is thus considered a promising technique to boost performance in future networks by effectively trading off bandwidth for storage. The original coded caching model introduced by Maddah-Ali and Niesen does not consider the case where some users involved in the placement phase, may be offline during the delivery phase. If so, the delivery may not start or it may be wasteful to perform the delivery with fictitious demands for the offline users. In addition, the active users may require their demand to be kept private. This paper formally defines a coded caching system where some users are offline, and investigates the optimal performance with and without demand privacy against colluding users. For this novel coded caching model with offline users, achievable and converse bounds are proposed. These bounds are shown to meet under certain conditions, and otherwise to be to within a constant multiplicative gap of one another. In addition, the proposed achievable schemes have lower subpacketization and lower load compared to baseline schemes (that trivially extend known schemes so as to accommodate for privacy) in some memory regimes.

cs.IT↗

arXiv2023

New optimal trade-off point for coded caching systems with limited cache size

This paper presents a new achievable scheme for coded caching systems with $\mathsf{N}$ files, $\mathsf{K}=\mathsf{N}$ users, and cache size $\mathsf{M}=1/(\mathsf{N}-1)$. The scheme employs linear coding during the cache placement phase, and a three-stage transmissions designed to eliminate interference in the delivery phase. The achievable load meets a known converse bound, which impose no constraint on the cache placement, and is thus optimal. This new result, together with known inner and outer bounds, shows optimality of linear coding placement for $\mathsf{M} \leq 1/(\mathsf{N}-1)$ when $\mathsf{K}=\mathsf{N}\geq 3$. Interestingly and surprisingly, the proposed scheme is relatively simple but requires operations on a finite field of size at least 3.

cs.IT↗

arXiv2023

Demand Privacy in Hotplug Caching Systems

Coded caching, introduced by Maddah-Ali and Niesen (MAN), is a model where a server broadcasts multicast packets to users with a local cache that is leveraged so as to reduce the peak network communication load. The original MAN model does not consider missing demands (i.e., some users may not request a file) or privacy issues (i.e., decoding the multicast packets may expose the users' demands). The former issue was captured by the hotplug model with offline users, where the server starts sending multicast packets after having received a certain number of file requests. The latter issue was addressed by devoting part of the cache to store privacy keys to help users decode their requested file while remaining completely ignorant about the demands of the remaining users. This paper investigates the problem of private demands against colluding users in the hotplug model with offline users. Two achievable schemes are proposed based on Maximum Distance Separable (MDS) codes. They achieve lower subpacketization, and lower load in the small memory regime compared to baseline schemes that trivially include demand privacy or offline users in known schemes.

cs.IT↗

arXiv2022

AutoShard: Automated Embedding Table Sharding for Recommender Systems

Embedding learning is an important technique in deep recommendation models to map categorical features to dense vectors. However, the embedding tables often demand an extremely large number of parameters, which become the storage and efficiency bottlenecks. Distributed training solutions have been adopted to partition the embedding tables into multiple devices. However, the embedding tables can easily lead to imbalances if not carefully partitioned. This is a significant design challenge of distributed systems named embedding table sharding, i.e., how we should partition the embedding tables to balance the costs across devices, which is a non-trivial task because 1) it is hard to efficiently and precisely measure the cost, and 2) the partition problem is known to be NP-hard. In this work, we introduce our novel practice in Meta, namely AutoShard, which uses a neural cost model to directly predict the multi-table costs and leverages deep reinforcement learning to solve the partition problem. Experimental results on an open-sourced large-scale synthetic dataset and Meta's production dataset demonstrate the superiority of AutoShard over the heuristics. Moreover, the learned policy of AutoShard can transfer to sharding tasks with various numbers of tables and different ratios of the unseen tables without any fine-tuning. Furthermore, AutoShard can efficiently shard hundreds of tables in seconds. The effectiveness, transferability, and efficiency of AutoShard make it desirable for production use. Our algorithms have been deployed in Meta production environment. A prototype is available at https://github.com/daochenzha/autoshard

cs.LG↗

arXiv2022

On Coded Caching Systems with Offline Users

Coded caching is a technique that leverages locally cached contents at the users to reduce the network's peak-time communication load. Coded caching achieves significant performance gains compared to uncoded caching schemes and is thus a promising technique to boost performance in future networks. In the original model introduced by Maddah-Ali and Niesen (MAN), a server stores multiple files and is connected to multiple cache-aided users through an error-free shared link; once the local caches have been filled and all users have sent their demand to the server, the server can start sending coded multicast messages to satisfy all users' demands. A practical limitation of the original MAN model is that it halts if the server does not receive all users' demands, which is the limiting case of asynchronous coded caching when the requests of some users arrive with infinite delay. In this paper we formally define a coded caching system where some users are offline. We propose achievable and converse bounds for this novel setting and show under which conditions they meet, thus providing an optimal solution, and when they are to within a constant multiplicative gap of two. Interestingly, when optimality can be be shown, the optimal load-memory tradeoff only depends on the number active users, and not on the total (active plus offline) number of users.

cs.IT↗

arXiv2021

Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models

Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms of time to solution over previous systems. We achieve this by (i) designing the ZionEX platform with dedicated scale-out network, provisioned with high bandwidth, optimal topology and efficient transport (ii) implementing an optimized PyTorch-based training stack supporting both model and data parallelism (iii) developing sharding algorithms capable of hierarchical partitioning of the embedding tables along row, column dimensions and load balancing them across multiple workers; (iv) adding high-performance core operators while retaining flexibility to support optimizers with fully deterministic updates (v) leveraging reduced precision communications, multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we develop and briefly comment on distributed data ingestion and other supporting services that are required for the robust and efficient end-to-end training in production environments.

cs.DC↗

arXiv2021

A General Coded Caching Scheme for Scalar Linear Function Retrieval

Coded caching aims to minimize the network's peak-time communication load by leveraging the information pre-stored in the local caches at the users. The original single file retrieval setting by Maddah-Ali and Niesen has been recently extended to general Scalar Linear Function Retrieval (SLFR) by Wan et al., who proposed a linear scheme that surprisingly achieves the same optimal load (under the constraint of uncoded cache placement) as in single file retrieval. This paper's goal is to characterize the conditions under which a general SLFR linear scheme is optimal and gain practical insights into why the specific choices made by Wan et al. work. This paper shows that the optimal decoding coefficients are necessarily the product of two terms, one only involving the encoding coefficients and the other only the demands. In addition, the relationships among the encoding coefficients are shown to be captured by the cycles of certain graphs. Thus, a general linear scheme for SLFR can be found by solving a spanning tree problem.

cs.IT↗