arXiv subjects

Yibo Jin

Publications and source records attributed to Yibo Jin.

9 recordsLinked to original sources

arXiv2026

TurboGR: An Accelerated Training System for Large-Scale Generative Recommendation

Generative recommendation (GR) has emerged as a promising paradigm that replaces fragmented, scenario-specific architectures with unified Transformer-based models, exhibiting scaling-law behavior where recommendation quality improves systematically with increased model capacity and training data. However, deploying GR at scale on Ascend NPUs faces fundamental system-level challenges. These challenges are further exacerbated on Ascend NPUs due to the absence of high-performance implementations for jagged operators and the architectural mismatch between irregular sparse primitives and NPU's dense-computation-optimized design. In this paper, we present \model, an Ascend-affinity training system for generative recommendation that systematically addresses these bottlenecks through three core innovations: (i) Ascend-affinity jagged acceleration, including fusion operators that eliminate padding redundancy and dynamic load balancing that reduces inter-device imbalance from 47\% to 2.4\%; (ii) distributed communication optimization, comprising hierarchical sparse parallelism, semi-asynchronous training with proven convergence guarantees, and fine-grained pipeline orchestration that sustains 94\% NPU utilization; and (iii) negative sampling optimization via asynchronous offloading, jaggedness-aware FP16 quantization, and intra-batch logit sharing that expand the effective negative space without additional embedding lookups. Evaluated on the KuaiRand-27K dataset, \model supports training at up to 0.2B parameters and achieves 54.71\% MFU with near-linear scalability (0.97).

cs.DC↗

arXiv2026

RelayGR: Scaling Long-Sequence Generative Recommendation via Cross-Stage Relay-Race Inference

Real-time recommender systems execute multi-stage cascades (retrieval, pre-processing, fine-grained ranking) under strict tail-latency SLOs, leaving only tens of milliseconds for ranking. Generative recommendation (GR) models can improve quality by consuming long user-behavior sequences, but in production their online sequence length is tightly capped by the ranking-stage P99 budget. We observe that the majority of GR tokens encode user behaviors that are independent of the item candidates, suggesting an opportunity to pre-infer a user-behavior prefix once and reuse it during ranking rather than recomputing it on the critical path. Realizing this idea at industrial scale is non-trivial: the prefix cache must survive across multiple pipeline stages before the final ranking instance is determined, the user population implies cache footprints far beyond a single device, and indiscriminate pre-inference would overload shared resources under high QPS. We present RelayGR, a production system that enables in-HBM relay-race inference for GR. RelayGR selectively pre-infers long-term user prefixes, keeps their KV caches resident in HBM over the request lifecycle, and ensures the subsequent ranking can consume them without remote fetches. RelayGR combines three techniques: 1) a sequence-aware trigger that admits only at-risk requests under a bounded cache footprint and pre-inference load, 2) an affinity-aware router that co-locates cache production and consumption by routing both the auxiliary pre-infer signal and the ranking request to the same instance, and 3) a memory-aware expander that uses server-local DRAM to capture short-term cross-request reuse while avoiding redundant reloads. We implement RelayGR on Huawei Ascend NPUs and evaluate it with real queries. Under a fixed P99 SLO, RelayGR supports up to 1.5$\times$ longer sequences and improves SLO-compliant throughput by up to 3.6$\times$.

cs.DC↗

arXiv2025

P/D-Device: Disaggregated Large Language Model between Cloud and Devices

Serving disaggregated large language models has been widely adopted in industrial practice for enhanced performance. However, too many tokens generated in decoding phase, i.e., occupying the resources for a long time, essentially hamper the cloud from achieving a higher throughput. Meanwhile, due to limited on-device resources, the time to first token (TTFT), i.e., the latency of prefill phase, increases dramatically with the growth on prompt length. In order to concur with such a bottleneck on resources, i.e., long occupation in cloud and limited on-device computing capacity, we propose to separate large language model between cloud and devices. That is, the cloud helps a portion of the content for each device, only in its prefill phase. Specifically, after receiving the first token from the cloud, decoupling with its own prefill, the device responds to the user immediately for a lower TTFT. Then, the following tokens from cloud are presented via a speed controller for smoothed TPOT (the time per output token), until the device catches up with the progress. On-device prefill is then amortized using received tokens while the resource usage in cloud is controlled. Moreover, during cloud prefill, the prompt can be refined, using those intermediate data already generated, to further speed up on-device inference. We implement such a scheme P/D-Device, and confirm its superiority over other alternatives. We further propose an algorithm to decide the best settings. Real-trace experiments show that TTFT decreases at least 60%, maximum TPOT is about tens of milliseconds, and cloud throughput increases by up to 15x.

cs.DC↗

arXiv2025

Subspecialty-Specific Foundation Model for Intelligent Gastrointestinal Pathology

Gastrointestinal (GI) diseases represent a clinically significant burden, necessitating precise diagnostic approaches to optimize patient outcomes. Conventional histopathological diagnosis suffers from limited reproducibility and diagnostic variability. To overcome these limitations, we develop Digepath, a specialized foundation model for GI pathology. Our framework introduces a dual-phase iterative optimization strategy combining pretraining with fine-screening, specifically designed to address the detection of sparsely distributed lesion areas in whole-slide images. Digepath is pretrained on over 353 million multi-scale images from 210,043 H&E-stained slides of GI diseases. It attains state-of-the-art performance on 33 out of 34 tasks related to GI pathology, including pathological diagnosis, protein expression status prediction, gene mutation prediction, and prognosis evaluation. We further translate the intelligent screening module for early GI cancer and achieve near-perfect 99.70% sensitivity across nine independent medical institutions. This work not only advances AI-driven precision pathology for GI diseases but also bridge critical gaps in histopathological practice.

eess.IV↗

arXiv2024

P/D-Serve: Serving Disaggregated Large Language Model at Scale

Serving disaggregated large language models (LLMs) over tens of thousands of xPU devices (GPUs or NPUs) with reliable performance faces multiple challenges. 1) Ignoring the diversity (various prefixes and tidal requests), treating all the prompts in a mixed pool is inadequate. To facilitate the similarity per scenario and minimize the inner mismatch on P/D (prefill and decoding) processing, fine-grained organization is required, dynamically adjusting P/D ratios for better performance. 2) Due to inaccurate estimation on workload (queue status or maintained connections), the global scheduler easily incurs unnecessary timeouts in prefill. 3) Block-fixed device-to-device (D2D) KVCache transfer over cluster-level RDMA (remote direct memory access) fails to achieve desired D2D utilization as expected. To overcome previous problems, this paper proposes an end-to-end system P/D-Serve, complying with the paradigm of MLOps (machine learning operations), which models end-to-end (E2E) P/D performance and enables: 1) fine-grained P/D organization, mapping the service with RoCE (RDMA over converged ethernet) as needed, to facilitate similar processing and dynamic adjustments on P/D ratios; 2) on-demand forwarding upon rejections for idle prefill, decoupling the scheduler from regular inaccurate reports and local queues, to avoid timeouts in prefill; and 3) efficient KVCache transfer via optimized D2D access. P/D-Serve is implemented upon Ascend and MindSpore, has been deployed over tens of thousands of NPUs for more than eight months in commercial use, and further achieves 60\%, 42\% and 46\% improvements on E2E throughput, time-to-first-token (TTFT) SLO (service level objective) and D2D transfer time. As the E2E system with optimizations, P/D-Serve achieves 6.7x increase on throughput, compared with aggregated LLMs.

cs.DC↗

arXiv2015

Spin-valve Effect in NiFe/MoS2/NiFe Junctions

Two-dimensional (2D) layered transition metal dichalcogenides (TMDs) have been recently proposed as appealing candidate materials for spintronic applications owing to their distinctive atomic crystal structure and exotic physical properties arising from the large bonding anisotropy. Here we introduce the first MoS2-based spin-valves that employ monolayer MoS2 as the nonmagnetic spacer. In contrast with what expected from the semiconducting band-structure of MoS2, the vertically sandwiched-MoS2 layers exhibit metallic behavior. This originates from their strong hybridization with the Ni and Fe atoms of the Permalloy (Py) electrode. The spin-valve effect is observed up to 240 K, with the highest magnetoresistance (MR) up to 0.73% at low temperatures. The experimental work is accompanied by the first principle electron transport calculations, which reveal an MR of ~ 9% for an ideal Py/MoS2/Py junction. Our results clearly identify TMDs as a promising spacer compound in magnetic tunnel junctions and may open a new avenue for the TMDs-based spintronic applications.

cond-mat.mtrl-sci↗

arXiv2015

Controllable Schottky Barriers between MoS2 and Permalloy

MoS2 is a layered two-dimensional material with strong spin-orbit coupling and long spin lifetime, which is promising for electronic and spintronic applications. However, because of its large band gap and small electron affinity, a considerable Schottky barrier exists between MoS2 and contact metal, hindering the further study of spin transport and spin injection in MoS2. Although substantial progress has been made in improving device performance, the existence of metal-semiconductor Schottky barrier has not yet been fully understood. Here, we investigate permalloy (Py) contacts to both multilayer and monolayer MoS2. Ohmic contact is developed between multilayer MoS2 and Py electrodes with a negative Schottky barrier, which yields a high field-effect mobility exceeding 55 cm2V-1s-1 at low temperature. Further, by applying back gate voltage and inserting different thickness of Al2O3 layer between the metal and monolayer MoS2, we have achieved a good tunability of the Schottky barrier height (down to zero). These results are important in improving the performance of MoS2 transistor devices; and it may pave the way to realize spin transport and spin injection in MoS2.

cond-mat.mtrl-sci↗

arXiv2015

ReS2-based field-effect transistors and photodetectors

Atomically-thin two-dimensional (2D) layered transition metal dichalcogenides (TMDs) have been extensively studied in recent years because of their appealing electrical and optical properties. Here, we report on the fabrication of ReS2 field-effect transistors via the encapsulation of ReS2 nanosheets in a high-\k{appa} Al2O3 dielectric environment. Low-temperature transport measurements allowed us to observe a direct metal-to-insulator transition originating from strong electron-electron interactions. Remarkably, the photodetectors based on ReS2 exhibit gate-tunable photoresponsivity up to 16.14 A/W and external quantum efficiency reaching 3,168 %, showing a competitive device performance to those reported in graphene, MoSe2, GaS and GaSe-based photodetectors. Our study unambiguously distinguishes ReS2 as a new candidate for future applications in electronics and optoelectronics.

cond-mat.mtrl-sci↗

arXiv2014

Tunable charge-trap memory based on few-layer MoS2

Charge-trap memory with high-\k dielectric materials is considered to be a promising candidate for next-generation memory devices. Ultrathin layered two-dimensional (2D) materials like graphene and MoS2 have been receiving much attention because of their novel physical properties and potential applications in electronic devices. Here, we report on a dual-gate charge-trap memory device composed of a few-layer MoS2 channel and a three-dimensional (3D) Al2O3/HfO2/Al2O3 charge-trap gate stack. Owing to the extraordinary trapping ability of both electrons and holes in HfO2, the MoS2 memory device exhibits an unprecedented memory window exceeding 20 V. More importantly, with a back gate the window size can be effectively tuned from 15.6 to 21 V; the program/erase current ratio can reach up to 104, far beyond Si-based flash memory, which allows for multi-bit information storage. Furthermore, the device shows a high mobility of 170 cm2V-1s-1, a good endurance of hundreds of cycles and a stable retention of ~28% charge loss after 10 years which is drastically lower than ever reported MoS2 flash memory. The combination of 2D materials with traditional high-\k charge-trap gate stacks opens up an exciting field of novel nonvolatile memory devices.

cond-mat.mtrl-sci↗