arXiv subjects

Min Shen

Publications and source records attributed to Min Shen.

16 recordsLinked to original sources

arXiv2026

Compiler-Grounded Hierarchical Diagnosis for LLM-Based Triton Kernel Optimization

Recent advances in large language models (LLMs) have enabled automated kernel generation and optimization, but most existing approaches rely on surface signals such as compilation feedback and profiling metrics. These signals reveal that a kernel is slow, but not why the backend compiler fails to realize a profitable optimization, especially on emerging accelerators such as NPUs. We therefore formulate kernel optimization as a progressive cross-layer diagnosis problem that links runtime symptoms to IR structure and compiler behavior before rewriting source. Based on this insight, we present our system, a compiler-grounded and hierarchical optimization framework for Triton kernels. the system escalates from lightweight pattern triage and profiling diagnosis to IR attribution and compiler-grounded analysis only when deeper evidence is needed, then proposes evidence-backed source-level rewrites. We implement the system on Triton for Ascend NPUs and evaluate it on 37 successfully converted entries from a standardized NPUKernelBench-derived Ascend 950 benchmark. Across these entries, the system attains a geometric-mean speedup of 4.35$\times$ and a median speedup of 2.73$\times$ from the initial to optimized Triton kernel; 22/37 exceed 2$\times$ and 13/37 exceed 5$\times$. The complete distribution ranges from near-baseline entries to large wins, motivating transparent reporting of the current system's scope and limitations.

cs.AI↗

arXiv2026

CoCoScale: Leveraging Layer-wise Scaling to Unlock the Potential of Online LLM Serving

Online large language model (LLM) serving has become the backbone of modern AI applications, powering diverse downstream services through shared hardware clusters. However, modern serving systems frequently encounter highly dynamic workloads characterized by severe workload skewness, where a small fraction of model instances receives the vast majority of traffic. Existing instance-level scaling mechanisms are limited by coarse-grained resource adjustment: scaling up requires the cold-start of full-model replicas, incurring substantial latency, while scaling down leaves the system vulnerable to performance degradation during sudden traffic surges. The key insight of this work is that LLM serving offers a unique opportunity for fine-grained scaling. In this paper, we propose CoCoScale, a layer-wise dynamic scaling mechanism that selectively expands the parallelism of hot layers onto idle resources reclaimed from underutilized devices, enabling elastic data parallelism without altering model architectures or adding hardware overhead. Evaluations demonstrate that CoCoScale significantly reduces cold start latency by 97.9%-99.3% compared to traditional scale up. Under production traces, CoCoScale reduces average latency by 20.7\%--28.1\% and achieves full Service Level Objective (SLO) attainment, demonstrating superior dynamic adaptability and resource efficiency.

cs.DC↗

arXiv2026

A specialized reasoning large language model for accelerating rare disease diagnosis: a randomized AI physician assistance trial

Rare diseases affect millions of individuals worldwide, yet timely diagnosis remains a major public health challenge due to scarcity of specialized clinical expertise. While large language models (LLMs) show promise to support rare disease diagnosis, current models are constrained by insufficient clinical deployability, limited clinically grounded evidence, and scarcity of training data. Here we present RaDaR (Rare Disease navigatoR), an open-source, compact reasoning LLM (32B parameters) for rare disease diagnosis. RaDaR was trained with 49,170 publicly available free-text cases and 104,666 synthetic cases with reasoning-enhanced training. RaDaR showed the strongest performance among evaluated open-source models, including the 671B DeepSeek-R1, across public benchmarks and four external validation centers. In a retrospective cohort, RaDaR prioritized the final diagnosis before documented clinical suspicion in 61.06 percent of cases, corresponding to a potential lead time of 1.87 months and 50.18 percent of the within-center interval. In a randomized physician-assistance trial, RaDaR assistance improved physicians' rare-disease diagnostic accuracy by 21.44 percentage points compared with internet search alone. Synthetic-data ablations suggested that phenotype-anchored narratives provide useful training signal for long-tail rare diseases, with a monotonic scaling trend within the tested data range. Together, RaDaR and its development and validation framework provide a deployable rare-disease reasoning model and a reproducible development framework for diagnostic AI under data scarcity.

cs.AI↗

arXiv2026

SwiftCache: Efficient LLM Serving for Multi-turn Conversations with Heterogeneous KV Cache Sharing

Multi-turn conversation is a fundamental scenario in LLM applications, widely used in chatbots and AI agents. As the conversation evolves, historical tokens accumulate continuously. Existing systems cache their key-value (KV) pairs to avoid redundant computation. However, limited GPU memory (HBM) capacity often forces these KV caches to be offloaded to CPU memory or SSD, making KV cache reloads increasingly costly in terms of latency as the context grows. Meanwhile, the constrained HBM capacity also limits the maximum inference length, thereby restricting the number of turns that can be supported in a conversation. To address these two challenges, we propose SwiftCache, a collaborative inference system that enables heterogeneous models to share underutilized GPU memory and NVLink bandwidth within a server. Specifically, models with low KV cache demand donate idle GPU memory to store the prefix cache of high-demand models, allowing cross-model KV cache sharing over NVLink and avoiding slow PCIe transfers. SwiftCache further reduces memory pressure by keeping only the KV cache of the currently active layer in local GPU memory, thereby enabling longer-context inference. Our experiments on real-world workloads show that SwiftCache reduces P99 time-to-first-token (TTFT) by up to 69% and extends maximum context length by up to 3.98x compared to vLLM and SGLang, with minimal interference to co-located models.

cs.DC↗

arXiv2025

BanaServe: Unified KV Cache and Dynamic Module Migration for Balancing Disaggregated LLM Serving in AI Infrastructure

Large language models (LLMs) are increasingly deployed in AI infrastructure, driving the need for high throughput, resource efficient serving systems. Disaggregated LLM serving, which separates prompt prefill from auto-regressive decode, has emerged as a promising architecture by isolating their heterogeneous compute and memory demands. However, current disaggregated systems face three key limitations: (i) static resource allocation cannot adapt to highly dynamic workloads, causing over-provisioning that wastes resources or under-provisioning that violates service level objectives (SLOs); (ii) inherent load imbalance between prefill and decode stages, where prefill is compute-bound and decode is memory-bound, causes under-utilization in one tier while the other becomes a bottleneck; and (iii) prefix cache aware routing skews load distribution, as high cache hit rate prefill nodes attract disproportionately more requests, further degrading balance and efficiency. To address these issues, we present BanaServe, a dynamic orchestration framework that continuously rebalances computational and memory resources across prefill and decode instances while eliminating hotspots induced by cache. BanaServe introduces layer level weight migration, attention level Key Value Cache (KV Cache) migration, and Global KV Cache Store sharing with layer wise overlapped transmission, enabling both coarse grained (layer level) and fine grained (attention level) load redistribution with minimal latency overhead. These mechanisms allow routers to perform purely load aware scheduling, unconstrained by cache placement. Compared to vLLM, BanaServe achieves 1.2x-3.9x higher throughput with 3.9%-78.4% lower total processing time, and outperforms DistServe by 1.1x-2.8x in throughput with 1.4%-70.1% latency reduction.

cs.DC↗

arXiv2025

Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM

Recent advancements in code large language models (LLMs) have demonstrated remarkable capabilities in code generation and understanding. It is still challenging to build a code LLM with comprehensive performance yet ultimate efficiency. Many attempts have been released in the open source community to break the trade-off between performance and efficiency, such as the Qwen Coder series and the DeepSeek Coder series. This paper introduces yet another attempt in this area, namely Ling-Coder-Lite. We leverage the efficient Mixture-of-Experts (MoE) architecture along with a set of high-quality data curation methods (especially those based on program analytics) to build an efficient yet powerful code LLM. Ling-Coder-Lite exhibits on-par performance on 12 representative coding benchmarks compared to state-of-the-art models of similar size, such as Qwen2.5-Coder-7B and DeepSeek-Coder-V2-Lite, while offering competitive latency and throughput. In practice, we achieve a 50\% reduction in deployment resources compared to the similar-sized dense model without performance loss. To facilitate further research and development in this area, we open-source our models as well as a substantial portion of high-quality data for the annealing and post-training stages. The models and data can be accessed at~\url{https://huggingface.co/inclusionAI/Ling-Coder-lite}.

cs.LG↗

arXiv2024

The Promise and Peril of Generative AI: Evidence from GPT as Sell-Side Analysts

Large language models (LLMs) promise to democratize financial analysis by reducing information-processing costs. Yet equal access does not ensure equal outcomes, as the locus of friction may shift from processing information to evaluating model outputs. We study GPT's earnings forecasts following corporate earnings releases and document two patterns. First, GPT's narrative attention is consistent and human-like but not always associated with higher forecast accuracy. Second, its quantitative reasoning varies substantially across contexts, challenging the view that LLMs are uniformly weak at numerical tasks. Building on these insights, we propose a diagnostic framework that links forecast accuracy to observable processing features (i.e., narrative focus, numerical reasoning, and self-assessed confidence). These indicators serve as proxies for this new form of information friction and alert investors when to exercise caution. Our study has implications for information frictions, regulatory oversight, and the economics of AI-mediated financial markets.

q-fin.GN↗

arXiv2023

MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning

Code LLMs have emerged as a specialized research field, with remarkable studies dedicated to enhancing model's coding capabilities through fine-tuning on pre-trained models. Previous fine-tuning approaches were typically tailored to specific downstream tasks or scenarios, which meant separate fine-tuning for each task, requiring extensive training resources and posing challenges in terms of deployment and maintenance. Furthermore, these approaches failed to leverage the inherent interconnectedness among different code-related tasks. To overcome these limitations, we present a multi-task fine-tuning framework, MFTcoder, that enables simultaneous and parallel fine-tuning on multiple tasks. By incorporating various loss functions, we effectively address common challenges in multi-task learning, such as data imbalance, varying difficulty levels, and inconsistent convergence speeds. Extensive experiments have conclusively demonstrated that our multi-task fine-tuning approach outperforms both individual fine-tuning on single tasks and fine-tuning on a mixed ensemble of tasks. Moreover, MFTcoder offers efficient training capabilities, including efficient data tokenization modes and PEFT fine-tuning, resulting in significantly improved speed compared to traditional fine-tuning methods. MFTcoder seamlessly integrates with several mainstream open-source LLMs, such as CodeLLama and Qwen. Leveraging the CodeLLama foundation, our MFTcoder fine-tuned model, \textsc{CodeFuse-CodeLLama-34B}, achieves an impressive pass@1 score of 74.4\% on the HumaneEval benchmark, surpassing GPT-4 performance (67\%, zero-shot). MFTCoder is open-sourced at \url{https://github.com/codefuse-ai/MFTCOder}

cs.LG↗

arXiv2023

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model

Code Large Language Models (Code LLMs) have gained significant attention in the industry due to their wide applications in the full lifecycle of software engineering. However, the effectiveness of existing models in understanding non-English inputs for multi-lingual code-related tasks is still far from well studied. This paper introduces CodeFuse-13B, an open-sourced pre-trained code LLM. It is specifically designed for code-related tasks with both English and Chinese prompts and supports over 40 programming languages. CodeFuse achieves its effectiveness by utilizing a high quality pre-training dataset that is carefully filtered by program analyzers and optimized during the training process. Extensive experiments are conducted using real-world usage scenarios, the industry-standard benchmark HumanEval-x, and the specially designed CodeFuseEval for Chinese prompts. To assess the effectiveness of CodeFuse, we actively collected valuable human feedback from the AntGroup's software development process where CodeFuse has been successfully deployed. The results demonstrate that CodeFuse-13B achieves a HumanEval pass@1 score of 37.10%, positioning it as one of the top multi-lingual code LLMs with similar parameter sizes. In practical scenarios, such as code generation, code translation, code comments, and testcase generation, CodeFuse performs better than other models when confronted with Chinese prompts.

cs.SE↗

arXiv2005

Spin dynamics in a compound semiconductor spintronic structure with a Schottky barrier

We demonstrate theoretically that spin dynamics of electrons injected into a GaAs semiconductor structure through a Schottky barrier possesses strong non-equilibrium features. Electrons injected are redistributed quickly among several valleys. Spin relaxation driven by the spin-orbital coupling in the semiconductor is very rapid. At T = 4.2 K, injected spin polarization decays on a distance of the order of 50 - 100 nm from the interface. This spin penetration depth reduces approximately by half at room temperature. The spin scattering length is different for different valleys.

cond-mat.mes-hall↗

arXiv2004

Monte Carlo modeling of spin injection through a Schottky barrier and spin transport in a semiconductor quantum well

We develop a Monte Carlo model to study injection of spin-polarized electrons through a Schottky barrier from a ferromagnetic metal contact into a non-magnetic low-dimensional semiconductor structure. Both mechanisms of thermionic emission and tunneling injection are included in the model. Due to the barrier shape, the injected electrons are non-thermalized. Spin dynamics in the semiconductor heterostructure is controlled by the Rashba and Dresselhaus spin-orbit interactions and described by a single electron spin density matrix formalism. In addition to the linear term, the third order term in momentum for the Dresselhaus interaction is included. Effect of the Schottky potential on the spin dynamics in a 2 dimensional semiconductor device channel is studied. It is found that the injected current can maintain substantial spin polarization to a length scale in the order of 1 micrometer at room temperature without external magnetic fields.

cond-mat.mes-hall↗

arXiv2004

Spin injection in spin FETs using a step-doping profile

We investigate effect of a step-doping profile on the spin injection from a ferromagnetic metal contact into a semiconductor quantum well (QW) in spin FETs using a Monte Carlo model. The considered scheme uses a heavily doped layer at the metal/semiconductor interface to vary the Schottky barrier shape and enhance the tunneling current. It is found that spin flux (spin current density) is enhanced proportionally to the total current, and the variation of current spin polarization does not exceed 20%.

cond-mat.mes-hall↗

arXiv2004

Modulation of spin dynamics in a channel of a non-ballistic spin field effect transistor

We have investigated the effect of the gate voltage on spin relaxation in an Al0.3Ga0.7As/GaAs/Al0.3Ga0.7As heterostructure. The study is motivated by a recent proposal for a non-ballistic spin field effect transistor that utilizes the interplay between the Rashba and the Dresselhaus spin-orbit interaction in the device channel. The model, which utilizes real material parameters, in order to calculate spin dynamics as a function of the gate voltage has been developed. From the obtained results we define the efficiency of the spin polarization and spin density modulation. The estimated modulation of the spin polarization at room temperature is of the order of 15-20%. The results show that the effect is not sufficient for device applications. However, it can be observed experimentally by optical pulse-probe techniques.

cond-mat.mes-hall↗

arXiv2003

Monte Carlo Modeling of Spin FETs Controlled by Spin-Orbit Interaction

A method for Monte Carlo simulation of 2D spin-polarized electron transport in III-V semiconductor heterojunction FETs is presented. In the simulation, the dynamics of the electrons in coordinate and momentum space is treated semiclassically. The density matrix description of the spin is incorporated in the Monte Carlo method to account for the spin polarization dynamics. The spin-orbit interaction in the spin FET leads to both coherent evolution and dephasing of the electron spin polarization. Spin-independent scattering mechanisms, including optical phonons, acoustic phonons and ionized impurities, are implemented in the simulation. The electric field is determined self-consistently from the charge distribution resulting from the electron motion. Description of the Monte Carlo scheme is given and simulation results are reported for temperatures in the range 77-300 K.

cond-mat↗

arXiv2003

Semiclassical Monte Carlo Model for In-Plane Transport of Spin-Polarized Electrons in III-V Heterostructures

We study the in-plane transport of spin-polarized electrons in III-V semiconductor quantum wells. The spin dynamics is controlled by the spin-orbit interaction, which arises via the Dresselhaus (bulk asymmetry) and Rashba (well asymmetry) mechanisms. This interaction, owing to its momentum dependence, causes rotation of the spin polarization vector, and also produces effective spin dephasing. The density matrix approach is used to describe the evolution of the electron spin polarization, while the spatial motion of the electrons is treated semiclassically. Monte Carlo simulations have been carried out for temperatures in the range 77-300 K.

cond-mat.mes-hall↗

arXiv2003

Monte Carlo Simulation of Spin-Polarized Transport

Monte Carlo simulations are performed to study the in-plane transport of spin-polarized electrons in III-V semiconductor quantum wells. The density matrix description of the spin polarization is incorporated in the simulation algorithm. The spin-orbit interaction terms generate coherent evolution of the electron spin polarization and also cause dephasing. The spatial motion of the electrons is treated semiclassically. Three different scattering mechanisms--optical phonons, acoustic phonons and ionized impurities--are considered. The electric field is calculated self-consistently from the charge distribution. The Monte Carlo scheme is described, and simulation results are reported for temperatures in the range 77-300 K.

cond-mat.mes-hall↗