arXiv subjects

Qi Hou

Publications and source records attributed to Qi Hou.

14 recordsLinked to original sources

arXiv2026

DITRON: Distributed Multi-level Tiling Compiler for Parallel Tensor Programs

The scaling of large language models (LLMs) is currently bottlenecked by the rigidity of distributed programming. While high-performance libraries like CuBLAS and NCCL provide optimized primitives, they lack the flexibility required for rapidly evolving model architectures. Conversely, existing tensor compilers fail to address the complex memory hierarchy of distributed clusters effectively. To bridge this gap, we propose DITRON, a scalable tile-level compiler that democratizes high-performance distributed kernel development. DITRON introduces a novel hierarchical programming abstraction spanning Core, Device, and Task levels to map tensor programs efficiently onto heterogeneous distributed hardware. This abstraction allows DITRON to support diverse parallelism strategies while abstracting away the complexity of inter-node and intra-node communication. Evaluated across large-scale clusters, DITRON achieves performance parity with or exceeding expert-tuned CUDA libraries, delivering speedups of $6\%-30\%$ on isolated kernels and $5\%-30\%$ on end-to-end inference in vLLM. Furthermore, DITRON demonstrates strong portability, achieving significant speedups on both NVIDIA and AMD platforms. \ours{} has been deployed at the enterprise level for both training and inference. It achieves an MFU improvement of over 10\% in training tasks, saving approximately 500,000 GPU hours of training cost per month. For inference tasks, it delivers an end-to-end gain of over 20\% and has been applied to cloud service inference and edge inference scenarios.

cs.PL↗

arXiv2025

Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler

In this report, we propose Triton-distributed, an extension of existing Triton compiler, to overcome the programming challenges in distributed AI systems. Triton-distributed is the first compiler that supports native overlapping optimizations for distributed AI workloads, providing a good coverage of existing optimizations from different frameworks. First, we integrate communication primitives compliant with the OpenSHMEM standard into the compiler. This enables programmers to utilize these primitives with a higher-level Python programming model. Second, we illustrate how to achieve complex joint optimization of computation, memory access, and communication with the assistance of the compiler. In particular, we show how to use overlapping techniques to hide latency and present our compiler-based programming methods in both single-node and multi-node scenarios. Finally, we showcase the performance of the code generated by our compiler. In a test environment with up to 64 devices, our compiler can fully utilize heterogeneous communication and computation resources to provide effective overlapping and high performance. In many cases, the performance of the generated code can even outperform hand-optimized code. Moreover, the development difficulty and the time cost for development using our compiler are far less than those of low-level programming such as CUDA/C++, which clearly demonstrates significant productivity advantages.

cs.DC↗

arXiv2025

Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning

We introduce Seed1.5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks. Seed1.5-Thinking achieves 86.7 on AIME 2024, 55.0 on Codeforces and 77.3 on GPQA, demonstrating excellent reasoning abilities in STEM and coding. Beyond reasoning tasks, the method demonstrates notable generalization across diverse domains. For instance, it surpasses DeepSeek R1 by 8% in win rate on non-reasoning tasks, indicating its broader applicability. Compared to other state-of-the-art reasoning models, Seed1.5-Thinking is a Mixture-of-Experts (MoE) model with a relatively small size, featuring 20B activated and 200B total parameters. As part of our effort to assess generalized reasoning, we develop two internal benchmarks, BeyondAIME and Codeforces, both of which will be publicly released to support future research. Model trial link: https://www.volcengine.com/experience/ark.

cs.CL↗

arXiv2025

TileLink: Generating Efficient Compute-Communication Overlapping Kernels using Tile-Centric Primitives

Large deep learning models have achieved state-of-the-art performance in a wide range of tasks. These models often necessitate distributed systems for efficient training and inference. The fundamental building blocks for distributed model execution are intra-layer parallel operators. The most effective approach to enhancing the performance of intra-layer parallel operators involves overlapping computation with communication. The overlapping can be achieved through either operator decomposition or kernel fusion. While decomposing operators is straightforward to implement, it often results in suboptimal performance. On the other hand, fusing communication kernels with compute kernels demands significant expertise and is error-prone. In this paper, we propose TileLink to enable efficient compilation and generation of overlapped compute-communication kernels. TileLink is composed of frontend and backend. In the frontend, TileLink decouples the design space of communication and computation, linking these two parts via tile-centric primitives. In the backend, TileLink translates these primitives into low-level communication instructions, integrating the communication and computation components to achieve overlapped execution. In experiments, TileLink achieves from $1.17\times$ to $20.76\times$ speedup to non-overlapping baseline and achieves performance comparable to state-of-the-art overlapping libraries on GPUs.

cs.DC↗

arXiv2025

Sub-elliptic diffusions on compact groups via Dirichlet form perturbation

This work provides an extension of parts of the classical finite dimensional sub-elliptic theory in the context of infinite dimensional compact connected metrizable groups. Given a well understood and well behaved bi-invariant Laplacian, $\Delta$, and a sub-Laplacian, $L$, to which intrinsic distances, $d_\Delta$, $d_L$, are naturally attached, we show that a comparison inequality of the form $d_L\le C(d_\Delta)^c$ (for some $0<c\le 1$) implies that the Dirichlet form of a fractional power of $\Delta$ is dominated by the Dirichlet form associated with $L$. We use this result to show that, under additional assumptions, certain good properties of the heat kernel for $\Delta$ are then passed to the heat kernel associated with $L$. Explicit examples on the infinite product of copies of $SU(2)$ are discussed to illustrate these results.

math.PR↗

arXiv2025

Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts

Mixture-of-experts (MoE) has been extensively employed to scale large language models to trillion-plus parameters while maintaining a fixed computational cost. The development of large MoE models in the distributed scenario encounters the problem of large communication overhead. The inter-device communication of a MoE layer can occupy 47% time of the entire model execution with popular models and frameworks. Therefore, existing methods suggest the communication in a MoE layer to be pipelined with the computation for overlapping. However, these coarse grained overlapping schemes introduce a notable impairment of computational efficiency and the latency concealing is sub-optimal. To this end, we present COMET, an optimized MoE system with fine-grained communication-computation overlapping. Leveraging data dependency analysis and task rescheduling, COMET achieves precise fine-grained overlapping of communication and computation. Through adaptive workload assignment, COMET effectively eliminates fine-grained communication bottlenecks and enhances its adaptability across various scenarios. Our evaluation shows that COMET accelerates the execution of a single MoE layer by $1.96\times$ and for end-to-end execution, COMET delivers a $1.71\times$ speedup on average. COMET has been adopted in the production environment of clusters with ten-thousand-scale of GPUs, achieving savings of millions of GPU hours.

cs.DC↗

arXiv2024

FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion

Large deep learning models have demonstrated strong ability to solve many tasks across a wide range of applications. Those large models typically require training and inference to be distributed. Tensor parallelism is a common technique partitioning computation of an operation or layer across devices to overcome the memory capacity limitation of a single processor, and/or to accelerate computation to meet a certain latency requirement. However, this kind of parallelism introduces additional communication that might contribute a significant portion of overall runtime. Thus limits scalability of this technique within a group of devices with high speed interconnects, such as GPUs with NVLinks in a node. This paper proposes a novel method, Flux, to significantly hide communication latencies with dependent computations for GPUs. Flux over-decomposes communication and computation operations into much finer-grained operations and further fuses them into a larger kernel to effectively hide communication without compromising kernel efficiency. Flux can potentially overlap up to 96% of communication given a fused kernel. Overall, it can achieve up to 1.24x speedups for training over Megatron-LM on a cluster of 128 GPUs with various GPU generations and interconnects, and up to 1.66x and 1.30x speedups for prefill and decoding inference over vLLM on a cluster with 8 GPUs with various GPU generations and interconnects.

cs.LG↗

arXiv2024

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

We present the design, implementation and engineering experience in building and deploying MegaScale, a production system for training large language models (LLMs) at the scale of more than 10,000 GPUs. Training LLMs at this scale brings unprecedented challenges to training efficiency and stability. We take a full-stack approach that co-designs the algorithmic and system components across model block and optimizer design, computation and communication overlapping, operator optimization, data pipeline, and network performance tuning. Maintaining high efficiency throughout the training process (i.e., stability) is an important consideration in production given the long extent of LLM training jobs. Many hard stability issues only emerge at large scale, and in-depth observability is the key to address them. We develop a set of diagnosis tools to monitor system components and events deep in the stack, identify root causes, and derive effective techniques to achieve fault tolerance and mitigate stragglers. MegaScale achieves 55.2% Model FLOPs Utilization (MFU) when training a 175B LLM model on 12,288 GPUs, improving the MFU by 1.34x compared to Megatron-LM. We share our operational experience in identifying and fixing failures and stragglers. We hope by articulating the problems and sharing our experience from a systems perspective, this work can inspire future LLM systems research.

cs.LG↗

arXiv2022

Motion of Lee-Yang zeros

We consider the zeros of the partition function of the Ising model with ferromagnetic pair interactions and complex external field. Under the assumption that the graph with strictly positive interactions is connected, we vary the interaction (denoted by $t$) at a fixed edge. It is already known that each zero is monotonic (either increasing or decreasing) in $t$; we prove that its motion is local: the entire trajectories of any two distinct zeros are disjoint. If the underlying graph is a complete graph and all interactions take the same value $t\geq 0$ (i.e., the Curie-Weiss model), we prove that all the principal zeros (those in $i[0,π/2)$) decrease strictly in $t$.

math-ph↗

arXiv2021

Perturbation results concerning Gaussian estimates and hypoellipticity for left-invariant Laplacians on compact groups

In this paper we study left-invariant Laplacians on compact connected groups that are form-comparable perturbations of bi-invariant Laplacians. Our results show that Gaussian bounds for derivatives of heat kernels enjoyed by certain bi-invariant Laplacians hold for their form-comparable perturbations. We further show that the parabolic operators associated with such left-invariant Laplacians, in particular, with the bi-invariant Laplacians, are hypoelliptic in various senses.

math.PR↗

arXiv2020

Time regularity for local weak solutions of the heat equation on local Dirichlet spaces

We study the time regularity of local weak solutions of the heat equation in the context of local regular symmetric Dirichlet spaces. Under two basic and rather minimal assumptions, namely, the existence of certain cut-off functions and a very weak $L^2$ Gaussian type upper-bound for the heat semigroup, we prove that the time derivatives of a local weak solution of the heat equation are themselves local weak solutions. This applies, for instance, to local weak solutions of parabolic equations with uniformly elliptic symmetric divergence form second order operators with measurable coefficients. We describe some applications to the structure of ancient local weak solutions of such equations which generalize recent results of [8] and [33].

math.AP↗

arXiv2020

Rough Hypoellipticity for the Heat Equation in Dirichlet Spaces

This paper aims at proving the local boundedness and continuity of solutions of the heat equation in the context of Dirichlet spaces under some rather weak additional assumptions. We consider symmetric local regular Dirichlet forms which satisfy mild assumptions concerning (a) the existence of cut-off functions, (b) a local ultracontractivity hypothesis, and (c) a weak off-diagonal upper bound. In this setting, local weak solutions of the heat equation, and their time derivatives, are shown to be locally bounded; they are further locally continuous, if the semigroup admits a locally continuous density function. Applications of the results are provided including discussion on the existence of locally bounded heat kernel; $L^\infty$ structure results for ancient solutions of the heat equation. The last section presents a special case where the $L^\infty$ off-diagonal upper bound follows from the ultracontractivity property of the semigroup. This paper is a continuation of [7].

math.AP↗

arXiv2018

Two-level Transmission Scheme for Cache-enabled Fog Radio Access Networks

In this paper, we investigate the downlink transmission for cache-enabled fog radio access networks aiming at maximizing the delivery rate under the constraints of fronthaul capacity, maximum transmit power, and size of files. To reduce the delivery latency and the burden on fronthaul links and make full use of the local cache and baseband signal processing capabilities of enhanced remote radio heads (eRRHs), a two-level transmission scheme including cache-level and network-level transmission is proposed. In cache-level transmission, only requested files cached at the local cache are transmitted to the corresponding users. The duration of cache-level transmission is the delay caused by the transfer between the baseband unit (BBU) and eRRHs as well as the signal processing at the BBU. The remaining requested files are jointly transmitted to the corresponding users at network-level transmission. For cache-level transmission, a centralized optimization algorithm is firstly presented and then a decentralized optimization algorithm is provided to avoid the exchange of signaling among eRRHs. Meanwhile, another centralized optimization algorithm is presented to tackle the optimization problem for network-level transmission. All presented algorithms are proved to converge to the Karush-Kuhn-Tucker (KKT) solutions of the problems. Numerical results are provided to validate the effectiveness of the proposed transmission scheme as well as evaluating the system performance.

cs.IT↗

arXiv2018

The Automatic Identification of Butterfly Species

The available butterfly data sets comprise a few limited species, and the images in the data sets are always standard patterns without the images of butterflies in their living environment. To overcome the aforementioned limitations in the butterfly data sets, we build a butterfly data set composed of all species of butterflies in China with 4270 standard pattern images of 1176 butterfly species, and 1425 images from living environment of 111 species. We propose to use the deep learning technique Faster-Rcnn to train an automatic butterfly identification system including butterfly position detection and species recognition. We delete those species with only one living environment image from data set, then partition the rest images from living environment into two subsets, one used as test subset, the other as training subset respectively combined with all standard pattern butterfly images or the standard pattern butterfly images with the same species of the images from living environment. In order to construct the training subset for FasterRcnn, nine methods were adopted to amplifying the images in the training subset including the turning of up and down, and left and right, rotation with different angles, adding noises, blurring, and contrast ratio adjusting etc. Three prediction models were trained. The mAP (Mean Average prediction) criterion was used to evaluate the performance of the prediction model. The experimental results demonstrate that our Faster-Rcnn based butterfly automatic identification system performed well, and its worst mAP is up to 60%, and can simultaneously detect the positions of more than one butterflies in one images from living environment and recognize the species of those butterflies as well.

cs.CV↗