arXiv subjects
Publications and source records attributed to Zhen Zheng.
We establish Schauder estimates for linearized very fast diffusion equations with Dirichlet boundary conditions in bounded smooth domains. These estimates provide an important ingredient for deriving optimal boundary regularity and long-time dynamics of solutions to very fast diffusion equations.
We prove the optimal global regularity of admissible solutions to a transformed very fast diffusion equation in the range $-1<p<0$, posed on smooth bounded domains with zero Dirichlet boundary data and initial data comparable to the distance function. More precisely, we establish existence and uniqueness and show that solutions belong to $C^{1,p+1}(\overline\Omega)$ in space for every positive time and are $C^\infty$ in time uniformly up to the boundary. Moreover, all their time derivatives belong to $C^{1,p+1}(\overline\Omega)$, and the exponent $p+1$ is optimal. These regularity estimates further yield fine long-time asymptotics toward the friendly giant solution, including a first-order expansion in the $C^{1,p+1}(\overline{\Omega})$ topology and an improved convergence rate for the relative error in $C^{p+1}(\overline\Omega)$.
The KV cache is a dominant memory bottleneck for LLM inference. While 4-bit KV quantization preserves accuracy, 2-bit often degrades it, especially on long-context reasoning. We close this gap via an algorithm-system co-design for mixed-precision KV caching: Kitty. On the algorithm side, extensive experiments show that Dynamic Channel-wise Precision Boost -- which ranks Key-cache channels by sensitivity and keeps only a small fraction at higher precision -- maintains near-zero loss in accuracy drop while approaching 2-bit memory. The main challenge is handling dynamic 4-bit channel boosts while keeping the page layout coalesced and the dequantization uniform, with no scattered reads or hard-coded masks. Kitty addresses these issues by decompose each mixed-precision Key page into two tensors with unified 2-bit precision. Based on this, Kitty provides a page-centric KV layout, Triton-compatible page dequantization kernels, and a lightweight runtime pipeline that preserves coalescing and avoids divergence. Across seven tasks and two model families (Qwen3, LLaMA3), Kitty cuts KV memory by nearly 8x with negligible accuracy loss, enabling up to 8x larger batches and 2.1x-4.1x higher throughput under the same memory budget. We release the full implementation of Kitty at https://github.com/Summer-Summer/Kitty.
The many-body physics of higher-spin systems is expected to host qualitatively new matter phases, but realizing them requires the controllable interactions between multispin components that can be tuned independently for each component. Here we propose a scheme that meets this demand in ultracold Fermi gases. By engineering the atom-cavity coupling, we generate cavity-induced effective interactions between pseudo-spin states via multiple Raman and cavity paths. Focusing on the simplest spin-1 case, we obtain two independent scattering channels whose relative strengths and signs are determined by the Clebsch-Gordan coefficients and optical-field parameters. The resulting Hamiltonian combines the on-site Cooper pairing with the off-site repulsion, and drives a continuous transition from the superfluid to the spin-density-wave phase. The coexistence region is reminiscent of a supersolid, yet the self-organized modulation appears in the spin density profile of a higher-spin representation, rather than in the number density profile. The proposal can be implemented with the existing techniques in ultracold atoms. Therefore it offers a versatile platform for quantum simulation of higher-spin many-body physics.
We report the discovery of several classes of novel topological insulators (TIs) with hybrid-order boundary states generated from the first-order TIs with additional crystalline symmetries. Unlike the current studies on hybrid-order TIs where different-order topology arises from merging different-order TIs in various energy, {\color{red} these novel TIs exhibit unique properties, featuring a remarkable coexistence of first-order gapless modes and higher-order Fermi arc states}, behaving as a hybrid between the first-order TIs and higher-order topological semimetals within a single bulk gap. Our findings establish a profound connection between these novel $d$-dimensional ($d$D) TIs and ($d-1$)D higher-order TIs (HOTIs), which can be understood as a result of stacking $(d-1)$D HOTIs to $d$D with $d=3,4$, revealing unconventional topological phase transitions by closing the gap in certain first-order boundaries rather than the bulk. The bulk-boundary correspondence between these higher-order Fermi-arcs and bulk topological invariants associated with additional crystalline symmetries is also demonstrated. We then address the conventional topological phase transitions from these novel TIs to nodal-line/nodal-surface semimetal phases, where the gapless phases host new kinds of topological responses. Meanwhile, we present the corresponding topological semimetal phases by stacking these unique TIs. Finally, we discuss potential ways to realize these novel phases in synthetic and real materials, with a particular focus on the feasible implementation in optical lattices using ultracold atoms.
Quantization has become one of the most effective methodologies to compress LLMs into smaller size. However, the existing quantization solutions still show limitations of either non-negligible accuracy drop or low system efficiency. In this paper, we propose MixLLM that explores the optimization space of mixed-precision quantization between output features, based on the insight that different features matter differently in the model. MixLLM identifies the important output features in the global view rather than within each single layer, effectively assigning larger bit-width to output features that need it the most to achieve high accuracy and low memory usage. We present the sweet spot of quantization configuration of algorithm-system co-design with high accuracy and system efficiency. To address the system challenge, we design the two-step dequantization to make use of the Tensor Core easily and fast data type conversion to reduce dequantization overhead, and present the software pipeline to overlap the memory access, dequantization and the MatMul to the best. Extensive experiments show that with only 10\% more bits, the perplexity increase can be reduced from about 0.5 in SOTA to within 0.2 for Llama 3.1 70B, while MMLU-Pro loss can be reduced from 1.92 to 0.99 over the SOTA of three popular models. Besides its superior accuracy, MixLLM also achieves state-of-the-art system efficiency. Code is released at https://github.com/microsoft/MixLLM.
Large language models (LLMs) increasingly play an important role in a wide range of information processing and management tasks in industry. Many of these tasks are performed in large batches or even offline, and the performance indicator for which is throughput. These tasks usually show the characteristic of prefix sharing, where different prompt input can partially show the common prefix. However, the existing LLM inference engines tend to optimize the streaming requests and show limitations of supporting the large batched tasks with the prefix sharing characteristic. The existing solutions use the LRU-based cache to reuse the KV context of common prefix between requests. The KV context that are about to be reused may be prematurely evicted with the implicit cache management. Besides, the streaming oriented systems do not leverage the request-batch information and can not mix the decoding tokens with the prefill chunks to the best for the batched scenarios, and thus fails to saturate the GPU. We propose BatchLLM to address the above problems. BatchLLM explicitly identifies the common prefixes globally. The requests sharing the same prefix will be scheduled together to reuse the KV context the best. BatchLLM reorders the requests and schedules the requests with larger ratio of decoding first to better mix the decoding tokens with the latter prefill chunks, and applies memory-centric token batching to enlarge the token-batch sizes, which helps to increase the GPU utilization. Extensive evaluation shows that BatchLLM outperforms vLLM and SGLang by $1.3\times$ to $10.8\times$ on a set of microbenchmarks and a typical industry workload under different hardware environments. Code is available at https://github.com/microsoft/MixLLM/tree/batchllm_vllm_064.
The essential role of synthetic spin-orbit coupling in discovering new topological matter phases with cold atoms is widely acknowledged. However, the engineering of spin-orbit coupling remains unclear for arbitrary-spin models due to the complexity of spin matrices. In this paper, we develop a more general but relatively straightforward method to achieve spin-orbit coupling for multispin models. Our approach hinges on controlling the coupling between distinct pseudo-spins through two intermediary states, resulting in tunneling with spin flips that have direction-dependent strength. The engineered spin-orbit coupling can facilitate topological phase transitions with Chern numbers over 1, a unique characteristic of multispin models compared to spin-1/2 models. By utilizing existing cold atom techniques, our proposed method provides an ideal platform for investigating topological properties related to large Chern numbers.
Topological phases associated with non-Abelian charges can exhibit a distinguished bulk-edge correspondence compared with Abelian phases, although elucidating this relationship remains challenging in traditional solid-state systems. In this paper, we propose a theoretical framework for synthesizing non-Abelian quaternion charges in ultracold atomic gases. By designing artificial spin-orbit coupling patterns, the topological edge modes demonstrate a clear correspondence with the band topology determined by various quaternion charges. This paves the way for observing the interface modes whose existence is attributed to the nonconservation multiplication relation, which is fundamental to non-Abelian charges. This scheme can be readily implemented using current ultracold atom techniques, offering a promising approach to explore the intriguing non-Abelian characteristics of the system.
Six-bit quantization (FP6) can effectively reduce the size of large language models (LLMs) and preserve the model quality consistently across varied applications. However, existing systems do not provide Tensor Core support for FP6 quantization and struggle to achieve practical performance improvements during LLM inference. It is challenging to support FP6 quantization on GPUs due to (1) unfriendly memory access of model weights with irregular bit-width and (2) high runtime overhead of weight de-quantization. To address these problems, we propose TC-FPx, the first full-stack GPU kernel design scheme with unified Tensor Core support of float-point weights for various quantization bit-width. We integrate TC-FPx kernel into an existing inference system, providing new end-to-end support (called FP6-LLM) for quantized LLM inference, where better trade-offs between inference cost and model quality are achieved. Experiments show that FP6-LLM enables the inference of LLaMA-70b using only a single GPU, achieving 1.69x-2.65x higher normalized inference throughput than the FP16 baseline. The source code is publicly available at https://github.com/usyd-fsalab/fp6_llm.
This study examines 4-bit quantization methods like GPTQ in large language models (LLMs), highlighting GPTQ's overfitting and limited enhancement in Zero-Shot tasks. While prior works merely focusing on zero-shot measurement, we extend task scope to more generative categories such as code generation and abstractive summarization, in which we found that INT4 quantization can significantly underperform. However, simply shifting to higher precision formats like FP6 has been particularly challenging, thus overlooked, due to poor performance caused by the lack of sophisticated integration and system acceleration strategies on current AI hardware. Our results show that FP6, even with a coarse-grain quantization scheme, performs robustly across various algorithms and tasks, demonstrating its superiority in accuracy and versatility. Notably, with the FP6 quantization, \codestar-15B model performs comparably to its FP16 counterpart in code generation, and for smaller models like the 406M it closely matches their baselines in summarization. Neither can be achieved by INT4. To better accommodate various AI hardware and achieve the best system performance, we propose a novel 4+2 design for FP6 to achieve similar latency to the state-of-the-art INT4 fine-grain quantization. With our design, FP6 can become a promising solution to the current 4-bit quantization methods used in LLMs.
With the fast growth of parameter size, it becomes increasingly challenging to deploy large generative models as they typically require large GPU memory consumption and massive computation. Unstructured model pruning has been a common approach to reduce both GPU memory footprint and the overall computation while retaining good model accuracy. However, the existing solutions do not provide a highly-efficient support for handling unstructured sparsity on modern GPUs, especially on the highly-structured Tensor Core hardware. Therefore, we propose Flash-LLM for enabling low-cost and highly-efficient large generative model inference with the sophisticated support of unstructured sparsity on high-performance but highly restrictive Tensor Cores. Based on our key observation that the main bottleneck of generative model inference is the several skinny matrix multiplications for which Tensor Cores would be significantly under-utilized due to low computational intensity, we propose a general Load-as-Sparse and Compute-as-Dense methodology for unstructured sparse matrix multiplication. The basic insight is to address the significant memory bandwidth bottleneck while tolerating redundant computations that are not critical for end-to-end performance on Tensor Cores. Based on this, we design an effective software framework for Tensor Core based unstructured SpMM, leveraging on-chip resources for efficient sparse data extraction and computation/memory-access overlapping. At SpMM kernel level, Flash-LLM significantly outperforms the state-of-the-art library, i.e., Sputnik and SparTA by an average of 2.9x and 1.5x, respectively. At end-to-end framework level on OPT-30B/66B/175B models, for tokens per GPU-second, Flash-LLM achieves up to 3.8x and 3.6x improvement over DeepSpeed and FasterTransformer, respectively, with significantly lower inference cost.
Nonlocal interaction between ultracold atoms trapped in optical lattices can give rise to interesting quantum many-body phenomena. However, its realization usually demands unconventional techniques, for example the artificial gauge fields or higher-orbit Feshbach resonances, and is not highly controllable. Here, we propose a valid and feasible scheme for realizing a tunable finite-range interaction for spinless fermions immersed into the bath of bosons. The strength of the effective interaction for the fermionic subsystem is artificially tunable by manipulating bosons, ranging from the repulsive to attractive regime. And the interaction distance is locked to the hopping of bosons, making the finite-range interaction perfectly clean for the fermionic subsystem. Specifically we find that, by introducing an additional staggered hopping of bosons, the proposal is readily applied to search the Majorana corner modes in such a spinless system, without implementation of complex artificial gauge fields, which is totally distinct from existing results reported in spinful systems. Therefore this scheme provides a potential platform for exploring the unconventional topological superfluids and other nontrivial phases induced by long-range interactions in ultracold atoms.
We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment. It transforms a tensor program written for a single device into an equivalent distributed program that is capable of scaling up to thousands of devices with no user configuration. Rhino firstly works on a semantically independent intermediate representation of tensor programs, which facilitates its generalization to unprecedented applications. Additionally, it implements a task-oriented controller and a distributed runtime for optimal performance. Rhino explores on a complete and systematic parallelization strategy space that comprises all the paradigms commonly employed in deep learning (DL), in addition to strided partitioning and pipeline parallelism on non-linear models. Aiming to efficiently search for a near-optimal parallel execution plan, our analysis of production clusters reveals general heuristics to speed up the strategy search. On top of it, two optimization levels are designed to offer users flexible trade-offs between the search time and strategy quality. Our experiments demonstrate that Rhino can not only re-discover the expert-crafted strategies of classic, research and production DL models, but also identify novel parallelization strategies which surpass existing systems for novel models.
Higher-dimensional topological phases play a key role in understanding the lower-dimensional topological phases and the related topological responses through a dimensional reduction procedure. In this work, we present a Dirac-type model of four-dimensional (4D) $\mathbb{Z}_2$ topological insulator (TI) protected by $\mathcal{CP}$-symmetry, whose 3D boundary supports an odd number of Dirac cones. A specific perturbation splits each bulk massive Dirac cone into two valleys separated in energy-momentum space with opposite second Chern numbers, in which the 3D boundary modes become a nodal sphere or a Weyl semimetallic phase. By introducing the electromagnetic (EM) and pseudo-EM fields, exotic topological responses of our 4D system are revealed, which are found to be described by the (4+1)D mixed Chern-Simons theories in the low-energy regime. Notably, several topological phase transitions occur from a $\mathcal{CP}$-broken $\mathbb{Z}_2$ TI to a $\mathbb{Z}$ TI when the bulk gap closes by giving rise to exotic double-nodal-line/nodal-hyper-torus gapless phases. Finally, we propose to probe experimentally these topological effects in cold atoms.
This paper proposes DisCo, an automatic deep learning compilation module for data-parallel distributed training. Unlike most deep learning compilers that focus on training or inference on a single device, DisCo optimizes a DNN model for distributed training over multiple GPU machines. Existing single-device compilation strategies do not work well in distributed training, due mainly to communication inefficiency that they incur. DisCo generates optimized, joint computation operator and communication tensor fusion strategies to enable highly efficient distributed training. A GNN-based simulator is built to effectively estimate per-iteration training time achieved by operator/tensor fusion candidates. A backtracking search algorithm is driven by the simulator, navigating efficiently in the large strategy space to identify good operator/tensor fusion strategies that minimize distributed training time. We compare DisCo with existing DL fusion schemes and show that it achieves good training speed-up close to the ideal, full computation-communication overlap case.
The scaling up of deep neural networks has been demonstrated to be effective in improving model quality, but also encompasses several training challenges in terms of training efficiency, programmability, and resource adaptability. We present Whale, a general and efficient distributed training framework for giant models. To support various parallel strategies and their hybrids, Whale generalizes the programming interface by defining two new primitives in the form of model annotations, allowing for incorporating user hints. The Whale runtime utilizes those annotations and performs graph optimizations to transform a local deep learning DAG graph for distributed multi-GPU execution. Whale further introduces a novel hardware-aware parallel strategy, which improves the performance of model training on heterogeneous GPUs in a balanced manner. Deployed in a production cluster with 512 GPUs, Whale successfully trains an industry-scale multimodal model with over ten trillion model parameters, named M6, demonstrating great scalability and efficiency.
We study non-Hermitian spatial symmetries -- a class of symmetries that have no counterparts in Hermitian systems -- and study how normal and exceptional semimetals can be stabilized by these symmetries. Different from internal ones, spatial symmetries act nonlocally in momentum space and enforce global constraints on both band degeneracies and topological quantities at different locations. In deriving general constraints on band degeneracies and topological invariants, we demonstrate that non-Hermitian spatial symmetries are on an equal footing with, but are essentially different from Hermitian ones. First, we discover the nonlocal Hermitian conjugate pair of exceptional or normal band degeneracies that are enforced by non-Hermitian spatial symmetries. Remarkably, we find that these pairs lead to the symmetry-enforced violation of the Fermion doubling theorem in the long-time limit. Second, with the topological constraints, we unravel that certain exceptional manifold is only compatible with and stabilized by non-Hermitian spatial symmetries but is intrinsically incompatible with Hermitian spatial symmetries. We illustrate these findings using two three-dimensional models of a non-Hermitian Weyl semimetal and an exceptional unconventional Weyl semimetal. Experimental cold-atom realizations of both models are also proposed.