Searcharxiv⌕ Search

arXiv subjects

Yuhao Hu

Publications and source records attributed to Yuhao Hu.

13 recordsLinked to original sources
arXiv2025

Remoe: Towards Efficient and Low-Cost MoE Inference in Serverless Computing

Mixture-of-Experts (MoE) has become a dominant architecture in large language models (LLMs) due to its ability to scale model capacity via sparse expert activation. Meanwhile, serverless computing, with its elasticity and pay-per-use billing, is well-suited for deploying MoEs with bursty workloads. However, the large number of experts in MoE models incurs high inference costs due to memory-intensive parameter caching. These costs are difficult to mitigate via simple model partitioning due to input-dependent expert activation. To address these issues, we propose Remoe, a heterogeneous MoE inference system tailored for serverless computing. Remoe assigns non-expert modules to GPUs and expert modules to CPUs, and further offloads infrequently activated experts to separate serverless functions to reduce memory overhead and enable parallel execution. We incorporate three key techniques: (1) a Similar Prompts Searching (SPS) algorithm to predict expert activation patterns based on semantic similarity of inputs; (2) a Main Model Pre-allocation (MMP) algorithm to ensure service-level objectives (SLOs) via worst-case memory estimation; and (3) a joint memory and replica optimization framework leveraging Lagrangian duality and the Longest Processing Time (LPT) algorithm. We implement Remoe on Kubernetes and evaluate it across multiple LLM benchmarks. Experimental results show that Remoe reduces inference cost by up to 57% and cold start latency by 47% compared to state-of-the-art baselines.

cs.DC↗
arXiv2025

BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation

Diffusion Transformers currently lead the field in high-quality video generation, but their slow iterative denoising process and prohibitive quadratic attention costs for long sequences create significant inference bottlenecks. While both step distillation and sparse attention mechanisms have shown promise as independent acceleration strategies, effectively combining these approaches presents critical challenges -- training-free integration yields suboptimal results, while separately training sparse attention after step distillation requires prohibitively expensive high-quality video data. To overcome these limitations, we propose BLADE, an innovative data-free joint training framework that introduces: (1) an Adaptive Block-Sparse Attention (ASA) mechanism for dynamically generating content-aware sparsity masks to focus computation on salient spatiotemporal features, and (2) a sparsity-aware step distillation paradigm, built upon Trajectory Distribution Matching (TDM), directly incorporates sparsity into the distillation process rather than treating it as a separate compression step and features fast convergence. We validate BLADE on text-to-video models like CogVideoX-5B and Wan2.1-1.3B, and our framework demonstrates remarkable efficiency gains across different scales. On Wan2.1-1.3B, BLADE achieves a 14.10x end-to-end inference acceleration over a 50-step baseline. Moreover, on models such as CogVideoX-5B with short video sequence lengths, our framework delivers a robust 8.89x speedup. Crucially, the acceleration is accompanied by a consistent quality improvement. On the VBench-2.0 benchmark, BLADE boosts the score of CogVideoX-5B to 0.569 (from 0.534) and Wan2.1-1.3B to 0.570 (from 0.563), results that are further corroborated by superior ratings in human evaluations. Project is available at http://ziplab.co/BLADE-Homepage/.

cs.CV↗
arXiv2025

Hyperbolic Monge-Amp\`ere systems with $S_1=0$

For hyperbolic Monge-Amp\`ere systems, a well-known solution of the equivalence problem yields two invariant tensors, ${S}_1$ and ${S}_2$, defined on the underlying $5$-manifold, where ${S}_2=0$ characterizes systems that are Euler-Lagrange. In this article, we consider the `opposite' case, ${S}_1 = 0$, and show that the local generality of such systems is `$2$ arbitrary functions of $3$ variables'. In addition, we classify all $S_1=0$ systems with cohomogeneity at most one, which turn out to be linear up to contact transformations.

math.DG↗
arXiv2025

SyzParam: Introducing Runtime Parameters into Kernel Driver Fuzzing

This paper introduces a novel fuzzing framework, SyzParam which incorporates runtime parameters into the fuzzing process. Achieving this objective requires addressing several key challenges, including valid value extraction, inter-device relation construction, and fuzz engine integration. By inspecting the data structures and functions associated with the LKDM, our tool can extract runtime parameters across various drivers through static analysis. Additionally, SyzParam collects inter-device relations and identifies associations between runtime parameters and drivers. Furthermore, SyzParam proposes a novel mutation strategy, which leverages these relations and prioritizes parameter modification during related driver execution. Our evaluation demonstrates that SyzParam outperforms existing fuzzing works in driver code coverage and bug-detection capabilities. To date, we have identified 30 unique bugs in the latest kernel upstreams, with 20 confirmed and 14 patched into the mainline kernel, including 9 CVEs.

cs.CR↗
arXiv2024

Ascend HiFloat8 Format for Deep Learning

This preliminary white paper proposes a novel 8-bit floating-point data format HiFloat8 (abbreviated as HiF8) for deep learning. HiF8 features tapered precision. For normal value encoding, it provides 7 exponent values with 3-bit mantissa, 8 exponent values with 2-bit mantissa, and 16 exponent values with 1-bit mantissa. For denormal value encoding, it extends the dynamic range by 7 extra powers of 2, from 31 to 38 binades (notice that FP16 covers 40 binades). Meanwhile, HiF8 encodes all the special values except that positive zero and negative zero are represented by only one bit-pattern. Thanks to the better balance between precision and dynamic range, HiF8 can be simultaneously used in both forward and backward passes of AI training. In this paper, we will describe the definition and rounding methods of HiF8, as well as the tentative training and inference solutions. To demonstrate the efficacy of HiF8, massive simulation results on various neural networks, including traditional neural networks and large language models (LLMs), will also be presented.

cs.LG↗
arXiv2024

Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition

Micro-expressions are subtle facial movements that occur spontaneously when people try to conceal real emotions. Micro-expression recognition is crucial in many fields, including criminal analysis and psychotherapy. However, micro-expression recognition is challenging since micro-expressions have low intensity and public datasets are small in size. To this end, a three-stream temporal-shift attention network based on self-knowledge distillation is proposed in this paper. Firstly, to address the low intensity of muscle movements, we utilize learning-based motion magnification modules to enhance the intensity of muscle movements. Secondly, we employ efficient channel attention modules in the local-spatial stream to make the network focus on facial regions that are highly relevant to micro-expressions. In addition, temporal shift modules are used in the dynamic-temporal stream, which enables temporal modeling with no additional parameters by mixing motion information from two different temporal domains. Furthermore, we introduce self-knowledge distillation into the micro-expression recognition task by introducing auxiliary classifiers and using the deepest section of the network for supervision, encouraging all blocks to fully explore the features of the training set. Finally, extensive experiments are conducted on five publicly available micro-expression datasets. The experimental results demonstrate that our network outperforms other existing methods and achieves new state-of-the-art performance. Our code is available at https://github.com/GuanghaoZhu663/SKD-TSTSAN.

cs.CV↗
arXiv2022

Rigidity of 3D spherical caps via $\mu$-bubbles

By using Gromov's $\mu$-bubble technique, we show that the $3$-dimensional spherical caps are rigid under perturbations that do not reduce the metric, the scalar curvature, and the mean curvature along its boundary. Several generalizations of this result will be discussed.

math.DG↗
arXiv2020

Rank 2 B\"acklund Transformations of Hyperbolic Monge-Amp\`ere Systems

There are two main types of rank 2 B\"acklund transformations relating a pair of hyperbolic Monge-Amp\`ere systems, which we call Type $\mathscr{A}$ and Type $\mathscr{B}$. For Type $\mathscr{A}$, we completely determine a subclass whose local invariants satisfy a specific but simple algebraic constraint; such B\"acklund transformations are parametrized by a finite number of constants, whose cohomogeneity can be either 2, 3 or 4. In addition, we present an invariantly formulated condition that determines whether a generic Type $\mathscr{B}$ B\"acklund transformation is one that, under suitable choices of local coordinates, relates solutions of two PDEs of the form $z_{xy} = F(x,y,z,z_x,z_y)$ and preserves the $x,y$ variables on solutions.

math.DG↗
arXiv2020

On Absolute Equivalence and Linearization I

In this paper, we study the absolute equivalence between Pfaffian systems with a degree 1 independence condition and obtain structural results, particularly for systems of corank 3. We apply these results to understanding dynamic feedback linearization of control systems with 2 inputs.

math.DG↗
arXiv2019

Geometry of B\"acklund Transformations II: Monge-Amp\`ere Invariants

This article is concerned with the question: For which pairs of hyperbolic Euler-Lagrange systems in the plane does there exist a rank-$1$ B\"acklund transformation relating them? We express some obstructions to such existence in terms of the local invariants of the Euler-Lagrange systems. In addition, we discover a class of B\"acklund transformations relating two hyperbolic Euler-Lagrange systems of distinct types.

math.AP↗
arXiv2019

Geometry of B\"acklund Transformations I: Generality

Using Cartan's Method of Equivalence, we prove an upper bound for the generality of generic rank-1 B\"acklund transformations relating two hyperbolic Monge-Amp\`ere systems. In cases when the B\"acklund transformation admits a symmetry group whose orbits have codimension 1, 2, or 3, we obtain classification results and new examples of auto-B\"acklund transformations.

math.DG↗