Searcharxiv⌕ Search

arXiv subjects

Hailong Zou

Publications and source records attributed to Hailong Zou.

10 recordsLinked to original sources

dit.xpu : Taming DiT Sparse Inference on Mobile SoCs

Diffusion Transformers (DiTs) enable on-device image generation and editing but incur high attention costs at high resolutions. Existing sparse attention reduces computation, yet dynamic sparsity introduces preprocessing and irregular data accesses that poorly fit mobile NPUs. We present DiT.XPU, a sparse DiT inference framework for heterogeneous mobile SoCs. DiT.XPU combines tile-aligned sparse attention with anchor-based error compensation by exploiting slowly varying dense-sparse output residuals across denoising steps. Heterogeneous XPU collaboration overlaps NPU dense attention with GPU sparse calibration, while mask-guided dataflow reduces data transfers and layout conversion by overlapping data movement with computation. Across five DiT models on two Snapdragon smartphones, DiT.XPU achieves up to 1.56x end-to-end denoising speedup over the Hexagon-NPU backend and 11.5x attention-kernel speedup over the fastest alternative, while preserving generation quality close to dense attention. Code is available at https://github.com/IF-LAB-PKU/dit.xpu.

cs.CV↗

FedHQ: Hybrid Runtime Quantization for Federated Learning

Federated Learning (FL) must balance training efficiency and model accuracy across heterogeneous clients. Post-training quantization (PTQ) avoids the extra computation of quantization-aware training (QAT), but can incur greater accuracy loss. We propose FedHQ, a hybrid quantization framework that assigns PTQ or QAT according to clients' hardware and data characteristics. Hardware-based speed estimates and local data-distribution indicators guide coarse-grained global initialization, followed by fine-grained training-based adjustment of selected client strategies. Experiments on three image-classification datasets show that FedHQ achieves up to 2.47x training speedup over FedAQT and up to 11.15 percentage points higher accuracy than FedFQ.

cs.LG↗

ActECC: Action Chunk Temporal Harnessing for Edge-Cloud Collaborative VLA Inference System

VLA models have emerged as the mainstream for embodied intelligence, which decouples reasoning and execution via action chunks. To enable real-time deployment of VLA models, existing studies investigate edge-cloud collaborative inference, which pose two challenges to the action chunk mechanism. (1) Edge-cloud collaboration complicates system latency components, hindering the determination of valid ranges for action chunks. (2) Latency fluctuations in cross-edge update links under edge-cloud scenarios make it difficult to schedule replanning requests. To address these, we analyze the key factors affecting action chunk usable ranges and replanning request timing, revealing phase dependence and edge-cloud latency constraints. Based on these insights, we propose ActECC, which enables Action Chunk Temporal Harnessing through kinematics-guided usable range assessment and delay-aware replanning scheduling. Experiments show that ActECC reduces per-action latency by up to 27.4% and cloud calls by up to 45.1%, while lowering average execution error compared with existing edge-cloud collaborative.

cs.DC↗

DyQ-VLA: Temporal-Dynamic-Aware Quantization for Embodied Vision-Language-Action Models

Vision-language-action (VLA) models achieve favorable task performance, yet runtime errors in closed-loop execution evolve with alternating updates of actions and observations. Existing VLA quantization methods mainly trade off inference speed and model performance while rarely investigating how quantization alters runtime errors. By comparing error distributions between full-precision and quantized models, we identify distinct evolutionary patterns for the two types of errors: quantization enlarges the variance of translation error distributions and increases their dispersion, whereas the distribution center of rotation errors gradually shifts across execution steps, demonstrating cumulative drift. Motivated by this observation, we rethink the optimal quantization strategy for VLA models and propose \textit{DyQ-VLA}, a runtime-error-aware quantization framework, which dynamically selects activation precision according to execution steps and tracks as well as compensates rotation errors via accumulated quantization residuals. Corresponding operators and runtime adaptation strategies are devised within the framework to enable dynamic-precision execution. Experiments show that \textit{DyQ-VLA} achieves a 1.88 to 1.93 inference speedups while maintaining comparable or higher average task success rates. Moreover, it reduces mean execution steps by 17.7% to 19.8%. Our code is here: https://anonymous.4open.science/r/DyQ-VLA-7F51/.

cs.LG↗

BigMoMo: Efficient Inference of Large-Scale MoE with Speculative Decoding on Mobile Devices

Mixture-of-Experts (MoE) models expand language model capacity on smartphones, but expert offloading remains constrained by limited DRAM capacity and costly data movement. Sequential token routing couples expert execution to fragmented flash reads and multistage NPU preparation, leaving sparse computation stalled on weight transfers. Each transfer serves few tokens before execution moves on. We exploit the multi-token verification window of speculative decoding to decouple expert movement from single-token execution, enabling weight reuse, contiguous flash reads, and load-compute overlap. We present \textsc{BigMoMo}, a mobile MoE runtime that exploits this window across the memory hierarchy. It prunes speculative branches and expert activations using acceptance rates, routing impact, and movement cost; reorganizes on-flash experts according to runtime co-loading patterns; and batches ready experts to overlap NPU computation with pending transfers. Across four MoE models and five benchmarks on two mobile platforms, \textsc{BigMoMo} achieves mean decoding speedups of $4.83\times$ over on-demand autoregressive offloading and $1.82\times$ over the best speculative MoE baseline, supporting MoE models up to 80B parameter.

cs.AR↗

EchoCache: Energy-Guided Cross-Modal Caching for Efficient Audio-Driven Video Generation

Audio-driven video generation (A2V) has achieved promising progress in synthesizing temporally coherent and audio-visually aligned videos, yet its inference remains expensive due to the iterative denoising process of diffusion models. Existing caching methods mainly exploit temporal redundancy in visual features while overlooking the cross-modal alignment of A2V, where audio drives visual generation with highly non-uniform temporal importance. In this paper, we identify two levels of misalignment in existing A2V caching methods: temporal-semantic and computation-storage misalignment. To address them, we propose EchoCache, an energy-guided cross-modal caching framework for efficient A2V generation. EchoCache leverages audio time-frequency energy as a saliency anchor to guide latent-level cache updates and further introduces a dynamic timestep-latent caching mechanism with quantized cache management for joint efficiency and memory optimization. Extensive experiments on mainstream A2V models show that EchoCache consistently improves the latency-quality trade-off while preserving generation quality and audio-visual consistency. In particular, on Wan2.2-S2V over the EMTD benchmark, EchoCache achieves a 2.46x speedup with the best overall performance. Code is available at https://github.com/IF-LAB-PKU/EchoCache.

cs.CV↗

MoECa: Aligning Feature Reuse with Expert Decomposition in Diffusion Transformers

Diffusion Transformers with Mixture-of-Experts (DiT-MoE) improve model capacity under sparse activation, but diffusion inference is still bottlenecked by redundant computation across timesteps. Existing caching methods mainly operate at the token level, which becomes suboptimal in DiT-MoE because each token update is internally decomposed into multiple routed expert branches. Our analysis shows that cross-timestep redundancy in DiT-MoE is better characterized at the expert-branch level than at the whole-token level. Based on this observation, we propose MoECa, a fine-grained caching framework that performs branch-level feature reuse across timesteps. MoECa further introduces expert-aware adaptive control and synchronized cache updates across MoE and attention paths to maintain stable intermediate states. Experiments on multiple DiT-MoE models show a favorable speed--quality trade-off, with up to 2.93$\times$ speedups while preserving generation quality.

cs.LG↗

When and Where to Look: Adaptive Visual Evidence Scheduling for Efficient Long Video Understanding

Efficient long-video understanding requires vision--language models (VLMs) to reason over a small number of frames selected as sparse visual evidence. Existing relevance-based methods rely on static one-shot selection with fixed frame budgets and candidate pools, while agent-based schedulers achieve adaptivity through costly multi-round reasoning and interactive search. We propose EcoFrame, a training-free framework for low-overhead query-adaptive visual evidence scheduling. EcoFrame leverages the VLM's inference feedback to determine when to increase the frame budget and where to search for additional candidate evidence. Specifically, entropy-gated budget scheduling uses output uncertainty to stop early when the current evidence is sufficient or progressively expand the frame budget otherwise. Meanwhile, attention-guided candidate proposal converts frame-level attention into a temporal prior, enabling dense local search in informative regions while preserving global coverage when attention is diffuse. Experiments on Video-MME, LongVideoBench, and MLVU demonstrate that EcoFrame achieves a better accuracy--efficiency trade-off across multiple VLM backbones. On Qwen2.5-VL, EcoFrame achieves an average accuracy of 64.4, surpassing BOLT at 63.5, while providing a $1.85\times$ speedup over AKS and BOLT. Compared with the agent-based A.I.R., EcoFrame maintains comparable accuracy with up to a $13.5\times$ inference speedup. Code will be available at https://github.com/AK-DREAM/EcoFrame.

cs.CV↗

VLN-Cache: Enabling Token Caching for VLN Models with Visual/Semantic Dynamics Awareness

Vision-and-Language Navigation (VLN) increasingly relies on large vision-language models, but their inference cost conflicts with real-time deployment. Token caching is a promising training-free strategy that avoids redundant computation by reusing stable visual tokens across frames. However, existing methods assume a static camera and fixed semantic focus, assumptions that VLN fundamentally violates. We identify two failure modes: (1) visual dynamics, where viewpoint shift displaces token positions across frames, causing position-wise matching to pair misaligned content; (2) semantic dynamics, where token relevance shifts across task stages as navigation progresses, making cached states stale. We propose VLN-Cache, a visual-dynamic-aware and semantic-dynamic-aware caching framework that introduces view-aligned remapping to recover geometric correspondences and a task-relevance saliency filter to veto reuse at semantic transitions. A layer-adaptive entropy policy further balances the per-layer reuse budget. Experiments on the R2R-CE simulation benchmark show up to 1.52x speedup while maintaining competitive navigation success rates.

cs.RO↗

GOMA: Geometrically Optimal Mapping via Analytical Modeling for Spatial Accelerators

General matrix multiplication (GEMM) on spatial accelerators is highly sensitive to mapping choices in both execution efficiency and energy consumption. However, the mapping space exhibits combinatorial explosion, which makes it extremely challenging to obtain optimal mappings within an acceptable time budget. Existing approaches typically face challenges: They often lack global-optimality guarantees and become prohibitively slow as the mapping space grows. To address these limitations, we propose \textsc{GOMA}, a geometric-abstraction-based, globally optimal GEMM mapping framework via analytical modeling, which achieves efficient solving while guaranteeing optimality. \textsc{GOMA} introduces, from first principles, a geometric abstraction for GEMM mapping, yielding an exact analytical energy objective with $O(1)$ evaluation for any given mapping. The objective is highly accurate. \textsc{GOMA} then formulates mapping selection as an integer optimization problem under hardware and mapping constraints, using the analytical energy model as the objective to automate mapping search. \textsc{GOMA} can quickly compute a global-optimal mapping for any (GEMM workload, target hardware) pair, achieving this for the first time in mapping space exploration. Experiments confirm that across representative accelerators and large language model prefill workloads, \textsc{GOMA} improves the energy--delay product (EDP) by $2.24$--$4.24\times$ over SOTA mappers, while accelerating time-to-solution by $3.83$--$73.6\times$.

cs.AR↗