arXiv subjects

Huixia Ben

Publications and source records attributed to Huixia Ben.

5 recordsLinked to original sources

arXiv2026

BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration

Training-free feature caching accelerates diffusion transformer (DiT) inference by reusing or forecasting intermediate features. However, fixed schedules make compute predictable but ignore prompt- and timestep-dependent risk, whereas hand-tuned error thresholds adapt locally but leave realized compute difficult to control. To address these limitations, we present Budgeted Cache Refresh (BeCARE), a training-free framework that, given a user-specified cap on full computations, adaptively determines when to refresh the cache during accelerated inference. Specifically, we first derive an error-amplification profile from the sampler's noise schedule to characterize the varying impact of approximation errors across denoising timesteps. We then combine this profile with prompt-specific extrapolation residuals and cache age to form an accumulated risk score, triggering a refresh when continued caching becomes harmful. Meanwhile, we use the same profile to construct an analytic spending reference and adjust the refresh threshold through feedback, thereby distributing the limited refresh budget over the sampling trajectory. Finally, structural safeguards (a fixed warmup, a budget-derived late-stage reserve, and a maximum cache-reuse length) prevent unreliable extrapolation and premature budget exhaustion. These designs enable prompt-adaptive refresh placement and allow a single calibration to transfer across budget tiers without per-tier tuning. Experiments on FLUX.1-dev with 200 prompts and measured FLOPs show that our method consistently outperforms representative training-free caching baselines across speedups from 3x to 6x. At the main operating point (3.3x acceleration), it improves paired PSNR by 2.7 dB over the strongest baseline using no more compute. On SD3.5 Large, it also substantially improves the same Taylor cache engine over fixed-interval scheduling at matched compute.

cs.CV↗

arXiv2025

Accelerating Controllable Generation via Hybrid-grained Cache

Controllable generative models have been widely used to improve the realism of synthetic visual content. However, such models must handle control conditions and content generation computational requirements, resulting in generally low generation efficiency. To address this issue, we propose a Hybrid-Grained Cache (HGC) approach that reduces computational overhead by adopting cache strategies with different granularities at different computational stages. Specifically, (1) we use a coarse-grained cache (block-level) based on feature reuse to dynamically bypass redundant computations in encoder-decoder blocks between each step of model reasoning. (2) We design a fine-grained cache (prompt-level) that acts within a module, where the fine-grained cache reuses cross-attention maps within consecutive reasoning steps and extends them to the corresponding module computations of adjacent steps. These caches of different granularities can be seamlessly integrated into each computational link of the controllable generation process. We verify the effectiveness of HGC on four benchmark datasets, especially its advantages in balancing generation efficiency and visual quality. For example, on the COCO-Stuff segmentation benchmark, our HGC significantly reduces the computational cost (MACs) by 63% (from 18.22T to 6.70T), while keeping the loss of semantic fidelity (quantized performance degradation) within 1.5%.

cs.CV↗

arXiv2025

StgcDiff: Spatial-Temporal Graph Condition Diffusion for Sign Language Transition Generation

Sign language transition generation seeks to convert discrete sign language segments into continuous sign videos by synthesizing smooth transitions. However,most existing methods merely concatenate isolated signs, resulting in poor visual coherence and semantic accuracy in the generated videos. Unlike textual languages,sign language is inherently rich in spatial-temporal cues, making it more complex to model. To address this,we propose StgcDiff, a graph-based conditional diffusion framework that generates smooth transitions between discrete signs by capturing the unique spatial-temporal dependencies of sign language. Specifically, we first train an encoder-decoder architecture to learn a structure-aware representation of spatial-temporal skeleton sequences. Next, we optimize a diffusion denoiser conditioned on the representations learned by the pre-trained encoder, which is tasked with predicting transition frames from noise. Additionally, we design the Sign-GCN module as the key component in our framework, which effectively models the spatial-temporal features. Extensive experiments conducted on the PHOENIX14T, USTC-CSL100,and USTC-SLR500 datasets demonstrate the superior performance of our method.

cs.CV↗

arXiv2024

Hierarchical Space-Time Attention for Micro-Expression Recognition

Micro-expression recognition (MER) aims to recognize the short and subtle facial movements from the Micro-expression (ME) video clips, which reveal real emotions. Recent MER methods mostly only utilize special frames from ME video clips or extract optical flow from these special frames. However, they neglect the relationship between movements and space-time, while facial cues are hidden within these relationships. To solve this issue, we propose the Hierarchical Space-Time Attention (HSTA). Specifically, we first process ME video frames and special frames or data parallelly by our cascaded Unimodal Space-Time Attention (USTA) to establish connections between subtle facial movements and specific facial areas. Then, we design Crossmodal Space-Time Attention (CSTA) to achieve a higher-quality fusion for crossmodal data. Finally, we hierarchically integrate USTA and CSTA to grasp the deeper facial cues. Our model emphasizes temporal modeling without neglecting the processing of special data, and it fuses the contents in different modalities while maintaining their respective uniqueness. Extensive experiments on the four benchmarks show the effectiveness of our proposed HSTA. Specifically, compared with the latest method on the CASME3 dataset, it achieves about 3% score improvement in seven-category classification.

cs.CV↗

arXiv2022

Image Captioning via Compact Bidirectional Architecture

Most current image captioning models typically generate captions from left-to-right. This unidirectional property makes them can only leverage past context but not future context. Though refinement-based models can exploit both past and future context by generating a new caption in the second stage based on pre-retrieved or pre-generated captions in the first stage, the decoder of these models generally consists of two networks~(i.e. a retriever or captioner in the first stage and a captioner in the second stage), which can only be executed sequentially. In this paper, we introduce a Compact Bidirectional Transformer model for image captioning that can leverage bidirectional context implicitly and explicitly while the decoder can be executed parallelly. Specifically, it is implemented by tightly coupling left-to-right(L2R) and right-to-left(R2L) flows into a single compact model to serve as a regularization for implicitly exploiting bidirectional context and optionally allowing explicit interaction of the bidirectional flows, while the final caption is chosen from either L2R or R2L flow in a sentence-level ensemble manner. We conduct extensive ablation studies on MSCOCO benchmark and find that the compact bidirectional architecture and the sentence-level ensemble play more important roles than the explicit interaction mechanism. By combining with word-level ensemble seamlessly, the effect of sentence-level ensemble is further enlarged. We further extend the conventional one-flow self-critical training to the two-flows version under this architecture and achieve new state-of-the-art results in comparison with non-vision-language-pretraining models. Finally, we verify the generality of this compact bidirectional architecture by extending it to LSTM backbone. Source code is available at https://github.com/YuanEZhou/cbtic.

cs.CV↗