arXiv subjects

Ziheng Gao

Publications and source records attributed to Ziheng Gao.

3 recordsLinked to original sources

arXiv2024

Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference

Large Language Models (LLMs) inherently use autoregressive decoding, which lacks parallelism in inference and results in significantly slow inference speed. While methods such as Medusa constructs parallelized heads, they lack adequate information interaction across different prediction positions. To overcome this limitation, we introduce Amphista, an enhanced speculative decoding framework that builds upon Medusa. Specifically, Amphista models an Auto-embedding Block capable of parallel inference, incorporating bi-directional attention to enable interaction between different drafting heads. Additionally, Amphista integrates Staged Adaptation Layers, which ensure a seamless transition of semantic information from the target model's autoregressive inference to the drafting heads' non-autoregressive inference, effectively achieving paradigm shift and feature fusion. Experimental results on Vicuna models using MT-Bench and Spec-Bench demonstrate that Amphista achieves substantial acceleration while maintaining generation quality. On MT-Bench, Amphista delivers up to 2.75$\times$ speedup over vanilla autoregressive decoding and 1.40$\times$ over Medusa on Vicuna 33B in wall-clock time.

cs.AI↗

arXiv2024

LADDER: An Efficient Framework for Video Frame Interpolation

Video Frame Interpolation (VFI) is a crucial technique in various applications such as slow-motion generation, frame rate conversion, video frame restoration etc. This paper introduces an efficient video frame interpolation framework that aims to strike a favorable balance between efficiency and quality. Our framework follows a general paradigm consisting of a flow estimator and a refinement module, while incorporating carefully designed components. First of all, we adopt depth-wise convolution with large kernels in the flow estimator that simultaneously reduces the parameters and enhances the receptive field for encoding rich context and handling complex motion. Secondly, diverging from a common design for the refinement module with a UNet-structure (encoder-decoder structure), which we find redundant, our decoder-only refinement module directly enhances the result from coarse to fine features, offering a more efficient process. In addition, to address the challenge of handling high-definition frames, we also introduce an innovative HD-aware augmentation strategy during training, leading to consistent enhancement on HD images. Extensive experiments are conducted on diverse datasets, Vimeo90K, UCF101, Xiph and SNU-FILM. The results demonstrate that our approach achieves state-of-the-art performance with clear improvement while requiring much less FLOPs and parameters, reaching to a better spot for balancing efficiency and quality.

cs.CV↗

arXiv2023

Giant phonon softening and avoided crossing in aliovalence-doped heavy-band thermoelectrics

Aliovalent doping has been adopted to optimize the electrical properties of semiconductors, while its impact on the phonon structure and propagation is seldom paid proper attention to. This work reveals that aliovalent doping can be much more effective in reducing the lattice thermal conductivity of thermoelectric semiconductors than the commonly employed isoelectronic alloying strategy. As demonstrated in the heavy-band NbFeSb system, a large reduction of 65% in the lattice thermal conductivity is achieved through only 10% aliovalent Hf-doping, compared to the 4 times higher isoelectronic Ta-alloying. It is elucidated that aliovalent doping introduces free charge carriers and enhances the screening, leading to the giant softening and deceleration of optical phonons. Moreover, the heavy dopant can induce the avoided-crossing of acoustic and optical phonon branches, further decelerating the acoustic phonons. These results highlight the significant role of aliovalent dopants in regulating the phonon structure and suppressing the phonon propagation of semiconductors.

cond-mat.mtrl-sci↗