arXiv subjects

Carol Jingyi Li

Publications and source records attributed to Carol Jingyi Li.

3 recordsLinked to original sources

arXiv2026

Not All Attention Heads Contribute to Critical Visual Token Selection: Head-Aware Pruning Matters More

Vision-Language Models (VLMs) have exhibited impressive performance across diverse visual scenarios. However, this success comes at the cost of explosive growth in visual tokens, which imposes substantial memory and computational overhead during inference, ultimately increasing latency. To improve VLM inference efficiency, a typical class of visual token pruning methods estimates token importance by aggregating attention scores across all heads in the pruning layer of the Large Language Model (LLM) backbone and prunes tokens based on aggregated scores. However, in this paper, we reveal a compelling phenomenon: the capability to pinpoint critical visual tokens is concentrated within a small fraction of heads. Aggregation exclusively on these heads can improve task performance. Inspired by this observation, we propose ProViP, a training-free progressive visual token pruning framework. ProViP first removes redundant visual tokens based on the embedding similarity of input tokens before reasoning of the LLM backbone, and then further prunes tokens during reasoning via head-aware pruning. Experiments demonstrate that ProViP delivers outstanding task performance and inference efficiency. For instance, when applied to LLaVA-1.5-7B, ProViP retains 95.9% of the original performance and achieves 1.62x inference speedup under an 88.9% pruning ratio.

cs.CV↗

arXiv2026

VersaQ-3D: Architecture Support for Visual Geometry Grounded Transformers via Versatile Quantization

3D reconstruction and view synthesis are fundamental to AR/VR, robotics, and digital twins. The Visual Geometry Grounded Transformer (VGGT) enables strong feed-forward 3D reconstruction while its billion-parameter scale limits on-device deployment. LLM-oriented quantization methods fail on VGGT due to saturated activation channels that resist low-bit quantization and diverse 3D semantics that impede calibration. VGGT further poses hardware challenges from multi-precision architecture support and long-sequence global attention with excessive memory demands. We propose VersaQ-3D, an algorithm-architecture co-design framework for efficient VGGT inference. At the algorithm level, we present the first calibration-free, input-agnostic quantization method for VGGT, leveraging transform coding to suppress outliers and preserve structural weight features, enabling robust low-bit inference down to 4 bits. At the architecture level, we design a reconfigurable accelerator with a hierarchical multi-precision compute unit (BF16/INT8/INT4) that executes both linear and non-linear operators within a shared systolic datapath, reducing end-to-end latency by 77%. A two-stage recomputation-based tiling strategy further cuts runtime by 7% by alleviating on-chip memory pressure for long-sequence attention. Evaluations across various datasets show that VersaQ-3D incurs negligible accuracy loss at W4A8 and consistently achieves leading accuracy at W4A4 over prior quantization methods across diverse scenes. The co-designed accelerator delivers 5.4$\times$-22.0$\times$ speedup over edge GPUs and 2.2$\times$-3.0$\times$ over prior quantization-based accelerators under iso-PE-area comparison, enabling instant and energy-efficient feed-forward 3D reconstruction on edge devices.

cs.AR↗

arXiv2025

AMD Versal Implementations of FAM and SSCA Estimators

Cyclostationary analysis is widely used in signal processing, particularly in the analysis of human-made signals, and spectral correlation density (SCD) is often used to characterise cyclostationarity. Unfortunately, for real-time applications, even utilising the fast Fourier transform (FFT), the high computational complexity associated with estimating the SCD limits its applicability. In this work, we present optimised, high-speed field-programmable gate array (FPGA) implementations of two SCD estimation techniques. Specifically, we present an implementation of the FFT accumulation method (FAM) running entirely on the AMD Versal AI engine (AIE) array. We also introduce an efficient implementation of the strip spectral correlation analyser (SSCA) that can be used for window sizes up to $2^{20}$. For both techniques, a generalised methodology is presented to parallelise the computation while respecting memory size and data bandwidth constraints. Compared to an NVIDIA GeForce RTX 3090 graphics processing unit (GPU) which uses a similar 7nm technology to our FPGA, for the same accuracy, our FAM/SSCA implementations achieve speedups of 4.43x/1.90x and a 30.5x/24.5x improvement in energy efficiency.

cs.AR↗