Searcharxiv⌕ Search

arXiv subjects

Shenhui Zhang

Publications and source records attributed to Shenhui Zhang.

3 recordsLinked to original sources

CALFA-KV: Joint Calibration and Fisher-Guided Anchors for 1-Bit KV Cache Compression

1-bit key-value (KV) cache compression can combine quantizer calibration with selective high-precision storage to reduce reconstruction error. We propose Calibration and Fisher-guided Anchors for KV cache compression (CALFA-KV). Offline, we jointly fit a key center, radial normalization parameters, and K/V vector codebooks to match the frozen model's projected attention output. We also estimate channel-wise Fisher-type sensitivities from squared gradients of a reference next-token cross-entropy loss. Online, we use these sensitivities to weight the squared reconstruction errors of each KV state. Here, a KV state is one token's key-value pair within one layer and KV head. We retain the highest-scoring states at high precision under a per-layer, per-KV-head budget. Under exact restoration, we bound the excess risk of score-based anchor selection relative to the optimum at the same anchor budget, where risk is the conditional mean square of the first-order loss change. With a budget of 56 additional FP16 anchors per layer and KV head, CALFA-KV (d8b8) outperforms all evaluated nominal 1-bit baselines on the mean score across seven benchmarks for each of the three models. At the same anchor budget, CALFA-KV (d8b10) uses a nominal index rate of 1.25 bits per channel and narrows the gap to BF16 to 0.90--2.01 points on the same metric.

cs.LG↗

Watch Before You Answer: Learning from Visually Grounded Post-Training

It is critical for vision-language models (VLMs) to comprehensively understand visual, temporal, and textual cues. However, despite rapid progress in multimodal modeling, video understanding performance still lags behind text-based reasoning. In this work, we find that progress is even worse than previously assumed: commonly reported long video understanding benchmarks contain 40-60% of questions that can be answered using text cues alone. Furthermore, we find that these issues are also pervasive in widely used post-training datasets, potentially undercutting the ability of post-training to improve VLM video understanding performance. Guided by this observation, we introduce VidGround as a simple yet effective solution: using only the actual visually grounded questions without any linguistic biases for post-training. When used in tandem with RL-based post-training algorithms, this simple technique improves performance by up to 6.2 points relative to using the full dataset, while using only 69.1% of the original post-training data. Moreover, we show that data curation with a simple post-training algorithm outperforms several more complex post-training techniques, highlighting that data quality is a major bottleneck for improving video understanding in VLMs. These results underscore the importance of curating post-training data and evaluation benchmarks that truly require visual grounding to advance the development of more capable VLMs. Project page: http://vidground.etuagi.com.

cs.CV↗

AuthROS: Secure Data Sharing Among Robot Operating Systems Based on Ethereum

The Robot Operating System (ROS) streamlines human processes, increasing the efficiency of various production tasks. However, the security of data transfer operations in ROS is still in its immaturity. Securing data exchange between several robots is a significant problem. This paper proposes \textit{AuthROS}, an Ethereum blockchain-based secure data sharing method, for robot communication. It is a ROS node authorization system capable of ensuring the immutability and security of private data flow between ROS nodes of any size. To ensure data security, AuthROS employs the smart contract for permission granting and identification, SM2-based key exchange, and SM4-based plaintext encryption techniques. In addition, we deploy a data digest upload technique to optimize data query and upload performance. Finally, the experimental findings reveal that AuthROS has strong security, time performance, and node forging in cases where data should be recorded and robots need to remain immobile.

cs.RO↗