Searcharxiv⌕ Search

arXiv subjects

Devanshu Jha

Publications and source records attributed to Devanshu Jha.

5 recordsLinked to original sources

Verify Before You Fetch: Reliable KV Cache Compression for LLM Inference

Long-context applications, long-lived agents, and multi-agent systems require large language models (LLMs) to process and retain extensive information. In these settings, the KV cache can exceed GPU high-bandwidth memory (HBM) capacity, necessitating offloading to host memory, where CPU-to-GPU transfers can dominate inference latency. While KV cache compression can reduce data movement, query-agnostic compression that guarantees accurate attention for arbitrary queries is extremely challenging, if not impossible. We introduce VerKV, a hybrid approach that provides $(ε, δ)$ guarantees for approximating attention on any given query. VerKV maintains a compact set of query-agnostic pivotal tokens on the GPU, together with a token-level sketch of the full KV cache, while retaining the complete cache in CPU memory. We introduce a novel stochastic sampling procedure, PoissonComp, to select the pivotal tokens. At inference time, our proposed KVFetch algorithm uses the pivotal tokens and the sketch to estimate the relative error of the resulting attention computation. If the estimate satisfies a user-specified $(ε, δ)$ tolerance, inference proceeds without accessing host memory. Otherwise, VerKV performs query-aware retrieval to fetch additional tokens from the CPU, refining the attention computation to meet the required tolerance. Thus, VerKV enables attention with bounded approximation error while minimizing CPU-to-GPU transfers. We show that PoissonComp outperforms state-of-the-art approaches to KV cache compression. Combined with KVFetch, it achieves near-dense performance (90-100% of dense quality at 32K context length, averaged over RULER and LOFT), using Qwen3-4B, Qwen3.5-9B and Ministral-3-8B, while reducing the GPU memory footprint by 8-32x and keeping CPU-to-GPU memory fetches within 2-23% of the cache per generated token, depending on the model.

cs.AI↗

Final Report for SAG 22: A Target Star Archive for Exoplanet Science

Present and upcoming NASA missions will be intensively observing a selected, partially overlapping set of stars for exoplanet studies. Key physical and chemical information about these stars and their systems is needed for planning observations and interpreting the results. A target star archive of such data would benefit a wide cross-section of the exoplanet community by enhancing the chances of mission success and improving the efficiency of mission observatories. It would also provide a common, accessible resource for scientific analysis based on standardized assumptions, while revealing gaps or deficiencies in existing knowledge of stellar properties necessary for exoplanetary system characterization.

astro-ph.IM↗

Captured Small Solar System Bodies in the Ice Giant Region

This white paper advocates for the inclusion of small, captured Outer Solar system objects, found in the Ice Giant region in the next Decadal Survey. These objects include the Trojans and Irregular satellite populations of Uranus and Neptune. The captured small bodies provide vital clues as to the formation of our Solar system. They have unique dynamical situations, which any model of Solar system formation needs to explain. The major issue is that so few of these objects have been discovered, with very little information known about them. The purpose of this document is to prioritize further discovery and characterization of these objects. This will require the use of NASA and NSF facilities over the 2023 2032 decade, including additional support for analysis. This is in preparation for potential future insitu missions in the following decades.

astro-ph.IM↗

Small Bodies Tell the Story of the Solar System: A Scientific Rationale for a Multi-Target Small Body Sample Return Program including the Earth-based Laboratory Analysis of Returned Samples

Small bodies are time-capsules of different eras of solar system history from the most primitive materials within the solar system to evolved pieces of larger bodies. A small body sample return program is an essential component of small body exploration, and such a program should include opportunities for both missions and laboratory analysis.

astro-ph.IM↗

The case for a multi-channel polarization sensitive LIDAR for investigation of insolation-driven ices and atmospheres

All LIDAR instruments are not the same, and advancement of LIDAR technology requires an ongoing interest and demand from the community to foster further development of the required components. The purpose of this paper is to make the community aware of the need for further technical development, and the potential payoff of investing experimental time, money and thought into the next generation of LIDARs.

astro-ph.IM↗