arXiv subjects

Olga Kondrateva

Publications and source records attributed to Olga Kondrateva.

6 recordsLinked to original sources

arXiv2026

DiffLUT-Net: Differentiable Training of FPGA LUT Networks with Learnable Connectivity

Field-programmable gate arrays (FPGAs) enable efficient neural-network inference, but most deployment flows either accelerate multiply-accumulate operations or convert pretrained quantized models into lookup tables (LUTs). We present DiffLUT-Net, an FPGA-native network connected by six-input LUTs that are trained from scratch. We jointly learn the 64 truth-table entries of a LUT and the source to each of its six input ports using a differentiable LUT function relaxation and hardware source selection. After training, the truth tables and connections are discretized, unused logic can be pruned, and the network is exported directly as synthesizable Verilog. Across five benchmarks, DiffLUT-Net achieves favorable accuracy-resource trade-offs. These results demonstrate the effectiveness of jointly learning LUT functions and sparse connectivity for compact FPGA-native inference. The code is available at https://github.com/TUDa-HWAI/DiffLUT-Network.

cs.LG↗

arXiv2026

CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

Long-context large language model (LLM) inference is increasingly constrained by the memory footprint and decoding cost of key-value (KV) caches, limiting sustainable deployment on resource-constrained hardware. Existing KV cache eviction methods typically apply heuristic token scoring over all heads in GQA-based LLMs. These methods ignore the different functionalities of attention heads, leading to the eviction of critical tokens and thus degrading the performance of LLMs. To address this issue, we propose CompressKV, a resource-efficient KV-cache compression framework for GQA-based LLMs. Instead of aggregating attention scores from all heads, CompressKV identifies Semantic Retrieval Heads (SRHs) that capture both the initial and final tokens of a prompt and semantically important mid-context evidence, and uses them to select tokens whose KV pairs should be retained. Furthermore, CompressKV allocates cache budgets across layers according to offline estimates of layer-wise eviction error. Experiments on LongBench and Needle-in-a-Haystack show that CompressKV consistently outperforms existing KV-cache eviction methods across memory budgets. Notably, it preserves over 97\% of full-cache performance using only 3\% of the KV cache on LongBench question-answering tasks and achieves 90\% accuracy with just 0.7\% KV storage on Needle-in-a-Haystack. These results demonstrate an improved resource--performance trade-off for long-context LLM inference. Our code is publicly available at: https://github.com/TUDa-HWAI/CompressKV

cs.AI↗

arXiv2025

CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation

Recent advances in large language models (LLMs) have significantly boosted long-context processing. However, the increasing key-value (KV) cache size poses critical challenges to memory and execution efficiency. Most KV cache compression methods rely on heuristic token eviction using all attention heads in Grouped Query Attention (GQA)-based LLMs. This method ignores the different functionalities of attention heads, leading to the eviction of critical tokens and thus degrades the performance of LLMs. To address the issue above, instead of using all the attention heads in GQA-based LLMs to determine important tokens as in the previous work, we first identify the attention heads in each layer that are not only capable of retrieving the initial and final tokens of a prompt, but also capable of retrieving important tokens within the text and attending to their surrounding semantic context. Afterwards, we exploit such heads to determine the important tokens and retain their corresponding KV cache pairs. Furthermore, we analyze the cache eviction error of each layer individually and introduce a layer-adaptive KV cache allocation strategy. Experimental results demonstrate the proposed CompressKV consistently outperforms state-of-the-art approaches under various memory budgets on LongBench and Needle-in-a-Haystack benchmarks. Our code is publicly available at: https://github.com/TUDa-HWAI/CompressKV.git.

cs.CL↗

arXiv2025

Deep Joint Source-Channel Coding for Small Satellite Applications

Small satellites used for Earth observation generate vast amounts of high-dimensional data, but their operation in low Earth orbit creates a significant communication bottleneck due to limited contact times and harsh, varying channel conditions. While deep joint source-channel coding (DJSCC) has emerged as a promising technique, its practical application to the complex satellite environment remains an open question. This paper presents a comprehensive DJSCC framework tailored for satellite communications. We first establish a basic system, DJSCC-SAT, and integrate a realistic, multi-state statistical channel model to guide its training and evaluation. To overcome the impracticality of using separate models for every channel condition, we then introduce an adaptable architecture, ADJSCC-SAT, which leverages attention modules to allow a single neural network to adjust to a wide range of channel states with minimal overhead. Through extensive evaluation on Sentinel-2 multi-spectral data, we demonstrate that our adaptable approach achieves performance comparable to using multiple specialized networks while significantly reducing model storage requirements. Furthermore, the adaptable model shows enhanced robustness to channel estimation errors, outperforming the non-adaptable baseline. The proposed framework is a practical and efficient step toward deploying robust, adaptive DJSCC systems for real-world satellite missions.

cs.NI↗

arXiv2024

Adaptable Deep Joint Source-and-Channel Coding for Small Satellite Applications

Earth observation with small satellites serves a wide range of relevant applications. However, significant advances in sensor technology (e.g., higher resolution, multiple spectrums beyond visible light) in combination with challenging channel characteristics lead to a communication bottleneck when transmitting the collected data to Earth. Recently, joint source coding, channel coding, and modulation based on neuronal networks has been proposed to combine image compression and communication. Though this approach achieves promising results when applied to standard terrestrial channel models, it remains an open question whether it is suitable for the more complicated and quickly varying satellite communication channel. In this paper, we consider a detailed satellite channel model accounting for different shadowing conditions and train an encoder-decoder architecture with realistic Sentinel-2 satellite imagery. In addition, to reduce the overhead associated with applying multiple neural networks for various channel states, we leverage attention modules and train a single adaptable neural network that covers a wide range of different channel conditions. Our evaluation results show that the proposed approach achieves similar performance when compared to less space-efficient schemes that utilize separate neuronal networks for differing channel conditions.

cs.NI↗

arXiv2024

Joint Source-and-Channel Coding for Small Satellite Applications

Small satellites are widely used today as cost effective means to perform Earth observation and other tasks that generate large amounts of high-dimensional data, such as multi-spectral imagery. These satellites typically operate in low earth orbit, which poses significant challenges for data transmission due to short contact times with ground stations, low bandwidth, and high packet loss probabilities. In this paper, we introduce JSCC-Sat, which applies joint source-and-channel coding using neural networks to provide efficient and robust transmission of compressed image data for satellite applications. We evaluate our mechanism against traditional transmission schemes with separate source and channel coding and demonstrate that it outperforms the existing approaches when applied to Earth observation data of the Sentinel-2 mission.

cs.NI↗