arXiv subjects

Alexey Frolov

Publications and source records attributed to Alexey Frolov.

At least 19 recordsLinked to original sources

arXiv2026

Learning LDPC codes with quantized density evolution over relaxed protographs

We consider the design of low-density parity-check (LDPC) codes for a given iterative decoder. Despite tools such as direct simulation, density evolution (DE), and EXIT-chart analysis, selecting a parity-check matrix remains a difficult combinatorial optimization problem. Existing approaches often rely on population-based search, random mutations, genetic algorithms, or related heuristics, which require careful parameter tuning and may be computationally expensive. Recent gradient descent (GD)-based methods optimize relaxed parity-check matrices by differentiating through decoder simulations. However, such decoder-in-the-loop strategies rely on noisy Monte Carlo estimates, require line search over soft matrix representations, and remain costly for long LDPC codes. Moreover, although optimization is performed in a relaxed domain, the loss is typically evaluated only at integer-valued parity-check matrices. In this work, we focus on the design of long protograph-based LDPC codes and propose a deterministic GD-based framework that operates directly on a relaxed protograph representation. Each protograph entry is interpreted as the probability that the corresponding element is equal to one. The loss function is based on DE bit error rate (BER) performance and can be evaluated directly for relaxed protographs. To justify this relaxation, we associate the relaxed representation with an ensemble of binary protographs and show that the proposed relaxed DE gives the ensemble-averaged DE performance. The resulting optimization procedure is fully autonomous and uses standard GD methods. Owing to deterministic DE evaluation and informative gradients, the proposed approach provides fast and reliable convergence. Numerical experiments for the min-sum decoder show that the optimized protographs outperform 5G LDPC codes with the same protograph dimensions.

cs.IT↗

arXiv2026

Towards Diverse and Comprehensive Benchmarks for Mutual Information Estimation

Mutual information (MI) estimation is a central problem in machine learning and statistics; however, existing benchmarks typically evaluate estimators on simplified, low-dimensional distributions, leaving their performance on complex, realistic data largely unexplored. We address this gap with a comprehensive benchmarking framework grounded in a unified copula-theoretic perspective that subsumes existing benchmarks as special cases. Within this framework, we propose two complementary families of tests: a copula-first family that systematically varies ground-truth MI, dimensionality, and marginal complexity using synthetic and flow-based transformations; and a marginals-first family that couples real-world image data with controlled dependency structures, extending the classic same-class-pairing paradigm. We use this suite to extensively evaluate three classes of estimators: non-parametric, discriminative, and generative. Contrary to prevailing assumptions, our results indicate that there is no universal winner: each category can systematically outperform all other estimators under specific setups. By analyzing these cases, we identify fundamental estimation barriers and propose new tests that more effectively stress these specific limitations. We share the open source code at https://github.com/VanessB/mutinfo.

cs.LG↗

arXiv2026

Recurrent Transformer-Based Near- and Far-Field THz Wideband Channel Estimation for UM-MIMO

The integration of terahertz communications and ultra-massive multiple-input multiple-output (UM-MIMO) systems in 6G networks is motivated by their ability to enable unprecedented data rates, mitigate spectrum congestion, and enhance overall network performance. However, the enlarged antenna apertures and higher carrier frequencies in these systems increase the Rayleigh distance, causing users to span both the near-field and conventional far-field regions. Accurate spatial precoding thus requires exact channel estimation at the base station - a task made more challenging by the hybrid coexistence of near- and far-field effects and the limited number of digital chains available in hybrid beamforming architectures. In this paper, we propose a block recurrent transformer model to address this challenge. We demonstrate that a single transformer block equipped with state memory can be trained once and then iteratively applied for hybrid-field channel estimation. Furthermore, we train the model such that it generalizes to wireless channels with varying scatterer distances, different numbers of propagation paths, and wideband operation. Simulation results show that the proposed method achieves performance gains of approximately 5 dB and 7.5 dB in normalized mean squared error (NMSE) over state-of-the-art solutions in narrowband and wideband scenarios, respectively.

eess.SP↗

arXiv2026

A Log-Domain Approximation of SOCS Decoding for Turbo Product Codes

This paper studies low-complexity soft-output decoding of turbo product codes with extended Bose--Chaudhuri--Hocquenghem component codes. Recent soft-output from covered space (SOCS) decoding substantially improves the quality of extrinsic information compared with the conventional Chase--Pyndiah decoder, but its probabilistic-domain implementation is less attractive for hardware-oriented realizations. We therefore propose a log-domain approximation of SOCS based on max-log approach. The proposed soft-input soft-output rule replaces probability-domain operations with a piecewise-linear function of reliability gaps between competing Chase-II decoding list and out of the list hypotheses, which preserves compatibility with the standard iterative TPC decoding loop. Numerical results for a TPC built from (256,239) eBCH component codes show that the proposed decoder clearly outperforms the baseline Chase--Pyndiah decoder with the same list size and approaches the performance of SOCS decoder.

cs.IT↗

arXiv2025

FMMI: Flow Matching Mutual Information Estimation

We introduce a novel Mutual Information (MI) estimator that fundamentally reframes the discriminative approach. Instead of training a classifier to discriminate between joint and marginal distributions, we learn a normalizing flow that transforms one into the other. This technique produces a computationally efficient and precise MI estimate that scales well to high dimensions and across a wide range of ground-truth MI values.

cs.LG↗

arXiv2025

Unsourced Random Access

Current wireless networks are designed to optimize spectral efficiency for human users, who typically require sustained connections for high-data-rate applications like file transfers and video streaming. However, these networks are increasingly inadequate for the emerging era of machine-type communications (MTC). With a vast number of devices exhibiting sporadic traffic patterns consisting of short packets, the grant-based multiple access procedures utilized by existing networks lead to significant delays and inefficiencies. To address this issue the unsourced random access (URA) paradigm has been proposed. This paradigm assumes the devices to share a common encoder thus simplifying the reception process by eliminating the identification procedure. The URA paradigm not only addresses the computational challenges but it also considers the random access (RA) as a coding problem, i.e., takes into account both medium access protocols and physical layer effects. In this monograph we provide a comprehensive overview of the URA problem in noisy channels, with the main task being to explain the major ideas rather than to list all existing solutions.

cs.IT↗

arXiv2025

Iterative Neural Rollback Chase-Pyndiah Decoding

Iterative decoding is essential in modern communication systems, especially optical communications, where error-correcting codes such as turbo product codes (TPC) and staircase codes are widely employed. A key factor in achieving high error correction performance is the use of soft-decision decoding for component codes. However, implementing optimal maximum a posteriori (MAP) probability decoding for commonly used component codes, such as BCH and Polar codes, is computationally prohibitive. Instead, practical systems rely on approximations, with the Chase-Pyndiah algorithm being a widely used suboptimal method. TPC are more powerful than their component codes and begin to function effectively at low signal-to-noise ratios. Consequently, during the initial iterations, the component codes do not perform well and introduce errors in the extrinsic information updates. This phenomenon limits the performance of TPC. This paper proposes a neural network-aided rollback Chase-Pyndiah decoding method to address this issue. A transformer-based neural network identifies cases where extrinsic updates are likely to introduce errors, triggering a rollback mechanism which prevents the update and keeps the component code message intact. Our results demonstrate that a neural network with a relatively small number of parameters can effectively distinguish destructive updates and improve decoding performance. We evaluate the proposed approach using TPC with (256, 239) extended BCH component codes. We show that the proposed method enhances the bit error rate performance of Chase-Pyndiah p=6 decoding, achieving a gain of approximately 0.145 dB in a TPC scheme with four full iterations, significantly outperforming conventional Chase p=7 decoding.

cs.IT↗

arXiv2025

Curse of Slicing: Why Sliced Mutual Information is a Deceptive Measure of Statistical Dependence

Sliced Mutual Information (SMI) is widely used as a scalable alternative to mutual information for measuring non-linear statistical dependence. Despite its advantages, such as faster convergence, robustness to high dimensionality, and nullification only under statistical independence, we demonstrate that SMI is highly susceptible to data manipulation and exhibits counterintuitive behavior. Through extensive benchmarking and theoretical analysis, we show that SMI saturates easily, fails to detect increases in statistical dependence, prioritizes redundancy over informative content, and in some cases, performs worse than correlation coefficient.

cs.LG↗

arXiv2025

Generalized LDPC codes with low-complexity decoding and fast convergence

We consider generalized low-density parity-check (GLDPC) codes with component codes that are duals of Cordaro-Wagner codes. Two efficient decoding algorithms are proposed: one based on Hartmann-Rudolph processing, analogous to Sum-Product decoding, and another based on evaluating two hypotheses per bit, referred to as the Min-Sum decoder. Both algorithms are derived using latent variables and an appropriate message-passing schedule. A quantized, protograph-based density evolution procedure is used to optimize GLDPC codes for Min-Sum decoding. Compared to 5G LDPC codes, the proposed GLDPC codes offer similar performance at 50 iterations and significantly better convergence and performance at 10 iterations.

cs.IT↗

arXiv2025

Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax

Deep InfoMax (DIM) is a well-established method for self-supervised representation learning (SSRL) based on maximization of the mutual information between the input and the output of a deep neural network encoder. Despite the DIM and contrastive SSRL in general being well-explored, the task of learning representations conforming to a specific distribution (i.e., distribution matching, DM) is still under-addressed. Motivated by the importance of DM to several downstream tasks (including generative modeling, disentanglement, outliers detection and other), we enhance DIM to enable automatic matching of learned representations to a selected prior distribution. To achieve this, we propose injecting an independent noise into the normalized outputs of the encoder, while keeping the same InfoMax training objective. We show that such modification allows for learning uniformly and normally distributed representations, as well as representations of other absolutely continuous distributions. Our approach is tested on various downstream tasks. The results indicate a moderate trade-off between the performance on the downstream tasks and quality of DM.

cs.LG↗

arXiv2024

Mutual Information Estimation via Normalizing Flows

We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which MI is easier to estimate. We additionally explore the target distributions with known closed-form expressions for MI. Theoretical guarantees are provided to demonstrate that our approach yields MI estimates for the original data. Experiments with high-dimensional data are conducted to highlight the practical advantages of the proposed method.

cs.LG↗

arXiv2024

Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression

The Information Bottleneck (IB) principle offers an information-theoretic framework for analyzing the training process of deep neural networks (DNNs). Its essence lies in tracking the dynamics of two mutual information (MI) values: between the hidden layer output and the DNN input/target. According to the hypothesis put forth by Shwartz-Ziv & Tishby (2017), the training process consists of two distinct phases: fitting and compression. The latter phase is believed to account for the good generalization performance exhibited by DNNs. Due to the challenging nature of estimating MI between high-dimensional random vectors, this hypothesis was only partially verified for NNs of tiny sizes or specific types, such as quantized NNs. In this paper, we introduce a framework for conducting IB analysis of general NNs. Our approach leverages the stochastic NN method proposed by Goldfeld et al. (2019) and incorporates a compression step to overcome the obstacles associated with high dimensionality. In other words, we estimate the MI between the compressed representations of high-dimensional random vectors. The proposed method is supported by both theoretical and practical justifications. Notably, we demonstrate the accuracy of our estimator through synthetic experiments featuring predefined MI values and comparison with MINE (Belghazi et al., 2018). Finally, we perform IB analysis on a close-to-real-scale convolutional DNN, which reveals new features of the MI dynamics.

cs.LG↗

arXiv2023

On the Error-Reducing Properties of Superposition Codes

Next-generation wireless communication systems impose much stricter requirements for transmission rate, latency, and reliability. The peak data rate of 6G networks should be no less than 1 Tb/s, which is comparable to existing long-haul optical transport networks. It is believed that using long error-correcting codes (ECC) with soft-decision decoding (SDD) is not feasible in this case due to the resulting high power consumption. On the other hand, ECC with hard-decision decoding (HDD) suffers from significant performance degradation. In this paper, we consider a concatenated solution consisting of an outer long HDD code and an inner short SDD code. The latter code is a crucial component of the system and the focus of our research. Due to its short length, the code cannot correct all errors, but it is designed to minimize the number of errors. Such codes are known as error-reducing codes. We investigate the error-reducing properties of superposition codes. Initially, we explore sparse regression codes (SPARCs) with Gaussian signals. This approach outperforms error-reducing binary LDPC codes optimized by Barakatain, et al. (2018) in terms of performance but faces limitations in practical applicability due to high implementation complexity. Subsequently, we propose an LDPC-based superposition code scheme with low-complexity soft successive interference cancellation (SIC) decoding. This scheme demonstrates comparable performance to SPARCs while maintaining manageable complexity. Numerical results were obtained for inner codes with an overhead (OH) of 8.24% within a concatenated scheme (15% OH) with an outer hard-decision decoded staircase code (6.25% OH).

cs.IT↗

arXiv2023

Soft-Output Deep Neural Network-Based Decoding

Deep neural network (DNN)-based channel decoding is widely considered in the literature. The existing solutions are investigated for the case of hard output, i.e. when the decoder returns the estimated information word. At the same time, soft-output decoding is of critical importance for iterative receivers and decoders. In this paper, we focus on the soft-output DNN-based decoding problem. We start with the syndrome-based approach proposed by Bennatan et al. (2018) and modify it to provide soft output in the AWGN channel. The new decoder can be considered as an approximation of the MAP decoder with smaller computation complexity. We discuss various regularization functions for joint DNN-MAP training and compare the resulting distributions for [64, 45] BCH code. Finally, to demonstrate the soft-output quality we consider the turbo-product code with [64, 45] BCH codes as row and column codes. We show that the resulting DNN-based scheme is very close to the MAP-based performance and significantly outperforms the solution based on the Chase decoder. We come to the conclusion that the new method is prospective for the challenging problem of DNN-based decoding of long codes consisting of short component codes.

cs.IT↗

arXiv2023

Energy Efficiency of Unsourced Random Access over the Binary-Input Gaussian Channel

We investigate the fundamental limits of the unsourced random access over the binary-input Gaussian channel. By fundamental limits, we mean the minimal energy per bit required to achieve the target per-user probability of error. The original method proposed by Y. Polyanskiy (2017) and based on Gallager's trick does not work well for binary signaling. We utilize Fano's method, which is based on the choice of the so-called ``good'' region. We apply this method for the cases of Gaussian and binary codebooks and obtain two achievability bounds. The first bound is very close to Polyanskiy's bound but does not lead to any improvement. At the same time, the numerical results show that the bound for the binary case practically coincides with the bound for the Gaussian codebook. Thus, we conclude that binary modulation does not lead to performance degradation, and energy-efficient schemes with binary modulation do exist.

cs.IT↗

arXiv2023

Unsourced Random Access with the MIMO Receiver: Projection Decoding Analysis

We consider unsourced random access with MIMO receiver - a crucial communication scenario for future 5G/6G wireless networks. We perform a projection-based decoder analysis and derive energy efficiency achievability bounds when channel state information is unknown at transmitters and the receiver (no-CSI scenario). The comparison to the maximum-likelihood (ML) achievability bounds by Gao et al. (2023) is performed. We show that there is a region where the new bound outperforms the ML bound. The latter fact should not surprise the reader as both decoding criteria are suboptimal when considering per-user probability of error (PUPE). Moreover, transition to projection decoding allows for significant dimensionality reduction, which greatly reduces the computation time.

cs.IT↗

arXiv2023

Federated Privacy-preserving Collaborative Filtering for On-Device Next App Prediction

In this study, we propose a novel SeqMF model to solve the problem of predicting the next app launch during mobile device usage. Although this problem can be represented as a classical collaborative filtering problem, it requires proper modification since the data are sequential, the user feedback is distributed among devices and the transmission of users' data to aggregate common patterns must be protected against leakage. According to such requirements, we modify the structure of the classical matrix factorization model and update the training procedure to sequential learning. Since the data about user experience are distributed among devices, the federated learning setup is used to train the proposed sequential matrix factorization model. One more ingredient of the proposed approach is a new privacy mechanism that guarantees the protection of the sent data from the users to the remote server. To demonstrate the efficiency of the proposed model we use publicly available mobile user behavior data. We compare our model with sequential rules and models based on the frequency of app launches. The comparison is conducted in static and dynamic environments. The static environment evaluates how our model processes sequential data compared to competitors. Therefore, the standard train-validation-test evaluation procedure is used. The dynamic environment emulates the real-world scenario, where users generate new data by running apps on devices, and evaluates our model in this case. Our experiments show that the proposed model provides comparable quality with other methods in the static environment. However, more importantly, our method achieves a better privacy-utility trade-off than competitors in the dynamic environment, which provides more accurate simulations of real-world usage.

cs.IR↗

arXiv2022

Coded Compressed Sensing with List Recoverable Codes for the Unsourced Random Access

We consider a coded compressed sensing approach for the unsourced random access and replace the outer tree code proposed by Amalladinne et al. with the list recoverable code capable of correcting t errors. A finite-length random coding bound for such codes is derived. The numerical experiments in the single antenna quasi-static Rayleigh fading MAC show that transition to list recoverable codes correcting t errors improves the performance of coded compressed sensing scheme by 7-10 dB compared to the tree code-based scheme. We propose two practical constructions of outer codes. The first is a modification of the tree code. It utilizes the same code structure, and a key difference is a decoder capable of correcting up to t errors. The second is based on the Reed-Solomon codes and Guruswami-Sudan list decoding algorithm. The first scheme provides an energy efficiency very close to the random coding bound when the decoding complexity is unbounded. But for the practical parameters, the second scheme is better and improves the performance of a tree code-based scheme when the number of active users is less than 200.

cs.IT↗