arXiv subjects

Huihui Wu

Publications and source records attributed to Huihui Wu.

At least 19 recordsLinked to original sources

arXiv2026

A Survey of Learn-to-Compute Paradigms for Rate-Distortion-Type Problems

Rate-distortion (RD) theory and its related formulations play a central role in understanding efficient information representation, but computing these quantities remains challenging in high-dimensional settings. Classical iterative methods such as the Blahut-Arimoto algorithm become impractical in high-dimensional domains due to the curse of dimensionality and the intractability of mutual-information terms. Recent advances in neural modeling and differentiable optimization offer a promising alternative through a learn-to-compute paradigm, in which probability distributions and objective functionals are represented by flexible neural parameterizations. This survey presents an overview of neural approaches for evaluating the RD-type objectives. We present three representative families of methods: variational inference, neural mutual-information estimation, and dual-form optimization. By reviewing their theoretical principles, algorithmic techniques, and consistency properties, we elucidate how these methods collectively transform classical RD-type problems into scalable differentiable objectives suitable for deep learning, though challenges remain in large-scale applications. Together, these perspectives offer promising avenues for scaling information-theoretic computation to complex, high-dimensional machine learning systems.

cs.IT↗

arXiv2025

An Optimal Transport-Based Method for Computing LM Rate and Its Convergence Analysis

The mismatch capacity characterizes the highest information rate of the channel under a prescribed decoding metric and serves as a critical performance indicator in numerous practical communication scenarios. Compared to the commonly used Generalized Mutual Information (GMI), the Lower bound on the Mismatch capacity (LM rate) generally provides a tighter lower bound on the mismatch capacity. However, the efficient computation of the LM rate is significantly more challenging than that of the GMI, particularly as the size of the channel input alphabet increases. This growth in complexity renders standard numerical methods (e.g., interior point methods) computationally intensive and, in some cases, impractical. In this work, we reformulate the computation of the LM rate as a special instance of the optimal transport (OT) problem with an additional constraint. Building on this formulation, we develop a novel numerical algorithm based on the Sinkhorn algorithm, which is well known for its efficiency in solving entropy regularized optimization problems. We further provide the convergence analysis of the proposed algorithm, revealing that the algorithm has a sub-linear convergence rate. Numerical experiments demonstrate the feasibility and efficiency of the proposed algorithm for the computation of the LM rate.

cs.IT↗

arXiv2025

Efficient Computation of Marton's Error Exponent via Constraint Decoupling

The error exponent in lossy source coding characterizes the asymptotic decay rate of error probability with respect to blocklength. The Marton's error exponent provides the theoretically optimal bound on this rate. However, computation methods of the Marton's error exponent remain underdeveloped due to its formulation as a non-convex optimization problem with limited efficient solvers. While a recent grid search algorithm can compute its inverse function, it incurs prohibitive computational costs from two-dimensional brute-force parameter grid searches. This paper proposes a composite maximization approach that effectively handles both Marton's error exponent and its inverse function. Through a constraint decoupling technique, the resulting problem formulations admit efficient solvers driven by an alternating maximization algorithm. By fixing one parameter via a one-dimensional line search, the remaining subproblem becomes convex and can be efficiently solved by alternating variable updates, thereby significantly reducing search complexity. Therefore, the global convergence of the algorithm can be guaranteed. Numerical experiments for simple sources and the Ahlswede's counterexample, demonstrates the superior efficiency of our algorithm in contrast to existing methods.

cs.IT↗

arXiv2025

Neural Estimation of the Information Bottleneck Based on a Mapping Approach

The information bottleneck (IB) method is a technique designed to extract meaningful information related to one random variable from another random variable, and has found extensive applications in machine learning problems. In this paper, neural network based estimation of the IB problem solution is studied, through the lens of a novel formulation of the IB problem. Via exploiting the inherent structure of the IB functional and leveraging the mapping approach, the proposed formulation of the IB problem involves only a single variable to be optimized, and subsequently is readily amenable to data-driven estimators based on neural networks. A theoretical analysis is conducted to guarantee that the neural estimator asymptotically solves the IB problem, and the numerical experiments on both synthetic and MNIST datasets demonstrate the effectiveness of the neural estimator.

cs.IT↗

arXiv2025

An Efficient Alternating Minimization Algorithm for Computing Quantum Rate-Distortion Function

We consider the computation of the entanglement-assisted quantum rate-distortion function, which plays a central role in quantum information theory. We propose an efficient alternating minimization algorithm based on the Lagrangian analysis. Instead of fixing the multiplier corresponding to the distortion constraint, we update the multiplier in each iteration. Hence the algorithm solves the original problem itself, rather than the Lagrangian relaxation of it. Moreover, all the other variables are iterated in closed form without solving multi-dimensional nonlinear equations or multivariate optimization problems. Numerical experiments show the accuracy of our proposed algorithm and its improved efficiency over existing methods.

cs.IT↗

arXiv2025

Estimating Rate-Distortion Functions Using the Energy-Based Model

The rate-distortion (RD) theory is one of the key concepts in information theory, providing theoretical limits for compression performance and guiding the source coding design, with both theoretical and practical significance. The Blahut-Arimoto (BA) algorithm, as a classical algorithm to compute RD functions, encounters computational challenges when applied to high-dimensional scenarios. In recent years, many neural methods have attempted to compute high-dimensional RD problems from the perspective of implicit generative models. Nevertheless, these approaches often neglect the reconstruction of the optimal conditional distribution or rely on unreasonable prior assumptions. In face of these issues, we propose an innovative energy-based modeling framework that leverages the connection between the RD dual form and the free energy in statistical physics, achieving effective reconstruction of the optimal conditional distribution.The proposed algorithm requires training only a single neural network and circumvents the challenge of computing the normalization factor in energy-based models using the Markov chain Monte Carlo (MCMC) sampling. Experimental results demonstrate the significant effectiveness of the proposed algorithm in estimating high-dimensional RD functions and reconstructing the optimal conditional distribution.

cs.IT↗

arXiv2025

RDD Function: A Tradeoff Between Rate and Distortion-in-Distortion

In this paper, we propose a novel function named Rate Distortion-in-Distortion (RDD) function as an extension of the classical rate-distortion (RD) function, where the expected distortion constraint is replaced by a Gromov-type distortion. This distortion, integral to the Gromov-Wasserstein (GW) distance, effectively defines the similarity in spaces of possibly different dimensions even without a direct metric between them. While the RDD function qualifies as an informational RD function, encoding theorems substantiate its status as an operational RD function, thereby underscoring its potential applicability in real-world source coding. Due to the high computational complexity associated with Gromov-type distortion, in general, the RDD function cannot be evaluated analytically. Consequently, we develop an alternating mirror descent algorithm that significantly reduces computational complexity by employing decomposition, linearization, and relaxation techniques. Numerical results on classical sources and different grids demonstrate the effectiveness of the developed algorithm. By exploring the relationship between the RDD function and the RD function, we suggest that the RDD function may have potential applications in future scenarios.

cs.IT↗

arXiv2024

CRC-Assisted Channel Codes for Integrated Passive Sensing and Communications

We propose a novel coded integrated passive sensing and communication (CIPSAC) system with orthogonal frequency division multiplexing (OFDM), where a multi-antenna base station (BS) passively senses the parameters of the targets and decodes the information bit sequences transmitted by a user. The transmitted signal is comprised of pilot and data OFDM symbols where the data symbols adopt cyclic redundancy check (CRC)-assisted channel codes to facilitate both the decoding and sensing procedures. In the proposed scheme, CRC not only enhances the reliability of communication but also provides guidance to the parameter sensing procedure at the BS. In particular, a novel iterative parameter sensing and channel decoding (IPSCD) algorithm is proposed, where the correctly decoded codewords that pass CRC are utilized for sensing to improve the parameter estimation accuracy, and in return, more accurate parameter estimates lead to a larger number of correctly decoded data symbols. Conventional sensing algorithms rely only on the received pilot signals, while we utilize both the data and pilot signals for sensing. We provide a detailed analysis of the optimal strategy, in which the wrongly decoded data packets are replaced by zero codewords. To further improve the performance, we introduce learning-based near-orthogonal superposition (NOS) codes, which exhibit superior error correction capability especially in the short block length regime. NOS codes are trained using a weighted loss function, where a hyper parameter is introduced to balance the sensing and the communication losses. Simulation results show the effectiveness of the proposed CIPSAC system and the IPSCD algorithm, where both the sensing and decoding performances are significantly improved with a few iterations. We also carry out extensive ablation studies for a comprehensive understanding of the proposed scheme.

eess.SP↗

arXiv2024

Alternating Maximization Algorithm for Mismatch Capacity with Oblivious Relaying

Reliable communication over a discrete memoryless channel with the help of a relay has aroused interest due to its widespread applications in practical scenarios. By considering the system with a mismatched decoder, previous works have provided optimization models to evaluate the mismatch capacity in these scenarios. The proposed models, however, are difficult due to the complicated structure of the mismatched decoding problem with the information flows in hops given by the relay. Existing methods, such as the grid search, become impractical as they involve finding all roots of a nonlinear system, with the growing size of the alphabet. To address this problem, we reformulate the max-min optimization model as a consistent maximization form, by considering the dual form of the inner minimization problem and the Lagrangian with a fixed multiplier. Based on the proposed formulation, an alternating maximization framework is designed, which provides the closed-form solution with simple iterations in each step by introducing a suitable variable transformation. The effectiveness of the proposed approach is demonstrated by the simulations over practical scenarios, including Quaternary and Gaussian channels. Moreover, the simulation results of the transitional probability also shed light on the promising application attribute to the quantizer design in the relay node.

cs.IT↗

arXiv2024

Enhancing Covert Communication in Relay Systems Using Multi-Antenna Technique

This paper exploits the multi-antenna technique to enhance the covert communication performance in a relay system, where a source S conducts covert communication with a destination D via a relay R, subjecting to the detections of transmissions in the two hops from a single-antenna warden W. To demonstrate the performance gain from adopting the multi-antenna technique, we first consider the scenario when S, R and D all adopt single antenna, and apply hypothesis testing and statistics theories to develop a theoretical framework for the covert performance modeling in terms of detection error probability (DEP) and covert throughput. We then consider the scenario when S, R and D all adopt multiple antennas, and apply the hypothesis testing, statistics and matrix theories to develop corresponding theoretical framework for performance modeling. We further explore the optimal designs of the target rate and transmit power for covert throughput maximization under above both scenarios, subjecting to the constraints of covertness, reliability and transmit power. To solve the optimization problems, we employ Karushi-Kuhn-Tucker (KKT) conditions method in the single antenna scenario and a search algorithm in the multi-antenna scenario. Finally, we provide extensive numerical results to illustrate how the multi-antenna technique can enhance the covert performance in two-hop relay systems.

cs.CR↗

arXiv2024

On Convergence of Discrete Schemes for Computing the Rate-Distortion Function of Continuous Source

Computing the rate-distortion function for continuous sources is commonly regarded as a standard continuous optimization problem. When numerically addressing this problem, a typical approach involves discretizing the source space and subsequently solving the associated discrete problem. However, existing literature has predominantly concentrated on the convergence analysis of solving discrete problems, usually neglecting the convergence relationship between the original continuous optimization and its associated discrete counterpart. This neglect is not rigorous, since the solution of a discrete problem does not necessarily imply convergence to the solution of the original continuous problem, especially for non-linear problems. To address this gap, our study employs rigorous mathematical analysis, which constructs a series of finite-dimensional spaces approximating the infinite-dimensional space of the probability measure, establishing that solutions from discrete schemes converge to those from the continuous problems.

cs.IT↗

arXiv2024

A Double Maximization Approach for Optimizing the LM Rate of Mismatched Decoding

An approach is established for maximizing the Lower bound on the Mismatch capacity (hereafter abbreviated as LM rate), a key performance bound in mismatched decoding, by optimizing the channel input probability distribution. Under a fixed channel input probability distribution, the computation of the corresponding LM rate is a convex optimization problem. When optimizing the channel input probability distribution, however, the corresponding optimization problem adopts a max-min formulation, which is generally non-convex and is intractable with standard approaches. To solve this problem, a novel dual form of the LM rate is proposed, thereby transforming the max-min formulation into an equivalent double maximization formulation. This new formulation leads to a maximization problem setup wherein each individual optimization direction is convex. Consequently, an alternating maximization algorithm is established to solve the resultant maximization problem setup. Each step of the algorithm only involves a closed-form iteration, which is efficiently implemented with standard optimization procedures. Numerical experiments show the proposed approach for optimizing the LM rate leads to noticeable rate gains.

cs.IT↗

arXiv2024

An Expectation-Maximization Relaxed Method for Privacy Funnel

The privacy funnel (PF) gives a framework of privacy-preserving data release, where the goal is to release useful data while also limiting the exposure of associated sensitive information. This framework has garnered significant interest due to its broad applications in characterization of the privacy-utility tradeoff. Hence, there is a strong motivation to develop numerical methods with high precision and theoretical convergence guarantees. In this paper, we propose a novel relaxation variant based on Jensen's inequality of the objective function for the computation of the PF problem. This model is proved to be equivalent to the original in terms of optimal solutions and optimal values. Based on our proposed model, we develop an accurate algorithm which only involves closed-form iterations. The convergence of our algorithm is theoretically guaranteed through descent estimation and Pinsker's inequality. Numerical results demonstrate the effectiveness of our proposed algorithm.

cs.IT↗

arXiv2024

Efficient and Provably Convergent Computation of Information Bottleneck: A Semi-Relaxed Approach

Information Bottleneck (IB) is a technique to extract information about one target random variable through another relevant random variable. This technique has garnered significant interest due to its broad applications in information theory and deep learning. Hence, there is a strong motivation to develop efficient numerical methods with high precision and theoretical convergence guarantees. In this paper, we propose a semi-relaxed IB model, where the Markov chain and transition probability condition are relaxed from the relevance-compression function. Based on the proposed model, we develop an algorithm, which recovers the relaxed constraints and involves only closed-form iterations. Specifically, the algorithm is obtained by analyzing the Lagrangian of the relaxed model with alternating minimization in each direction. The convergence property of the proposed algorithm is theoretically guaranteed through descent estimation and Pinsker's inequality. Numerical experiments across classical and discrete distributions corroborate the analysis. Moreover, our proposed algorithm demonstrates notable advantages in terms of computational efficiency, evidenced by significantly reduced run times compared to existing methods with comparable accuracy.

cs.IT↗

arXiv2024

Integrated Sensing and Communications in Clutter Environment

In this paper, we propose a practical integrated sensing and communications (ISAC) framework to sense dynamic targets from clutter environment while ensuring users communications quality. To implement communications function and sensing function simultaneously, we design multiple communications beams that can communicate with the users as well as one sensing beam that can rotate and scan the entire space. To minimize the interference of sensing beam on existing communications systems, we divide the service area into sensing beam for sensing (S4S) sector and communications beam for sensing (C4S) sector, and provide beamforming design and power allocation optimization strategies for each type sector. Unlike most existing ISAC studies that ignore the interference of static environmental clutter on target sensing, we construct a mixed sensing channel model that includes both static environment and dynamic targets. When base station receives the echo signals, the mean phasor cancellation (MPC) method is employed to filter out the interference from static environmental clutter and to extract the effective dynamic target echoes. Then a complete and practical dynamic target sensing scheme is designed to detect the presence of dynamic targets and to estimate their angles, distances, and velocities. In particular, dynamic target detection and angle estimation are realized through angle-Doppler spectrum estimation (ADSE) and joint detection over multiple subcarriers (MSJD), while distance and velocity estimation are realized through the extended subspace algorithm. Simulation results demonstrate the effectiveness of the proposed scheme and its superiority over the existing methods that ignore environmental clutter.

eess.SP↗

arXiv2024

A Constrained BA Algorithm for Rate-Distortion and Distortion-Rate Functions

The Blahut-Arimoto (BA) algorithm has played a fundamental role in the numerical computation of rate-distortion (RD) functions. This algorithm possesses a desirable monotonic convergence property by alternatively minimizing its Lagrangian with a fixed multiplier. In this paper, we propose a novel modification of the BA algorithm, wherein the multiplier is updated through a one-dimensional root-finding step using a monotonic univariate function, efficiently implemented by Newton's method in each iteration. Consequently, the modified algorithm directly computes the RD function for a given target distortion, without exploring the entire RD curve as in the original BA algorithm. Moreover, this modification presents a versatile framework, applicable to a wide range of problems, including the computation of distortion-rate (DR) functions. Theoretical analysis shows that the outputs of the modified algorithms still converge to the solutions of the RD and DR functions with rate $O(1/n)$, where $n$ is the number of iterations. Additionally, these algorithms provide $\varepsilon$-approximation solutions with $O\left(\frac{MN\log N}{\varepsilon}(1+\log |\log \varepsilon|)\right)$ arithmetic operations, where $M,N$ are the sizes of source and reproduced alphabets respectively. Numerical experiments demonstrate that the modified algorithms exhibit significant acceleration compared with the original BA algorithms and showcase commendable performance across classical source distributions such as discretized Gaussian, Laplacian and uniform sources.

cs.IT↗

arXiv2023

Moving Target Sensing for ISAC Systems in Clutter Environment

In this paper, we consider the moving target sensing problem for integrated sensing and communication (ISAC) systems in clutter environment. Scatterers produce strong clutter, deteriorating the performance of ISAC systems in practice. Given that scatterers are typically stationary and the targets of interest are usually moving, we here focus on sensing the moving targets. Specifically, we adopt a scanning beam to search for moving target candidates. For the received signal in each scan, we employ high-pass filtering in the Doppler domain to suppress the clutter within the echo, thereby identifying candidate moving targets according to the power of filtered signal. Then, we adopt root-MUSIC-based algorithms to estimate the angle, range, and radial velocity of these candidate moving targets. Subsequently, we propose a target detection algorithm to reject false targets. Simulation results validate the effectiveness of these proposed methods.

eess.SP↗

arXiv2023

Information Bottleneck Revisited: Posterior Probability Perspective with Optimal Transport

Information bottleneck (IB) is a paradigm to extract information in one target random variable from another relevant random variable, which has aroused great interest due to its potential to explain deep neural networks in terms of information compression and prediction. Despite its great importance, finding the optimal bottleneck variable involves a difficult nonconvex optimization problem due to the nonconvexity of mutual information constraint. The Blahut-Arimoto algorithm and its variants provide an approach by considering its Lagrangian with fixed Lagrange multiplier. However, only the strictly concave IB curve can be fully obtained by the BA algorithm, which strongly limits its application in machine learning and related fields, as strict concavity cannot be guaranteed in those problems. To overcome the above difficulty, we derive an entropy regularized optimal transport (OT) model for IB problem from a posterior probability perspective. Correspondingly, we use the alternating optimization procedure and generalize the Sinkhorn algorithm to solve the above OT model. The effectiveness and efficiency of our approach are demonstrated via numerical experiments.

cs.IT↗