arXiv subjects

Farhad Mirkarimi

Publications and source records attributed to Farhad Mirkarimi.

5 recordsLinked to original sources

arXiv2026

VBO-MI: A Fully Gradient-Based Bayesian Optimization Framework Using Variational Mutual Information Estimation

Many real-world tasks require optimizing expensive black-box functions accessible only through noisy evaluations, a setting commonly addressed with Bayesian optimization (BO). While Bayesian neural networks (BNNs) have recently emerged as scalable alternatives to Gaussian Processes (GPs), traditional BNN-BO frameworks remain burdened by expensive posterior sampling and acquisition function optimization. In this work, we propose {VBO-MI} (Variational Bayesian Optimization with Mutual Information), a fully gradient-based BO framework that leverages recent advances in variational mutual information estimation. To enable end-to-end gradient flow, we employ an actor-critic architecture consisting of an {action-net} to navigate the input space and a {variational critic} to estimate information gain. This formulation effectively eliminates the traditional inner-loop acquisition optimization bottleneck, achieving up to a {$10^2 \times$ reduction in FLOPs} compared to BNN-BO baselines. We evaluate our method on a diverse suite of benchmarks, including high-dimensional synthetic functions and complex real-world tasks such as PDE optimization, the Lunar Lander control problem, and categorical Pest Control. Our experiments demonstrate that VBO-MI consistently provides the same or superior optimization performance and computational scalability over the baselines.

cs.LG↗

arXiv2022

A Perspective on Neural Capacity Estimation: Viability and Reliability

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. These estimators ar referred to as neural mutual information estimation (NMIE)s. NMIEs differ from other approaches as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. In order to test the performance across various NMIEs, it is desirable to establish a benchmark encompassing the different challenges of capacity estimation. This is the objective of this paper. In particular, we consider three scenarios for benchmarking:i the classic AWGN channel, ii channels continuous inputs optical intensity and peak-power constrained AWGN channel iii channels with a discrete output, i.e., Poisson channel. We also consider the extension to the multi-terminal case with iv the AWGN and optical MAC models. We argue that benchmarking a certain NMIE across these four scenarios provides a substantive test of performance. In this paper we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE). and provide insights on the performance of other methods as well. To summarize our benchmarking results, MINE provides the most reliable performance.

cs.IT↗

arXiv2022

Neural Capacity Estimators: How Reliable Are They?

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowing closed form distribution of the data. This class of estimators is referred to as neural mutual information estimators. Although very promising, such techniques have yet to be rigorously bench-marked so as to establish their efficacy, ease of implementation, and stability for capacity estimation which is joint maximization frame-work. In this paper, we compare the different techniques proposed in the literature for estimating capacity and provide a practitioner perspective on their effectiveness. In particular, we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE) and provide insights on InfoNCE. We evaluated these algorithms in terms of their ability to learn the input distributions that are capacity approaching for the AWGN channel, the optical intensity channel, and peak power-constrained AWGN channel. For both scenarios, we provide insightful comments on various aspects of the training process, such as stability, sensitivity to initialization.

cs.IT↗

arXiv2021

Neural Computation of Capacity Region of Memoryless Multiple Access Channels

This paper provides a numerical framework for computing the achievable rate region of memoryless multiple access channel (MAC) with a continuous alphabet from data. In particular, we use recent results on variational lower bounds on mutual information and KL-divergence to compute the boundaries of the rate region of MAC using a set of functions parameterized by neural networks. Our method relies on a variational lower bound on KL-divergence and an upper bound on KL-divergence based on the f-divergence inequalities. Unlike previous work, which computes an estimate on mutual information, which is neither a lower nor an upper bound, our method estimates a lower bound on mutual information. Our numerical results show that the proposed method provides tighter estimates compared to the MINE-based estimator at large SNRs while being computationally more efficient. Finally, we apply the proposed method to the optical intensity MAC and obtain a new achievable rate boundary tighter than prior works.

cs.IT↗

arXiv2020

On the Capacity of the Joint Time and Concentration Modulation for Molecular Communications

Most diffusion based molecular channels suffer from low information capacity due to the structure of the diffusion environment. To address this issue, this paper studies the capacity of the diffusion based molecular communication by exploiting both time and concentration level of the released molecules for information transfer. The transmitter releases molecules in one of the sub-intervals with a level of concentration both determined by input data, hereby applying joint time and concentration (JTAC) modulation. The observation time, at the receiver, which is equal to symbol period, is divided to some sub-intervals, not necessarily equal to the number of sub-intervals in the transmitter, and the number of received molecules in each sub-interval is counted. We propose three practical schemes, depending on how the receiver uses the number of molecules counted in the sub-intervals and find the lower bound on capacity in each case. Moreover, the symmetric Kullback-Liebler (KL) divergence metric is used to obtain a computable upper bound on the JTAC channel capacity. Finally, the Blahut-Arimoto algorithm is used to compute the capacity numerically, and to determine how tight the derived bounds are. Our numerical results indicate that our bounds are tight especially in environments with high diffusion coefficient. The improvements compared to the conventional concentration based modulation and timing based modulation are also demonstrated.

cs.IT↗