arXiv subjects

Amin Gohari

Publications and source records attributed to Amin Gohari.

At least 37 records · Page 2Linked to original sources

arXiv2025

On the Source Model Key Agreement Problem

We consider the source model key agreement problem involving two legitimate parties and an eavesdropper who observe n i.i.d. samples of X, Y, and Z, respectively. In this paper, we focus on one of the simplest instances where the key capacity remains open, specifically when X and Y are binary random variables and Z is a function of the pair (X, Y). The best-known upper bound on the key capacity is characterized by an inf-max optimization problem that generally lacks a closed-form solution. We provide general conditions under which the upper bound reduces to I(X;Y). As an example, we consider the XOR setting in which X and Y are binary, and Z is the XOR of X and Y. The upper bound reduces to I(X;Y) for this source. Next, we conjecture that the rate I(X;Y) is not achievable for the XOR source and provide some ideas that might be useful for developing a new upper bound on the source model problem.

cs.IT↗

arXiv2025

A New Upper Bound for Distributed Hypothesis Testing Using the Auxiliary Receiver Approach

This paper employs the add-and-subtract technique of the auxiliary receiver approach to establish a new upper bound for the distributed hypothesis testing problem. This new bound has fewer assumptions than the upper bound proposed by Rahman and Wagner, is at least as tight as the bound by Rahman and Wagner, and can outperform it in certain Gaussian settings. Conceptually speaking, unlike Rahman and Wagner, who view their additional receiver as side information, we view it as an auxiliary receiver and use a different manipulation for single-letterization.

cs.IT↗

arXiv2024

On the Ratio of Shannon Numbers of Graphs

Let $Γ$ be a function that maps two arbitrary graphs $G$ and $H$ to a non-negative real number such that $$α(G^{\boxtimes n})\leq α(H^{\boxtimes n})Γ(G,H)^n$$ where $n$ is any natural number and $G^{\boxtimes n}$ is the strong product of $G$ with itself $n$ times. We establish the equivalence of two different approaches for finding such a function $Γ$. The common solution obtained through either approach is termed ``the relative fractional independence number of a graph $G$ with respect to another graph $H$". We show this function by $α^*(G|H)$ and discuss some of its properties. In particular, we show that $α^*(G|H)\geq \frac{X(G)}{X(H)} \geq \frac{1}{α^*(H|G)},$ where $X(G)$ can be the independence number, the Shannon capacity, the fractional independence number, the Lovász number, or the Schrijver's or Szegedy's variants of the Lovász number of a graph $G$. This inequality is the first explicit non-trivial upper bound on the ratio of the invariants of two arbitrary graphs, as mentioned earlier, which can also be used to obtain upper or lower bounds for these invariants. As explicit applications, we present new upper bounds for the ratio of the Shannon capacity of two Cayley graphs and compute new lower bounds on the Shannon capacity of certain Johnson graphs (yielding the exact value of their Haemers number). Moreover, we show that $α^*(G|H)$ can be used to present a stronger version of the well-known No-Homomorphism Lemma.

math.CO↗

arXiv2024

On the Inductive Biases of Demographic Parity-based Fair Learning Algorithms

Fair supervised learning algorithms assigning labels with little dependence on a sensitive attribute have attracted great attention in the machine learning community. While the demographic parity (DP) notion has been frequently used to measure a model's fairness in training fair classifiers, several studies in the literature suggest potential impacts of enforcing DP in fair learning algorithms. In this work, we analytically study the effect of standard DP-based regularization methods on the conditional distribution of the predicted label given the sensitive attribute. Our analysis shows that an imbalanced training dataset with a non-uniform distribution of the sensitive attribute could lead to a classification rule biased toward the sensitive attribute outcome holding the majority of training data. To control such inductive biases in DP-based fair learning, we propose a sensitive attribute-based distributionally robust optimization (SA-DRO) method improving robustness against the marginal distribution of the sensitive attribute. Finally, we present several numerical results on the application of DP-based learning methods to standard centralized and distributed learning problems. The empirical findings support our theoretical results on the inductive biases in DP-based fair learning algorithms and the debiasing effects of the proposed SA-DRO method.

cs.LG↗

arXiv2024

Impact of Fairness Regulations on Institutions' Policies and Population Qualifications

The proliferation of algorithmic systems has fueled discussions surrounding the regulation and control of their social impact. Herein, we consider a system whose primary objective is to maximize utility by selecting the most qualified individuals. To promote demographic parity in the selection algorithm, we consider penalizing discrimination across social groups. We examine conditions under which a discrimination penalty can effectively reduce disparity in the selection. Additionally, we explore the implications of such a penalty when individual qualifications may evolve over time in response to the imposed penalizing policy. We identify scenarios where the penalty could hinder the natural attainment of equity within the population. Moreover, we propose certain conditions that can counteract this undesirable outcome, thus ensuring fairness.

cs.LG↗

arXiv2023

f-divergences and their applications in lossy compression and bounding generalization error

In this paper, we provide three applications for $f$-divergences: (i) we introduce Sanov's upper bound on the tail probability of the sum of independent random variables based on super-modular $f$-divergence and show that our generalized Sanov's bound strictly improves over ordinary one, (ii) we consider the lossy compression problem which studies the set of achievable rates for a given distortion and code length. We extend the rate-distortion function using mutual $f$-information and provide new and strictly better bounds on achievable rates in the finite blocklength regime using super-modular $f$-divergences, and (iii) we provide a connection between the generalization error of algorithms with bounded input/output mutual $f$-information and a generalized rate-distortion problem. This connection allows us to bound the generalization error of learning algorithms using lower bounds on the $f$-rate-distortion function. Our bound is based on a new lower bound on the rate-distortion function that (for some examples) strictly improves over previously best-known bounds.

cs.IT↗

arXiv2022

Learning under Distribution Mismatch and Model Misspecification

We study learning algorithms when there is a mismatch between the distributions of the training and test datasets of a learning algorithm. The effect of this mismatch on the generalization error and model misspecification are quantified. Moreover, we provide a connection between the generalization error and the rate-distortion theory, which allows one to utilize bounds from the rate-distortion theory to derive new bounds on the generalization error and vice versa. In particular, the rate-distortion based bound strictly improves over the earlier bound by Xu and Raginsky even when there is no mismatch. We also discuss how "auxiliary loss functions" can be utilized to obtain upper bounds on the generalization error.

cs.IT↗

arXiv2022

Rate-Distortion Theoretic Generalization Bounds for Stochastic Learning Algorithms

Understanding generalization in modern machine learning settings has been one of the major challenges in statistical learning theory. In this context, recent years have witnessed the development of various generalization bounds suggesting different complexity notions such as the mutual information between the data sample and the algorithm output, compressibility of the hypothesis space, and the fractal dimension of the hypothesis space. While these bounds have illuminated the problem at hand from different angles, their suggested complexity notions might appear seemingly unrelated, thereby restricting their high-level impact. In this study, we prove novel generalization bounds through the lens of rate-distortion theory, and explicitly relate the concepts of mutual information, compressibility, and fractal dimensions in a single mathematical framework. Our approach consists of (i) defining a generalized notion of compressibility by using source coding concepts, and (ii) showing that the `compression error rate' can be linked to the generalization error both in expectation and with high probability. We show that in the `lossless compression' setting, we recover and improve existing mutual information-based bounds, whereas a `lossy compression' scheme allows us to link generalization to the rate-distortion dimension -- a particular notion of fractal dimension. Our results bring a more unified perspective on generalization and open up several future research directions.

stat.ML↗

arXiv2022

A Strengthened Cutset Upper Bound on the Capacity of the Relay Channel and Applications

We develop a new upper bound on the capacity of the relay channel that is tighter than previously known upper bounds. This upper bound is proved using traditional weak converse techniques involving mutual information inequalities and Gallager-type explicit identification of auxiliary random variables. We show that the new upper bound is strictly tighter than all previous bounds for the Gaussian relay channel with non-zero channel gains. When specialized to the relay channel with orthogonal receiver components, the bound resolves a conjecture by Kim on a class of deterministic relay channels. When further specialized to the class of product-form relay channels with orthogonal receiver components, the bound resolves a generalized version of Cover's relay channel problem, recovers the recent upper bound for the Gaussian case by Wu et al., and improves upon the recent bounds for the binary symmetric case by Wu et al. and Barnes et al., which were obtained using non-traditional geometric proof techniques. For the special class of a relay channel with orthogonal receiver components, we develop another upper bound on the capacity which utilizes an auxiliary receiver and show that it is strictly tighter than the bound by Tandon and Ulukus. Finally, we show through the Gaussian relay channel with i.i.d. relay output sequence that the bound with the auxiliary receiver can be strictly tighter than our main bound.

cs.IT↗

arXiv2021

An Analytical Model for Molecular Communication over a Non-linear Reaction-Diffusion Medium

One of the main challenges in diffusion-based molecular communication is dealing with the non-linearity of reaction-diffusion chemical equations. While numerical methods can be used to solve these equations, a change in the input signals or the parameters of the medium requires one to redo the simulations. This makes it difficult to design modulation schemes and practically impossible to prove the optimality of a given transmission strategy. In this paper, we provide an analytical technique for modeling the non-linearity of chemical reaction equations based on the perturbation method. The perturbation method expresses the solution in terms of an infinite power series. An approximate solution can be found by keeping the leading terms of the power series. The approximate solution is shown to track the true solution if either the simulation time interval or the reaction rate is sufficiently small. Approximate solutions for long time intervals are also discussed. An illustrative example is given. For this example, it is shown that when the reaction rate (or the total time interval) is low, instead of using a continuous release waveform, it is optimal for the transmitters to release molecules at two time instances.

cs.IT↗

arXiv2021

Transmission of a Bit over a Discrete Poisson Channel with Memory

A coding scheme for transmission of a bit maps a given bit to a sequence of channel inputs (called the codeword associated to the transmitted bit). In this paper, we study the problem of designing the best code for a discrete Poisson channel with memory (under peak-power and total-power constraints). The outputs of a discrete Poisson channel with memory are Poisson distributed random variables with a mean comprising of a fixed additive noise and a linear combination of past input symbols. Assuming a maximum-likelihood (ML) decoder, we search for a codebook that has the smallest possible error probability. This problem is challenging because error probability of a code does not have a closed-form analytical expression. For the case of having only a total-power constraint, the optimal code structure is obtained, provided that the blocklength is greater than the memory length of the channel. For the case of having only a peak-power constraint, the optimal code is derived for arbitrary memory and blocklength in the high-power regime. For the case of having both the peak-power and total-power constraints, the optimal code is derived for memoryless Poisson channels when both the total-power and the peak-power bounds are large.

cs.IT↗

arXiv2020

Coding for Positive Rate in the Source Model Key Agreement Problem

A two-party key agreement problem with public discussion, known as the source model problem, is considered. By relating key agreement to hypothesis testing, a new coding scheme is developed that yields a sufficient condition to achieve a positive secret-key (SK) rate in terms of Rényi divergence. The merits of this coding scheme are illustrated by applying it to an erasure model for Eve's side information, and by deriving an upper bound on Eve's erasure probabilities for which the SK capacity is zero. This bound strictly improves on the best known single-letter lower bound on the SK capacity. Moreover, the bound is tight when Alice's or Bob's source is binary, which extends a previous result for a doubly symmetric binary source. The results motivate a new measure for the correlation between two random variables, which is of independent interest.

cs.IT↗

arXiv2019

Playing Games with Bounded Entropy: Convergence Rate and Approximate Equilibria

We consider zero-sum repeated games in which the players are restricted to strategies that require only a limited amount of randomness. Let $v_n$ be the max-min value of the $n$ stage game; previous works have characterized $\lim_{n\rightarrow\infty}v_n$, i.e., the long-run max-min value. Our first contribution is to study the convergence rate of $v_n$ to its limit. To this end, we provide a new tool for simulation of a source (target source) from another source (coin source). Considering the total variation distance as the measure of precision, this tool offers an upper bound for the precision of simulation, which is vanishing exponentially in the difference of Rényi entropies of the coin and target sources. In the second part of paper, we characterize the set of all approximate Nash equilibria achieved in long run. It turns out that this set is in close relation with the long-run max-min value.

cs.GT↗

arXiv2018

Playing Games with Bounded Entropy

In this paper, we consider zero-sum repeated games in which the maximizer is restricted to strategies requiring no more than a limited amount of randomness. Particularly, we analyze the maxmin payoff of the maximizer in two models: the first model forces the maximizer to randomize her action in each stage just by conditioning her decision to outcomes of a given sequence of random source, whereas, in the second model, the maximizer is a team of players who are free to privately randomize their corresponding actions but do not have access to any explicit source of shared randomness needed for cooperation. The works of Gossner and Vieille, and Gossner and Tomala adopted the method of types to establish their results; however, we utilize the idea of random hashing which is the core of randomness extractors in the information theory literature. In addition, we adopt the well-studied tool of simulation of a source from another source. By utilizing these tools, we are able to simplify the prior results and generalize them as well. We characterize the maxmin payoff of the maximizer in the repeated games under study. Particularly, the maxmin payoff of the first model is fully described by the function $J(h)$ which is the maximum payoff that the maximizer can secure in a one-shot game by choosing mixed strategies of entropy at most $h$. In the second part of the paper, we study the computational aspects of $J(h)$. We offer three explicit lower bounds on the entropy-payoff trade-off curve. To do this, we provide and utilize new results for the set of distributions that guarantee a certain payoff for Alice. In particular, we study how this set of distributions shrinks as we increase the security level. While the use of total variation distance is common in game theory, our derivation indicates the suitability of utilizing the Renyi-divergence of order two.

cs.IT↗

arXiv2018

A Correlation Measure Based on Vector-Valued $L_p$-Norms

In this paper, we introduce a new measure of correlation for bipartite quantum states. This measure depends on a parameter $α$, and is defined in terms of vector-valued $L_p$-norms. The measure is within a constant of the exponential of $α$-Rényi mutual information, and reduces to the trace norm (total variation distance) for $α=1$. We will prove some decoupling type theorems in terms of this measure of correlation, and present some applications in privacy amplification as well as in bounding the random coding exponents. In particular, we establish a bound on the secrecy exponent of the wiretap channel (under the total variation metric) in terms of the $α$-Rényi mutual information according to \emph{Csiszár's proposal}.

cs.IT↗

arXiv2018

On Medium Chemical Reaction in Diffusion-Based Molecular Communication: a Two-Way Relaying Example

Chemical reactions are a prominent feature of molecular communication (MC) systems, with no direct parallels in wireless communications. While chemical reactions may be used inside the transmitter nodes, receiver nodes or the communication medium, we focus on its utility in the medium in this paper. Such chemical reactions can be used to perform computation over the medium as molecules diffuse and react with each other (physical-layer computation). We propose the use of chemical reactions for the following purposes: (i) to reduce signal-dependent observation noise of receivers by reducing the signal density, (ii) to realize molecular physical-layer network coding (molecular PNC) by performing the natural XOR operation inside the medium, and (iii) to reduce the inter-symbol interference (ISI) of other transmitters by canceling out the remaining molecules from previous transmissions. To make the ideas formal, we consider an explicit two-way relaying example with a transparent receiver (which has a signal-dependent noise). The proposed ideas are used to define a modulation scheme (which we call the PNC scheme). We compare the PNC with a previously proposed scheme for this problem where the XOR operation is performed at the relay node (using a molecular logic gate). We call the latter, the straightforward network coding (SNC). It is observed that in addition to the simplicity of the proposed PNC scheme, it outperforms the SNC scheme especially when we consider ISI.

cs.IT↗

arXiv2018

Diffusion Based Molecular Communication with Limited Molecule Production Rate

This paper studies the impact of a transmitter's molecule generation process on the capacity of a concentration based Molecular Communication (MC) system. Constraints caused by the molecule generation process affect the availability of the molecules at the transmitter. The transmitter has a storage of molecules, and should decide whether to release or save the currently produced molecules. As a result, the MC system has conceptual connections with energy harvesting systems. In this paper, we consider two scenarios on the propagation channel. The first scenario assumes a channel with no Inter-symbol Interference (ISI), \emph{i.e.,} a memoryless channel. We derive bounds on the capacity of the MC system in this scenario. The second scenario assumes the MC network with ISI, in which the output of the channel depends on the history of released molecules in the previous time-slots. Based on the assumptions that either the transmitter or the receiver knows the channel statistics, we compute a lower bound on the channel capacity.

cs.IT↗

arXiv2018

How Compressible are Innovation Processes?

The sparsity and compressibility of finite-dimensional signals are of great interest in fields such as compressed sensing. The notion of compressibility is also extended to infinite sequences of i.i.d. or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this work, we use the entropy measure to study the compressibility of continuous-domain innovation processes (alternatively known as white noise). Specifically, we define such a measure as the entropy limit of the doubly quantized (time and amplitude) process. This provides a tool to compare the compressibility of various innovation processes. It also allows us to identify an analogue of the concept of "entropy dimension" which was originally defined by Rényi for random variables. Particular attention is given to stable and impulsive Poisson innovation processes. Here, our results recognize Poisson innovations as the more compressible ones with an entropy measure far below that of stable innovations. While this result departs from the previous knowledge regarding the compressibility of fat-tailed distributions, our entropy measure ranks stable innovations according to their tail decay.

cs.IT↗