arXiv subjects

Amin Gohari

Publications and source records attributed to Amin Gohari.

At least 19 recordsLinked to original sources

arXiv2026

Exact Rate Exponent Tradeoff for New Classes of Distributed Hypothesis Testing Problems

We characterize the exact rate--exponent tradeoff for new classes of one-way distributed hypothesis testing problems by demonstrating that a recent upper bound, derived via the auxiliary-receiver technique, coincides with known lower bounds. We achieve this by relaxing the upper bound on the type-II error exponent into a form that shares the same inner functional as Han's lower bound, differing only in the outer rate constraint. Furthermore, we prove that this upper bound is tight for testing against dependence and for the doubly symmetric binary source (DSBS) with crossover probabilities $\kappa_0$ under the null hypothesis and $\kappa_1$ under the alternative hypothesis, provided $0 < \kappa_1 < \kappa_0 < \frac12$. The characterization of the exact error exponent for the DSBS source holds for every communication rate $R \geq 0$.

cs.IT↗

arXiv2026

Imaginative Generative AI: Crossing the Entropy Wall into Worlds Beyond Imitation

Generative AI models are primarily designed to imitate the data distribution, an objective that neither corrects diversity lost by a learned generator nor defines how generation should extend beyond the diversity of the data itself. We introduce Imaginative Generative AI (IGA), a framework that makes diversity part of the target-distribution design problem: among distributions close to a reference, IGA selects one whose spectral diversity reaches a prescribed level. Diversity is measured by the von Neumann entropy of the generated distribution's kernel covariance operator in a fixed representation space, providing a reference-free representation-guided measure of how broadly probability mass occupies embedding directions. The spectral entropy of the population data distribution defines an Entropy Wall. Below the wall, IGA performs diversity repair, recovering variation that a learned generator has lost while remaining within the diversity level of the data. Beyond the wall, the data distribution itself becomes infeasible, and IGA deliberately departs from it to produce distributions with greater representation-relative spectral diversity, an operational notion of imaginative generation. These regimes form a single regularization path from imitation to imagination and define an i.i.d. target distribution at each prescribed diversity level. We develop the theory of this entropy-constrained projection and show that, under a KL anchor to a pretrained generator, the optimum satisfies a self-consistent exponential-tilt relation. This characterization leads to IGA Guidance, a retraining-free inference-time method for score-based and diffusion models, including DDPM and DDIM samplers. Experiments on synthetic and vision benchmarks demonstrate diversity repair below the Entropy Wall and controlled spectral extrapolation beyond it.

cs.LG↗

arXiv2026

The Capacity Region for Classes of Sum-Broadcast Channels

We compute the capacity region of a sum of broadcast channels whose components are degraded, less-noisy, more-capable, deterministic, or semi-deterministic. We achieve this by showing that an auxiliary-receiver outer bound, previously introduced by some of the authors, matches Marton's inner bound. This result generalizes a previously known result for the sum of two reversely degraded broadcast channels due to El Gamal (1980). Moreover, we define a class of primary broadcast channels and show an analogous result for the sum of primary broadcast channels.

cs.IT↗

arXiv2026

Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search

Proving Shannon-type entropy inequalities is a fundamental task in information theory that often requires constructing non-trivial linear combinations of known constraints, which is a combinatorial search problem that scales poorly with the number of random variables. We investigate whether small-scale large language models (0.6B--1.7B parameters), fine-tuned on atomic proof steps and combined with guided beam search, can automate this process. On a held-out test set of 60 inequalities spanning n=10 to 15 variables, our 0.6B fine-tuned model achieves an 85\% proof success rate with tree search. GPT-5.5 solves 1.7\% samples under zero-shot prompting while Psitip solves 33.3\% samples. A systematic ablation study across training context length (4096 vs.\ 8192 tokens) and data distribution (n=9-skewed vs not skewed) reveals that a 4096-token not skewed training distribution yields the best performance, with extended context and skewed data providing no marginal benefit. We further identify two dominant failure modes -- format failures and step quality degradation -- and verify that the beam-scoring heuristic is essential via a controlled ablation (random scoring reduces success from 83\% to 23\%).

cs.IT↗

arXiv2026

List Estimation

Classical estimation outputs a single point estimate of an unknown $d$-dimensional vector from an observation. In this paper, we study \emph{$k$-list estimation}, in which a single observation is used to produce a list of $k$ candidate estimates and performance is measured by the expected squared distance from the true vector to the closest candidate. We compare this centralized setting with a symmetric decentralized MMSE benchmark in which $k$ agents observe conditionally i.i.d.\ measurements and each agent outputs its own MMSE estimate. On the centralized side, we show that optimal $k$-list estimation is equivalent to fixed-rate $k$-point vector quantization of the posterior distribution and, under standard regularity conditions, admits an exact high-rate asymptotic expansion with explicit constants and decay rate $k^{-2/d}$. On the decentralized side, we derive lower bounds in terms of the small-ball behavior of the single-agent MMSE error; in particular, when the conditional error density is bounded near the origin, the benchmark distortion cannot decay faster than order $k^{-2/d}$. We further show that if the error density vanishes at the origin, then the decentralized benchmark is provably unable to match the centralized $k^{-2/d}$ exponent, whereas the centralized estimator retains that scaling. Gaussian specializations yield explicit formulas and numerical experiments corroborate the predicted asymptotic behavior. Overall, the results show that, in the scaling with $k$, one observation combined with $k$ carefully chosen candidates can be asymptotically as effective as -- and in some regimes strictly better than -- this MMSE-based decentralized benchmark with $k$ independent observations.

cs.IT↗

arXiv2026

Subjective Distortion: Achievability and Outer Bounds for Distortion Functions with Memory

In some rate-distortion-type problems, the required fidelity of information is affected by past actions. As a result, the distortion function depends not only on the instantaneous distortion between a source symbol and its representation symbol, but also on past representations. In this paper, we give a formal definition of this problem and introduce both inner (achievable) and outer bounds on the rate-distortion tradeoff. We also discuss convexification of the problem, which makes it easier to find bounds. Problems of this type arise in biological information processing, as well as in recommendation engines; we provide an example applied to a simplified biological information processing problem.

cs.IT↗

arXiv2026

Ribbons from Independence Structure: Hypercontractivity, $\Phi$-Mutual Information, and Matrix $\Phi$-Entropy

We study the hypercontractivity ribbon and the $\Phi$-ribbon for joint distributions that obey a given independence structure, obtaining tight bounds in some basic regimes. For general independence structures, modeled as a hypergraph whose hyperedges specify mutually independent subcollections of random variables, we provide an explicit inner bound on the $\Phi$-ribbon described by a simple convex hull of incidence vectors. We also provide a new multipartite generalization version and a $\Phi$-mutual information analogue of the Zhang--Yeung inequality, which implies nontrivial points in the hypercontractivity ribbon and the $\Phi$-ribbon respectively. Finally, we propose the matrix $\Phi$-ribbon based on matrix $\Phi$-entropy and establish the tensorization and data processing properties, together with the calculation of an exact matrix SDPI constant for the doubly symmetric binary source.

cs.IT↗

arXiv2026

Stable Source Coding

A source encoder is stable if a small change in the source sequence (e.g., changing a few symbols) results in a small (or bounded) change in the output codeword. By this definition, the common technique of random binning is unstable; because the mapping is random, two nearly identical source sequences can be assigned to completely unrelated bin indices. We study compression rates of stable lossless source codes. Using combinatorial arguments, we derive information-theoretic limits on the achievable rate as a function of the stability parameters.

cs.IT↗

arXiv2026

A Class of Subadditive Information Measures and their Applications

We introduce a two-parameter family of discrepancy measures, termed \emph{$(G,f)$-divergences}, obtained by applying a non-decreasing function $G$ to an $f$-divergence $D_f$. Building on Csisz\'ar's formulation of mutual $f$-information, we define a corresponding $(G,f)$-information measure $ I_{G,f}(X;Y)$. A central theme of the paper is subadditivity over product distributions and product channels. We develop reduction principles showing that, for broad classes of $G$, it suffices to verify divergence subadditivity on binary alphabets. Specializing to the functions $G(x)\in\{x,\log(1+x),-\log(1-x)\}$, we derive tractable sufficient conditions on $f$ that guarantee subadditivity, covering many standard $f$-divergences. Finally, we present applications to finite-blocklength converses for channel coding, bounds in binary hypothesis testing, and an extension of the Shannon--Gallager--Berlekamp sphere-packing exponent framework to subadditive $(G,f)$-divergences.

cs.IT↗

arXiv2025

SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score

Diffusion models have demonstrated remarkable success in high-fidelity image synthesis and prompt-guided generative modeling. However, ensuring adequate diversity in generated samples of prompt-guided diffusion models remains a challenge, particularly when the prompts span a broad semantic spectrum and the diversity of generated data needs to be evaluated in a prompt-aware fashion across semantically similar prompts. Recent methods have introduced guidance via diversity measures to encourage more varied generations. In this work, we extend the diversity measure-based approaches by proposing the Scalable Prompt-Aware R\'eny Kernel Entropy Diversity Guidance (SPARKE) method for prompt-aware diversity guidance. SPARKE utilizes conditional entropy for diversity guidance, which dynamically conditions diversity measurement on similar prompts and enables prompt-aware diversity control. While the entropy-based guidance approach enhances prompt-aware diversity, its reliance on the matrix-based entropy scores poses computational challenges in large-scale generation settings. To address this, we focus on the special case of Conditional latent RKE Score Guidance, reducing entropy computation and gradient-based optimization complexity from the $O(n^3)$ of general entropy measures to $O(n)$. The reduced computational complexity allows for diversity-guided sampling over potentially thousands of generation rounds on different prompts. We numerically test the SPARKE method on several text-to-image diffusion models, demonstrating that the proposed method improves the prompt-aware diversity of the generated data without incurring significant computational costs. We release our code on the project page: https://mjalali.github.io/SPARKE

cs.CV↗

arXiv2025

A Differential Equation Approach to the Most-Informative Boolean Function Conjecture

We study the most-informative Boolean function conjecture using a differential equation approach. This leads to a formulation of a functional inequality on finite-dimensional random variables. We also develop a similar inequality in the case of the Hellinger conjecture. Finally, we conjecture a specific finite dimensional inequality that, if proved, will lead to a proof of the Boolean function conjecture in the balanced case. We further show that the above inequality holds modulo four explicit inequalities (all of which seems to hold via numerical simulation) with the first three containing just two variables and a final one involving four variables.

cs.IT↗

arXiv2025

On Phi-entropic Dependence Measures and Non-local Correlations

We say that a measure of dependence between two random variables $X$ and $Y$, denoted as $\rho(X;Y)$, satisfies the data processing property if $\rho(X;Y)\geq \rho(X';Y')$ for every $X'\rightarrow X\rightarrow Y\rightarrow Y'$, and satisfies the tensorization property if $\rho(X_1X_2;Y_1Y_2)=\max\{\rho(X_1;Y_1),\rho(X_2;Y_2)\}$ when $(X_1,Y_1)$ is independent of $(X_2,Y_2)$. It is known that measures of dependence defined based on $\Phi$-entropy satisfy these properties. These measures are important because they generalize R{\'e}nyi's maximal correlation and the hypercontractivity ribbon. The data processing and tensorization properties are special cases of monotonicity under wirings of non-local boxes. We show that ribbons defined using $\Phi$-entropic measures of dependence are monotone under wiring of non-local no-signaling boxes, generalizing an earlier result. In addition, we also discuss the evaluation of $\Phi$-strong data processing inequality constant for joint distributions obtained from a $Z$-channel.

cs.IT↗

arXiv2025

On the Source Model Key Agreement Problem

We consider the source model key agreement problem involving two legitimate parties and an eavesdropper who observe n i.i.d. samples of X, Y, and Z, respectively. In this paper, we focus on one of the simplest instances where the key capacity remains open, specifically when X and Y are binary random variables and Z is a function of the pair (X, Y). The best-known upper bound on the key capacity is characterized by an inf-max optimization problem that generally lacks a closed-form solution. We provide general conditions under which the upper bound reduces to I(X;Y). As an example, we consider the XOR setting in which X and Y are binary, and Z is the XOR of X and Y. The upper bound reduces to I(X;Y) for this source. Next, we conjecture that the rate I(X;Y) is not achievable for the XOR source and provide some ideas that might be useful for developing a new upper bound on the source model problem.

cs.IT↗

arXiv2024

Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

Generative models guided by text prompts are widely evaluated for fidelity and prompt alignment, yet their ability to produce outputs remains underexplored. Existing diversity metrics such as Vendi and RKE, which are based on the von Neumann and R\'enyi entropies of kernel matrices, were developed for unconditional models and cannot distinguish prompt-induced from model-induced variability. We address this gap by introducing \textit{Conditional-Vendi} and \textit{Conditional-RKE}, diversity measures derived from the conditional entropy of positive semidefinite matrices. These scores isolate model-induced diversity in prompt-guided generation, with Conditional-RKE enjoying an $O(1/\sqrt{n})$ convergence rate. For Conditional-Vendi, we introduce a truncated-spectrum approximation that yields scalable and consistent estimates. Experiments on text-to-image, image-captioning, and LLM tasks show that the conditional scores recover ground-truth diversity orderings and can also guide diffusion models toward more diverse samples. The codebase is available at https://github.com/mjalali/conditional-vendi.

cs.LG↗

arXiv2024

A New Upper Bound for Distributed Hypothesis Testing Using the Auxiliary Receiver Approach

This paper employs the add-and-subtract technique of the auxiliary receiver approach to establish a new upper bound for the distributed hypothesis testing problem. This new bound has fewer assumptions than the upper bound proposed by Rahman and Wagner, is at least as tight as the bound by Rahman and Wagner, and can outperform it in certain Gaussian settings. Conceptually speaking, unlike Rahman and Wagner, who view their additional receiver as side information, we view it as an auxiliary receiver and use a different manipulation for single-letterization.

cs.IT↗

arXiv2024

Impact of Fairness Regulations on Institutions' Policies and Population Qualifications

The proliferation of algorithmic systems has fueled discussions surrounding the regulation and control of their social impact. Herein, we consider a system whose primary objective is to maximize utility by selecting the most qualified individuals. To promote demographic parity in the selection algorithm, we consider penalizing discrimination across social groups. We examine conditions under which a discrimination penalty can effectively reduce disparity in the selection. Additionally, we explore the implications of such a penalty when individual qualifications may evolve over time in response to the imposed penalizing policy. We identify scenarios where the penalty could hinder the natural attainment of equity within the population. Moreover, we propose certain conditions that can counteract this undesirable outcome, thus ensuring fairness.

cs.LG↗

arXiv2024

On the Inductive Biases of Demographic Parity-based Fair Learning Algorithms

Fair supervised learning algorithms assigning labels with little dependence on a sensitive attribute have attracted great attention in the machine learning community. While the demographic parity (DP) notion has been frequently used to measure a model's fairness in training fair classifiers, several studies in the literature suggest potential impacts of enforcing DP in fair learning algorithms. In this work, we analytically study the effect of standard DP-based regularization methods on the conditional distribution of the predicted label given the sensitive attribute. Our analysis shows that an imbalanced training dataset with a non-uniform distribution of the sensitive attribute could lead to a classification rule biased toward the sensitive attribute outcome holding the majority of training data. To control such inductive biases in DP-based fair learning, we propose a sensitive attribute-based distributionally robust optimization (SA-DRO) method improving robustness against the marginal distribution of the sensitive attribute. Finally, we present several numerical results on the application of DP-based learning methods to standard centralized and distributed learning problems. The empirical findings support our theoretical results on the inductive biases in DP-based fair learning algorithms and the debiasing effects of the proposed SA-DRO method.

cs.LG↗

arXiv2023

On the Ratio of Shannon Numbers of Graphs

Let $\Gamma$ be a function that maps two arbitrary graphs $G$ and $H$ to a non-negative real number such that $$\alpha(G^{\boxtimes n})\leq \alpha(H^{\boxtimes n})\Gamma(G,H)^n$$ where $n$ is any natural number and $G^{\boxtimes n}$ is the strong product of $G$ with itself $n$ times. We establish the equivalence of two different approaches for finding such a function $\Gamma$. The common solution obtained through either approach is termed ``the relative fractional independence number of a graph $G$ with respect to another graph $H$". We show this function by $\alpha^*(G|H)$ and discuss some of its properties. In particular, we show that $\alpha^*(G|H)\geq \frac{X(G)}{X(H)} \geq \frac{1}{\alpha^*(H|G)},$ where $X(G)$ can be the independence number, the Shannon capacity, the fractional independence number, the Lov\'{a}sz number, or the Schrijver's or Szegedy's variants of the Lov\'{a}sz number of a graph $G$. This inequality is the first explicit non-trivial upper bound on the ratio of the invariants of two arbitrary graphs, as mentioned earlier, which can also be used to obtain upper or lower bounds for these invariants. As explicit applications, we present new upper bounds for the ratio of the Shannon capacity of two Cayley graphs and compute new lower bounds on the Shannon capacity of certain Johnson graphs (yielding the exact value of their Haemers number). Moreover, we show that $\alpha^*(G|H)$ can be used to present a stronger version of the well-known No-Homomorphism Lemma.

math.CO↗