arXiv subjects

Ali Moharrer

Publications and source records attributed to Ali Moharrer.

7 recordsLinked to original sources

arXiv2020

Latent Factor Analysis of Gaussian Distributions under Graphical Constraints

We explore the algebraic structure of the solution space of convex optimization problem Constrained Minimum Trace Factor Analysis (CMTFA), when the population covariance matrix $Σ_x$ has an additional latent graphical constraint, namely, a latent star topology. In particular, we have shown that CMTFA can have either a rank $ 1 $ or a rank $ n-1 $ solution and nothing in between. The special case of a rank $ 1 $ solution, corresponds to the case where just one latent variable captures all the dependencies among the observables, giving rise to a star topology. We found explicit conditions for both rank $ 1 $ and rank $n- 1$ solutions for CMTFA solution of $Σ_x$. As a basic attempt towards building a more general Gaussian tree, we have found a necessary and a sufficient condition for multiple clusters, each having rank $ 1 $ CMTFA solution, to satisfy a minimum probability to combine together to build a Gaussian tree. To support our analytical findings we have presented some numerical demonstrating the usefulness of the contributions of our work.

cs.IT↗

arXiv2019

Algebraic Properties of Wyner Common Information Solution under Graphical Constraints

The Constrained Minimum Determinant Factor Analysis (CMDFA) setting was motivated by Wyner's common information problem where we seek a latent representation of a given Gaussian vector distribution with the minimum mutual information under certain generative constraints. In this paper, we explore the algebraic structures of the solution space of the CMDFA, when the underlying covariance matrix $Σ_x$ has an additional latent graphical constraint, namely, a latent star topology. In particular, sufficient and necessary conditions in terms of the relationships between edge weights of the star graph have been found. Under such conditions and constraints, we have shown that the CMDFA problem has either a rank one solution or a rank $n-1$ solution where $n$ is the dimension of the observable vector. Numerical results are provided to demonstrate the difference between the optimal mutual information and that derived under a naive star constraint.

cs.IT↗

arXiv2019

Algebraic Properties of Wyner Common Information Solution under Graphical Constraints

The Constrained Minimum Determinant Factor Analysis (CMDFA) setting was motivated by Wyner's common information problem where we seek a latent representation of a given Gaussian vector distribution with the minimum mutual information under certain generative constraints. In this paper, we explore the algebraic structures of the solution space of the CMDFA, when the underlying covariance matrix $Σ_x$ has an additional latent graphical constraint, namely, a latent star topology. In particular, sufficient and necessary conditions in terms of the relationships between edge weights of the star graph have been found. Under such conditions and constraints, we have shown that the CMDFA problem has either a rank one solution or a rank $n-1$ solution where $n$ is the dimension of the observable vector. Further results are given in regards to the solution to the CMDFA with $n-1$ latent factors.

cs.IT↗

arXiv2018

Latent Factor Analysis of Gaussian Distributions under Graphical Constraints

In this paper, we explore the algebraic structures of solution spaces for Gaussian latent factor analysis when the population covariance matrix $Σ_x$ has an additional latent graphical constraint, namely, a latent star topology. In particular, we give sufficient and necessary conditions under which the solutions to constrained minimum trace factor analysis (CMTFA) is still star. We further show that the solution to CMTFA under the star constraint can only have two cases, i.e. the number of latent variable can be only one (star) or $n-1$ where $n$ is the dimension of the observable vector, and characterize the solution for both the cases.

cs.IT↗

arXiv2017

Layered Synthesis of Latent Gaussian Trees

A new synthesis scheme is proposed to generate a random vector with prescribed joint density that induces a (latent) Gaussian tree structure. The quality of synthesis is shown by vanishing total variation distance between the synthesized and desired statistics. The proposed layered and successive synthesis scheme relies on the learned structure of tree to use sufficient number of common random variables to synthesize the desired density. We characterize the achievable rate region for the rate tuples of multi-layer latent Gaussian tree, through which the number of bits needed to synthesize such Gaussian joint density are determined. The random sources used in our algorithm are the latent variables at the top layer of tree, the additive independent Gaussian noises, and the Bernoulli sign inputs that capture the ambiguity of correlation signs between the variables. We have shown that such ambiguity can further help in reducing the synthesis rates for the underlying Gaussian trees.

cs.IT↗

arXiv2017

Tree Structured Synthesis of Gaussian Trees

A new synthesis scheme is proposed to effectively generate a random vector with prescribed joint density that induces a (latent) Gaussian tree structure. The quality of synthesis is measured by total variation distance between the synthesized and desired statistics. The proposed layered and successive encoding scheme relies on the learned structure of tree to use minimal number of common random variables to synthesize the desired density. We characterize the achievable rate region for the rate tuples of multi-layer latent Gaussian tree, through which the number of bits needed to simulate such Gaussian joint density are determined. The random sources used in our algorithm are the latent variables at the top layer of tree, the additive independent Gaussian noises, and the Bernoulli sign inputs that capture the ambiguity of correlation signs between the variables.

cs.IT↗

arXiv2016

Synthesis of Gaussian Trees with Correlation Sign Ambiguity: An Information Theoretic Approach

In latent Gaussian trees the pairwise correlation signs between the variables are intrinsically unrecoverable. Such information is vital since it completely determines the direction in which two variables are associated. In this work, we resort to information theoretical approaches to achieve two fundamental goals: First, we quantify the amount of information loss due to unrecoverable sign information. Second, we show the importance of such information in determining the maximum achievable rate region, in which the observed output vector can be synthesized, given its probability density function. In particular, we model the graphical model as a communication channel and propose a new layered encoding framework to synthesize observed data using upper layer Gaussian inputs and independent Bernoulli correlation sign inputs from each layer. We find the achievable rate region for the rate tuples of multi-layer latent Gaussian messages to synthesize the desired observables.

cs.IT↗