arXiv subjects

Bohdan Kivva

Publications and source records attributed to Bohdan Kivva.

10 recordsLinked to original sources

arXiv2022

Identifiability of deep generative models without auxiliary information

We prove identifiability of a broad class of deep latent variable models that (a) have universal approximation capabilities and (b) are the decoders of variational autoencoders that are commonly used in practice. Unlike existing work, our analysis does not require weak supervision, auxiliary information, or conditioning in the latent space. Specifically, we show that for a broad class of generative (i.e. unsupervised) models with universal approximation capabilities, the side information $u$ is not necessary: We prove identifiability of the entire generative model where we do not observe $u$ and only observe the data $x$. The models we consider match autoencoder architectures used in practice that leverage mixture priors in the latent space and ReLU/leaky-ReLU activations in the encoder, such as VaDE and MFC-VAE. Our main result is an identifiability hierarchy that significantly generalizes previous work and exposes how different assumptions lead to different "strengths" of identifiability, and includes certain "vanilla" VAEs with isotropic Gaussian priors as a special case. For example, our weakest result establishes (unsupervised) identifiability up to an affine transformation, and thus partially resolves an open problem regarding model identifiability raised in prior work. These theoretical results are augmented with experiments on both simulated and real data.

cs.LG↗

arXiv2021

Learning latent causal graphs via mixture oracles

We study the problem of reconstructing a causal graphical model from data in the presence of latent variables. The main problem of interest is recovering the causal structure over the latent variables while allowing for general, potentially nonlinear dependence between the variables. In many practical problems, the dependence between raw observations (e.g. pixels in an image) is much less relevant than the dependence between certain high-level, latent features (e.g. concepts or objects), and this is the setting of interest. We provide conditions under which both the latent representations and the underlying latent causal model are identifiable by a reduction to a mixture oracle. These results highlight an intriguing connection between the well-studied problem of learning the order of a mixture model and the problem of learning the bipartite structure between observables and unobservables. The proof is constructive, and leads to several algorithms for explicitly reconstructing the full graphical model. We discuss efficient algorithms and provide experiments illustrating the algorithms in practice.

cs.LG↗

arXiv2021

Exact nuclear norm, completion and decomposition for random overcomplete tensors via degree-4 SOS

In this paper we show that simple semidefinite programs inspired by degree $4$ SOS can exactly solve the tensor nuclear norm, tensor decomposition, and tensor completion problems on tensors with random asymmetric components. More precisely, for tensor nuclear norm and tensor decomposition, we show that w.h.p. these semidefinite programs can exactly find the nuclear norm and components of an $(n\times n\times n)$-tensor $\mathcal{T}$ with $m\leq n^{3/2}/polylog(n)$ random asymmetric components. Unlike most of the previous algorithms, our algorithm provides a certificate for the decomposition, does not require knowledge about the number of components in the decomposition and does not make any assumptions on the sizes of the coefficients in the decomposition. As a byproduct, we show that w.h.p. the nuclear norm decomposition exactly coincides with the minimum rank decomposition for tensors with $m\leq n^{3/2}/polylog(n)$ random asymmetric components. For tensor completion, we show that w.h.p. the semidefinite program, introduced by Potechin & Steurer (2017) for tensors with orthogonal components, can exactly recover an $(n\times n\times n)$-tensor $\mathcal{T}$ with $m$ random asymmetric components from only $n^{3/2}m polylog(n)$ randomly observed entries. For non-orthogonal tensors, this improves the dependence on $m$ of the number of entries needed for exact recovery over all previously known algorithms and provides the first theoretical guarantees for exact tensor completion in the overcomplete regime.

cs.DS↗

arXiv2021

Structure learning in polynomial time: Greedy algorithms, Bregman information, and exponential families

Greedy algorithms have long been a workhorse for learning graphical models, and more broadly for learning statistical models with sparse structure. In the context of learning directed acyclic graphs, greedy algorithms are popular despite their worst-case exponential runtime. In practice, however, they are very efficient. We provide new insight into this phenomenon by studying a general greedy score-based algorithm for learning DAGs. Unlike edge-greedy algorithms such as the popular GES and hill-climbing algorithms, our approach is vertex-greedy and requires at most a polynomial number of score evaluations. We then show how recent polynomial-time algorithms for learning DAG models are a special case of this algorithm, thereby illustrating how these order-based algorithms can be rigourously interpreted as score-based algorithms. This observation suggests new score functions and optimality conditions based on the duality between Bregman divergences and exponential families, which we explore in detail. Explicit sample and computational complexity bounds are derived. Finally, we provide extensive experiments suggesting that this algorithm indeed optimizes the score in a variety of settings.

cs.LG↗

arXiv2021

On the automorphism groups of rank-4 primitive coherent configurations

The minimal degree of a permutation group $G$ is the minimum number of points not fixed by non-identity elements of $G$. Lower bounds on the minimal degree have strong structural consequences on $G$. Babai conjectured that if a primitive coherent configuration with $n$ vertices is not a Cameron scheme, then its automorphism group has minimal degree $\geq cn$ for some constant $c>0$. In 2014, Babai proved the desired lower bound on the minimal degree of the automorphism groups of strongly regular graphs, thus confirming the conjecture for primitive coherent configurations of rank 3. In this paper, we extend Babai's result to primitive coherent configurations of rank 4, confirming the conjecture in this special case. The proofs combine structural and spectral methods.

math.CO↗

arXiv2021

Improved upper bounds for the rigidity of Kronecker products

The rigidity of a matrix $A$ for target rank $r$ is the minimum number of entries of $A$ that need to be changed in order to obtain a matrix of rank at most $r$. At MFCS'77, Valiant introduced matrix rigidity as a tool to prove circuit lower bounds for linear functions and since then this notion received much attention and found applications in other areas of complexity theory. The problem of constructing an explicit family of matrices that are sufficiently rigid for Valiant's reduction (Valiant-rigid) still remains open. Moreover, since 2017 most of the long-studied candidates have been shown not to be Valiant-rigid. Some of those former candidates for rigidity are Kronecker products of small matrices. In a recent paper (STOC'21), Alman gave a general non-rigidity result for such matrices: he showed that if an $n\times n$ matrix $A$ (over any field) is a Kronecker product of $d\times d$ matrices $M_1,\dots, M_k$ (so $n=d^k$) $(d\ge 2)$ then changing only $n^{1+\varepsilon}$ entries of $A$ one can reduce its rank to $\le n^{1-γ}$, where $1/γ$ is roughly $2^d/\varepsilon^2$. In this note we improve this result in two directions. First, we do not require the matrices $M_i$ to have equal size. Second, we reduce $1/γ$ from exponential in $d$ to roughly $d^{3/2}/\varepsilon^2$ (where $d$ is the maximum size of the matrices $M_i$), and to nearly linear (roughly $d/\varepsilon^2$) for matrices $M_i$ of sizes within a constant factor of each other. As an application of our results we significantly expand the class of Hadamard matrices that are known not to be Valiant-rigid; these now include the Kronecker products of Paley-Hadamard matrices and Hadamard matrices of bounded size.

cs.DS↗

arXiv2019

A characterization of Johnson and Hamming graphs and proof of Babai's conjecture

One of the central results in the representation theory of distance-regular graphs classifies distance-regular graphs with $μ\geq 2$ and second largest eigenvalue $θ_1= b_1-1$. In this paper we give a classification under the (weaker) approximate eigenvalue constraint $θ_1\geq (1-\varepsilon)b_1$ for the class of geometric distance-regular graphs. As an application, we confirm Babai's conjecture on the minimal degree of the automorphism group of distance-regular graphs.

math.CO↗

arXiv2019

On the spectral gap and the automorphism group of distance-regular graphs

We prove that a distance-regular graph with a dominant distance is a spectral expander. The key ingredient of the proof is a new inequality on the intersection numbers. We use the spectral gap bound to study the structure of the automorphism group. The minimal degree of a permutation group $G$ is the minimum number of points not fixed by non-identity elements of $G$. Lower bounds on the minimal degree have strong structural consequences on $G$. In 2014 Babai proved that the automorphism group of a strongly regular graph with $n$ vertices has minimal degree $\geq c n$, with known exceptions. Strongly regular graphs correspond to distance-regular graphs of diameter 2. Babai conjectured that Hamming and Johnson graphs are the only primitive distance-regular graphs of diameter $d\geq 3$ whose automorphism group has sublinear minimal degree. We confirm this conjecture for non-geometric primitive distance-regular graphs of bounded diameter. We also show if the primitivity assumption is removed, then only one additional family of exceptions arises, the cocktail-party graphs. We settle the geometric case in a companion paper.

math.CO↗

arXiv2018

On the automorphism groups of distance-regular graphs and rank-4 primitive coherent configurations

The minimal degree of a permutation group $G$ is the minimum number of points not fixed by non-identity elements of $G$. Lower bounds on the minimal degree have strong structural consequences on $G$. In 2014 Babai proved that the automorphism group of a strongly regular graph with $n$ vertices has minimal degree $\geq c n$, with known exceptions. Strongly regular graphs correspond to primitive coherent configurations of rank 3. We extend Babai's result to primitive coherent configurations of rank 4. We also show that the result extends to non-geometric distance-regular graphs of bounded diameter. The proofs combine structural and spectral methods.

math.CO↗

arXiv2017

Automaton groups and complete square complexes

The first example of a non-residually finite group in the classes of finitely presented small-cancelation groups, automatic groups, and CAT(0) groups was constructed by Wise as the fundamental group of a complete square complex (CSC for short) with twelve squares. At the same time, Janzen and Wise proved that CSCs with at most three squares, five or seven squares have residually finite fundamental group. The smallest open cases were CSCs with four squares and directed complete VH complexes with six squares. We prove that the CSC with four squares studied by Janzen and Wise has a non-residually finite fundamental group. In particular, this gives a non-residually finite CAT(0) group isometric to $F_2\times F_2$. For the class of complete directed VH complexes, we prove that there are exactly two complexes with six squares having a non-residually finite fundamental group. In particular, this positively answers to a question of Wise on whether the main example from his PhD thesis is non-residually finite. As a by-product, we get finitely presented torsion-free simple groups which decompose into an amalgamated free product of free groups $F_7*_{F_{49}}F_7$. Our approach relies on the connection between square complexes and automata discovered by Glasner and Mozes, where complete VH complexes with one vertex correspond to bireversible automata. We prove that the square complex associated to a bireversible automaton with two states or over the binary alphabet generating an infinite automaton group has a non-residually finite fundamental group. We describe automaton groups associated to CSCs with four squares and get two simple automaton representations of the free group $F_2$ and the first automaton representation of the free product $C_3*C_3$.

math.GR↗