arXiv subjects

Piotr Zwiernik

Publications and source records attributed to Piotr Zwiernik.

At least 19 recordsLinked to original sources

arXiv2026

Bayesian Networks of Density Operators

We study quantum analogues of Bayesian networks on a directed acyclic graph (DAG), distinguishing two constructions for positive definite density operators on finite-dimensional tensor-product Hilbert spaces. The intrinsic construction starts from a joint state and its conditional-independence properties. The extrinsic construction assembles a state sequentially from prescribed local quantum kernels, following an ordering compatible with the arrows of the DAG. For the intrinsic construction, we prove the equivalence of the ordered, local, and global directed Markov properties, together with entropy, recursive-factorization, and logarithmic characterizations. The extrinsic construction always gives a normalized state and recovers each kernel as a conditional on all preceding systems. The same kernel, however, need not be recovered from the marginal on the vertex and its parents; a three-qubit example exhibits this obstruction. We prove that independence of the chosen topological ordering is sufficient exactly for transitive DAGs: every order-invariant kernel family then yields an intrinsically directed Markov state. Finally, we associate a logarithmic candidate with every positive definite state and DAG, prove that it is subnormalized, and show that the trace-one candidate is a directed Markov state. Both the candidate and the excess global information are invariant under DAG Markov equivalence.

quant-ph↗

arXiv2026

Universality of Benign Overfitting in Binary Linear Classification

The practical success of deep learning has led to the discovery of several surprising phenomena. One of these phenomena, that has spurred intense theoretical research, is ``benign overfitting'': deep neural networks seem to generalize well in the over-parametrized regime even though the networks show a perfect fit to noisy training data. It is now known that benign overfitting also occurs in various classical statistical models. For linear maximum margin classifiers, benign overfitting has been established theoretically in a class of mixture models with very strong assumptions on the covariate distribution. However, even in this simple setting, many questions remain open. For instance, most of the existing literature focuses on the noiseless case where all true class labels are observed without errors, whereas the more interesting noisy case remains poorly understood. We provide a comprehensive study of benign overfitting for linear maximum margin classifiers. We discover a phase transition in test error bounds for the noisy model which was previously unknown and provide some geometric intuition behind it. We further considerably relax the required covariate assumptions in both the noisy and noiseless cases. Our results demonstrate that benign overfitting of maximum margin classifiers holds in a much wider range of scenarios than was previously known and provide new insights into the underlying mechanisms.

cs.LG↗

arXiv2026

The Markov Marginal Problem for Density Operators

We study when local reduced density operators, viewed as quantum marginals, can be assembled into a global quantum state with a prescribed Markov structure. The starting point is a canonical logarithmic construction $T(\mathcal R)$, the noncommutative analogue of the junction-tree formula for decomposable graphical models. Unlike in the classical case, this formal construction may fail: noncommutativity can prevent it from being a normalized state with the prescribed marginals. We prove that this obstruction is captured exactly by a trace condition. For two overlapping marginals, and for clique marginals on a chordal graph, the condition ${\rm Tr}(T(\mathcal R))=1$ is equivalent to the existence of a quantum Markov completion. When it exists, the completion is unique, equal to $T(\mathcal R)$, and selected by the maximum entropy principle. In the two-clique case, we also give an equivalent conditional reconstruction characterization: the two natural one-sided sandwich reconstructions agree if and only if the trace condition holds. We introduce the global quantum information $g{\rm I}(\mathcal{G})_ρ$ associated with a chordal graph $\mathcal{G}$ and show that it is a relative-entropy discrepancy from $ρ$ to the logarithmic candidate, with a trace correction when the candidate is not normalized. We also prove an intersection property for strictly positive quantum conditional independence. Three-qubit Pauli examples illustrate how the quantum obstructions are real: local consistency, feasibility, Markov feasibility, and maximum entropy can all separate.

quant-ph↗

arXiv2026

Testing properties of trees in graphical models with covariance queries

We consider the problem of testing properties of graphs underlying high-dimensional graphical models. We adopt the model of covariance queries introduced by Lugosi, Truszkowski, Velona, and Zwiernik (2021). We study the case when the underlying graph is a tree. The main results of the paper show that, while reconstructing the entire tree may be costly, certain global structural properties can be tested efficiently. In particular, we design randomized tests for global structural properties that use a sub-quadratic number of queries. We develop testing procedures for several fundamental properties, including the number of leaves, the maximum degree, the typical distance, and the diameter of the tree. For each property, we obtain explicit query complexity bounds that depend on the target threshold and tolerance parameters.

stat.ML↗

arXiv2026

Causal discovery under mean independence and linearity

Causal discovery methods such as LiNGAM identify causal structure from observational data by assuming mutually independent disturbances. This assumption is fragile: shared volatility, common scale effects, or other forms of dependence can cause the methods to recover the wrong causal order, even with infinite data. We introduce the Linear Mean-Independent Acyclic Model (LiMIAM), which replaces full independence with weaker one-sided mean-independence restrictions on the disturbances. Under finite-order consequences of these restrictions, source nodes are generically identifiable, and hence a compatible causal order can be recovered recursively. Our proof is constructive and leads to DirectLiMIAM, a sequential residual-based algorithm for causal discovery under dependent noise. In simulations with mean-independent but dependent disturbances, DirectLiMIAM outperforms LiNGAM methods. A large-scale empirical application to the oil market highlights the implausibility of the independence assumption and the ability of DirectLiMIAM to recover a realistic causal ordering, from policy to production and from prices to inflation.

stat.ME↗

arXiv2026

Improving variable selection properties with data integration and transfer learning

We study variable selection (also called support recovery) in high-dimensional sparse linear regression when one has external information on which variables are likely to be associated with the response. Consistent recovery is only possible under somewhat restrictive conditions on sample size, dimension, signal strength, and sparsity. We investigate how these conditions can be relaxed by incorporating said external information. A key application that we consider is structural transfer learning, where variables selected in one or more source datasets are used to guide variable selection in a target dataset. We introduce a family of likelihood penalties that depend on the external information, motivated by connections to Bayesian variable selection. We show that these methods achieve variable selection consistency in regimes where any method ignoring external information fails, and that they achieve consistency at faster rates. We first quantify the potential gains under ideal, oracle-chosen, penalties. We then propose computationally efficient empirical Bayes procedures that learn suitable penalties from the data. We prove that these procedures have improved variable selection properties compared to methods that do not use external information. We illustrate our approach using simulations and a genomics application, where results from mouse experiments are used to inform variable selection for gene expression data in humans.

math.ST↗

arXiv2025

Probabilistic PCA on tensors

In probabilistic principal component analysis (PPCA), an observed vector is modeled as a linear transformation of a low-dimensional Gaussian factor plus isotropic noise. We generalize PPCA to tensors by constraining the loading operator to have Tucker structure, yielding a probabilistic multilinear PCA model that enables uncertainty quantification and naturally accommodates multiple, possibly heterogeneous, tensor observations. We develop the associated theory: we establish identifiability of the loadings and noise variance and show that-unlike in matrix PPCA-the maximum likelihood estimator (MLE) exists even from a single tensor sample. We then study two estimators. First, we consider the MLE and propose an expectation maximization (EM) algorithm to compute it. Second, exploiting that Tucker maps correspond to rank-one elements after a Kronecker lifting, we design a computationally efficient estimator for which we provide provable finite-sample guarantees. Together, these results provide a coherent probabilistic framework and practical algorithms for learning from tensor-valued data.

math.ST↗

arXiv2025

Beyond independent component analysis: identifiability and algorithms

Independent Component Analysis (ICA) is a classical method for recovering latent variables with useful identifiability properties. For independent variables, cumulant tensors are diagonal; relaxing independence yields tensors whose zero structure generalizes diagonality. These models have been the subject of recent work in non-independent component analysis. We show that pairwise mean independence answers the question of how much one can relax independence: it is identifiable, any weaker notion is non-identifiable, and it contains the models previously studied as special cases. Our results apply to distributions with the required zero pattern at any cumulant tensor. We propose an algebraic recovery algorithm based on least-squares optimization over the orthogonal group. Simulations highlight robustness: enforcing full independence can harm estimation, while pairwise mean independence enables more stable recovery. These findings extend the classical ICA framework and provide a rigorous basis for blind source separation beyond independence.

math.ST↗

arXiv2025

Learning latent tree models with small query complexity

We consider the problem of structure recovery in a graphical model of a tree where some variables are latent. Specifically, we focus on the Gaussian case, which can be reformulated as a well-studied problem: recovering a semi-labeled tree from a distance metric. We introduce randomized procedures that achieve query complexity of optimal order. Additionally, we provide statistical analysis for scenarios where the tree distances are noisy. The Gaussian setting can be extended to other situations, including the binary case and non-paranormal distributions.

math.ST↗

arXiv2025

Orthogonal eigenvectors and singular vectors of tensors

The spectral theorem says that a real symmetric matrix has an orthogonal basis of eigenvectors and that, for a matrix with distinct eigenvalues, the basis is unique (up to signs). In this paper, we study the symmetric tensors with an orthogonal basis of eigenvectors and show that, for a generic such tensor, the orthogonal basis is unique. This resolves a conjecture by Mesters and Zwiernik. We also study the non-symmetric setting. The singular value decomposition says that a real matrix has an orthogonal basis of singular vector pairs and that, for a matrix with distinct singular values, the basis is unique (up to signs). We describe the tensors with an orthogonal basis of singular vectors and show that a generic such tensor has a unique orthogonal basis, with one exceptional format: order four binary tensors. We use these results to propose a new tensor decomposition that generalizes an orthogonally decomposable decomposition and specializes the Tucker decomposition.

math.SP↗

arXiv2025

The Identification Problem for Linear Rational Expectations Models

This version corrects a number of mistakes that appeared in the previous draft. In particular, the (EU-LREM) condition is sufficient for existence and uniqueness but not necessary, as we had claimed. We are grateful to P. C. B. Phillips and to three anonymous referees for the substantial improvements to the paper since it first appeared. Any remaining errors are our own responsibility.

math.ST↗

arXiv2024

Tensors in algebraic statistics

Tensors are ubiquitous in statistics and data analysis. The central object that links data science to tensor theory and algebra is that of a model with latent variables. We provide an overview of tensor theory, with a particular emphasis on its applications in algebraic statistics. This high-level treatment is supported by numerous examples to illustrate key concepts. Additionally, an extensive literature review is included to guide readers toward more detailed studies on the subject.

math.ST↗

arXiv2024

Property testing in graphical models: testing small separation numbers

In many statistical applications, the dimension is too large to handle for standard high-dimensional machine learning procedures. This is particularly true for graphical models, where the interpretation of a large graph is difficult and learning its structure is often computationally impossible either because the underlying graph is not sufficiently sparse or the number of vertices is too large. To address this issue, we develop a procedure to test a property of a graph underlying a graphical model that requires only a subquadratic number of correlation queries (i.e., we require that the algorithm only can access a tiny fraction of the covariance matrix). This provides a conceptually simple test to determine whether the underlying graph is a tree or, more generally, if it has a small separation number, a quantity closely related to the treewidth of the graph. The proposed method is a divide-and-conquer algorithm that can be applied to quite general graphical models.

math.ST↗

arXiv2024

Entropic covariance models

In covariance matrix estimation, one of the challenges lies in finding a suitable model and an efficient estimation method. Two commonly used modelling approaches in the literature involve imposing linear restrictions on the covariance matrix or its inverse. Another approach considers linear restrictions on the matrix logarithm of the covariance matrix. In this paper, we present a general framework for linear restrictions on different transformations of the covariance matrix, including the mentioned examples. Our proposed estimation method solves a convex problem and yields an $M$-estimator, allowing for relatively straightforward asymptotic (in general) and finite sample analysis (in the Gaussian case). In particular, we recover standard $\sqrt{n/d}$ rates, where $d$ is the dimension of the underlying model. Our geometric insights allow to extend various recent results in covariance matrix modelling. This includes providing unrestricted parametrizations of the space of correlation matrices, which is alternative to a recent result utilizing the matrix logarithm.

math.ST↗

arXiv2024

Non-Independent Components Analysis

A seminal result in the ICA literature states that for $AY = \varepsilon$, if the components of $\varepsilon$ are independent and at most one is Gaussian, then $A$ is identified up to sign and permutation of its rows (Comon, 1994). In this paper we study to which extent the independence assumption can be relaxed by replacing it with restrictions on higher order moment or cumulant tensors of $\varepsilon$. We document new conditions that establish identification for several non-independent component models, e.g. common variance models, and propose efficient estimation methods based on the identification results. We show that in situations where independence cannot be assumed the efficiency gains can be significant relative to methods that rely on independence.

math.ST↗

arXiv2023

Graphical model inference with external network data

We consider two applications where we study how dependence structure between many variables is linked to external network data. We first study the interplay between social media connectedness and the co-evolution of the COVID-19 pandemic across USA counties. We next study study how the dependence between stock market returns across firms relates to similarities in economic and policy indicators from text regulatory filings. Both applications are modelled via Gaussian graphical models where one has external network data. We develop spike-and-slab and graphical LASSO frameworks to integrate the network data, both facilitating the interpretation of the graphical model and improving inference. The goal is to detect when the network data relates to the graphical model and, if so, explain how. We found that counties strongly connected on Facebook are more likely to have similar COVID-19 evolution (positive partial correlations), accounting for various factors driving the mean. We also found that the association in stock market returns depends in a stronger fashion on economic than on policy indicators. The examples show that data integration can improve interpretation, statistical accuracy, and out-of-sample prediction, in some instances using significantly sparser graphical models.

stat.AP↗

arXiv2023

Total positivity in multivariate extremes

Positive dependence is present in many real world data sets and has appealing stochastic properties that can be exploited in statistical modeling and in estimation. In particular, the notion of multivariate total positivity of order 2 ($ \mathrm{MTP}_{2} $) is a convex constraint and acts as an implicit regularizer in the Gaussian case. We study positive dependence in multivariate extremes and introduce $ \mathrm{EMTP}_{2} $, an extremal version of $ \mathrm{MTP}_{2} $. This notion turns out to appear prominently in extremes, and in fact, it is satisfied by many classical models. For a Hüsler--Reiss distribution, the analogue of a Gaussian distribution in extremes, we show that it is $ \mathrm{EMTP}_{2} $ if and only if its precision matrix is a Laplacian of a connected graph. We propose an estimator for the parameters of the Hüsler--Reiss distribution under $ \mathrm{EMTP}_{2} $ as the solution of a convex optimization problem with Laplacian constraint. We prove that this estimator is consistent and typically yields a sparse model with possibly nondecomposable extremal graphical structure. Applying our methods to a data set of Danube River flows, we illustrate this regularization and the superior performance compared to existing methods.

math.ST↗

arXiv2023

Positivity in Linear Gaussian Structural Equation Models

We study a notion of positivity of Gaussian directed acyclic graphical models corresponding to a non-negativity constraint on the coefficients of the associated structural equation model. We prove that this constraint is equivalent to the distribution being conditionally increasing in sequence (CIS), a well-known subclass of positively associated random variables. These distributions require knowledge of a permutation, a CIS ordering, of the nodes for which the constraint of non-negativity holds. We provide an algorithm and prove in the noise-less setting that a CIS ordering can be recovered when it exists. We extend this result to the noisy setting and provide assumptions for recovering the CIS orderings. In addition, we provide a characterization of Markov equivalence for CIS DAG models. Further, we show that when a CIS ordering is known, the corresponding class of Gaussians lies in a family of distributions in which maximum likelihood estimation is a convex problem.

math.ST↗