arXiv subjects

Diego Granziol

Publications and source records attributed to Diego Granziol.

21 records · Page 2Linked to original sources

arXiv2017

Entropic Determinants

The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example. In this paper we demonstrate the optimality of Maximum Entropy methods in approximating such calculations. We prove the equivalence between mean value constraints and sample expectations in the big data limit, that Covariance matrix eigenvalue distributions can be completely defined by moment information and that the reduction of the self entropy of a maximum entropy proposal distribution, achieved by adding more moments reduces the KL divergence between the proposal and true eigenvalue distribution. We empirically verify our results on a variety of SparseSuite matrices and establish best practices.

stat.ML↗

arXiv2017

Entropic Trace Estimates for Log Determinants

The scalable calculation of matrix determinants has been a bottleneck to the widespread application of many machine learning methods such as determinantal point processes, Gaussian processes, generalised Markov random fields, graph models and many others. In this work, we estimate log determinants under the framework of maximum entropy, given information in the form of moment constraints from stochastic trace estimation. The estimates demonstrate a significant improvement on state-of-the-art alternative methods, as shown on a wide variety of UFL sparse matrices. By taking the example of a general Markov random field, we also demonstrate how this approach can significantly accelerate inference in large-scale learning methods involving the log determinant.

math.NA↗

arXiv2017

An information and field theoretic approach to the grand canonical ensemble

We present a novel derivation of the constraints required to obtain the underlying principles of statistical mechanics using a maximum entropy framework. We derive the mean value constraints by use of the central limit theorem and the scaling properties of Lagrange multipliers. We then arrive at the same result using a quantum free field theory and the Ward identities. The work provides a principled footing for maximum entropy methods in statistical physics, adding the body of work aligned to Jaynes's vision of statistical mechanics as a form of inference rather than a physical theory dependent on ergodicity, metric transitivity and equal a priori probabilities. We show that statistical independence, in the macroscopic limit, is the unifying concept that leads to all these derivations.

cond-mat.stat-mech↗