arXiv subjects

Nigel J. Newton

Publications and source records attributed to Nigel J. Newton.

6 recordsLinked to original sources

arXiv2019

A Class of Non-Parametric Statistical Manifolds modelled on Sobolev Space

We construct a family of non-parametric (infinite-dimensional) manifolds of finite measures on $R^d$. The manifolds are modelled on a variety of weighted Sobolev spaces, including Hilbert-Sobolev spaces and mixed-norm spaces. Each supports the Fisher-Rao metric as a weak Riemannian metric. Densities are expressed in terms of a deformed exponential function having linear growth. Unusually for the Sobolev context, and as a consequence of its linear growth, this "lifts" to a nonlinear superposition (Nemytskii) operator that acts continuously on a particular class of mixed-norm model spaces, and on the fixed norm space $W^{2,1}$; i.e. it maps each of these spaces continuously into itself. It also maps continuously between other fixed-norm spaces with a loss of Lebesgue exponent that increases with the number of derivatives. Some of the results make essential use of a log-Sobolev embedding theorem. Each manifold contains a smoothly embedded submanifold of probability measures. Applications to the stochastic partial differential equations of nonlinear filtering (and hence to the Fokker-Planck equation) are outlined.

math.PR↗

arXiv2018

Manifolds of Differentiable Densities

We develop a family of infinite-dimensional (non-parametric) manifolds of probability measures. The latter are defined on underlying Banach spaces, and have densities of class $C_b^k$ with respect to appropriate reference measures. The case $k=\infty$, in which the manifolds are modelled on Fréchet spaces, is included. The manifolds admit the Fisher-Rao metric and, unusually for the non-parametric setting, Amari's $α$-covariant derivatives for all $α\in R$. By construction, they are $C^\infty$-embedded submanifolds of particular manifolds of finite measures. The statistical manifolds are dually ($α=\pm 1$) flat, and admit mixture and exponential representations as charts. Their curvatures with respect to the $α$-covariant derivatives are derived. The likelihood function associated with a finite sample is a continuous function on each of the manifolds, and the $α$-divergences are of class $C^\infty$.

math.PR↗

arXiv2016

Transfer Entropy and Directed Information in Gaussian diffusion processes

Transfer Entropy and Directed Information are information-theoretic measures of the directional dependency between stochastic processes. Following the definitions of Schreiber and Massey in discrete time, we define and evaluate these measures for the components of multidimensional Gaussian diffusion processes. When the components are jointly Markov, the Transfer Entropy and Directed Information are both measures of influence according to a simple physical principle. More generally, the effect of other components has to be accounted for, and this can be achieved in more than one way. We propose two definitions, one of which preserves the properties of influence of the jointly Markov case. The Transfer Entropy and Directed Information are expressed in terms of the solutions of matrix Riccati equations, and so are easy to compute. The definition of continuous-time Directed Information we propose differs from that previously appearing in the literature. We argue that the latter is not strictly directional.

math.PR↗

arXiv2016

Infinite-dimensional statistical manifolds based on a balanced chart

We develop a family of infinite-dimensional Banach manifolds of measures on an abstract measurable space, employing charts that are "balanced" between the density and log-density functions. The manifolds, $(\tilde{M}_λ,λ\in [2,\infty))$, retain many of the features of finite-dimensional information geometry; in particular, the $α$-divergences are of class $C^{\lceilλ\rceil-1}$, enabling the definition of the Fisher metric and $α$-derivatives of particular classes of vector fields. Manifolds of probability measures, $(M_λ,λ\in [2,\infty))$, based on centred versions of the charts are shown to be $C^{\lceilλ\rceil-1}$-embedded submanifolds of the $\tilde{M}_λ$. The Fisher metric is a pseudo-Riemannian metric on $\tilde{M}_λ$. However, when restricted to finite-dimensional embedded submanifolds it becomes a Riemannian metric, allowing the full development of the geometry of $α$-covariant derivatives. $\tilde{M}_λ$ and $M_λ$ provide natural settings for the study and comparison of approximations to posterior distributions in problems of Bayesian estimation.

math.PR↗

arXiv2015

Information Geometric Nonlinear Filtering

This paper develops information geometric representations for nonlinear filters in continuous time. The posterior distribution associated with an abstract nonlinear filtering problem is shown to satisfy a stochastic differential equation on a Hilbert information manifold. This supports the Fisher metric as a pseudo-Riemannian metric. Flows of Shannon information are shown to be connected with the quadratic variation of the process of posterior distributions in this metric. Apart from providing a suitable setting in which to study such information-theoretic properties, the Hilbert manifold has an appropriate topology from the point of view of multi-objective filter approximations. A general class of finite-dimensional exponential filters is shown to fit within this framework, and an intrinsic evolution equation, involving Amari's $-1$-covariant derivative, is developed for such filters. Three example systems, one of infinite dimension, are developed in detail.

math.PR↗

arXiv2014

Maximum work extraction and implementation costs for non-equilibrium Maxwell's demons

In this theoretical study, we determine the maximum amount of work extractable in finite time by a demon performing continuous measurements on a quadratic Hamiltonian system subjected to thermal fluctuations, in terms of the information extracted from the system. This is in contrast to many recent studies that focus on demons' maximizing the extracted work over received information, and operate close to equilibrium. The maximum work demon is found to apply a high-gain continuous feedback using a Kalman-Bucy estimate of the system state. A simple and concrete electrical implementation of the feedback protocol is proposed, which allows for analytic expressions of the flows of energy and entropy inside the demon. This let us show that any implementation of the demon must necessarily include an external power source, which we prove both from classical thermodynamics arguments and from a version of Landauer's memory erasure argument extended to non-equilibrium linear systems.

cond-mat.stat-mech↗