arXiv subjects

James G. Dowty

Publications and source records attributed to James G. Dowty.

8 recordsLinked to original sources

arXiv2017

Chentsov's theorem for exponential families

Chentsov's theorem characterizes the Fisher information metric on statistical models as essentially the only Riemannian metric that is invariant under sufficient statistics. This implies that each statistical model is naturally equipped with a geometry, so Chentsov's theorem explains why many statistical properties can be described in geometric terms. However, despite being one of the foundational theorems of statistics, Chentsov's theorem has only been proved previously in very restricted settings or under relatively strong regularity and invariance assumptions. We therefore prove a version of this theorem for the important case of exponential families. In particular, we characterise the Fisher information metric as the only Riemannian metric (up to rescaling) on an exponential family and its derived families that is invariant under independent and identically distributed extensions and canonical sufficient statistics. Our approach is based on the central limit theorem, so it gives a unified proof for both discrete and continuous exponential families, and it is less technical than previous approaches.

math.ST↗

arXiv2015

Higher-order asymptotics for the parametric complexity

The parametric complexity is the key quantity in the minimum description length (MDL) approach to statistical model selection. Rissanen and others have shown that the parametric complexity of a statistical model approaches a simple function of the Fisher information volume of the model as the sample size $n$ goes to infinity. This paper derives higher-order asymptotic expansions for the parametric complexity, in the case of exponential families and independent and identically distributed data. These higher-order approximations are calculated for some examples and are shown to have better finite-sample behaviour than Rissanen's approximation. The higher-order terms are given as expressions involving cumulants (or, more naturally, the Amari-Chentsov tensors), and these terms are likely to be interesting in themselves since they arise naturally from the general information-theoretic principles underpinning MDL. The derivation given here specializes to an alternative and arguably simpler proof of Rissanen's result (for the case considered here), proving for the first time that his approximation is $O(n^{-1})$.

cs.IT↗

arXiv2014

Volumes of logistic regression models with applications to model selection

Logistic regression models with $n$ observations and $q$ linearly-independent covariates are shown to have Fisher information volumes which are bounded below by $π^q$ and above by ${n \choose q} π^q$. This is proved with a novel generalization of the classical theorems of Pythagoras and de Gua, which is of independent interest. The finding that the volume is always finite is new, and it implies that the volume can be directly interpreted as a measure of model complexity. The volume is shown to be a continuous function of the design matrix $X$ at generic $X$, but to be discontinuous in general. This means that models with sparse design matrices can be significantly less complex than nearby models, so the resulting model-selection criterion prefers sparse models. This is analogous to the way that $\ell^1$-regularisation tends to prefer sparse model fits, though in our case this behaviour arises spontaneously from general principles. Lastly, an unusual topological duality is shown to exist between the ideal boundaries of the natural and expectation parameter spaces of logistic regression models.

math.ST↗

arXiv2014

SMML estimators for exponential families with continuous sufficient statistics

The minimum message length principle is an information theoretic criterion that links data compression with statistical inference. This paper studies the strict minimum message length (SMML) estimator for $d$-dimensional exponential families with continuous sufficient statistics, for all $d \ge 1$. The partition of an SMML estimator is shown to consist of convex polytopes (i.e. convex polygons when $d=2$) which can be described explicitly in terms of the assertions and coding probabilities. While this result is known, we give a new proof based on the calculus of variations, and this approach gives some interesting new inequalities for SMML estimators. We also use this result to construct an SMML estimator for a $2$-dimensional normal random variable with known variance and a normal prior on its mean.

cs.IT↗

arXiv2014

SMML estimators for linear regression and tessellations of hyperbolic space

The strict minimum message length (SMML) principle links data compression with inductive inference. The corresponding estimators have many useful properties but they can be hard to calculate. We investigate SMML estimators for linear regression models and we show that they have close connections to hyperbolic geometry. When equipped with the Fisher information metric, the linear regression model with $p$ covariates and a sample size of $n$ becomes a Riemannian manifold, and we show that this is isometric to $(p+1)$-dimensional hyperbolic space $\mathbb{H}^{p+1}$ equipped with a metric tensor which is $2n$ times the usual metric tensor on $\mathbb{H}^{p+1}$. A natural identification then allows us to also view the set of sufficient statistics for the linear regression model as a hyperbolic space. We show that the partition of an SMML estimator corresponds to a tessellation of this hyperbolic space.

cs.IT↗

arXiv2013

The time-evolution of DCIS size distributions with applications to breast cancer growth and progression

Ductal carcinoma {\em in situ} (DCIS) lesions are non-invasive tumours of the breast which are thought to precede most invasive breast cancers (IBC). As individual DCIS lesions are initiated, grow and invade (i.e. become IBC) the size distribution of the DCIS lesions present in a given human population will evolve. We derive a differential equation governing this evolution and show, for given assumptions about growth and invasion, that there is a unique distribution which does not vary with time. Further, we show that any initial distribution converges to this stationary distribution exponentially quickly. It is therefore reasonable to assume that the stationary distribution is equal to the true DCIS size distribution, at least for human populations which are relatively stable with respect to the determinants of breast cancer. Based on this assumption and the size data of 110 DCIS lesions detected in a mammographic screening program between 1993 and 2000, we produce maximum likelihood estimates for certain growth and invasion parameters. Assuming that DCIS size is proportional to a positive power $p$ of the time since tumour initiation we estimate $p$ to be 0.50 with a 95% confidence interval of $(0.35, 0.71)$. Therefore we estimate that DCIS lesions follow a square-root growth law and hence that they grow rapidly when small and relatively slowly when large. Our approach and results should be useful for other mathematical studies of cancer, especially those investigating biological mechanisms of invasion.

q-bio.PE↗

arXiv2012

SMML estimators for 1-dimensional continuous data

A method is given for calculating the strict minimum message length (SMML) estimator for 1-dimensional exponential families with continuous sufficient statistics. A set of $n$ equations are found that the $n$ cut-points of the SMML estimator must satisfy. These equations can be solved using Newton's method and this approach is used to produce new results and to replicate results that C. S. Wallace obtained using his boundary rules for the SMML estimator. A rigorous proof is also given that, despite being composed of step functions, the posterior probability corresponding to the SMML estimator is a continuous function of the data.

cs.IT↗

arXiv2002

A new invariant on hyperbolic Dehn surgery space

In this paper we define a new invariant of the incomplete hyperbolic structures on a 1-cusped finite volume hyperbolic 3-manifold M, called the ortholength invariant. We show that away from a (possibly empty) subvariety of excluded values this invariant both locally parameterises equivalence classes of hyperbolic structures and is a complete invariant of the Dehn fillings of M which admit a hyperbolic structure. We also give an explicit formula for the ortholength invariant in terms of the traces of the holonomies of certain loops in M. Conjecturally this new invariant is intimately related to the boundary of the hyperbolic Dehn surgery space of M.

math.GT↗