arXiv subjects
Publications and source records attributed to Jim Pitman.
In a classic paper Schröder posed four combinatorial problems about the number of certain types of bracketings of words and sets. Here we address what these bracketings look like on average. For each of the four problems we prove that a uniform pick from the appropriate set of bracketings, when considered as a tree, has the Brownian continuum random tree as its scaling limit as the size of the word or set goes to infinity.
We examine a new path transform on 1-dimensional simple random walks and Brownian motion, the quantile transform. This transformation relates to identities in fluctuation theory due to Wendel, Port, Dassios and others, and to discrete and Brownian versions of Tanaka's formula. For an n-step random walk, the quantile transform reorders increments according to the value of the walk at the start of each increment. We describe the distribution of the quantile transform of a simple random walk of n steps, using a bijection to characterize the number of pre-images of each possible transformed path. We deduce, both for simple random walks and for Brownian motion, that the quantile transform has the same distribution as Vervaat's transform. For Brownian motion, the quantile transforms of the embedded simple random walks converge to a time change of the local time profile. We characterize the distribution of the local time profile, giving rise to an identity that generalizes a variant of Jeulin's description of the local time profile of a Brownian bridge or excursion.
The problem of inferring a clustering of a data set has been the subject of much research in Bayesian analysis, and there currently exists a solid mathematical foundation for Bayesian approaches to clustering. In particular, the class of probability distributions over partitions of a data set has been characterized in a number of ways, including via exchangeable partition probability functions (EPPFs) and the Kingman paintbox. Here, we develop a generalization of the clustering problem, called feature allocation, where we allow each data point to belong to an arbitrary, non-negative integer number of groups, now called features or topics. We define and study an "exchangeable feature probability function" (EFPF)---analogous to the EPPF in the clustering setting---for certain types of feature models. Moreover, we introduce a "feature paintbox" characterization---analogous to the Kingman paintbox for clustering---of the class of exchangeable feature models. We provide a further characterization of the subclass of feature allocations that have EFPF representations.
We offer a unified approach to the theory of convex minorants of Lévy processes with continuous distributions. New results include simple explicit constructions of the convex minorant of a Lévy process on both finite and infinite time intervals, and of a Poisson point process of excursions above the convex minorant up to an independent exponential time. The Poisson-Dirichlet distribution of parameter 1 is shown to be the universal law of ranked lengths of excursions of a Lévy process with continuous distributions above its convex minorant on the interval $[0,1]$.
The early history of our department was dominated by Jerzy Neyman (1894-1981), while the next phase was largely in the hands of Neyman's students, with Erich Lehmann (1917-2009) being a central, long-lived and much-loved member of this group. We are very fortunate in having Constance Reid's biography "Neyman -- From Life" and Erich's "Reminiscences of a Statistician: The Company I Kept" and other historical material documenting the founding and growth of the department, and the people in it. In what follows, we will draw heavily from these sources, describing what seems to us to be a remarkable success story: one person starting "a cell of statistical research and teaching ... not being hampered by any existing traditions and routines" and seeing that cell grow rapidly into a major force in academic statistics worldwide. That it has remained so for (at least) the half-century after its founding is a testament to the strength of Neyman's model for a department of statistics.
We discuss a characterization of the centered Gaussian distribution which can be read from results of Archimedes and Maxwell, and relate it to Charles Stein's well-known characterization of the same distribution. These characterizations fit into a more general framework involving the beta-gamma algebra, which explains some other characterizations appearing in the Stein's method literature.
The beta-Bernoulli process provides a Bayesian nonparametric prior for models involving collections of binary-valued features. A draw from the beta process yields an infinite collection of probabilities in the unit interval, and a draw from the Bernoulli process turns these into binary-valued features. Recent work has provided stick-breaking representations for the beta process analogous to the well-known stick-breaking representation for the Dirichlet process. We derive one such stick-breaking representation directly from the characterization of the beta process as a completely random measure. This approach motivates a three-parameter generalization of the beta process, and we study the power laws that can be obtained from this generalized beta process. We present a posterior inference algorithm for the beta-Bernoulli process that exploits the stick-breaking representation, and we present experimental results for a discrete factor-analysis model.
We offer a unified approach to the theory of concave majorants of random walks by providing a path transformation for a walk of finite length that leaves the law of the walk unchanged whilst providing complete information about the concave majorant. This leads to a description of a walk of random geometric length as a Poisson point process of excursions away from its concave majorant, which is then used to find a complete description of the concave majorant for a walk of infinite length. In the case where subsets of increments may have the same arithmetic mean, we investigate three nested compositions that naturally arise from our construction of the concave majorant.
This article provides an overview of recent work on descriptions and properties of the convex minorant of random walks and Lévy processes which summarize and extend the literature on these subjects. The results surveyed include point process descriptions of the convex minorant of random walks and Lévy processes on a fixed finite interval, up to an independent exponential time, and in the infinite horizon case. These descriptions follow from the invariance of these processes under an adequate path transformation. In the case of Brownian motion, we note how further special properties of this process, including time-inversion, imply a sequential description for the convex minorant of the Brownian meander.
This article contains both a point process and a sequential description of the greatest convex minorant of Brownian motion on a finite interval. We use these descriptions to provide new analysis of various features of the convex minorant such as the set of times where the Brownian motion meets its minorant. The equivalence of the these descriptions is non-trivial, which leads to many interesting identities between quantities derived from our analysis. The sequential description can be viewed as a Markov chain for which we derive some fundamental properties.
We prove a long-standing conjecture which characterises the Ewens-Pitman two-parameter family of exchangeable random partitions, plus a short list of limit and exceptional cases, by the following property: for each $n = 2,3, >...$, if one of $n$ individuals is chosen uniformly at random, independently of the random partition $π_n$ of these individuals into various types, and all individuals of the same type as the chosen individual are deleted, then for each $r > 0$, given that $r$ individuals remain, these individuals are partitioned according to $π_r'$ for some sequence of random partitions $(π_r')$ that does not depend on $n$. An analogous result characterizes the associated Poisson-Dirichlet family of random discrete distributions by an independence property related to random deletion of a frequency chosen by a size-biased pick. We also survey the regenerative properties of members of the two-parameter family, and settle a question regarding the explicit arrangement of intervals with lengths given by the terms of the Poisson-Dirichlet random sequence into the interval partition induced by the range of a neutral-to-the right process.
We provide a representation of the maximal difference between a standard Brownian bridge and its concave majorant on the unit interval, from which we deduce expressions for the distribution and density functions and moments of this difference. This maximal difference has an application in nonparametric statistics where it arises in testing monotonicity of a density or regression curve.
We use a natural ordered extension of the Chinese Restaurant Process to grow a two-parameter family of binary self-similar continuum fragmentation trees. We provide an explicit embedding of Ford's sequence of alpha model trees in the continuum tree which we identified in a previous article as a distributional scaling limit of Ford's trees. In general, the Markov branching trees induced by the two-parameter growth rule are not sampling consistent, so the existence of compact limiting trees cannot be deduced from previous work on the sampling consistent case. We develop here a new approach to establish such limits, based on regenerative interval partitions and the urn-model description of sampling from Dirichlet random distributions.
We develop some theory of spinal decompositions of discrete and continuous fragmentation trees. Specifically, we consider a coarse and a fine spinal integer partition derived from spinal tree decompositions. We prove that for a two-parameter Poisson--Dirichlet family of continuous fragmentation trees, including the stable trees of Duquesne and Le Gall, the fine partition is obtained from the coarse one by shattering each of its parts independently, according to the same law. As a second application of spinal decompositions, we prove that among the continuous fragmentation trees, stable trees are the only ones whose distribution is invariant under uniform re-rooting.
We study fragmentation trees of Gibbs type. In the binary case, we identify the most general Gibbs-type fragmentation tree with Aldous' beta-splitting model, which has an extended parameter range $β>-2$ with respect to the ${\rm beta}(β+1,β+1)$ probability distributions on which it is based. In the multifurcating case, we show that Gibbs fragmentation trees are associated with the two-parameter Poisson--Dirichlet models for exchangeable random partitions of $\mathbb {N}$, with an extended parameter range $0\leα\le1$, $θ\ge-2α$ and $α<0$, $θ=-mα$, $m\in \mathbb {N}$.
Given any regularly varying dislocation measure, we identify a natural self-similar fragmentation tree as scaling limit of discrete fragmentation trees with unit edge lengths. As an application, we obtain continuum random tree limits of Aldous's beta-splitting models and Ford's alpha models for phylogenetic trees. This confirms in a strong way that the whole trees grow at the same speed as the mean height of a randomly chosen leaf.
We solve the moment problem for convex distribution functions on $[0,1]$ in terms of completely alternating sequences. This complements a recent solution of this problem by Diaconis and Freedman, and relates this work to the Lévy-Khintchine formula for the Laplace transform of a subordinator, and to regenerative composition structures.
The boundary problem for graphs like Pascal's but with general multiplicities of edges is related to a `backward' problem of moments of the Hausdorff type.