arXiv subjects

Alexander L Young

Publications and source records attributed to Alexander L Young.

4 recordsLinked to original sources

arXiv2024

Targeted stochastic gradient Markov chain Monte Carlo for hidden Markov models with rare latent states

Markov chain Monte Carlo (MCMC) algorithms for hidden Markov models often rely on the forward-backward sampler. This makes them computationally slow as the length of the time series increases, motivating the development of sub-sampling-based approaches. These approximate the full posterior by using small random subsequences of the data at each MCMC iteration within stochastic gradient MCMC. In the presence of imbalanced data resulting from rare latent states, subsequences often exclude rare latent state data, leading to inaccurate inference and prediction/detection of rare events. We propose a targeted sub-sampling (TASS) approach that over-samples observations corresponding to rare latent states when calculating the stochastic gradient of parameters associated with them. TASS uses an initial clustering of the data to construct subsequence weights that reduce the variance in gradient estimation. This leads to improved sampling efficiency, in particular in settings where the rare latent states correspond to extreme observations. We demonstrate substantial gains in predictive and inferential accuracy on real and synthetic examples.

stat.ML↗

arXiv2019

Consistent Entropy Estimation for Stationary Time Series

Entropy estimation, due in part to its connection with mutual information, has seen considerable use in the study of time series data including causality detection and information flow. In many cases, the entropy is estimated using $k$-nearest neighbor (Kozachenko-Leonenko) based methods. However, analytic results on this estimator are limited to independent data. In the article, we show rigorous bounds on the rate of decay of the bias in the number of samples, $N$, assuming they are drawn from a stationary process which satisfies a suitable mixing condition. Numerical examples are presented which demonstrate the efficiency of the estimator when applied to a Markov process with stationary Gaussian density. These results support the asymptotic rates derived in the theoretical work.

math.ST↗

arXiv2018

Bayesian Constraint Relaxation

Prior information often takes the form of parameter constraints. Bayesian methods include such information through prior distributions having constrained support. By using posterior sampling algorithms, one can quantify uncertainty without relying on asymptotic approximations. However, sharply constrained priors are (a) not necessary in some settings; and (b) tend to limit modeling scope to a narrow set of distributions that are tractable computationally. Inspired by the vast literature that replaces the slab-and-spike prior with a continuous approximation, we propose to replace the sharp indicator function of the constraint with an exponential kernel, thereby creating a close-to-constrained neighborhood within the Euclidean space in which the constrained subspace is embedded. This kernel decays with distance from the constrained space at a rate depending on a relaxation hyperparameter. By avoiding the sharp constraint, we enable use of off-the-shelf posterior sampling algorithms, such as Hamiltonian Monte Carlo, facilitating automatic computation in broad models. We study the constrained and relaxed distributions under multiple settings, and theoretically quantify their differences. We illustrate the method through multiple novel modeling examples.

stat.ME↗

arXiv2017

On collisions times of self-sorting interacting particles in one-dimension with random initial positions and velocities

We investigate a one-dimensional system of $N$ particles, initially distributed with random positions and velocities, interacting through binary collisions. The collision rule is such that there is a time after which the $N$ particles do not interact and become sorted according to their velocities. When the collisions are elastic, we derive asymptotic distributions for the final collision time of a single particle and the final collision time of the system as the number of particles approaches infinity, under different assumptions for the initial distributions of the particles' positions and velocities. For comparison, a numerical investigation is carried out to determine how a non-elastic collision rule, which conserves neither momentum nor energy, affects the median collision time of a particle and the median final collision time of the system.

math-ph↗