arXiv subjects

Tobias Rydén

Publications and source records attributed to Tobias Rydén.

6 recordsLinked to original sources

arXiv2025

Inverse Covariance and Partial Correlation Matrix Estimation via Joint Partial Regression

We present a method for estimating sparse high-dimensional inverse covariance and partial correlation matrices, which exploits the connection between the inverse covariance matrix and linear regression. The method is a two-stage estimation method wherein each individual feature is regressed on all other features while positive semi-definiteness is enforced simultaneously. We derive non-asymptotic estimation rates for both inverse covariance and partial correlation matrix estimation. An efficient proximal splitting algorithm for numerically computing the estimate is also dervied. The effectiveness of the proposed method is demonstrated on both synthetic and real-world data.

stat.ML↗

arXiv2010

Metropolising forward particle filtering backward sampling and Rao-Blackwellisation of Metropolised particle smoothers

Smoothing in state-space models amounts to computing the conditional distribution of the latent state trajectory, given observations, or expectations of functionals of the state trajectory with respect to this distributions. For models that are not linear Gaussian or possess finite state space, smoothing distributions are in general infeasible to compute as they involve intergrals over a space of dimensionality at least equal to the number of observations. Recent years have seen an increased interest in Monte Carlo-based methods for smoothing, often involving particle filters. One such method is to approximate filter distributions with a particle filter, and then to simulate backwards on the trellis of particles using a backward kernel. We show that by supplementing this procedure with a Metropolis-Hastings step deciding whether to accept a proposed trajectory or not, one obtains a Markov chain Monte Carlo scheme whose stationary distribution is the exact smoothing distribution. We also show that in this procedure, backward sampling can be replaced by backward smoothing, which effectively means averaging over all possible trajectories. In an example we compare these approaches to a similar one recently proposed by Andrieu, Doucet and Holenstein, and show that the new methods can be more efficient in terms of precision (inverse variance) per computation time.

stat.CO↗

arXiv2009

Subspace estimation and prediction methods for hidden Markov models

Hidden Markov models (HMMs) are probabilistic functions of finite Markov chains, or, put in other words, state space models with finite state space. In this paper, we examine subspace estimation methods for HMMs whose output lies a finite set as well. In particular, we study the geometric structure arising from the nonminimality of the linear state space representation of HMMs, and consistency of a subspace algorithm arising from a certain factorization of the singular value decomposition of the estimated linear prediction matrix. For this algorithm, we show that the estimates of the transition and emission probability matrices are consistent up to a similarity transformation, and that the $m$-step linear predictor computed from the estimated system matrices is consistent, i.e., converges to the true optimal linear $m$-step predictor.

math.ST↗

arXiv2006

Fast simulated annealing in $\R^d$ and an application to maximum likelihood estimation

Using classical simulated annealing to maximise a function $ψ$ defined on a subset of $\R^d$, the probability $\p(ψ(θ\_n)\leq ψ\_{\max}-ε)$ tends to zero at a logarithmic rate as $n$ increases; here $θ\_n$ is the state in the $n$-th stage of the simulated annealing algorithm and $ψ\_{\max}$ is the maximal value of $ψ$. We propose a modified scheme for which this probability is of order $n^{-1/3}\log n$, and hence vanishes at an algebraic rate. To obtain this faster rate, the exponentially decaying acceptance probability of classical simulated annealing is replaced by a more heavy-tailed function, and the system is cooled faster. We also show how the algorithm may be applied to functions that cannot be computed exactly but only approximated, and give an example of maximising the log-likelihood function for a state-space model.

math.PR↗

arXiv2006

Nonparametric estimation of mixing densities for discrete distributions

By a mixture density is meant a density of the form $π_μ(\cdot)=\intπ_θ(\cdot)\timesμ(dθ)$, where $(π_θ)_{θ\inΘ}$ is a family of probability densities and $μ$ is a probability measure on $Θ$. We consider the problem of identifying the unknown part of this model, the mixing distribution $μ$, from a finite sample of independent observations from $π_μ$. Assuming that the mixing distribution has a density function, we wish to estimate this density within appropriate function classes. A general approach is proposed and its scope of application is investigated in the case of discrete distributions. Mixtures of power series distributions are more specifically studied. Standard methods for density estimation, such as kernel estimators, are available in this context, and it has been shown that these methods are rate optimal or almost rate optimal in balls of various smoothness spaces. For instance, these results apply to mixtures of the Poisson distribution parameterized by its mean. Estimators based on orthogonal polynomial sequences have also been proposed and shown to achieve similar rates. The general approach of this paper extends and simplifies such results. For instance, it allows us to prove asymptotic minimax efficiency over certain smoothness classes of the above-mentioned polynomial estimator in the Poisson case. We also study discrete location mixtures, or discrete deconvolution, and mixtures of discrete uniform distributions.

math.ST↗

arXiv2002

Hidden Markov and state space models: asymptotic analysis of exact and approximate methods for prediction, filtering, smoothing and statistical inference

State space models have long played an important role in signal processing. The Gaussian case can be treated algorithmically using the famous Kalman filter. Similarly since the 1970s there has been extensive application of Hidden Markov models in speech recognition with prediction being the most important goal. The basic theoretical work here, in the case $X$ and $Y$ finite (small) providing both algorithms and asymptotic analysis for inference is that of Baum and colleagues. During the last 30-40 years these general models have proved of great value in applications ranging from genomics to finance. Unless the $X,Y$ are jointly Gaussian or $X$ is finite and small the problem of calculating the distributions discussed and the likelihood exactly are numerically intractable and if $Y$ is not finite asymptotic analysis becomes much more difficult. Some new developments have been the construction of so-called ``particle filters'' (Monte Carlo type) methods for approximate calculation of these distributions (see Doucet et al. [4]) for instance and general asymptotic methods for analysis of statistical methods in HMM [2] and other authors. We will discuss these methods and results in the light of exponential mixing properties of the conditional (posterior) distribution of $(X_1,X_2,...)$ given $(Y_1,Y_2,...)$ already noted by Baum and Petrie and recent work of the authors Bickel, Ritov and Ryden, Del Moral and Jacod, Douc and Matias.

math.ST↗