arXiv subjects

Takeru Matsuda

Publications and source records attributed to Takeru Matsuda.

At least 37 records · Page 2Linked to original sources

arXiv2023

Adapting to general quadratic loss via singular value shrinkage

The Gaussian sequence model is a canonical model in nonparametric estimation. In this study, we introduce a multivariate version of the Gaussian sequence model and investigate adaptive estimation over the multivariate Sobolev ellipsoids, where adaptation is not only to unknown smoothness but also to arbitrary quadratic loss. First, we derive an oracle inequality for the singular value shrinkage estimator by Efron and Morris, which is a matrix generalization of the James--Stein estimator. Next, we develop an asymptotically minimax estimator on the multivariate Sobolev ellipsoid for each quadratic loss, which can be viewed as a generalization of Pinsker's theorem. Then, we show that the blockwise Efron--Morris estimator is exactly adaptive minimax over the multivariate Sobolev ellipsoids under the corresponding quadratic loss. It attains sharp adaptive estimation of any linear combination of the mean sequences simultaneously.

math.ST↗

arXiv2023

Minimaxity under half-Cauchy type priors

This is a follow-up paper of Polson and Scott (2012, Bayesian Analysis), which claimed that the half-Cauchy prior is a sensible default prior for a scale parameter in hierarchical models. For estimation of a normal mean vector under the quadratic loss, they showed that the Bayes estimator with respect to the half-Cauchy prior seems to be minimax through numerical experiments. In terms of the shrinkage coefficient, the half-Cauchy prior has a U-shape and can be interpreted as a continuous spike and slab prior. In this paper, we consider a general class of priors with U-shapes and theoretically establish sufficient conditions for the minimaxity of the corresponding (generalized) Bayes estimators. We also develop an algorithm for posterior sampling and present numerical results.

math.ST↗

arXiv2023

Modelling the discretization error of initial value problems using the Wishart distribution

This paper presents a new discretization error quantification method for the numerical integration of ordinary differential equations. The error is modelled by using the Wishart distribution, which enables us to capture the correlation between variables. Error quantification is achieved by solving an optimization problem under the order constraints for the covariance matrices. An algorithm for the optimization problem is also established in a slightly broader context.

stat.ME↗

arXiv2023

Matrix quadratic risk of orthogonally invariant estimators for a normal mean matrix

In estimation of a normal mean matrix under the matrix quadratic loss, we develop a general formula for the matrix quadratic risk of orthogonally invariant estimators. The derivation is based on several formulas for matrix derivatives of orthogonally invariant functions of matrices. As an application, we calculate the matrix quadratic risk of a singular value shrinkage estimator motivated by Stein's proposal for improving on the Efron--Morris estimator 50 years ago.

math.ST↗

arXiv2023

Inadmissibility of the corrected Akaike information criterion

For the multivariate linear regression model with unknown covariance, the corrected Akaike information criterion is the minimum variance unbiased estimator of the expected Kullback--Leibler discrepancy. In this study, based on the loss estimation framework, we show its inadmissibility as an estimator of the Kullback--Leibler discrepancy itself, instead of the expected Kullback--Leibler discrepancy. We provide improved estimators of the Kullback--Leibler discrepancy that work well in reduced-rank situations and examine their performance numerically.

math.ST↗

arXiv2021

Information criteria for non-normalized models

Many statistical models are given in the form of non-normalized densities with an intractable normalization constant. Since maximum likelihood estimation is computationally intensive for these models, several estimation methods have been developed which do not require explicit computation of the normalization constant, such as noise contrastive estimation (NCE) and score matching. However, model selection methods for general non-normalized models have not been proposed so far. In this study, we develop information criteria for non-normalized models estimated by NCE or score matching. They are approximately unbiased estimators of discrepancy measures for non-normalized models. Simulation results and applications to real data demonstrate that the proposed criteria enable selection of the appropriate non-normalized model in a data-driven manner.

math.ST↗

arXiv2021

Estimation under matrix quadratic loss and matrix superharmonicity

We investigate estimation of a normal mean matrix under the matrix quadratic loss. Improved estimation under the matrix quadratic loss implies improved estimation of any linear combination of the columns. First, an unbiased estimate of risk is derived and the Efron--Morris estimator is shown to be minimax. Next, a notion of \textit{matrix superharmonicity} for matrix-variate functions is introduced and shown to have analogous properties with usual superharmonic functions, which may be of independent interest. Then, we show that the generalized Bayes estimator with respect to a matrix superharmonic prior is minimax. We also provide a class of matrix superharmonic priors that includes the previously proposed generalization of Stein's prior. Numerical results demonstrate that matrix superharmonic priors work well for low rank matrices.

math.ST↗

arXiv2021

Singular Value Shrinkage Priors for Bayesian Prediction

We develop singular value shrinkage priors for the mean matrix parameters in the matrix-variate normal model with known covariance matrices. Our priors are superharmonic and put more weight on matrices with smaller singular values. They are a natural generalization of the Stein prior. Bayes estimators and Bayesian predictive densities based on our priors are minimax and dominate those based on the uniform prior in finite samples. In particular, our priors work well when the true value of the parameter has low rank.

math.ST↗

arXiv2021

Interpretable Stein Goodness-of-fit Tests on Riemannian Manifolds

In many applications, we encounter data on Riemannian manifolds such as torus and rotation groups. Standard statistical procedures for multivariate data are not applicable to such data. In this study, we develop goodness-of-fit testing and interpretable model criticism methods for general distributions on Riemannian manifolds, including those with an intractable normalization constant. The proposed methods are based on extensions of kernel Stein discrepancy, which are derived from Stein operators on Riemannian manifolds. We discuss the connections between the proposed tests with existing ones and provide a theoretical analysis of their asymptotic Bahadur efficiency. Simulation results and real data applications show the validity of the proposed methods.

stat.ME↗

arXiv2021

Information geometry of operator scaling

Matrix scaling is a classical problem with a wide range of applications. It is known that the Sinkhorn algorithm for matrix scaling is interpreted as alternating e-projections from the viewpoint of classical information geometry. Recently, a generalization of matrix scaling to completely positive maps called operator scaling has been found to appear in various fields of mathematics and computer science, and the Sinkhorn algorithm has been extended to operator scaling. In this study, the operator Sinkhorn algorithm is studied from the viewpoint of quantum information geometry through the Choi representation of completely positive maps. The operator Sinkhorn algorithm is shown to coincide with alternating e-projections with respect to the symmetric logarithmic derivative metric, which is a Riemannian metric on the space of quantum states relevant to quantum estimation theory. Other types of alternating e-projections algorithms are also provided by using different information geometric structures on the positive definite cone.

math.OC↗

arXiv2021

Estimation of ordinary differential equation models with discretization error quantification

We consider parameter estimation of ordinary differential equation (ODE) models from noisy observations. For this problem, one conventional approach is to fit numerical solutions (e.g., Euler, Runge--Kutta) of ODEs to data. However, such a method does not account for the discretization error in numerical solutions and has limited estimation accuracy. In this study, we develop an estimation method that quantifies the discretization error based on data. The key idea is to model the discretization error as random variables and estimate their variance simultaneously with the ODE parameter. The proposed method has the form of iteratively reweighted least squares, where the discretization error variance is updated with the isotonic regression algorithm and the ODE parameter is updated by solving a weighted least squares problem using the adjoint system. Experimental results demonstrate that the proposed method attains robust estimation with at least comparable accuracy to the conventional method by successfully quantifying the reliability of the numerical solutions.

stat.ME↗

arXiv2020

Wasserstein Statistics in One-dimensional Location-Scale Model

Wasserstein geometry and information geometry are two important structures to be introduced in a manifold of probability distributions. Wasserstein geometry is defined by using the transportation cost between two distributions, so it reflects the metric of the base manifold on which the distributions are defined. Information geometry is defined to be invariant under reversible transformations of the base space. Both have their own merits for applications. In particular, statistical inference is based upon information geometry, where the Fisher metric plays a fundamental role, whereas Wasserstein geometry is useful in computer vision and AI applications. In this study, we analyze statistical inference based on the Wasserstein geometry in the case that the base space is one-dimensional. By using the location-scale model, we further derive the W-estimator that explicitly minimizes the transportation cost from the empirical distribution to a statistical model and study its asymptotic behaviors. We show that the W-estimator is consistent and explicitly give its asymptotic distribution by using the functional delta method. The W-estimator is Fisher efficient in the Gaussian case.

math.ST↗

arXiv2020

Adjoint-based exact Hessian computation

We consider a scalar function depending on a numerical solution of an initial value problem, and its second-derivative (Hessian) matrix for the initial value. The need to extract the information of the Hessian or to solve a linear system having the Hessian as a coefficient matrix arises in many research fields such as optimization, Bayesian estimation, and uncertainty quantification. From the perspective of memory efficiency, these tasks often employ a Krylov subspace method that does not need to hold the Hessian matrix explicitly and only requires computing the multiplication of the Hessian and a given vector. One of the ways to obtain an approximation of such Hessian-vector multiplication is to integrate the so-called second-order adjoint system numerically. However, the error in the approximation could be significant even if the numerical integration to the second-order adjoint system is sufficiently accurate. This paper presents a novel algorithm that computes the intended Hessian-vector multiplication exactly and efficiently. For this aim, we give a new concise derivation of the second-order adjoint system and show that the intended multiplication can be computed exactly by applying a particular numerical method to the second-order adjoint system. In the discussion, symplectic partitioned Runge--Kutta methods play an essential role.

math.NA↗

arXiv2020

Imputation estimators for unnormalized models with missing data

Several statistical models are given in the form of unnormalized densities, and calculation of the normalization constant is intractable. We propose estimation methods for such unnormalized models with missing data. The key concept is to combine imputation techniques with estimators for unnormalized models including noise contrastive estimation and score matching. In addition, we derive asymptotic distributions of the proposed estimators and construct confidence intervals. Simulation results with truncated Gaussian graphical models and the application to real data of wind direction reveal that the proposed methods effectively enable statistical inference with unnormalized models from missing data.

stat.ML↗

arXiv2020

Unified estimation framework for unnormalized models with statistical efficiency

The parameter estimation of unnormalized models is a challenging problem. The maximum likelihood estimation (MLE) is computationally infeasible for these models since normalizing constants are not explicitly calculated. Although some consistent estimators have been proposed earlier, the problem of statistical efficiency remains. In this study, we propose a unified, statistically efficient estimation framework for unnormalized models and several efficient estimators, whose asymptotic variance is the same as the MLE. The computational cost of these estimators is also reasonable and they can be employed whether the sample space is discrete or continuous. The loss functions of the proposed estimators are derived by combining the following two methods: (1) density-ratio matching using Bregman divergence, and (2) plugging-in nonparametric estimators. We also analyze the properties of the proposed estimators when the unnormalized models are misspecified. The experimental results demonstrate the advantages of our method over existing approaches.

stat.ML↗

arXiv2020

Generalization of partitioned Runge--Kutta methods for adjoint systems

This study computes the gradient of a function of numerical solutions of ordinary differential equations (ODEs) with respect to the initial condition. The adjoint method computes the gradient approximately by solving the corresponding adjoint system numerically. In this context, Sanz-Serna [SIAM Rev., 58 (2016), pp. 3--33] showed that when the initial value problem is solved by a Runge--Kutta (RK) method, the gradient can be exactly computed by applying an appropriate RK method to the adjoint system. Focusing on the case where the initial value problem is solved by a partitioned RK (PRK) method, this paper presents a numerical method, which can be seen as a generalization of PRK methods, for the adjoint system that gives the exact gradient.

math.NA↗

arXiv2020

A Stein Goodness-of-fit Test for Directional Distributions

In many fields, data appears in the form of direction (unit vector) and usual statistical procedures are not applicable to such directional data. In this study, we propose non-parametric goodness-of-fit testing procedures for general directional distributions based on kernel Stein discrepancy. Our method is based on Stein's operator on spheres, which is derived by using Stokes' theorem. Notably, the proposed method is applicable to distributions with an intractable normalization constant, which commonly appear in directional statistics. Experimental results demonstrate that the proposed methods control type-I error well and have larger power than existing tests, including the test based on the maximum mean discrepancy.

stat.ME↗

arXiv2019

Predictive density estimation under the Wasserstein loss

We investigate predictive density estimation under the $L^2$ Wasserstein loss for location families and location-scale families. We show that plug-in densities form a complete class and that the Bayesian predictive density is given by the plug-in density with the posterior mean of the location and scale parameters. We provide Bayesian predictive densities that dominate the best equivariant one in normal models.

math.ST↗