arXiv subjects

Genshiro Kitagawa

Publications and source records attributed to Genshiro Kitagawa.

14 recordsLinked to original sources

arXiv2026

Maximum Likelihood and Bayesian Estimation for State-Space Models Using the Non-Gaussian Filter

The non-Gaussian filter provides a deterministic numerical method for nonlinear and non-Gaussian state-space models, but its application has long been limited due to the computational cost of numerical integration. Advances in computing power and memory capacity have substantially reduced this limitation for low and moderate dimensional models. This paper re-examines the non-Gaussian filter and demonstrates its usefulness for maximum likelihood estimation and Bayesian inference. Numerical experiments with linear, nonlinear and radar-tracking models show that log-likelihood obtained by non-Gaussian filter is smooth and can be optimized reliably, whereas the ones obtained by particle filter are affected strongly by Monte Carlo variability even with many particles. Bayesian estimation is performed using a self-organizing state-space model, in which unknown parameters are incorporated into the state vector and estimated jointly with the latent states. These results demonstrate that the current computing technology has renewed the practical value of deterministic filtering for statistical inference in state-space models.

stat.CO↗

arXiv2026

Backward Smoothing versus Fixed-Lag Smoothing in Particle Filters

Particle smoothing enables state estimation in nonlinear and non-Gaussian state-space models, but its practical use is often limited by high computational cost. Backward smoothing methods such as the Forward Filter Backward Smoother (FFBS) and its marginal form (FFBSm) can achieve high accuracy, yet typically require quadratic computational complexity in the number of particles. This paper examines the accuracy--computational cost trade-offs of particle smoothing methods through a trend-estimation example. Fixed-lag smoothing, FFBS, and FFBSm are compared under Gaussian and heavy-tailed (Cauchy-type) system noise, with particular attention to O(m) approximations of FFBSm based on subsampling and local neighborhood restrictions. The results show that FFBS and FFBSm outperform fixed-lag smoothing at a fixed particle number, while fixed-lag smoothing often achieves higher accuracy under equal computational time. Moreover, efficient FFBSm approximations are effective for Gaussian transitions but become less advantageous for heavy-tailed dynamics.

stat.ME↗

arXiv2026

Bayesian Optimization of Noisy Log-Likelihoods Evaluated by Particle Filters -- One Parameter Case --

Likelihood functions evaluated using particle filters are typically noisy, computationally expensive, and non-differentiable due to Monte Carlo variability. These characteristics make conventional optimization methods difficult to apply directly or potentially unreliable. This paper investigates the use of Bayesian optimization for maximizing log-likelihood functions estimated by particle filters. By modeling the noisy log-likelihood surface with a Gaussian process surrogate and employing an acquisition function that balances exploration and exploitation, the proposed approach identifies the maximizer using a limited number of likelihood evaluations. Through numerical experiments, we demonstrate that Bayesian optimization provides robust and stable estimation in the presence of observation noise. The results suggest that Bayesian optimization is a promising alternative for likelihood maximization problems where exhaustive search or gradient-based methods are impractical. The estimation accuracy is quantitatively assessed using mean squared error metrics by comparison with the exact maximum likelihood solution obtained via the Kalman filter.

stat.ME↗

arXiv2025

Gaussian Process State-Space Modeling and Particle Filtering for Time Series Decomposition and Nonlinear Signal Extraction

Gaussian-process state-space models (GP-SSMs) provide a flexible nonparametric alternative for modeling time-series dynamics that are nonlinear or difficult to specify parametrically. While the Kalman filter is effective for linear-Gaussian trend and seasonal components, many real-world systems require more expressive representations. GP-SSMs address this need by learning transition functions directly from data, while particle filtering enables Bayesian state estimation even when posterior distributions deviate from Gaussianity. This paper develops a particle-filtering framework for GP-SSM inference and compares its performance with the Kalman filter in trend extraction and seasonal adjustment. We further evaluate nonlinear signal-extraction tasks, demonstrating that GP-SSMs can recover latent states under sharp or asymmetric dynamics. The results highlight the utility of combining GP modeling with sequential Monte Carlo methods for complex time-series analysis.

stat.ME↗

arXiv2025

Enhancing Seasonal Adjustment Space Models: Constraints and Regularization for Improved Trend and AR Decomposition

This paper investigates enhancements to model-based methods for seasonal adjustment, with a particular focus on the state space modeling framework. It addresses limitations of the standard Decomp model; specifically, the tendency to produce overly smooth trend components and the misattribution of long-term variation to the AR component when the eigenvalues of the AR model are close to unity. To mitigate these issues, the paper proposes imposing constraints on the modulus and argument of the AR eigenvalues, as well as applying regularization techniques ($L_1$ and $L_2$). These approaches are evaluated using real-world datasets. The paper is structured as follows: an overview of the Decomp model, a comparison with its noise-free variant, empirical assessment of constrained AR models, an exploration of regularization methods, and a concluding discussion of key insights.

stat.ME↗

arXiv2025

Information Criterion for the Gaussian and/or Laplace Distribution Models

The information criterion AIC has been used successfully in many areas of statistical modeling, and since it is derived based on the Taylor expansion of the log-likelihood function and the asymptotic distribution of the maximum likelihood estimator, it is not directly justified for likelihood functions that include non-differentiable points such as the Laplace distribution. In fact, it is known to work effectively in many such cases. In this paper, we attempt to evaluate the bias correction directly for the case where the true model or the model to be estimated is a simple Laplace distribution model. As a result, an approximate expression for the bias correction term was obtained. Numerical results show that the AIC approximations are relatively good except when the Gauss distribution model is fitted to data following the Laplace distribution.

stat.ME↗

arXiv2024

Emperical Study on Various Symmetric Distributions for Modeling Time Series

This study evaluated probability distributions for modeling time series with abrupt structural changes. The Pearson type VII distribution, with an adjustable shape parameter $b$, proved versatile. The generalized Laplace distribution performed similarly to the Pearson model, occasionally surpassing it in terms of likelihood and AIC. Mixture models, including the mixture of $\delta$-function and Gaussian distribution, showed potential but were less stable. Pearson type VII and extended Laplace models were deemed more reliable for general cases. Model selection depends on data characteristics and goals.

stat.CO↗

arXiv2024

Enhancing Computational Efficiency in State-Space Models Using Rao-Blackwellization and 2-Step Approximation

This paper explores a Bayesian self-organization method for state-space models, enabling simultaneous state and parameter estimation without repeated likelihood calculations. While efficient for low-dimensional models, high-dimensional cases like seasonal adjustment require many particles. Using Rao-Blackwellization and a 2-step approximation, the method reduces particle use and computation time while maintaining accuracy, as shown in Monte Carlo evaluations.

stat.CO↗

arXiv2023

Extended Relative Power Contribution that Allows to Evaluate the Effect of Correlated Noise

We proposed an extension of Akaike's relative power contribution that could be applied to data with correlations between noises. This method decomposes the power spectrum into a contribution of the terms caused by correlation between two noises, in addition to the contributions of the independent noises. Numerical examples confirm that some of the correlated noise has the effect of reducing the power spectrum.

stat.ME↗

arXiv2023

Information Criterion for a Large Scale Subset Regression Models

The information criterion for determining the number of explanatory variables in a subset regression modeling is discussed. Information criterion such as AIC is effective and frequently used in model selection for ordinary regression models and statistical models. With the recent prosperity of data science, analysis of large-scale data has become important. When constructing models heuristically from a very large number of candidate explanatory variables, there is a possibility of picking up apparent correlations and adopting inappropriate variables. In this paper, we point out the problems specific to subset regression from the viewpoint of bias correction for log-likelihood and present a correction method that takes this into account.

stat.ME↗

arXiv2022

Fitting State-space Model for Long-term Prediction of the Log-likelihood of Nonstationary Time Series Models

The goodness of the long-term prediction in the state-space model was evaluated using the squared long-term prediction error. In order to estimate the model parameters suitable for long-term prediction, we devised a modified log-likelihood corresponding to the long-term prediction error variance. Trend models and seasonally adjusted models with and without AR component are examined as examples.

stat.ME↗

arXiv2022

An implimentation of the Differential Filter for Computing Gradient and Hessian of the Log-likelihood of Nonstationary Time Series Models

The state-space model and the Kalman filter provide us with unified and computationaly efficient procedure for computing the log-likelihood of the diverse type of time series models. This paper presents an algorithm for computing the gradient and the Hessian matrix of the log-likelihood by extending the Kalman filter without resorting to the numerical difference. Different from the previous paper(Kitagawa 2020), it is assumed that the observation noise variance R=1. It is known that for univariate time series, by maximizing the log-likelihood of this restricted model, we can obtain the same estimates as the ones for the original state-space model. By this modification, the algorithm for computing the gradient and the Hessian becomes somewhat complicated. However, the dimension of the parameter vector is reduce by one and thus has a significant merit in estimating the parameter of the state-space model especially for relatively low dimentional parameter vector. Three examples of nonstationary time seirres models, i.e., trend model, statndard seasonal adjustment model and the seasonal adjustment model with AR componet are presented to exemplified the specification of structural matrices.

stat.ME↗

arXiv2022

The Information Criterion GIC of Trend and Seasonal Adjustment Models

This paper presents an algorithm for computing the GIC and the TIC of the nonstationary state-space models. The gradient and Hessian of the log-likelihood neccesary in computing the GIC are obtained by the differential filter that is derived by extending the Kalman filter. Three examples of the nonstationary time series models, i.e., the trend model, statndard seasonal adjustment model and the seasonal adjustment model with stationary AR component are presented to exemplified the specification of structural matrices.

stat.ME↗

arXiv2020

Pearson chi^2-divergence Approach to Gaussian Mixture Reduction and its Application to Gaussian-sum Filter and Smoother

The Gaussian mixture distribution is important in various statistical problems. In particular it is used in the Gaussian-sum filter and smoother for linear state-space model with non-Gaussian noise inputs. However, for this method to be practical, an efficient method of reducing the number of Gaussian components is necessary. In this paper, we show that a closed form expression of Pearson chi^2-divergence can be obtained and it can apply to the determination of the pair of two Gaussian components in sequential reduction of Gaussian components. By numerical examples for one dimensional and two dimensional distribution models, it will be shown that in most cases the proposed criterion performed almost equally as the Kullback-Libler divergence, for which computationally costly numerical integration is necessary. Application to Gaussian-sum filtering and smoothing is also shown.

stat.ME↗