arXiv subjects

Daisuke Murakami

Publications and source records attributed to Daisuke Murakami.

At least 37 records · Page 2Linked to original sources

arXiv2021

Adaptively Robust Geographically Weighted Regression

We develop a new robust geographically weighted regression method in the presence of outliers. We embed the standard geographically weighted regression in robust objective function based on $γ$-divergence. A novel feature of the proposed approach is that two tuning parameters that control robustness and spatial smoothness are automatically tuned in a data-dependent manner. Further, the proposed method can produce robust standard error estimates of the robust estimator and give us a reasonable quantity for local outlier detection. We demonstrate that the proposed method is superior to the existing robust version of geographically weighted regression through simulation and data analysis.

stat.ME↗

arXiv2021

Transformation-based generalized spatial regression using the spmoran package: Case study examples

This study presents application examples of generalized spatial regression modeling for count data and continuous non-Gaussian data using the spmoran package (version 0.2.2 onward). Section 2 introduces the model. The subsequent sections demonstrate applications of the model for disease mapping, spatial prediction and uncertainty modeling, and hedonic analysis. The R codes used in this vignette are available from https://github.com/dmuraka/spmoran. Another vignette focusing on Gaussian spatial regression modeling is also available from the same GitHub page.

stat.AP↗

arXiv2021

Analysis of COVID-19 evolution based on testing closeness of sequential data

A practical algorithm has been developed for closeness analysis of sequential data that combines closeness testing with algorithms based on the Markov chain tester. It was applied to reported sequential data for COVID-19 to analyze the evolution of COVID-19 during a certain time period (week, month, etc.).

stat.ME↗

arXiv2021

Compositionally-warped additive mixed modeling for a wide variety of non-Gaussian spatial data

As with the advancement of geographical information systems, non-Gaussian spatial data sets are getting larger and more diverse. This study develops a general framework for fast and flexible non-Gaussian regression, especially for spatial/spatiotemporal modeling. The developed model, termed the compositionally-warped additive mixed model (CAMM), combines an additive mixed model (AMM) and the compositionally-warped Gaussian process to model a wide variety of non-Gaussian continuous data including spatial and other effects. A specific advantage of the proposed CAMM is that it requires no explicit assumption of data distribution unlike existing AMMs. Monte Carlo experiments show the estimation accuracy and computational efficiency of CAMM for modeling non-Gaussian data including fat-tailed and/or skewed distributions. Finally, the model is applied to crime data to examine the empirical performance of the regression analysis and prediction. The result shows that CAMM provides intuitively reasonable coefficient estimates and outperforms AMM in terms of prediction accuracy. CAMM is verified to be a fast and flexible model that potentially covers a wide variety of non-Gaussian data modeling. The proposed approach is implemented in an R package spmoran.

stat.ME↗

arXiv2021

Balancing spatial and non-spatial variation in varying coefficient modeling: a remedy for spurious correlation

This study discusses the importance of balancing spatial and non-spatial variation in spatial regression modeling. Unlike spatially varying coefficients (SVC) modeling, which is popular in spatial statistics, non-spatially varying coefficients (NVC) modeling has largely been unexplored in spatial fields. Nevertheless, as we will explain, consideration of non-spatial variation is needed not only to improve model accuracy but also to reduce spurious correlation among varying coefficients, which is a major problem in SVC modeling. We consider a Moran eigenvector approach modeling spatially and non-spatially varying coefficients (S&NVC). A Monte Carlo simulation experiment comparing our S&NVC model with existing SVC models suggests both modeling accuracy and computational efficiency for our approach. Beyond that, somewhat surprisingly, our approach identifies true and spurious correlations among coefficients nearly perfectly, even when usual SVC models suffer from severe spurious correlations. It implies that S&NVC model should be used even when the analysis purpose is modeling SVCs. Finally, our S&NVC model is employed to analyze a residential land price dataset. Its results suggest existence of both spatial and non-spatial variation in regression coefficients in practice. The S&NVC model is now implemented in the R package spmoran.

stat.AP↗

arXiv2021

Spatially Clustered Regression

Spatial regression or geographically weighted regression models have been widely adopted to capture the effects of auxiliary information on a response variable of interest over a region. In contrast, relationships between response and auxiliary variables are expected to exhibit complex spatial patterns in many applications. This paper proposes a new approach for spatial regression, called spatially clustered regression, to estimate possibly clustered spatial patterns of the relationships. We combine K-means-based clustering formulation and penalty function motivated from a spatial process known as Potts model for encouraging similar clustering in neighboring locations. We provide a simple iterative algorithm to fit the proposed method, scalable for large spatial datasets. Through simulation studies, the proposed method demonstrates its superior performance to existing methods even under the true structure does not admit spatial clustering. Finally, the proposed method is applied to crime event data in Tokyo and produces interpretable results for spatial patterns. The R code is available at https://github.com/sshonosuke/SCR.

stat.ME↗

arXiv2020

Scalable model selection for spatial additive mixed modeling: application to crime analysis

A rapid growth in spatial open datasets has led to a huge demand for regression approaches accommodating spatial and non-spatial effects in big data. Regression model selection is particularly important to stably estimate flexible regression models. However, conventional methods can be slow for large samples. Hence, we develop a fast and practical model-selection approach for spatial regression models, focusing on the selection of coefficient types that include constant, spatially varying, and non-spatially varying coefficients. A pre-processing approach, which replaces data matrices with small inner products through dimension reduction dramatically accelerates the computation speed of model selection. Numerical experiments show that our approach selects the model accurately and computationally efficiently, highlighting the importance of model selection in the spatial regression context. Then, the present approach is applied to open data to investigate local factors affecting crime in Japan. The results suggest that our approach is useful not only for selecting factors influencing crime risk but also for predicting crime events. This scalable model selection will be key to appropriately specifying flexible and large-scale spatial regression models in the era of big data. The developed model selection approach was implemented in the R package spmoran.

stat.AP↗

arXiv2020

Scalable GWR: A linear-time algorithm for large-scale geographically weighted regression with polynomial kernels

Although a number of studies have developed fast geographically weighted regression (GWR) algorithms for large samples, none of them has achieved linear-time estimation, which is considered a requisite for big data analysis in machine learning, geostatistics, and related domains. Against this backdrop, this study proposes a scalable GWR (ScaGWR) for large datasets. The key improvement is the calibration of the model through a pre-compression of the matrices and vectors whose size depends on the sample size, prior to the leave-one-out cross-validation, which is the heaviest computational step in conventional GWR. This pre-compression allows us to run the proposed GWR extension so that its computation time increases linearly with the sample size. With this improvement, the ScaGWR can be calibrated with one million observations without parallelization. Moreover, the ScaGWR estimator can be regarded as an empirical Bayesian estimator that is more stable than the conventional GWR estimator. We compare the ScaGWR with the conventional GWR in terms of estimation accuracy and computational efficiency using a Monte Carlo simulation. Then, we apply these methods to a US income analysis. The code for ScaGWR is available in the R package scgwr. The code is embedded into C++ code and implemented in another R package, GWmodel.

stat.ME↗

arXiv2020

The GWR route map: a guide to the informed application of Geographically Weighted Regression

Geographically Weighted Regression (GWR) is increasingly used in spatial analyses of social and environmental data. It allows spatial heterogeneities in processes and relationships to be investigated through a series of local regression models rather than a global one. Standard GWR assumes that the relationships between the response and predictor variables operate at the same spatial scale, which is frequently not the case. To address this, several GWR variants have been proposed. This paper describes a route map to inform the choice of whether to use a GWR model or not, and if so which of three core variants to apply: a standard GWR, a mixed GWR or a multiscale GWR (MS-GWR). The route map comprises primary steps: a basic linear regression, a MS-GWR, and investigations of the results of these. The paper provides guidance for deciding whether to use a GWR approach, and if so for determining the appropriate GWR variant. It describes the importance of investigating a number of secondary issues at global and local scales including collinearity, the influence of outliers, and dependent error terms. Code and data for the case study used to illustrate the route map are provided, and further considerations are described in an extensive Appendix.

stat.ME↗

arXiv2020

Spatiotemporal analysis of urban heatwaves using Tukey g-and-h random field models

The statistical quantification of temperature processes for the analysis of urban heat island (UHI) effects and local heat-waves is an increasingly important application domain in smart city dynamic modelling. This leads to the increased importance of real-time heatwave risk management on a fine-grained spatial resolution. This study attempts to analyze and develop new methods for modelling the spatio-temporal behavior of ground temperatures. The developed models consider higher-order stochastic spatial properties such as skewness and kurtosis, which are key components for understanding and describing local temperature fluctuations and UHI's. The developed models are applied to the greater Tokyo metropolitan area for a detailed real-world data case study. The analysis also demonstrates how to statistically incorporate a variety of real data sets. This includes remotely sensed imagery and a variety of ground-based monitoring site data to build models linking city and urban covariates to air temperature. The air temperature models are then used to capture high-resolution spatial emulator outputs for ground surface temperature modelling. The main class of processes studied includes the Tukey g-and-h processes for capturing spatial and temporal aspects of heat processes in urban environments.

stat.AP↗

arXiv2019

A memory-free spatial additive mixed modeling for big spatial data

This study develops a spatial additive mixed modeling (AMM) approach estimating spatial and non-spatial effects from large samples, such as millions of observations. Although fast AMM approaches are already well-established, they are restrictive in that they assume an known spatial dependence structure. To overcome this limitation, this study develops a fast AMM with the estimation of spatial structure in residuals and regression coefficients together with non-spatial effects. We rely on a Moran coefficient-based approach to estimate the spatial structure. The proposed approach pre-compresses large matrices whose size grows with respect to the sample size N before the model estimation; thus, the computational complexity for the estimation is independent of the sample size. Furthermore, the pre-compression is done through a block-wise procedure that makes the memory consumption independent of N. Eventually, the spatial AMM is memory-free and fast even for millions of observations. The developed approach is compared to alternatives through Monte Carlo simulation experiments. The result confirms the accuracy and computational efficiency of the developed approach. The developed approaches are implemented in an R package spmoran.

stat.ME↗

arXiv2018

Spatially filtered unconditional quantile regression: Application to a hedonic analysis

Unconditional quantile regression (UQR) attracts attention in various fields to investigate the impacts of explanatory variables on quantiles of the marginal distribution of an explained variable. This study attempts to introduce spatial dependence into the UQR within the framework of random effects eigenvector spatial filtering, resulting in the model that we term the spatially filtered UQR (SF-UQR). We then develop a computationally efficient approach for SF-UQR estimation. Finally, the performance of the SF-UQR is tested with a hedonic land price model for the Tokyo metropolitan area. SF-UQR is implemented in an R package, "spmoran."

stat.ME↗

arXiv2018

Low rank spatial econometric models

This article presents a re-structuring of spatial econometric models in a linear mixed model framework. To that end, it proposes low rank spatial econometric models that are robust to the existence of noise (i.e., measurement error), and can enjoy fast parameter estimation and inference by Type II restricted likelihood maximization (empirical Bayes) techniques. The small sample properties of the proposed low rank spatial econometric models are examined using Monte Carlo simulation experiments, the results of these experiments confirm that direct effects and indirect effects a la LeSage and Pace (2009) can be estimated with a high degree of accuracy. Also, when data are noisy, estimators for coefficients in the proposed models have lower root mean squared errors compared to conventional specifications, despite them being low rank approximations. The proposed approach is implemented in an R package "spmoran".

stat.ME↗

arXiv2018

Spatially varying coefficient modeling for large datasets: Eliminating N from spatial regressions

While spatially varying coefficient (SVC) modeling is popular in applied science, its computational burden is substantial. This is especially true if a multiscale property of SVC is considered. Given this background, this study develops a Moran's eigenvector-based spatially varying coefficients (M-SVC) modeling approach that estimates multiscale SVCs computationally efficiently. This estimation is accelerated through a (i) rank reduction, (ii) pre-compression, and (iii) sequential likelihood maximization. Steps (i) and (ii) eliminate the sample size N from the likelihood function; after these steps, the likelihood maximization cost is independent of N. Step (iii) further accelerates the likelihood maximization so that multiscale SVCs can be estimated even if the number of SVCs, K, is large. The M-SVC approach is compared with geographically weighted regression (GWR) through Monte Carlo simulation experiments. These simulation results show that our approach is far faster than GWR when N is large, despite numerically estimating 2K parameters while GWR numerically estimates only 1 parameter. Then, the proposed approach is applied to a land price analysis as an illustration. The developed SVC estimation approach is implemented in the R package "spmoran."

stat.ME↗

arXiv2017

Eigenvector spatial filtering for large data sets: fixed and random effects approaches

Eigenvector spatial filtering (ESF) is a spatial modeling approach, which has been applied in urban and regional studies, ecological studies, and so on. However, it is computationally demanding, and may not be suitable for large data modeling. The objective of this study is developing fast ESF and random effects ESF (RE-ESF), which are capable of handling very large samples. To achieve it, we accelerate eigen-decomposition and parameter estimation, which make ESF and RE-ESF slow. The former is accelerated by utilizing the Nystrom extension, whereas the latter is by small matrix tricks. The resulting fast ESF and fast RE-ESF are compared with non-approximated ESF and RE-ESF in Monte Carlo simulation experiments. The result shows that, while ESF and RE-ESF are slow for several thousand samples, fast ESF and RE-ESF require only several seconds for the samples. They also suggest that the proposed approaches effectively remove positive spatial dependence in the residuals with very small approximation errors when the number of eigenvectors considered is 200 or more. Note that these approaches cannot deal with negative spatial dependence. The proposed approaches are implemented in an R package "spmoran."

stat.ME↗

arXiv2017

The importance of scale in spatially varying coefficient modeling

While spatially varying coefficient (SVC) models have attracted considerable attention in applied science, they have been criticized as being unstable. The objective of this study is to show that capturing the "spatial scale" of each data relationship is crucially important to make SVC modeling more stable, and in doing so, adds flexibility. Here, the analytical properties of six SVC models are summarized in terms of their characterization of scale. Models are examined through a series of Monte Carlo simulation experiments to assess the extent to which spatial scale influences model stability and the accuracy of their SVC estimates. The following models are studied: (i) geographically weighted regression (GWR) with a fixed distance or (ii) an adaptive distance bandwidth (GWRa), (iii) flexible bandwidth GWR (FB-GWR) with fixed distance or (iv) adaptive distance bandwidths (FB-GWRa), (v) eigenvector spatial filtering (ESF), and (vi) random effects ESF (RE-ESF). Results reveal that the SVC models designed to capture scale dependencies in local relationships (FB-GWR, FB-GWRa and RE-ESF) most accurately estimate the simulated SVCs, where RE-ESF is the most computationally efficient. Conversely GWR and ESF, where SVC estimates are naively assumed to operate at the same spatial scale for each relationship, perform poorly. Results also confirm that the adaptive bandwidth GWR models (GWRa and FB-GWRa) are superior to their fixed bandwidth counterparts (GWR and FB-GWR).

stat.ME↗

arXiv2017

Estimation of gridded population and GDP scenarios with spatially explicit statistical downscaling

This study downscales the population and gross domestic product (GDP) scenarios given under Shared Socioeconomic Pathways (SSPs) into 0.5-degree grids. Our downscale approach has the following features: (i) it explicitly considers spatial and socioeconomic interactions among cities; (ii) it utilizes auxiliary variables, including, road network and land cover; (iii) it endogenously estimates influence from each factor by a model ensemble approach; (iv) it allows us controlling urban shrinkage/dispersion depending on SSPs. It is confirmed that our downscaling results are consistent with scenario assumptions (e.g., concentration in SSP1 and dispersion in SSP3). Besides, while existing grid-level scenario tends to have overly-smoothed population distributions in non-urban areas, ours does not suffer from the problem, and captures difference in urban and non-urban areas in a more reasonable manner.

stat.AP↗

arXiv2016

A Moran coefficient-based mixed effects approach to investigate spatially varying relationships

This study develops a spatially varying coefficient model by extending the random effects eigenvector spatial filtering model. The developed model has the following properties: its coefficients are interpretable in terms of the Moran coefficient; each of its coefficients can have a different degree of spatial smoothness; and it yields a variant of a Bayesian spatially varying coefficient model. Also, parameter estimation of the model can be executed with a relatively small computationally burden. Results of a Monte Carlo simulation reveal that our model outperforms a conventional eigenvector spatial filtering (ESF) model and geographically weighted regression (GWR) models in terms of the accuracy of the coefficient estimates and computational time. We empirically apply our model to the hedonic land price analysis of flood risk in Japan.

stat.ME↗