arXiv subjects

Hidetoshi Shimodaira

Publications and source records attributed to Hidetoshi Shimodaira.

53 records · Page 3Linked to original sources

arXiv2019

Selective Inference for Testing Trees and Edges in Phylogenetics

Selective inference is considered for testing trees and edges in phylogenetic tree selection from molecular sequences. This improves the previously proposed approximately unbiased test by adjusting the selection bias when testing many trees and edges at the same time. The newly proposed selective inference $p$-value is useful for testing selected edges to claim that they are significantly supported if $p>1-α$, whereas the non-selective $p$-value is still useful for testing candidate trees to claim that they are rejected if $p<α$. The selective $p$-value controls the type-I error conditioned on the selection event, whereas the non-selective $p$-value controls it unconditionally. The selective and non-selective approximately unbiased $p$-values are computed from two geometric quantities called signed distance and mean curvature of the region representing tree or edge of interest in the space of probability distributions. These two geometric quantities are estimated by fitting a model of scaling-law to the non-parametric multiscale bootstrap probabilities. Our general method is applicable to a wider class of problems; phylogenetic tree selection is an example of model selection, and it is interpreted as the variable selection of multiple regression, where each edge corresponds to each predictor. Our method is illustrated in a previously controversial phylogenetic analysis of human, rabbit and mouse.

stat.AP↗

arXiv2019

An information criterion for auxiliary variable selection in incomplete data analysis

Statistical inference is considered for variables of interest, called primary variables, when auxiliary variables are observed along with the primary variables. We consider the setting of incomplete data analysis, where some primary variables are not observed. Utilizing a parametric model of joint distribution of primary and auxiliary variables, it is possible to improve the estimation of parametric model for the primary variables when the auxiliary variables are closely related to the primary variables. However, the estimation accuracy reduces when the auxiliary variables are irrelevant to the primary variables. For selecting useful auxiliary variables, we formulate the problem as model selection, and propose an information criterion for predicting primary variables by leveraging auxiliary variables. The proposed information criterion is an asymptotically unbiased estimator of the Kullback-Leibler divergence for complete data of primary variables under some reasonable conditions. We also clarify an asymptotic equivalence between the proposed information criterion and a variant of leave-one-out cross validation. Performance of our method is demonstrated via a simulation study and a real data example.

stat.ME↗

arXiv2019

Graph Embedding with Shifted Inner Product Similarity and Its Improved Approximation Capability

We propose shifted inner-product similarity (SIPS), which is a novel yet very simple extension of the ordinary inner-product similarity (IPS) for neural-network based graph embedding (GE). In contrast to IPS, that is limited to approximating positive-definite (PD) similarities, SIPS goes beyond the limitation by introducing bias terms in IPS; we theoretically prove that SIPS is capable of approximating not only PD but also conditionally PD (CPD) similarities with many examples such as cosine similarity, negative Poincare distance and negative Wasserstein distance. Since SIPS with sufficiently large neural networks learns a variety of similarities, SIPS alleviates the need for configuring the similarity function of GE. Approximation error rate is also evaluated, and experiments on two real-world datasets demonstrate that graph embedding using SIPS indeed outperforms existing methods.

stat.ML↗

arXiv2019

Robust Graph Embedding with Noisy Link Weights

We propose $β$-graph embedding for robustly learning feature vectors from data vectors and noisy link weights. A newly introduced empirical moment $β$-score reduces the influence of contamination and robustly measures the difference between the underlying correct expected weights of links and the specified generative model. The proposed method is computationally tractable; we employ a minibatch-based efficient stochastic algorithm and prove that this algorithm locally minimizes the empirical moment $β$-score. We conduct numerical experiments on synthetic and real-world datasets.

stat.ML↗

arXiv2018

PAFit: an R Package for the Non-Parametric Estimation of Preferential Attachment and Node Fitness in Temporal Complex Networks

Many real-world systems are profitably described as complex networks that grow over time. Preferential attachment and node fitness are two simple growth mechanisms that not only explain certain structural properties commonly observed in real-world systems, but are also tied to a number of applications in modeling and inference. While there are statistical packages for estimating various parametric forms of the preferential attachment function, there is no such package implementing non-parametric estimation procedures. The non-parametric approach to the estimation of the preferential attachment function allows for comparatively finer-grained investigations of the `rich-get-richer' phenomenon that could lead to novel insights in the search to explain certain nonstandard structural properties observed in real-world networks. This paper introduces the R package PAFit, which implements non-parametric procedures for estimating the preferential attachment function and node fitnesses in a growing network, as well as a number of functions for generating complex networks from these two mechanisms. The main computational part of the package is implemented in C++ with OpenMP to ensure scalability to large-scale networks. We first introduce the main functionalities of PAFit through simulated examples, and then use the package to analyze a collaboration network between scientists in the field of complex networks. The results indicate the joint presence of `rich-get-richer' and `fit-get-richer' phenomena in the collaboration network. The estimated attachment function is observed to be near-linear, which we interpret as meaning that the chance an author gets a new collaborator is proportional to their current number of collaborators. Furthermore, the estimated author fitnesses reveal a host of familiar faces from the complex networks community among the field's topmost fittest network scientists.

physics.data-an↗

arXiv2018

On representation power of neural network-based graph embedding and beyond

We consider the representation power of siamese-style similarity functions used in neural network-based graph embedding. The inner product similarity (IPS) with feature vectors computed via neural networks is commonly used for representing the strength of association between two nodes. However, only a little work has been done on the representation capability of IPS. A very recent work shed light on the nature of IPS and reveals that IPS has the capability of approximating any positive definite (PD) similarities. However, a simple example demonstrates the fundamental limitation of IPS to approximate non-PD similarities. We then propose a novel model named Shifted IPS (SIPS) that approximates any Conditionally PD (CPD) similarities arbitrary well. CPD is a generalization of PD with many examples such as negative Poincaré distance and negative Wasserstein distance, thus SIPS has a potential impact to significantly improve the applicability of graph embedding without taking great care in configuring the similarity function. Our numerical experiments demonstrate the SIPS's superiority over IPS. In theory, we further extend SIPS beyond CPD by considering the inner product in Minkowski space so that it approximates more general similarities.

stat.ML↗

arXiv2018

A probabilistic framework for multi-view feature learning with many-to-many associations via neural networks

A simple framework Probabilistic Multi-view Graph Embedding (PMvGE) is proposed for multi-view feature learning with many-to-many associations so that it generalizes various existing multi-view methods. PMvGE is a probabilistic model for predicting new associations via graph embedding of the nodes of data vectors with links of their associations. Multi-view data vectors with many-to-many associations are transformed by neural networks to feature vectors in a shared space, and the probability of new association between two data vectors is modeled by the inner product of their feature vectors. While existing multi-view feature learning techniques can treat only either of many-to-many association or non-linear transformation, PMvGE can treat both simultaneously. By combining Mercer's theorem and the universal approximation theorem, we prove that PMvGE learns a wide class of similarity measures across views. Our likelihood-based estimator enables efficient computation of non-linear transformations of data vectors in large-scale datasets by minibatch SGD, and numerical experiments illustrate that PMvGE outperforms existing multi-view methods.

stat.ML↗

arXiv2018

Selective inference for the problem of regions via multiscale bootstrap

A general approach to selective inference is considered for hypothesis testing of the null hypothesis represented as an arbitrary shaped region in the parameter space of multivariate normal model. This approach is useful for hierarchical clustering where confidence levels of clusters are calculated only for those appeared in the dendrogram, thus subject to heavy selection bias. Our computation is based on a raw confidence measure, called bootstrap probability, which is easily obtained by counting how many times the same cluster appears in bootstrap replicates of the dendrogram. We adjust the bias of the bootstrap probability by utilizing the scaling-law in terms of geometric quantities of the region in the abstract parameter space, namely, signed distance and mean curvature. Although this idea has been used for non-selective inference of hierarchical clustering, its selective inference version has not been discussed in the literature. Our bias-corrected $p$-values are asymptotically second-order accurate in the large sample theory of smooth boundary surfaces of regions, and they are also justified for nonsmooth surfaces such as polyhedral cones. The $p$-values are asymptotically equivalent to those of the iterated bootstrap but with less computation.

math.ST↗

arXiv2016

An information criterion for model selection with missing data via complete-data divergence

We derive an information criterion to select a parametric model of complete-data distribution when only incomplete or partially observed data is available. Compared with AIC, our new criterion has an additional penalty term for missing data, which is expressed by the Fisher information matrices of complete data and incomplete data. We prove that our criterion is an asymptotically unbiased estimator of complete-data divergence, namely, the expected Kullback-Leibler divergence between the true distribution and the estimated distribution for complete data, whereas AIC is that for the incomplete data. Information criteria PDIO (Shimodaira 1994) and AICcd (Cavanaugh and Shumway 1998) have been previously proposed to estimate complete-data divergence, and they have the same penalty term. The additional penalty term of our criterion for missing data turns out to be only half the value of that in PDIO and AICcd. The difference in the penalty term is attributed to the fact that our criterion is derived under a weaker assumption. A simulation study with the weaker assumption shows that our criterion is unbiased while the other two criteria are biased. In addition, we review the geometrical view of alternating minimizations of the EM algorithm. This geometrical view plays an important role in deriving our new criterion.

stat.ME↗

arXiv2015

Cross-validation of matching correlation analysis by resampling matching weights

The strength of association between a pair of data vectors is represented by a nonnegative real number, called matching weight. For dimensionality reduction, we consider a linear transformation of data vectors, and define a matching error as the weighted sum of squared distances between transformed vectors with respect to the matching weights. Given data vectors and matching weights, the optimal linear transformation minimizing the matching error is solved by the spectral graph embedding of Yan et al. (2007). This method is a generalization of the canonical correlation analysis, and will be called as matching correlation analysis (MCA). In this paper, we consider a novel sampling scheme where the observed matching weights are randomly sampled from underlying true matching weights with small probability, whereas the data vectors are treated as constants. We then investigate a cross-validation by resampling the matching weights. Our asymptotic theory shows that the cross-validation, if rescaled properly, computes an unbiased estimate of the matching error with respect to the true matching weights. Existing ideas of cross-validation for resampling data vectors, instead of resampling matching weights, are not applicable here. MCA can be used for data vectors from multiple domains with different dimensions via an embarrassingly simple idea of coding the data vectors. This method will be called as cross-domain matching correlation analysis (CDMCA), and an interesting connection to the classical associative memory model of neural networks is also discussed.

stat.ML↗

arXiv2015

A simple coding for cross-domain matching with dimension reduction via spectral graph embedding

Data vectors are obtained from multiple domains. They are feature vectors of images or vector representations of words. Domains may have different numbers of data vectors with different dimensions. These data vectors from multiple domains are projected to a common space by linear transformations in order to search closely related vectors across domains. We would like to find projection matrices to minimize distances between closely related data vectors. This formulation of cross-domain matching is regarded as an extension of the spectral graph embedding to multi-domain setting, and it includes several multivariate analysis methods of statistics such as multiset canonical correlation analysis, correspondence analysis, and principal component analysis. Similar approaches are very popular recently in pattern recognition and vision. In this paper, instead of proposing a novel method, we will introduce an embarrassingly simple idea of coding the data vectors for explaining all the above mentioned approaches. A data vector is concatenated with zero vectors from all other domains to make an augmented vector. The cross-domain matching is solved by applying the single-domain version of spectral graph embedding to these augmented vectors of all the domains. An interesting connection to the classical associative memory model of neural networks is also discussed by noticing a coding for association. A cross-validation method for choosing the dimension of the common space and a regularization parameter will be discussed in an illustrative numerical example.

stat.ML↗

arXiv2014

Higher-order accuracy of multiscale-double bootstrap for testing regions

We consider hypothesis testing for the null hypothesis being represented as an arbitrary-shaped region in the parameter space. We compute an approximate p-value by counting how many times the null hypothesis holds in bootstrap replicates. This frequency, known as bootstrap probability, is widely used in evolutionary biology, but often reported as biased in the literature. Based on the asymptotic theory of bootstrap confidence intervals, there have been some new attempts for adjusting the bias via bootstrap probability without direct access to the parameter value. One such an attempt is the double bootstrap which adjusts the bias by bootstrapping the bootstrap probability. Another new attempt is the multiscale bootstrap which is similar to the m-out-of-n bootstrap but very unusually extrapolating the bootstrap probability to $m=-n$. In this paper, we employ these two attempts at the same time, and call the new procedure as multiscale-double bootstrap. By focusing on the multivariate normal model, we investigate higher-order asymptotics up to fourth-order accuracy. Geometry of the region plays important roles in the asymptotic theory. It was known in the literature that the curvature of the boundary surface of the region determines the bias of bootstrap probability. We found out that the curvature-of-curvature determines the remaining bias of double bootstrap. The multiscale bootstrap removes these biases. The multiscale-double bootstrap is fourth order accurate with coverage probability erring only $O(n^{-2})$, and it is robust against computational error of parameter estimation used for generating bootstrap replicates from the null distribution.

math.ST↗

arXiv2013

Technical details of the multistep-multiscale bootstrap resampling

The technical details of the new bootstrap method of Shimodaira (2004) are given here in mathematical proofs as well as a supporting computer program. Approximately unbiased tests based on the bootstrap probabilities are considered in Shimodaira (2004) for the exponential family of distributions with unknown expectation parameter vector, where the null hypothesis is represented as an arbitrary-shaped region with smooth boundaries. It has been described in the lemmas of Shimodaira (2004) that the newly developed three-step multiscale bootstrap method calculates an asymptotically third-order accurate $p$-value. All the mathematical proofs of these lemmas are shown here. The straightforward, though very tedious, calculations involving tensor notations are verified in Shimodaira (2003), which is, in fact, a computer program for Mathematica. Here we also give a brief explanation of this program.

math.ST↗

arXiv2010

Computing p-values of LiNGAM outputs via Multiscale Bootstrap

Structural equation models and Bayesian networks have been widely used to study causal relationships between continuous variables. Recently, a non-Gaussian method called LiNGAM was proposed to discover such causal models and has been extended in various directions. An important problem with LiNGAM is that the results are affected by the random sampling of the data as with any statistical method. Thus, some analysis of the statistical reliability or confidence level should be conducted. A common method to evaluate a confidence level is a bootstrap method. However, a confidence level computed by ordinary bootstrap method is known to be biased as a probability-value ($p$-value) of hypothesis testing. In this paper, we propose a new procedure to apply an advanced bootstrap method called multiscale bootstrap to compute confidence levels, i.e., p-values, of LiNGAM outputs. The multiscale bootstrap method gives unbiased $p$-values with asymptotic much higher accuracy. Experiments on artificial data demonstrate the utility of our approach.

stat.ML↗

arXiv2008

Frequentist and Bayesian measures of confidence via multiscale bootstrap for testing three regions

A new computation method of frequentist $p$-values and Bayesian posterior probabilities based on the bootstrap probability is discussed for the multivariate normal model with unknown expectation parameter vector. The null hypothesis is represented as an arbitrary-shaped region. We introduce new parametric models for the scaling-law of bootstrap probability so that the multiscale bootstrap method, which was designed for one-sided test, can also computes confidence measures of two-sided test, extending applicability to a wider class of hypotheses. Parameter estimation is improved by the two-step multiscale bootstrap and also by including higher-order terms. Model selection is important not only as a motivating application of our method, but also as an essential ingredient in the method. A compromise between frequentist and Bayesian is attempted by showing that the Bayesian posterior probability with an noninformative prior is interpreted as a frequentist $p$-value of ``zero-sided'' test.

stat.ME↗

arXiv2008

A preferential attachment model with Poisson growth for scale-free networks

We propose a scale-free network model with a tunable power-law exponent. The Poisson growth model, as we call it, is an offshoot of the celebrated model of Barabási and Albert where a network is generated iteratively from a small seed network; at each step a node is added together with a number of incident edges preferentially attached to nodes already in the network. A key feature of our model is that the number of edges added at each step is a random variable with Poisson distribution, and, unlike the Barabási-Albert model where this quantity is fixed, it can generate any network. Our model is motivated by an application in Bayesian inference implemented as Markov chain Monte Carlo to estimate a network; for this purpose, we also give a formula for the probability of a network under our model.

stat.AP↗

arXiv2005

Approximately unbiased tests of regions using multistep-multiscale bootstrap resampling

Approximately unbiased tests based on bootstrap probabilities are considered for the exponential family of distributions with unknown expectation parameter vector, where the null hypothesis is represented as an arbitrary-shaped region with smooth boundaries. This problem has been discussed previously in Efron and Tibshirani [Ann. Statist. 26 (1998) 1687-1718], and a corrected p-value with second-order asymptotic accuracy is calculated by the two-level bootstrap of Efron, Halloran and Holmes [Proc. Natl. Acad. Sci. U.S.A. 93 (1996) 13429-13434] based on the ABC bias correction of Efron [J. Amer. Statist. Assoc. 82 (1987) 171-185]. Our argument is an extension of their asymptotic theory, where the geometry, such as the signed distance and the curvature of the boundary, plays an important role. We give another calculation of the corrected p-value without finding the ``nearest point'' on the boundary to the observation, which is required in the two-level bootstrap and is an implementational burden in complicated problems. The key idea is to alter the sample size of the replicated dataset from that of the observed dataset. The frequency of the replicates falling in the region is counted for several sample sizes, and then the p-value is calculated by looking at the change in the frequencies along the changing sample sizes. This is the multiscale bootstrap of Shimodaira [Systematic Biology 51 (2002) 492-508], which is third-order accurate for the multivariate normal model. Here we introduce a newly devised multistep-multiscale bootstrap, calculating a third-order accurate p-value for the exponential family of distributions.

math.ST↗