arXiv subjects

Takeru Matsuda

Publications and source records attributed to Takeru Matsuda.

40 records · Page 3Linked to original sources

arXiv2018

Analysis of Noise Contrastive Estimation from the Perspective of Asymptotic Variance

There are many models, often called unnormalized models, whose normalizing constants are not calculated in closed form. Maximum likelihood estimation is not directly applicable to unnormalized models. Score matching, contrastive divergence method, pseudo-likelihood, Monte Carlo maximum likelihood, and noise contrastive estimation (NCE) are popular methods for estimating parameters of such models. In this paper, we focus on NCE. The estimator derived from NCE is consistent and asymptotically normal because it is an M-estimator. NCE characteristically uses an auxiliary distribution to calculate the normalizing constant in the same spirit of the importance sampling. In addition, there are several candidates as objective functions of NCE. We focus on how to reduce asymptotic variance. First, we propose a method for reducing asymptotic variance by estimating the parameters of the auxiliary distribution. Then, we determine the form of the objective functions, where the asymptotic variance takes the smallest values in the original estimator class and the proposed estimator classes. We further analyze the robustness of the estimator.

stat.ML↗

arXiv2018

Game-theoretic derivation of upper hedging prices of multivariate contingent claims and submodularity

We investigate upper and lower hedging prices of multivariate contingent claims from the viewpoint of game-theoretic probability and submodularity. By considering a game between "Market" and "Investor" in discrete time, the pricing problem is reduced to a backward induction of an optimization over simplexes. For European options with payoff functions satisfying a combinatorial property called submodularity or supermodularity, this optimization is solved in closed form by using the Lovász extension and the upper and lower hedging prices can be calculated efficiently. This class includes the options on the maximum or the minimum of several assets. We also study the asymptotic behavior as the number of game rounds goes to infinity. The upper and lower hedging prices of European options converge to the solutions of the Black-Scholes-Barenblatt equations. For European options with submodular or supermodular payoff functions, the Black-Scholes-Barenblatt equation is reduced to the linear Black-Scholes equation and it is solved in closed form. Numerical results show the validity of the theoretical results.

q-fin.PR↗

arXiv2018

Estimation of Non-Normalized Mixture Models and Clustering Using Deep Representation

We develop a general method for estimating a finite mixture of non-normalized models. Here, a non-normalized model is defined to be a parametric distribution with an intractable normalization constant. Existing methods for estimating non-normalized models without computing the normalization constant are not applicable to mixture models because they contain more than one intractable normalization constant. The proposed method is derived by extending noise contrastive estimation (NCE), which estimates non-normalized models by discriminating between the observed data and some artificially generated noise. We also propose an extension of NCE with multiple noise distributions. Then, based on the observation that conventional classification learning with neural networks is implicitly assuming an exponential family as a generative model, we introduce a method for clustering unlabeled data by estimating a finite mixture of distributions in an exponential family. Estimation of this mixture model is attained by the proposed extensions of NCE where the training data of neural networks are used as noise. Thus, the proposed method provides a probabilistically principled clustering method that is able to utilize a deep representation. Application to image clustering using a deep neural network gives promising results.

stat.ML↗

arXiv2017

Empirical Bayes Matrix Completion

We develop an empirical Bayes (EB) algorithm for the matrix completion problems. The EB algorithm is motivated from the singular value shrinkage estimator for matrix means by Efron and Morris (1972). Since the EB algorithm is essentially the EM algorithm applied to a simple model, it does not require heuristic parameter tuning other than tolerance. Numerical results demonstrated that the EB algorithm achieves a good trade-off between accuracy and efficiency compared to existing algorithms and that it works particularly well when the difference between the number of rows and columns is large. Application to real data also shows the practical utility of the EB algorithm.

stat.ML↗