arXiv subjects

Naonori Ueda

Publications and source records attributed to Naonori Ueda.

23 records · Page 2Linked to original sources

arXiv2017

Multi-output Polynomial Networks and Factorization Machines

Factorization machines and polynomial networks are supervised polynomial models based on an efficient low-rank decomposition. We extend these models to the multi-output setting, i.e., for learning vector-valued functions, with application to multi-class or multi-task problems. We cast this as the problem of learning a 3-way tensor whose slices share a common basis and propose a convex formulation of that problem. We then develop an efficient conditional gradient algorithm and prove its global convergence, despite the fact that it involves a non-convex basis selection step. On classification tasks, we show that our algorithm achieves excellent accuracy with much sparser models than existing methods. On recommendation system tasks, we show how to combine our algorithm with a reduction from ordinal regression to multi-output classification and show that the resulting algorithm outperforms simple baselines in terms of ranking accuracy.

stat.ML↗

arXiv2016

Higher-Order Factorization Machines

Factorization machines (FMs) are a supervised learning approach that can use second-order feature combinations even when the data is very high-dimensional. Unfortunately, despite increasing interest in FMs, there exists to date no efficient training algorithm for higher-order FMs (HOFMs). In this paper, we present the first generic yet efficient algorithms for training arbitrary-order HOFMs. We also present new variants of HOFMs with shared parameters, which greatly reduce model size and prediction times while maintaining similar accuracy. We demonstrate the proposed approaches on four different link prediction tasks.

stat.ML↗

arXiv2016

Machine-learning Selection of Optical Transients in Subaru/Hyper Suprime-Cam Survey

We present an application of machine-learning (ML) techniques to source selection in the optical transient survey data with Hyper Suprime-Cam (HSC) on the Subaru telescope. Our goal is to select real transient events accurately and in a timely manner out of a large number of false candidates, obtained with the standard difference-imaging method. We have developed the transient selector which is based on majority voting of three ML machines of AUC Boosting, Random Forest, and Deep Neural Network. We applied it to our observing runs of Subaru-HSC in 2015 May and August, and proved it to be efficient in selecting optical transients. The false positive rate was 1.0% at the true positive rate of 90% in the magnitude range of 22.0--25.0 mag for the former data. For the latter run, we successfully detected and reported ten candidates of supernovae within the same day as the observation. From these runs, we learned the following lessons: (1) the training using artificial objects is effective in filtering out false candidates, especially for faint objects, and (2) combination of ML by majority voting is advantageous.

astro-ph.IM↗

arXiv2016

Polynomial Networks and Factorization Machines: New Insights and Efficient Training Algorithms

Polynomial networks and factorization machines are two recently-proposed models that can efficiently use feature interactions in classification and regression tasks. In this paper, we revisit both models from a unified perspective. Based on this new view, we study the properties of both models and propose new efficient training algorithms. Key to our approach is to cast parameter learning as a low-rank symmetric tensor estimation problem, which we solve by multi-convex optimization. We demonstrate our approach on regression and recommender system tasks.

stat.ML↗

arXiv2014

Collapsed Variational Bayes Inference of Infinite Relational Model

The Infinite Relational Model (IRM) is a probabilistic model for relational data clustering that partitions objects into clusters based on observed relationships. This paper presents Averaged CVB (ACVB) solutions for IRM, convergence-guaranteed and practically useful fast Collapsed Variational Bayes (CVB) inferences. We first derive ordinary CVB and CVB0 for IRM based on the lower bound maximization. CVB solutions yield deterministic iterative procedures for inferring IRM given the truncated number of clusters. Our proposal includes CVB0 updates of hyperparameters including the concentration parameter of the Dirichlet Process, which has not been studied in the literature. To make the CVB more practically useful, we further study the CVB inference in two aspects. First, we study the convergence issues and develop a convergence-guaranteed algorithm for any CVB-based inferences called ACVB, which enables automatic convergence detection and frees non-expert practitioners from difficult and costly manual monitoring of inference processes. Second, we present a few techniques for speeding up IRM inferences. In particular, we describe the linear time inference of CVB0, allowing the IRM for larger relational data uses. The ACVB solutions of IRM showed comparable or better performance compared to existing inference methods in experiments, and provide deterministic, faster, and easier convergence detection.

cs.LG↗