Searcharxiv⌕ Search

arXiv subjects

Takanori Maehara

Publications and source records attributed to Takanori Maehara.

At least 37 records · Page 2Linked to original sources
arXiv2019

State-Space Network Topology Identification from Partial Observations

In this work, we explore the state-space formulation of a network process to recover, from partial observations, the underlying network topology that drives its dynamics. To do so, we employ subspace techniques borrowed from system identification literature and extend them to the network topology identification problem. This approach provides a unified view of the traditional network control theory and signal processing on graphs. In addition, it provides theoretical guarantees for the recovery of the topological structure of a deterministic continuous-time linear dynamical system from input-output observations even though the input and state interaction networks might be different. The derived mathematical analysis is accompanied by an algorithm for identifying, from data, a network topology consistent with the dynamics of the system and conforms to the prior information about the underlying structure. The proposed algorithm relies on alternating projections and is provably convergent. Numerical results corroborate the theoretical findings and the applicability of the proposed algorithm.

eess.SP↗
arXiv2019

Data Cleansing for Models Trained with SGD

Data cleansing is a typical approach used to improve the accuracy of machine learning models, which, however, requires extensive domain knowledge to identify the influential instances that affect the models. In this paper, we propose an algorithm that can suggest influential instances without using any domain knowledge. With the proposed method, users only need to inspect the instances suggested by the algorithm, implying that users do not need extensive knowledge for this procedure, which enables even non-experts to conduct data cleansing and improve the model. The existing methods require the loss function to be convex and an optimal model to be obtained, which is not always the case in modern machine learning. To overcome these limitations, we propose a novel approach specifically designed for the models trained with stochastic gradient descent (SGD). The proposed method infers the influential instances by retracing the steps of the SGD while incorporating intermediate models computed in each step. Through experiments, we demonstrate that the proposed method can accurately infer the influential instances. Moreover, we used MNIST and CIFAR10 to show that the models can be effectively improved by removing the influential instances suggested by the proposed method.

stat.ML↗
arXiv2019

Revisiting Graph Neural Networks: All We Have is Low-Pass Filters

Graph neural networks have become one of the most important techniques to solve machine learning problems on graph-structured data. Recent work on vertex classification proposed deep and distributed learning models to achieve high performance and scalability. However, we find that the feature vectors of benchmark datasets are already quite informative for the classification task, and the graph structure only provides a means to denoise the data. In this paper, we develop a theoretical framework based on graph signal processing for analyzing graph neural networks. Our results indicate that graph neural networks only perform low-pass filtering on feature vectors and do not have the non-linear manifold learning property. We further investigate their resilience to feature noise and propose some insights on GCN-based graph neural network design.

stat.ML↗
arXiv2019

Stochastic Packing Integer Programs with Few Queries

We consider a stochastic variant of the packing-type integer linear programming problem, which contains random variables in the objective vector. We are allowed to reveal each entry of the objective vector by conducting a query, and the task is to find a good solution by conducting a small number of queries. We propose a general framework of adaptive and non-adaptive algorithms for this problem, and provide a unified methodology for analyzing the performance of those algorithms. We also demonstrate our framework by applying it to a variety of stochastic combinatorial optimization problems such as matching, matroid, and stable set problems.

cs.DS↗
arXiv2019

Optimal Algorithm to Reconstruct a Tree from a Subtree Distance

This paper addresses the problem of finding a representation of a subtree distance, which is an extension of the tree metric. We show that a minimal representation is uniquely determined by a given subtree distance, and give a linear time algorithm that finds such a representation. This algorithm achieves the optimal time complexity.

cs.DS↗
arXiv2019

Submodular Stochastic Probing with Prices

We introduce Stochastic Probing with Prices (SPP), a variant of the Stochastic Probing (SP) model in which we must pay a price to probe an element. A SPP problem involves two set systems $(N,\mathcal{I}_{in})$ and $(N,\mathcal{I}_{out})$ where each $e\in N$ is active with probability $p_e$. To discover whether $e$ is active, it must be probed by paying the price $Δ_e$. If it is probed and active, then it is irrevocably added to the solution. Moreover, at all times, the set of probed elements must lie in $\mathcal{I}_{out}$, and the solution (the set of probed and active elements) must lie in $\mathcal{I}_{in}$. The goal is to maximize a set function $f$ minus the cost of the probes. We give a bi-criteria approximation algorithm to the online version of this problem, in which the elements are shown to the algorithm in a possibly adversarial order. Our results translate to state-of-the-art approximations for the traditional (online) stochastic probing problem.

cs.DS↗
arXiv2018

Convex Hull Approximation of Nearly Optimal Lasso Solutions

In an ordinary feature selection procedure, a set of important features is obtained by solving an optimization problem such as the Lasso regression problem, and we expect that the obtained features explain the data well. In this study, instead of the single optimal solution, we consider finding a set of diverse yet nearly optimal solutions. To this end, we formulate the problem as finding a small number of solutions such that the convex hull of these solutions approximates the set of nearly optimal solutions. The proposed algorithm consists of two steps: First, we randomly sample the extreme points of the set of nearly optimal solutions. Then, we select a small number of points using a greedy algorithm. The experimental results indicate that the proposed algorithm can approximate the solution set well. The results also indicate that we can obtain Lasso solutions with a large diversity.

stat.ML↗
arXiv2018

A Simple Way to Deal with Cherry-picking

Statistical hypothesis testing serves as statistical evidence for scientific innovation. However, if the reported results are intentionally biased, hypothesis testing no longer controls the rate of false discovery. In particular, we study such selection bias in machine learning models where the reporter is motivated to promote an algorithmic innovation. When the number of possible configurations (e.g., datasets) is large, we show that the reporter can falsely report an innovation even if there is no improvement at all. We propose a `post-reporting' solution to this issue where the bias of the reported results is verified by another set of results. The theoretical findings are supported by experimental results with synthetic and real-world datasets.

stat.ME↗
arXiv2018

Linear Pseudo-Polynomial Factor Algorithm for Automaton Constrained Tree Knapsack Problem

The automaton constrained tree knapsack problem is a variant of the knapsack problem in which the items are associated with the vertices of the tree, and we can select a subset of items that is accepted by a top-down tree automaton. If the capacities or the profits of items are integers, the problem can be solved in pseudo-polynomial time using the dynamic programming algorithm. However, the natural implementation of this algorithm has a quadratic pseudo-polynomial factor in its complexity because of the max-plus convolution. In this study, we propose a new dynamic programming technique, called \emph{heavy-light recursive dynamic programming}, to obtain pseudo-polynomial time algorithms having linear pseudo-polynomial factors in the complexity. Such algorithms can be used for solving the problems with polynomially small capacities/profits efficiently, and used for deriving efficient fully polynomial-time approximation schemes. We also consider the $k$-subtree version problem that finds $k$ disjoint subtrees and a solution in each subtree that maximizes total profit under a budget constraint. We show that this problem can be solved in almost the same order as the original problem.

cs.DS↗
arXiv2018

Computing a tree having a small vertex cover

We consider a new Steiner tree problem, called vertex-cover-weighted Steiner tree problem. This problem defines the weight of a Steiner tree as the minimum weight of vertex covers in the tree, and seeks a minimum-weight Steiner tree in a given vertex-weighted undirected graph. Since it is included by the Steiner tree activation problem, the problem admits an O(log n)-approximation algorithm in general graphs with n vertices. This approximation factor is tight up to a constant because it is NP-hard to achieve an o(log n)-approximation for the vertex-cover-weighted Steiner tree problem on general graphs even if the given vertex weights are uniform and a spanning tree is required instead of a Steiner tree. In this paper, we present constant-factor approximation algorithms for the problem with unit disk graphs and with graphs excluding a fixed minor. For the latter graph class, our algorithm can be also applied for the Steiner tree activation problem.

cs.DS↗
arXiv2018

Algorithmic Meta-Theorems for Monotone Submodular Maximization

We consider a monotone submodular maximization problem whose constraint is described by a logic formula on a graph. Formally, we prove the following three `algorithmic metatheorems.' (1) If the constraint is specified by a monadic second-order logic on a graph of bounded treewidth, the problem is solved in $n^{O(1)}$ time with an approximation factor of $O(\log n)$. (2) If the constraint is specified by a first-order logic on a graph of low degree, the problem is solved in $O(n^{1 + ε})$ time for any $ε> 0$ with an approximation factor of $2$. (3) If the constraint is specified by a first-order logic on a graph of bounded expansion, the problem is solved in $n^{O(\log k)}$ time with an approximation factor of $O(\log k)$, where $k$ is the number of variables and $O(\cdot)$ suppresses only constants independent of $k$.

cs.DS↗
arXiv2018

Maximally Invariant Data Perturbation as Explanation

While several feature scoring methods are proposed to explain the output of complex machine learning models, most of them lack formal mathematical definitions. In this study, we propose a novel definition of the feature score using the maximally invariant data perturbation, which is inspired from the idea of adversarial example. In adversarial example, one seeks the smallest data perturbation that changes the model's output. In our proposed approach, we consider the opposite: we seek the maximally invariant data perturbation that does not change the model's output. In this way, we can identify important input features as the ones with small allowable data perturbations. To find the maximally invariant data perturbation, we formulate the problem as linear programming. The experiment on the image classification with VGG16 shows that the proposed method could identify relevant parts of the images effectively.

stat.ML↗
arXiv2018

Neural Inverse Rendering for General Reflectance Photometric Stereo

We present a novel convolutional neural network architecture for photometric stereo (Woodham, 1980), a problem of recovering 3D object surface normals from multiple images observed under varying illuminations. Despite its long history in computer vision, the problem still shows fundamental challenges for surfaces with unknown general reflectance properties (BRDFs). Leveraging deep neural networks to learn complicated reflectance models is promising, but studies in this direction are very limited due to difficulties in acquiring accurate ground truth for training and also in designing networks invariant to permutation of input images. In order to address these challenges, we propose a physics based unsupervised learning framework where surface normals and BRDFs are predicted by the network and fed into the rendering equation to synthesize observed images. The network weights are optimized during testing by minimizing reconstruction loss between observed and synthesized images. Thus, our learning process does not require ground truth normals or even pre-training on external images. Our method is shown to achieve the state-of-the-art performance on a challenging real-world scene benchmark.

cs.CV↗
arXiv2018

Subspace Selection via DR-Submodular Maximization on Lattices

The subspace selection problem seeks a subspace that maximizes an objective function under some constraint. This problem includes several important machine learning problems such as the principal component analysis and sparse dictionary selection problem. Often, these problems can be solved by greedy algorithms. Here, we are interested in why these problems can be solved by greedy algorithms, and what classes of objective functions and constraints admit this property. To answer this question, we formulate the problems as optimization problems on lattices. Then, we introduce a new class of functions, directional DR-submodular functions, to characterize the approximability of problems. We see that the principal component analysis, sparse dictionary selection problem, and these generalizations have directional DR-submodularities. We show that, under several constraints, the directional DR-submodular function maximization problem can be solved efficiently with provable approximation factors.

cs.DS↗
arXiv2018

ClassiNet -- Predicting Missing Features for Short-Text Classification

The fundamental problem in short-text classification is \emph{feature sparseness} -- the lack of feature overlap between a trained model and a test instance to be classified. We propose \emph{ClassiNet} -- a network of classifiers trained for predicting missing features in a given instance, to overcome the feature sparseness problem. Using a set of unlabeled training instances, we first learn binary classifiers as feature predictors for predicting whether a particular feature occurs in a given instance. Next, each feature predictor is represented as a vertex $v_i$ in the ClassiNet where a one-to-one correspondence exists between feature predictors and vertices. The weight of the directed edge $e_{ij}$ connecting a vertex $v_i$ to a vertex $v_j$ represents the conditional probability that given $v_i$ exists in an instance, $v_j$ also exists in the same instance. We show that ClassiNets generalize word co-occurrence graphs by considering implicit co-occurrences between features. We extract numerous features from the trained ClassiNet to overcome feature sparseness. In particular, for a given instance $\vec{x}$, we find similar features from ClassiNet that did not appear in $\vec{x}$, and append those features in the representation of $\vec{x}$. Moreover, we propose a method based on graph propagation to find features that are indirectly related to a given short-text. We evaluate ClassiNets on several benchmark datasets for short-text classification. Our experimental results show that by using ClassiNet, we can statistically significantly improve the accuracy in short-text classification tasks, without having to use any external resources such as thesauri for finding related features.

cs.CL↗
arXiv2018

Typical Approximation Performance for Maximum Coverage Problem

This study investigated typical performance of approximation algorithms known as belief propagation, greedy algorithm, and linear-programming relaxation for maximum coverage problems on sparse biregular random graphs. After using the cavity method for a corresponding hard-core lattice--gas model, results show that two distinct thresholds of replica-symmetry and its breaking exist in the typical performance threshold of belief propagation. In the low-density region, the superiority of three algorithms in terms of a typical performance threshold is obtained by some theoretical analyses. Although the greedy algorithm and linear-programming relaxation have the same approximation ratio in worst-case performance, their typical performance thresholds are mutually different, indicating the importance of typical performance. Results of numerical simulations validate the theoretical analyses and imply further mutual relations of approximation algorithms.

cond-mat.dis-nn↗
arXiv2017

On Tensor Train Rank Minimization: Statistical Efficiency and Scalable Algorithm

Tensor train (TT) decomposition provides a space-efficient representation for higher-order tensors. Despite its advantage, we face two crucial limitations when we apply the TT decomposition to machine learning problems: the lack of statistical theory and of scalable algorithms. In this paper, we address the limitations. First, we introduce a convex relaxation of the TT decomposition problem and derive its error bound for the tensor completion task. Next, we develop an alternating optimization method with a randomization technique, in which the time complexity is as efficient as the space complexity is. In experiments, we numerically confirm the derived bounds and empirically demonstrate the performance of our method with a real higher-order tensor.

stat.ML↗