arXiv subjects

Jinwoo Shin

Publications and source records attributed to Jinwoo Shin.

At least 217 records · Page 12Linked to original sources

arXiv2018

Three-dimensional Ricci-degenerate Riemannian manifolds satisfying geometric equations

In this paper, we study a three-dimensional Ricci-degenerate Riemannian manifold $(M^3,g)$ that admits a smooth nonzero solution $f$ to the equation \begin{align} \label{a1a} \nabla df=ψRc+ϕg, \end{align} where $ψ,ϕ$ are given smooth functions of $f$, $Rc$ is the Ricci tensor of $g$. Spaces of this type include various interesting classes, namely gradient Ricci solitons, $m$-quasi Einstein metrics, (vacuum) static spaces, $V$-static spaces, and critical point metrics. The $m$-quasi Einstein metrics and vacuum static spaces were previously studied in \cite{JJ,JEK}, respectively. In this paper, we refine them and develop a general approach for the solutions of (\ref{a1a}); we specify the shape of the metric $g$ satisfying (\ref{a1a}) when $\nabla f$ is not a Ricci-eigen vector. Then we focus on the remaining three classes, namely gradient Ricci solitons, $V$-static spaces, and critical point metrics. Furthermore, we present classifications of local three-dimensional Ricci-degenerate spaces of these three classes by explicitly describing the metric $g$ and the potential function $f$.

math.DG↗

arXiv2018

Gauged Mini-Bucket Elimination for Approximate Inference

Computing the partition function $Z$ of a discrete graphical model is a fundamental inference challenge. Since this is computationally intractable, variational approximations are often used in practice. Recently, so-called gauge transformations were used to improve variational lower bounds on $Z$. In this paper, we propose a new gauge-variational approach, termed WMBE-G, which combines gauge transformations with the weighted mini-bucket elimination (WMBE) method. WMBE-G can provide both upper and lower bounds on $Z$, and is easier to optimize than the prior gauge-variational algorithm. We show that WMBE-G strictly improves the earlier WMBE approximation for symmetric models including Ising models with no magnetic field. Our experimental results demonstrate the effectiveness of WMBE-G even for generic, nonsymmetric models.

stat.ML↗

arXiv2018

Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples

The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their predictions, i.e., do not distinguish in- and out-of-distributions. Recently, to handle this issue, several threshold-based detectors have been proposed given pre-trained neural classifiers. However, the performance of prior works highly depends on how to train the classifiers since they only focus on improving inference procedures. In this paper, we develop a novel training method for classifiers so that such inference algorithms can work better. In particular, we suggest two additional terms added to the original loss (e.g., cross entropy). The first one forces samples from out-of-distribution less confident by the classifier and the second one is for (implicitly) generating most effective training samples for the first one. In essence, our method jointly trains both classification and generative neural networks for out-of-distribution. We demonstrate its effectiveness using deep convolutional neural networks on various popular image datasets.

stat.ML↗

arXiv2018

A Graphical Transformation for Belief Propagation: Maximum Weight Matchings and Odd-Sized Cycles

We study the Maximum Weight Matching (MWM) problem for general graphs through the max-product Belief Propagation (BP) and related Linear Programming (LP). The BP approach provides distributed heuristics for finding the Maximum A Posteriori (MAP) assignment in a joint probability distribution represented by a Graphical Model (GM) and respective LPs can be considered as continuous relaxations of the discrete MAP problem. It was recently shown that a BP algorithm converges to the correct MWM assignment under a simple GM formulation of MAP/MWM as long as the corresponding LP relaxation is tight. First, under the motivation for forcing the tightness condition, we consider a new GM formulation of MWM, say C-GM, using non-intersecting odd-sized cycles in the graph: the new corresponding LP relaxation, say C-LP, becomes tight for more MWM instances. However, the tightness of C-LP now does not guarantee such convergence and correctness of the new BP on C-GM. To address the issue, we introduce a novel graph transformation applied to C-GM, which results in another GM formulation of MWM, and prove that the respective BP on it converges to the correct MAP/MWM assignment as long as C-LP is tight. Finally, we also show that C-LP always has half-integral solutions, which leads to an efficient BP-based MWM heuristic consisting of making sequential, `cutting plane', modifications to the underlying GM. Our experiments show that this BP-based cutting plane heuristic performs as well as that based on traditional LP solvers.

cs.DS↗

arXiv2017

Three dimensional m-quasi Einstein manifolds with degenerate Ricci tensor

In this article we give a classification of three dimensional m-quasi Einstein manifolds with two distinct Ricci-eigen values. Our study provides explicit description of local and complete metrics and potential functions. We also describe the associated warped product Einstein manifolds in detail. For the proof we present a Codazzi tensor on any three dimensional $m$-quasi Einstein manifold and use geometric properties of the tensor which help to analyze the m-quasi Einstein equation effectively. A technical advance over preceding studies is made by resolving the case when the gradient $\nabla f$ of the potential function is not a Ricci-eigen vector field.

math.DG↗

arXiv2017

Confident Multiple Choice Learning

Ensemble methods are arguably the most trustworthy techniques for boosting the performance of machine learning models. Popular independent ensembles (IE) relying on naive averaging/voting scheme have been of typical choice for most applications involving deep neural networks, but they do not consider advanced collaboration among ensemble models. In this paper, we propose new ensemble methods specialized for deep neural networks, called confident multiple choice learning (CMCL): it is a variant of multiple choice learning (MCL) via addressing its overconfidence issue.In particular, the proposed major components of CMCL beyond the original MCL scheme are (i) new loss, i.e., confident oracle loss, (ii) new architecture, i.e., feature sharing and (iii) new training method, i.e., stochastic labeling. We demonstrate the effect of CMCL via experiments on the image classification on CIFAR and SVHN, and the foreground-background segmentation on the iCoseg. In particular, CMCL using 5 residual networks provides 14.05% and 6.60% relative reductions in the top-1 error rates from the corresponding IE scheme for the classification task on CIFAR and SVHN, respectively.

cs.LG↗

arXiv2017

Gauging Variational Inference

Computing partition function is the most important statistical inference task arising in applications of Graphical Models (GM). Since it is computationally intractable, approximate methods have been used to resolve the issue in practice, where mean-field (MF) and belief propagation (BP) are arguably the most popular and successful approaches of a variational type. In this paper, we propose two new variational schemes, coined Gauged-MF (G-MF) and Gauged-BP (G-BP), improving MF and BP, respectively. Both provide lower bounds for the partition function by utilizing the so-called gauge transformation which modifies factors of GM while keeping the partition function invariant. Moreover, we prove that both G-MF and G-BP are exact for GMs with a single loop of a special structure, even though the bare MF and BP perform badly in this case. Our extensive experiments, on complete GMs of relatively small size and on large GM (up-to 300 variables) confirm that the newly proposed algorithms outperform and generalize MF and BP.

stat.ML↗

arXiv2017

Max-Product Belief Propagation for Linear Programming: Applications to Combinatorial Optimization

The max-product {belief propagation} (BP) is a popular message-passing heuristic for approximating a maximum-a-posteriori (MAP) assignment in a joint distribution represented by a graphical model (GM). In the past years, it has been shown that BP can solve a few classes of linear programming (LP) formulations to combinatorial optimization problems including maximum weight matching, shortest path and network flow, i.e., BP can be used as a message-passing solver for certain combinatorial optimizations. However, those LPs and corresponding BP analysis are very sensitive to underlying problem setups, and it has been not clear what extent these results can be generalized to. In this paper, we obtain a generic criteria that BP converges to the optimal solution of given LP, and show that it is satisfied in LP formulations associated to many classical combinatorial optimization problems including maximum weight perfect matching, shortest path, traveling salesman, cycle packing, vertex/edge cover and network flow.

cs.AI↗

arXiv2017

Faster Greedy MAP Inference for Determinantal Point Processes

Determinantal point processes (DPPs) are popular probabilistic models that arise in many machine learning tasks, where distributions of diverse sets are characterized by matrix determinants. In this paper, we develop fast algorithms to find the most likely configuration (MAP) of large-scale DPPs, which is NP-hard in general. Due to the submodular nature of the MAP objective, greedy algorithms have been used with empirical success. Greedy implementations require computation of log-determinants, matrix inverses or solving linear systems at each iteration. We present faster implementations of the greedy algorithms by utilizing the complementary benefits of two log-determinant approximation schemes: (a) first-order expansions to the matrix log-determinant function and (b) high-order expansions to the scalar log function with stochastic trace estimators. In our experiments, our algorithms are orders of magnitude faster than their competitors, while sacrificing marginal accuracy.

cs.DM↗

arXiv2017

Simplified Stochastic Feedforward Neural Networks

It has been believed that stochastic feedforward neural networks (SFNNs) have several advantages beyond deterministic deep neural networks (DNNs): they have more expressive power allowing multi-modal mappings and regularize better due to their stochastic nature. However, training large-scale SFNN is notoriously harder. In this paper, we aim at developing efficient training methods for SFNN, in particular using known architectures and pre-trained parameters of DNN. To this end, we propose a new intermediate stochastic model, called Simplified-SFNN, which can be built upon any baseline DNNand approximates certain SFNN by simplifying its upper latent units above stochastic ones. The main novelty of our approach is in establishing the connection between three models, i.e., DNN->Simplified-SFNN->SFNN, which naturally leads to an efficient training procedure of the stochastic models utilizing pre-trained parameters of DNN. Using several popular DNNs, we show how they can be effectively transferred to the corresponding stochastic models for both multi-modal and classification tasks on MNIST, TFD, CASIA, CIFAR-10, CIFAR-100 and SVHN datasets. In particular, we train a stochastic model of 28 layers and 36 million parameters, where training such a large-scale stochastic network is significantly challenging without using Simplified-SFNN

cs.LG↗

arXiv2017

Rapid Mixing Swendsen-Wang Sampler for Stochastic Partitioned Attractive Models

The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs). These tasks are computationally intractable in general, and the Gibbs sampler often suffers from slow mixing. In this paper, we study the Swendsen-Wang dynamics which is a more sophisticated Markov chain designed to overcome bottlenecks that impede the Gibbs sampler. We prove O(\log n) mixing time for attractive binary pairwise GMs (i.e., ferromagnetic Ising models) on stochastic partitioned graphs having n vertices, under some mild conditions, including low temperature regions where the Gibbs sampler provably mixes exponentially slow. Our experiments also confirm that the Swendsen-Wang sampler significantly outperforms the Gibbs sampler when they are used for learning parameters of attractive GMs.

cs.LG↗

arXiv2017

Sequential Local Learning for Latent Graphical Models

Learning parameters of latent graphical models (GM) is inherently much harder than that of no-latent ones since the latent variables make the corresponding log-likelihood non-concave. Nevertheless, expectation-maximization schemes are popularly used in practice, but they are typically stuck in local optima. In the recent years, the method of moments have provided a refreshing angle for resolving the non-convex issue, but it is applicable to a quite limited class of latent GMs. In this paper, we aim for enhancing its power via enlarging such a class of latent GMs. To this end, we introduce two novel concepts, coined marginalization and conditioning, which can reduce the problem of learning a larger GM to that of a smaller one. More importantly, they lead to a sequential learning framework that repeatedly increases the learning portion of given latent GM, and thus covers a significantly broader and more complicated class of loopy latent GMs which include convolutional and random regular models.

cs.LG↗

arXiv2017

Approximating the Spectral Sums of Large-scale Matrices using Chebyshev Approximations

Computation of the trace of a matrix function plays an important role in many scientific computing applications, including applications in machine learning, computational physics (e.g., lattice quantum chromodynamics), network analysis and computational biology (e.g., protein folding), just to name a few application areas. We propose a linear-time randomized algorithm for approximating the trace of matrix functions of large symmetric matrices. Our algorithm is based on coupling function approximation using Chebyshev interpolation with stochastic trace estimators (Hutchinson's method), and as such requires only implicit access to the matrix, in the form of a function that maps a vector to the product of the matrix and the vector. We provide rigorous approximation error in terms of the extremal eigenvalue of the input matrix, and the Bernstein ellipse that corresponds to the function at hand. Based on our general scheme, we provide algorithms with provable guarantees for important matrix computations, including log-determinant, trace of matrix inverse, Estrada index, Schatten p-norm, and testing positive definiteness. We experimentally evaluate our algorithm and demonstrate its effectiveness on matrices with tens of millions dimensions.

cs.DS↗

arXiv2017

Adiabatic Persistent Contrastive Divergence Learning

This paper studies the problem of parameter learning in probabilistic graphical models having latent variables, where the standard approach is the expectation maximization algorithm alternating expectation (E) and maximization (M) steps. However, both E and M steps are computationally intractable for high dimensional data, while the substitution of one step to a faster surrogate for combating against intractability can often cause failure in convergence. We propose a new learning algorithm which is computationally efficient and provably ensures convergence to a correct optimum. Its key idea is to run only a few cycles of Markov Chains (MC) in both E and M steps. Such an idea of running incomplete MC has been well studied only for M step in the literature, called Contrastive Divergence (CD) learning. While such known CD-based schemes find approximated gradients of the log-likelihood via the mean-field approach in E step, our proposed algorithm does exact ones via MC algorithms in both steps due to the multi-time-scale stochastic approximation theory. Despite its theoretical guarantee in convergence, the proposed scheme might suffer from the slow mixing of MC in E step. To tackle it, we also propose a hybrid approach applying both mean-field and MC approximation in E step, where the hybrid approach outperforms the bare mean-field CD scheme in our experiments on real-world datasets.

cs.LG↗

arXiv2017

Optimal Inference in Crowdsourced Classification via Belief Propagation

Crowdsourcing systems are popular for solving large-scale labelling tasks with low-paid workers. We study the problem of recovering the true labels from the possibly erroneous crowdsourced labels under the popular Dawid-Skene model. To address this inference problem, several algorithms have recently been proposed, but the best known guarantee is still significantly larger than the fundamental limit. We close this gap by introducing a tighter lower bound on the fundamental limit and proving that Belief Propagation (BP) exactly matches this lower bound. The guaranteed optimality of BP is the strongest in the sense that it is information-theoretically impossible for any other algorithm to correctly label a larger fraction of the tasks. Experimental results suggest that BP is close to optimal for all regimes considered and improves upon competing state-of-the-art algorithms.

cs.LG↗

arXiv2016

Scheduling using Interactive Optimization Oracles for Constrained Queueing Networks

Ever since Tassiulas and Ephremides (1992) proposed the maximum weight scheduling algorithm of throughput-optimality for constrained queueing networks that arise in the context of communication networks, extensive efforts have been devoted to resolving its most important drawback: high complexity. This paper proposes a generic framework for designing throughput- optimal and low-complexity scheduling algorithms for constrained queueing networks. Under our framework, a scheduling algorithm updates current schedules by interacting with a given oracle system that generates an approximate solution to a related optimization task. One can utilize our framework to design a variety of scheduling algorithms by choosing an oracle system such as random search, Markov chain, belief propagation, and primal-dual methods. The complexity of the resulting scheduling algorithm is determined by the number of operations required for an oracle to process a single query, which is typically small. We provide sufficient conditions for throughput-optimality of the scheduling algorithm in general constrained queueing network models. The linear-time algorithm of Tassiulas (1998) and the random access algorithm of Shah and Shin (2012) correspond to special cases of our framework using random search and Markov chain oracles, respectively. Our generic framework, however, provides a unified proof with milder assumptions.

math.PR↗

arXiv2016

On the classification of 4-dimensional $(m,ρ)$-quasi-Einstein manifolds with harmonic Weyl curvature

In this paper we study 4-dimensional $(m,ρ)$-quasi-Einstein manifolds with harmonic Weyl curvature when $m\notin\{0,\pm1,-2,\pm\infty\}$ and $ρ\notin\{\frac{1}{4},\frac{1}{6}\}$. We prove that a non-trivial $(m,ρ)$-quasi-Einstein metric $g$ (not necessarily complete) is locally isometric to one of the followings: (i) $\mathcal{B}^2_\frac{R}{2(m+2)}\times \mathbb{N}^2_\frac{R(m+1)}{2(m+2)}$ where $\mathcal{B}^2_\frac{R}{2(m+2)}$ is a northern hemisphere in the 2-dimensional sphere $\mathbb{S}^2_\frac{R}{2(m+2)}$, $\mathbb{N}_δ$ is the 2-dimensional Riemannian manifold with constant curvature $δ$ and $R$ is the constant scalar curvature of $g$, (ii) $\mathcal{D}^2_\frac{R}{2(m+2)}\times\mathbb{N}^2_\frac{R(m+1)}{2(m+2)}$ where $\mathcal{D}^2_\frac{R}{2(m+2)}$ is one half (cut by a hyperbolic line) of the hyperbolic plane $\mathbb{H}^2_\frac{R}{2(m+2)}$, (iii) $\mathbb{H}^2_\frac{R}{2(m+2)}\times\mathbb{N}^2_\frac{R(m+1)}{2(m+2)}$, (iv) a certain singular metric with $ρ=0$, (vi) a locally conformally flat metric. By applying this local classification, we obtain a classification of complete $(m,ρ)$-quasi-Einstein manifolds under the harmonic Weyl curvature condition. Our result can be viewed as a local classification of gradient Einstein-type manifolds. One corollary of our result is the classification of $(λ,4+m)$-Einstein manifolds which can be viewed as $(m,0)$-quasi-Einstein manifolds.

math.DG↗

arXiv2016

Four dimensional static and related critical spaces with harmonic curvature

In this article we study any 4-dimensional Riemannian manifold $(M,g)$ with harmonic curvature which admits a smooth nonzero solution $f$ to the following equation \begin{eqnarray} \label{0002bx} \nabla df = f(Rc -\frac{R}{n-1} g) + x Rc+ y(R) g. \end{eqnarray} where $Rc$ is the Ricci tensor of $g$, $x$ is a constant and $y(R)$ a function of the scalar curvature $R$. We show that a neighborhood of any point in some open dense subset of $M$ is locally isometric to one of the following five types; {\rm (i)} $ \mathbb{S}^2(\frac{R}{6}) \times \mathbb{S}^2(\frac{R}{3})$ with $R>0$, {\rm (ii)} $ \mathbb{H}^2(\frac{R}{6}) \times \mathbb{H}^2(\frac{R}{3}) $ with $R<0$, where $\mathbb{S}^2(k) $ and $\mathbb{H}^2(k) $ are the two-dimensional Riemannian manifold with constant sectional curvature $k>0$ and $k<0$, respectively, {\rm (iii)} the static spaces in Example 3 below, {\rm (iv)} conformally flat static spaces described in Kobayashi's \cite{Ko}, and {\rm (v)} a Ricci flat metric. We then get a number of Corollaries, including the classification of the following four dimensional spaces with harmonic curvature; static spaces, Miao-Tam critical metrics and $V$-static spaces. The proof is based on the argument from a preceding study of gradient Ricci solitons \cite{Ki}. Some Codazzi-tensor properties of Ricci tensor, which come from the harmonicity of curvature, are effectively used.

math.DG↗