arXiv subjects

Gustavo K. Rohde

Publications and source records attributed to Gustavo K. Rohde.

28 records · Page 2Linked to original sources

arXiv2019

Neural Networks, Hypersurfaces, and Radon Transforms

Connections between integration along hypersufaces, Radon transforms, and neural networks are exploited to highlight an integral geometric mathematical interpretation of neural networks. By analyzing the properties of neural networks as operators on probability distributions for observed data, we show that the distribution of outputs for any node in a neural network can be interpreted as a nonlinear projection along hypersurfaces defined by level surfaces over the input data space. We utilize these descriptions to provide new interpretation for phenomena such as nonlinearity, pooling, activation functions, and adversarial examples in neural network-based learning problems.

stat.ML↗

arXiv2019

Generalized Sliced Wasserstein Distances

The Wasserstein distance and its variations, e.g., the sliced-Wasserstein (SW) distance, have recently drawn attention from the machine learning community. The SW distance, specifically, was shown to have similar properties to the Wasserstein distance, while being much simpler to compute, and is therefore used in various applications including generative modeling and general supervised/unsupervised learning. In this paper, we first clarify the mathematical connection between the SW distance and the Radon transform. We then utilize the generalized Radon transform to define a new family of distances for probability measures, which we call generalized sliced-Wasserstein (GSW) distances. We also show that, similar to the SW distance, the GSW distance can be extended to a maximum GSW (max-GSW) distance. We then provide the conditions under which GSW and max-GSW distances are indeed distances. Finally, we compare the numerical performance of the proposed distances on several generative modeling tasks, including SW flows and SW auto-encoders.

cs.LG↗

arXiv2018

Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model

In this paper we study generative modeling via autoencoders while using the elegant geometric properties of the optimal transport (OT) problem and the Wasserstein distances. We introduce Sliced-Wasserstein Autoencoders (SWAE), which are generative models that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or defining a closed-form for the distribution. In short, we regularize the autoencoder loss with the sliced-Wasserstein distance between the distribution of the encoded training samples and a predefined samplable distribution. We show that the proposed formulation has an efficient numerical solution that provides similar capabilities to Wasserstein Autoencoders (WAE) and Variational Autoencoders (VAE), while benefiting from an embarrassingly simple implementation.

cs.LG↗

arXiv2018

Transport-Based Pattern Theory: A Signal Transformation Approach

In many scientific fields imaging is used to relate a certain physical quantity to other dependent variables. Therefore, images can be considered as a map from a real-world coordinate system to the non-negative measurements being acquired. In this work we describe an approach for simultaneous modeling and inference of such data, using the mathematics of optimal transport. To achieve this, we describe a numerical implementation of the linear optimal transport transform, based on the solution of the Monge-Ampere equation, which uses Brenier's theorem to characterize the solution of the Monge functional as the derivative of a convex potential function. We use our implementation of the transform to compute a curl-free mapping between two images, and show that it is able to match images with lower error that existing methods. Moreover, we provide theoretical justification for properties of the linear optimal transport framework observed in the literature, including a theorem for the linear separation of data classes. Finally, we use our optimal transport method to empirically demonstrate that the linear separability theorem holds, by rendering non-linearly separable data as linearly separable following transform to transport space.

cs.CV↗

arXiv2017

Sliced Wasserstein Distance for Learning Gaussian Mixture Models

Gaussian mixture models (GMM) are powerful parametric tools with many applications in machine learning and computer vision. Expectation maximization (EM) is the most popular algorithm for estimating the GMM parameters. However, EM guarantees only convergence to a stationary point of the log-likelihood function, which could be arbitrarily worse than the optimal solution. Inspired by the relationship between the negative log-likelihood function and the Kullback-Leibler (KL) divergence, we propose an alternative formulation for estimating the GMM parameters using the sliced Wasserstein distance, which gives rise to a new algorithm. Specifically, we propose minimizing the sliced-Wasserstein distance between the mixture model and the data distribution with respect to the GMM parameters. In contrast to the KL-divergence, the energy landscape for the sliced-Wasserstein distance is more well-behaved and therefore more suitable for a stochastic gradient descent scheme to obtain the optimal GMM parameters. We show that our formulation results in parameter estimates that are more robust to random initializations and demonstrate that it can estimate high-dimensional data distributions more faithfully than the EM algorithm.

cs.CV↗

arXiv2017

An optimal transport model for imaging in atmospheric turbulence

We describe a new model for image propagation through open air in the presence of changes in the index of refraction (e.g. due to turbulence) using the theory of optimal transport. We describe the relationship between photon density, or image intensity, and the phase of the traveling wave and, together with a least action principle, suggest a method for approximately recovering the solution of the photon flow. By linking atmospheric propagation solutions to optimal transport, we provide a physics-based (as opposed to phenomenological) model for predicting turbulence-induced changes to sequences of images. Simulated and real data are utilized to validate and compare the model to other existing methods typically used to model this type of data. Given its superior performance in describing experimental data, the new model suggests new algorithms for a variety of atmospheric imaging applications.

physics.optics↗

arXiv2016

A Transportation $L^p$ Distance for Signal Analysis

Transport based distances, such as the Wasserstein distance and earth mover's distance, have been shown to be an effective tool in signal and image analysis. The success of transport based distances is in part due to their Lagrangian nature which allows it to capture the important variations in many signal classes. However these distances require the signal to be nonnegative and normalized. Furthermore, the signals are considered as measures and compared by redistributing (transporting) them, which does not directly take into account the signal intensity. Here we study a transport-based distance, called the $TL^p$ distance, that combines Lagrangian and intensity modelling and is directly applicable to general, non-positive and multi-channelled signals. The framework allows the application of existing numerical methods. We give an overview of the basic properties of this distance and applications to classification, with multi-channelled, non-positive one and two-dimensional signals, and color transfer.

cs.CV↗

arXiv2016

Transport-based analysis, modeling, and learning from signal and data distributions

Transport-based techniques for signal and data analysis have received increased attention recently. Given their abilities to provide accurate generative models for signal intensities and other data distributions, they have been used in a variety of applications including content-based retrieval, cancer detection, image super-resolution, and statistical machine learning, to name a few, and shown to produce state of the art in several applications. Moreover, the geometric characteristics of transport-related metrics have inspired new kinds of algorithms for interpreting the meaning of data distributions. Here we provide an overview of the mathematical underpinnings of mass transport-related methods, including numerical implementation, as well as a review, with demonstrations, of several applications.

cs.CV↗

arXiv2015

Sliced Wasserstein Kernels for Probability Distributions

Optimal transport distances, otherwise known as Wasserstein distances, have recently drawn ample attention in computer vision and machine learning as a powerful discrepancy measure for probability distributions. The recent developments on alternative formulations of the optimal transport have allowed for faster solutions to the problem and has revamped its practical applications in machine learning. In this paper, we exploit the widely used kernel methods and provide a family of provably positive definite kernels based on the Sliced Wasserstein distance and demonstrate the benefits of these kernels in a variety of learning tasks. Our work provides a new perspective on the application of optimal transport flavored distances through kernel methods in machine learning tasks.

cs.LG↗

arXiv2015

The Radon cumulative distribution transform and its application to image classification

Invertible image representation methods (transforms) are routinely employed as low-level image processing operations based on which feature extraction and recognition algorithms are developed. Most transforms in current use (e.g. Fourier, Wavelet, etc.) are linear transforms, and, by themselves, are unable to substantially simplify the representation of image classes for classification. Here we describe a nonlinear, invertible, low-level image processing transform based on combining the well known Radon transform for image data, and the 1D Cumulative Distribution Transform proposed earlier. We describe a few of the properties of this new transform, and with both theoretical and experimental results show that it can often render certain problems linearly separable in transform space.

cs.CV↗