Searcharxiv⌕ Search

arXiv subjects

Ambedkar Dukkipati

Publications and source records attributed to Ambedkar Dukkipati.

At least 55 records · Page 3Linked to original sources
arXiv2017

Unsupervised feature learning with discriminative encoder

In recent years, deep discriminative models have achieved extraordinary performance on supervised learning tasks, significantly outperforming their generative counterparts. However, their success relies on the presence of a large amount of labeled data. How can one use the same discriminative models for learning useful features in the absence of labels? We address this question in this paper, by jointly modeling the distribution of data and latent features in a manner that explicitly assigns zero probability to unobserved data. Rather than maximizing the marginal probability of observed data, we maximize the joint probability of the data and the latent features using a two step EM-like procedure. To prevent the model from overfitting to our initial selection of latent features, we use adversarial regularization. Depending on the task, we allow the latent features to be one-hot or real-valued vectors and define a suitable prior on the features. For instance, one-hot features correspond to class labels and are directly used for the unsupervised and semi-supervised classification task, whereas real-valued feature vectors are fed as input to simple classifiers for auxiliary supervised discrimination tasks. The proposed model, which we dub discriminative encoder (or DisCoder), is flexible in the type of latent features that it can capture. The proposed model achieves state-of-the-art performance on several challenging tasks.

cs.CV↗
arXiv2017

Attentive Recurrent Comparators

Rapid learning requires flexible representations to quickly adopt to new evidence. We develop a novel class of models called Attentive Recurrent Comparators (ARCs) that form representations of objects by cycling through them and making observations. Using the representations extracted by ARCs, we develop a way of approximating a \textit{dynamic representation space} and use it for one-shot learning. In the task of one-shot classification on the Omniglot dataset, we achieve the state of the art performance with an error rate of 1.5\%. This represents the first super-human result achieved for this task with a generic model that uses only pixel information.

cs.CV↗
arXiv2017

Uniform Hypergraph Partitioning: Provable Tensor Methods and Sampling Techniques

In a series of recent works, we have generalised the consistency results in the stochastic block model literature to the case of uniform and non-uniform hypergraphs. The present paper continues the same line of study, where we focus on partitioning weighted uniform hypergraphs---a problem often encountered in computer vision. This work is motivated by two issues that arise when a hypergraph partitioning approach is used to tackle computer vision problems: (i) The uniform hypergraphs constructed for higher-order learning contain all edges, but most have negligible weights. Thus, the adjacency tensor is nearly sparse, and yet, not binary. (ii) A more serious concern is that standard partitioning algorithms need to compute all edge weights, which is computationally expensive for hypergraphs. This is usually resolved in practice by merging the clustering algorithm with a tensor sampling strategy---an approach that is yet to be analysed rigorously. We build on our earlier work on partitioning dense unweighted uniform hypergraphs (Ghoshdastidar and Dukkipati, ICML, 2015), and address the aforementioned issues by proposing provable and efficient partitioning algorithms. Our analysis justifies the empirical success of practical sampling techniques. We also complement our theoretical findings by elaborate empirical comparison of various hypergraph partitioning schemes.

cs.LG↗
arXiv2017

Generative Adversarial Residual Pairwise Networks for One Shot Learning

Deep neural networks achieve unprecedented performance levels over many tasks and scale well with large quantities of data, but performance in the low-data regime and tasks like one shot learning still lags behind. While recent work suggests many hypotheses from better optimization to more complicated network structures, in this work we hypothesize that having a learnable and more expressive similarity objective is an essential missing component. Towards overcoming that, we propose a network design inspired by deep residual networks that allows the efficient computation of this more expressive pairwise similarity objective. Further, we argue that regularization is key in learning with small amounts of data, and propose an additional generator network based on the Generative Adversarial Networks where the discriminator is our residual pairwise network. This provides a strong regularizer by leveraging the generated data samples. The proposed model can generate plausible variations of exemplars over unseen classes and outperforms strong discriminative baselines for few shot classification tasks. Notably, our residual pairwise network design outperforms previous state-of-theart on the challenging mini-Imagenet dataset for one shot learning by getting over 55% accuracy for the 5-way classification task over unseen classes.

cs.CV↗
arXiv2017

On Gröbner Bases and Krull Dimension of Residue Class Rings of Polynomial Rings over Integral Domains

Given an ideal $\mathfrak{a}$ in $A[x_1, \ldots, x_n]$, where $A$ is a Noetherian integral domain, we propose an approach to compute the Krull dimension of $A[x_1,\ldots,x_n]/\mathfrak{a}$, when the residue class polynomial ring is a free $A$-module. When $A$ is a field, the Krull dimension of $A[x_1,\ldots,x_n]/\mathfrak{a}$ has several equivalent algorithmic definitions by which it can be computed. But this is not true in the case of arbitrary Noetherian rings. For a Noetherian integral domain, $A$ we introduce the notion of combinatorial dimension of $A[x_1, \ldots,x_n]/\mathfrak{a}$ and give a Gröbner basis method to compute it for residue class polynomial rings that have a free $A$-module representation w.r.t. a lexicographic ordering. For such $A$-algebras, we derive a relation between Krull dimension and combinatorial dimension of $A[x_1, \ldots, x_n]/\mathfrak{a}$. An immediate application of this relation is that it gives a uniform method, the first of its kind, to compute the dimension of $A[x_1, \ldots, x_n]/\mathfrak{a}$ without having to consider individual properties of the ideal. For $A$-algebras that have a free $A$-module representation w.r.t. degree compatible monomial orderings, we introduce the concepts of Hilbert function, Hilbert series and Hilbert polynomials and show that Gröbner basis methods can be used to compute these quantities. We then proceed to show that the combinatorial dimension of such $A$-algebras is equal to the degree of the Hilbert polynomial. This enables us to extend the relation between Krull dimension and combinatorial dimension to $A$-algebras with a free $A$-module representation w.r.t. a degree compatible ordering as well.

cs.SC↗
arXiv2017

Discriminative Neural Topic Models

We propose a neural network based approach for learning topics from text and image datasets. The model makes no assumptions about the conditional distribution of the observed features given the latent topics. This allows us to perform topic modelling efficiently using sentences of documents and patches of images as observed features, rather than limiting ourselves to words. Moreover, the proposed approach is online, and hence can be used for streaming data. Furthermore, since the approach utilizes neural networks, it can be implemented on GPU with ease, and hence it is very scalable.

cs.LG↗
arXiv2017

Border Bases for Polynomial Rings over Noetherian Rings

The theory of border bases for zero-dimensional ideals has attracted several researchers in symbolic computation due to their numerical stability and mathematical elegance. As shown in (Francis & Dukkipati, J. Symb. Comp., 2014), one can extend the concept of border bases over Noetherian rings whenever the corresponding residue class ring is finitely generated and free. In this paper we address the following problem: Can the concept of border basis over Noetherian rings exists for ideals when the corresponding residue class rings are finitely generated but need not necessarily be free modules? We present a border division algorithm and prove the termination of the algorithm for a special class of border bases. We show the existence of such border bases over Noetherian rings and present some characterizations in this regard. We also show that certain reduced Gröbner bases over Noetherian rings are contained in this class of border bases.

cs.SC↗
arXiv2017

A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference

In this work we use the recent advances in representation learning to propose a neural architecture for the problem of natural language inference. Our approach is aligned to mimic how a human does the natural language inference process given two statements. The model uses variants of Long Short Term Memory (LSTM), attention mechanism and composable neural networks, to carry out the task. Each part of our model can be mapped to a clear functionality humans do for carrying out the overall task of natural language inference. The model is end-to-end differentiable enabling training by stochastic gradient descent. On Stanford Natural Language Inference(SNLI) dataset, the proposed model achieves better accuracy numbers than all published models in literature.

cs.CL↗
arXiv2017

Analytic Connectivity in General Hypergraphs

In this paper we extend the known results of analytic connectivity to non-uniform hypergraphs. We prove a modified Cheeger's inequality and also give a bound on analytic connectivity with respect to the degree sequence and diameter of a hypergraph.

cs.DM↗
arXiv2017

Image Generation and Editing with Variational Info Generative AdversarialNetworks

Recently there has been an enormous interest in generative models for images in deep learning. In pursuit of this, Generative Adversarial Networks (GAN) and Variational Auto-Encoder (VAE) have surfaced as two most prominent and popular models. While VAEs tend to produce excellent reconstructions but blurry samples, GANs generate sharp but slightly distorted images. In this paper we propose a new model called Variational InfoGAN (ViGAN). Our aim is two fold: (i) To generated new images conditioned on visual descriptions, and (ii) modify the image, by fixing the latent representation of image and varying the visual description. We evaluate our model on Labeled Faces in the Wild (LFW), celebA and a modified version of MNIST datasets and demonstrate the ability of our model to generate new images as well as to modify a given image by changing attributes.

cs.CV↗
arXiv2016

Deep Variational Inference Without Pixel-Wise Reconstruction

Variational autoencoders (VAEs), that are built upon deep neural networks have emerged as popular generative models in computer vision. Most of the work towards improving variational autoencoders has focused mainly on making the approximations to the posterior flexible and accurate, leading to tremendous progress. However, there have been limited efforts to replace pixel-wise reconstruction, which have known shortcomings. In this work, we use real-valued non-volume preserving transformations (real NVP) to exactly compute the conditional likelihood of the data given the latent distribution. We show that a simple VAE with this form of reconstruction is competitive with complicated VAE structures, on image modeling tasks. As part of our model, we develop powerful conditional coupling layers that enable real NVP to learn with fewer intermediate layers.

stat.ML↗
arXiv2016

On Ideal Lattices, Gröbner Bases and Generalized Hash Functions

In this paper, we draw connections between ideal lattices and multivariate polynomial rings over integers using Gröbner bases. Ideal lattices are ideals in the residue class ring, $\mathbb{Z}[x]/\langle f \rangle$ (here $f$ is a monic polynomial), and cryptographic primitives have been built based on these objects. As ideal lattices in the univariate case are generalizations of cyclic lattices, we introduce the notion of multivariate cyclic lattices and show that multivariate ideal lattices are indeed a generalization of them. Based on multivariate ideal lattices, we establish the existence of collision resistant hash functions using Gröbner basis techniques. For the construction of hash functions, we define a worst case problem, shortest substitution problem w.r.t. an ideal in $\mathbb{Z}[x_1,\ldots, x_n]$, and establish hardness results using functional fields.

cs.SC↗
arXiv2016

Variational methods for Conditional Multimodal Deep Learning

In this paper, we address the problem of conditional modality learning, whereby one is interested in generating one modality given the other. While it is straightforward to learn a joint distribution over multiple modalities using a deep multimodal architecture, we observe that such models aren't very effective at conditional generation. Hence, we address the problem by learning conditional distributions between the modalities. We use variational methods for maximizing the corresponding conditional log-likelihood. The resultant deep model, which we refer to as conditional multimodal autoencoder (CMMA), forces the latent representation obtained from a single modality alone to be `close' to the joint representation obtained from multiple modalities. We use the proposed model to generate faces from attributes. We show that the faces generated from attributes using the proposed model, are qualitatively and quantitatively more representative of the attributes from which they were generated, than those obtained by other deep generative models. We also propose a secondary task, whereby the existing faces are modified by modifying the corresponding attributes. We observe that the modifications in face introduced by the proposed model are representative of the corresponding modifications in attributes.

cs.CV↗
arXiv2016

On collapsed representation of hierarchical Completely Random Measures

The aim of the paper is to provide an exact approach for generating a Poisson process sampled from a hierarchical CRM, without having to instantiate the infinitely many atoms of the random measures. We use completely random measures~(CRM) and hierarchical CRM to define a prior for Poisson processes. We derive the marginal distribution of the resultant point process, when the underlying CRM is marginalized out. Using well known properties unique to Poisson processes, we were able to derive an exact approach for instantiating a Poisson process with a hierarchical CRM prior. Furthermore, we derive Gibbs sampling strategies for hierarchical CRM models based on Chinese restaurant franchise sampling scheme. As an example, we present the sum of generalized gamma process (SGGP), and show its application in topic-modelling. We show that one can determine the power-law behaviour of the topics and words in a Bayesian fashion, by defining a prior on the parameters of SGGP.

math.ST↗
arXiv2016

Reduced Gröbner Bases and Macaulay-Buchberger Basis Theorem over Noetherian Rings

In this paper, we extend the characterization of $\mathbb{Z}[x]/\ < f \ >$, where $f \in \mathbb{Z}[x]$ to be a free $\mathbb{Z}$-module to multivariate polynomial rings over any commutative Noetherian ring, $A$. The characterization allows us to extend the Gröbner basis method of computing a $\Bbbk$-vector space basis of residue class polynomial rings over a field $\Bbbk$ (Macaulay-Buchberger Basis Theorem) to rings, i.e. $A[x_1,\ldots,x_n]/\mathfrak{a}$, where $\mathfrak{a} \subseteq A[x_1,\ldots,x_n]$ is an ideal. We give some insights into the characterization for two special cases, when $A = \mathbb{Z}$ and $A = \Bbbk[θ_1,\ldots,θ_m]$. As an application of this characterization, we show that the concept of border bases can be extended to rings when the corresponding residue class ring is a finitely generated, free $A$-module.

cs.SC↗
arXiv2016

Consistency of Spectral Hypergraph Partitioning under Planted Partition Model

Hypergraph partitioning lies at the heart of a number of problems in machine learning and network sciences. Many algorithms for hypergraph partitioning have been proposed that extend standard approaches for graph partitioning to the case of hypergraphs. However, theoretical aspects of such methods have seldom received attention in the literature as compared to the extensive studies on the guarantees of graph partitioning. For instance, consistency results of spectral graph partitioning under the stochastic block model are well known. In this paper, we present a planted partition model for sparse random non-uniform hypergraphs that generalizes the stochastic block model. We derive an error bound for a spectral hypergraph partitioning algorithm under this model using matrix concentration inequalities. To the best of our knowledge, this is the first consistency result related to partitioning non-uniform hypergraphs.

stat.ML↗
arXiv2015

Coloring Random Non-Uniform Bipartite Hypergraphs

Let $H_{n,(p_m)_{m=2,\ldots,M}}$ be a random non-uniform hypergraph of dimension $M$ on $2n$ vertices, where the vertices are split into two disjoint sets of size $n$, and colored by two distinct colors. Each non-monochromatic edge of size $m=2,\ldots,M$ is independently added with probability $p_m$. We show that if $p_2,\ldots,p_M$ are such that the expected number of edges in the hypergraph is at least $dn\ln n$, for some $d>0$ sufficiently large, then with probability $(1-o(1))$, one can find a proper 2-coloring of $H_{n,(p_m)_{m=2,\ldots,M}}$ in polynomial time. We present a polynomial time algorithm for hypergraph 2-coloring, and provide discussions on extension of the approach for $k$-coloring of non-uniform hypergraphs.

math.CO↗