arXiv subjects

Rahul Gupta

Publications and source records attributed to Rahul Gupta.

194 records · Page 11Linked to original sources

arXiv2018

A Re-ranker Scheme for Integrating Large Scale NLU models

Large scale Natural Language Understanding (NLU) systems are typically trained on large quantities of data, requiring a fast and scalable training strategy. A typical design for NLU systems consists of domain-level NLU modules (domain classifier, intent classifier and named entity recognizer). Hypotheses (NLU interpretations consisting of various intent+slot combinations) from these domain specific modules are typically aggregated with another downstream component. The re-ranker integrates outputs from domain-level recognizers, returning a scored list of cross domain hypotheses. An ideal re-ranker will exhibit the following two properties: (a) it should prefer the most relevant hypothesis for the given input as the top hypothesis and, (b) the interpretation scores corresponding to each hypothesis produced by the re-ranker should be calibrated. Calibration allows the final NLU interpretation score to be comparable across domains. We propose a novel re-ranker strategy that addresses these aspects, while also maintaining domain specific modularity. We design optimization loss functions for such a modularized re-ranker and present results on decreasing the top hypothesis error rate as well as maintaining the model calibration. We also experiment with an extension involving training the domain specific re-rankers on datasets curated independently by each domain to allow further asynchronization. %The proposed re-ranker design showcases the following: (i) improved NLU performance over an unweighted aggregation strategy, (ii) cross-domain calibrated performance and, (iii) support for use cases involving training each re-ranker on datasets curated by each domain independently.

cs.CL↗

arXiv2018

On Enhancing Speech Emotion Recognition using Generative Adversarial Networks

Generative Adversarial Networks (GANs) have gained a lot of attention from machine learning community due to their ability to learn and mimic an input data distribution. GANs consist of a discriminator and a generator working in tandem playing a min-max game to learn a target underlying data distribution; when fed with data-points sampled from a simpler distribution (like uniform or Gaussian distribution). Once trained, they allow synthetic generation of examples sampled from the target distribution. We investigate the application of GANs to generate synthetic feature vectors used for speech emotion recognition. Specifically, we investigate two set ups: (i) a vanilla GAN that learns the distribution of a lower dimensional representation of the actual higher dimensional feature vector and, (ii) a conditional GAN that learns the distribution of the higher dimensional feature vectors conditioned on the labels or the emotional class to which it belongs. As a potential practical application of these synthetically generated samples, we measure any improvement in a classifier's performance when the synthetic data is used along with real data for training. We perform cross-validation analyses followed by a cross-corpus study.

cs.CL↗

arXiv2018

Semi-supervised and Transfer learning approaches for low resource sentiment classification

Sentiment classification involves quantifying the affective reaction of a human to a document, media item or an event. Although researchers have investigated several methods to reliably infer sentiment from lexical, speech and body language cues, training a model with a small set of labeled datasets is still a challenge. For instance, in expanding sentiment analysis to new languages and cultures, it may not always be possible to obtain comprehensive labeled datasets. In this paper, we investigate the application of semi-supervised and transfer learning methods to improve performances on low resource sentiment classification tasks. We experiment with extracting dense feature representations, pre-training and manifold regularization in enhancing the performance of sentiment classification systems. Our goal is a coherent implementation of these methods and we evaluate the gains achieved by these methods in matched setting involving training and testing on a single corpus setting as well as two cross corpora settings. In both the cases, our experiments demonstrate that the proposed methods can significantly enhance the model performance against a purely supervised approach, particularly in cases involving a handful of training data.

cs.IR↗

arXiv2018

Adversarial Auto-encoders for Speech Based Emotion Recognition

Recently, generative adversarial networks and adversarial autoencoders have gained a lot of attention in machine learning community due to their exceptional performance in tasks such as digit classification and face recognition. They map the autoencoder's bottleneck layer output (termed as code vectors) to different noise Probability Distribution Functions (PDFs), that can be further regularized to cluster based on class information. In addition, they also allow a generation of synthetic samples by sampling the code vectors from the mapped PDFs. Inspired by these properties, we investigate the application of adversarial autoencoders to the domain of emotion recognition. Specifically, we conduct experiments on the following two aspects: (i) their ability to encode high dimensional feature vector representations for emotional utterances into a compressed space (with a minimal loss of emotion class discriminability in the compressed space), and (ii) their ability to regenerate synthetic samples in the original feature space, to be later used for purposes such as training emotion recognition classifiers. We demonstrate the promise of adversarial autoencoders with regards to these aspects on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) corpus and present our analysis.

stat.ML↗

arXiv2018

An ADMM-based Coordination and Control Strategy for PV and Storage to Dispatch Stochastic Prosumers: Theory and Experimental Validation

This paper describes a two-layer control and coordination framework for distributed energy resources. The lower layer is a real-time model predictive control (MPC) executed at 10 s resolution to achieve fine tuning of a given energy set-point. The upper layer is a slower MPC coordination mechanism based on distributed optimization, and solved with the alternating direction method of multipliers (ADMM) at 5 minutes resolution. It is needed to coordinate the power flow among the controllable resources such that enough power is available in real-time to achieve a pre-established energy trajectory in the long term. Although the formulation is generic, it is developed for the case of a battery system and a curtailable PV facility to dispatch stochastic prosumption according to a trajectory at 5 minutes resolution established the day before the operation. The proposed method is experimentally validated in a real-life setup to dispatch the operation of a building with rooftop PV generation (i.e., 101 kW average load, 350 kW peak demand, 82 kW peak PV generation) by controlling a 560 kWh/720 kVA battery and a 13 kW peak curtailable PV facility.

eess.SY↗

arXiv2018

Deep Reinforcement Learning for Programming Language Correction

Novice programmers often struggle with the formal syntax of programming languages. To assist them, we design a novel programming language correction framework amenable to reinforcement learning. The framework allows an agent to mimic human actions for text navigation and editing. We demonstrate that the agent can be trained through self-exploration directly from the raw input, that is, program text itself, without any knowledge of the formal syntax of the programming language. We leverage expert demonstrations for one tenth of the training data to accelerate training. The proposed technique is evaluated on 6975 erroneous C programs with typographic errors, written by students during an introductory programming course. Our technique fixes 14% more programs and 29% more compiler error messages relative to those fixed by a state-of-the-art tool, DeepFix, which uses a fully supervised neural machine translation approach.

cs.AI↗

arXiv2017

SLING: A framework for frame semantic parsing

We describe SLING, a framework for parsing natural language into semantic frames. SLING supports general transition-based, neural-network parsing with bidirectional LSTM input encoding and a Transition Based Recurrent Unit (TBRU) for output decoding. The parsing model is trained end-to-end using only the text tokens as input. The transition system has been designed to output frame graphs directly without any intervening symbolic representation. The SLING framework includes an efficient and scalable frame store implementation as well as a neural network JIT compiler for fast inference during parsing. SLING is implemented in C++ and it is available for download on GitHub.

cs.CL↗

arXiv2017

A Partial Order Reduction Technique for Event-driven Multi-threaded Programs

Event-driven multi-threaded programming is fast becoming a preferred style of developing efficient and responsive applications. In this concurrency model, multiple threads execute concurrently, communicating through shared objects as well as by posting asynchronous events that are executed in their order of arrival. In this work, we consider partial order reduction (POR) for event-driven multi-threaded programs. The existing POR techniques treat event queues associated with threads as shared objects and thereby, reorder every pair of events handled on the same thread even if reordering them does not lead to different states. We do not treat event queues as shared objects and propose a new POR technique based on a novel backtracking set called the dependence-covering set. Events handled by the same thread are reordered by our POR technique only if necessary. We prove that exploring dependence-covering sets suffices to detect all deadlock cycles and assertion violations defined over local variables. To evaluate effectiveness of our POR scheme, we have implemented a dynamic algorithm to compute dependence-covering sets. On execution traces obtained from a few Android applications, we demonstrate that our technique explores many fewer transitions ---often orders of magnitude fewer--- compared to exploration based on persistent sets, wherein, event queues are considered as shared objects.

cs.PL↗

arXiv2016

Inferring object rankings based on noisy pairwise comparisons from multiple annotators

Ranking a set of objects involves establishing an order allowing for comparisons between any pair of objects in the set. Oftentimes, due to the unavailability of a ground truth of ranked orders, researchers resort to obtaining judgments from multiple annotators followed by inferring the ground truth based on the collective knowledge of the crowd. However, the aggregation is often ad-hoc and involves imposing stringent assumptions in inferring the ground truth (e.g. majority vote). In this work, we propose Expectation-Maximization (EM) based algorithms that rely on the judgments from multiple annotators and the object attributes for inferring the latent ground truth. The algorithm learns the relation between the latent ground truth and object attributes as well as annotator specific probabilities of flipping, a metric to assess annotator quality. We further extend the EM algorithm to allow for a variable probability of flipping based on the pair of objects at hand. We test our algorithms on two data sets with synthetic annotations and investigate the impact of annotator quality and quantity on the inferred ground truth. We also obtain the results on two other data sets with annotations from machine/human annotators and interpret the output trends based on the data characteristics.

stat.ML↗

arXiv2014

A Novel Method to Calculate Click Through Rate for Sponsored Search

Sponsored search adopts generalized second price (GSP) auction mechanism which works on the concept of pay per click which is most commonly used for the allocation of slots in the searched page. Two main aspects associated with GSP are the bidding amount and the click through rate (CTR). The CTR learning algorithms currently being used works on the basic principle of (#clicks_i/ #impressions_i) under a fixed window of clicks or impressions or time. CTR are prone to fraudulent clicks, resulting in sudden increase of CTR. The current algorithms are unable to find the solutions to stop this, although with the use of machine learning algorithms it can be detected that fraudulent clicks are being generated. In our paper, we have used the concept of relative ranking which works on the basic principle of (#clicks_i /#clicks_t). In this algorithm, both the numerator and the denominator are linked. As #clicks_t is higher than previous algorithms and is linked to the #clicks_i, the small change in the clicks which occurs in the normal scenario have a very small change in the result but in case of fraudulent clicks the number of clicks increases or decreases rapidly which will add up with the normal clicks to increase the denominator, thereby decreasing the CTR.

cs.IR↗

arXiv2014

A Moments' Analysis of Quasi-Exactly Solvable Systems: A New Perspective on the Sextic Anharmonic and Bender-Dunne Potentials

There continues to be great interest in understanding quasi-exactly solvable (QES) systems. In one dimension, QES states assume the form $Ψ(x) =x^γP_d(x) {\cal A}(x)$, where ${\cal A}(x) > 0$ is known in closed form, and $P_d(x)$ is a polynomial to be determined. That is ${{Ψ(x)}\over {x^γ{\cal A}(x)}} = \sum_{n=0}^\infty a_nx^n$ truncates. The extension of this "truncation" procedure to non-QES states corresponds to the Hill determinant method, which is unstable when the {\it reference} function assumes the physical asymptotic form. Recently, Handy and Vrinceanu introduced the Orthogonal Polynomial Projection Quantization (OPPQ) method which has non of these problems, allowing for a unified analysis of QES and non-QES states. OPPQ uses a non-orthogonal basis constructed from the orthonormal polynomials of ${\cal A}$: $Ψ(x) = \sum_{j=0}^\infty Ω_j {\cal P}^{(j)}(x) {\cal A}(x)$, where $\langle {\cal P}^{(j_1)}|{\cal A}|{\cal P}^{(j_2)} \rangle = δ_{j_1,j_2}$, and $Ω_j = \langle {\cal P}^{(j)}|Ψ\rangle$. For systems admitting a moment equation representation, such as those considered here, these coefficients can be readily determined. The OPPQ quantization condition, $Ω_{j} = 0$, is exact for QES states (provided $j \geq d+1$); and is computationally stable, and exponentially convergent, for non-QES states. OPPQ provides an alternate explanation to the Bender-Dunne (BD) orthogonal polynomial formalism for identifying QES states: they correlate with an anomalous kink behavior in the order of the finite difference moment equation associated with the $Φ= x^γ{\cal A}(x) Ψ(x)$ {\it Bessis}-representation (i.e. a spontaneous change in the degrees of freedom of the system).

math-ph↗

arXiv2012

Time Temperature Superposition in Soft Glassy Materials

Soft glassy materials are out of thermodynamic equilibrium and show time dependent slowing down of the relaxation dynamics. Under such situation these materials follow Boltzmann superposition principle only in the effective time domain, wherein time dependent relaxation processes are scaled by a constant relaxation time. In this work we extend effective time framework to successfully demonstrate time - temperature superposition of creep and stress relaxation data of a model soft glassy system comprised of clay suspension. Such superposition is possible when average relaxation time of the material changes with time and temperature without affecting shape of the spectrum. We show that variation in relaxation time as a function of temperature facilitates prediction of long and short time rheological behavior through time - temperature superposition from the experiments carried out over experimentally accessible timescales.

cond-mat.soft↗

arXiv2010

Joint Structured Models for Extraction from Overlapping Sources

We consider the problem of jointly training structured models for extraction from sources whose instances enjoy partial overlap. This has important applications like user-driven ad-hoc information extraction on the web. Such applications present new challenges in terms of the number of sources and their arbitrary pattern of overlap not seen by earlier collective training schemes applied on two sources. We present an agreement-based learning framework and alternatives within it to trade-off tractability, robustness to noise, and extent of agreement. We provide a principled scheme to discover low-noise agreement sets in unlabeled data across the sources. Through extensive experiments over 58 real datasets, we establish that our method of additively rewarding agreement over maximal segments of text provides the best trade-offs, and also scores over alternatives such as collective inference, staged training, and multi-view learning.

cs.AI↗

arXiv2009

Generalized Collective Inference with Symmetric Clique Potentials

Collective graphical models exploit inter-instance associative dependence to output more accurate labelings. However existing models support very limited kind of associativity which restricts accuracy gains. This paper makes two major contributions. First, we propose a general collective inference framework that biases data instances to agree on a set of {\em properties} of their labelings. Agreement is encouraged through symmetric clique potentials. We show that rich properties leads to bigger gains, and present a systematic inference procedure for a large class of such properties. The procedure performs message passing on the cluster graph, where property-aware messages are computed with cluster specific algorithms. This provides an inference-only solution for domain adaptation. Our experiments on bibliographic information extraction illustrate significant test error reduction over unseen domains. Our second major contribution consists of algorithms for computing outgoing messages from clique clusters with symmetric clique potentials. Our algorithms are exact for arbitrary symmetric potentials on binary labels and for max-like and majority-like potentials on multiple labels. For majority potentials, we also provide an efficient Lagrangian Relaxation based algorithm that compares favorably with the exact algorithm. We present a 13/15-approximation algorithm for the NP-hard Potts potential, with runtime sub-quadratic in the clique size. In contrast, the best known previous guarantee for graphs with Potts potentials is only 1/2. We empirically show that our method for Potts potentials is an order of magnitude faster than the best alternatives, and our Lagrangian Relaxation based algorithm for majority potentials beats the best applicable heuristic -- ICM.

cs.AI↗