arXiv subjects

Mayank Mishra

Publications and source records attributed to Mayank Mishra.

50 records · Page 3Linked to original sources

arXiv2023

Joint Reasoning on Hybrid-knowledge sources for Task-Oriented Dialog

Traditional systems designed for task oriented dialog utilize knowledge present only in structured knowledge sources to generate responses. However, relevant information required to generate responses may also reside in unstructured sources, such as documents. Recent state of the art models such as HyKnow and SeKnow aimed at overcoming these challenges make limiting assumptions about the knowledge sources. For instance, these systems assume that certain types of information, such as a phone number, is always present in a structured knowledge base (KB) while information about aspects such as entrance ticket prices, would always be available in documents. In this paper, we create a modified version of the MutliWOZ-based dataset prepared by SeKnow to demonstrate how current methods have significant degradation in performance when strict assumptions about the source of information are removed. Then, in line with recent work exploiting pre-trained language models, we fine-tune a BART based model using prompts for the tasks of querying knowledge sources, as well as, for response generation, without making assumptions about the information present in each knowledge source. Through a series of experiments, we demonstrate that our model is robust to perturbations to knowledge modality (source of information), and that it can fuse information from structured as well as unstructured knowledge to generate responses.

cs.CL↗

arXiv2022

Escaping Saddle Points for Effective Generalization on Class-Imbalanced Data

Real-world datasets exhibit imbalances of varying types and degrees. Several techniques based on re-weighting and margin adjustment of loss are often used to enhance the performance of neural networks, particularly on minority classes. In this work, we analyze the class-imbalanced learning problem by examining the loss landscape of neural networks trained with re-weighting and margin-based techniques. Specifically, we examine the spectral density of Hessian of class-wise loss, through which we observe that the network weights converge to a saddle point in the loss landscapes of minority classes. Following this observation, we also find that optimization methods designed to escape from saddle points can be effectively used to improve generalization on minority classes. We further theoretically and empirically demonstrate that Sharpness-Aware Minimization (SAM), a recent technique that encourages convergence to a flat minima, can be effectively used to escape saddle points for minority classes. Using SAM results in a 6.2\% increase in accuracy on the minority classes over the state-of-the-art Vector Scaling Loss, leading to an overall average increase of 4\% across imbalanced datasets. The code is available at: https://github.com/val-iisc/Saddle-LongTail.

cs.LG↗

arXiv2022

Demonstration of Entanglement-Enhanced Covert Sensing

The laws of quantum physics endow superior performance and security for information processing: quantum sensing harnesses nonclassical resources to enable measurement precision unmatched by classical sensing, whereas quantum cryptography aims to unconditionally protect the secrecy of the processed information. Here, we present the theory and experiment for entanglement-enhanced covert sensing, a paradigm that simultaneously offers high measurement precision and data integrity by concealing the probe signal in an ambient noise background so that the execution of the protocol is undetectable with a high probability. We show that entanglement offers a performance boost in estimating the imparted phase by a probed object, as compared to a classical protocol at the same covertness level. The implemented entanglement-enhanced covert sensing protocol operates close to the fundamental quantum limit by virtue of its near-optimum entanglement source and quantum receiver. Our work is expected to create ample opportunities for quantum information processing at unprecedented security and performance levels.

quant-ph↗

arXiv2022

A Closer Look at Smoothness in Domain Adversarial Training

Domain adversarial training has been ubiquitous for achieving invariant representations and is used widely for various domain adaptation tasks. In recent times, methods converging to smooth optima have shown improved generalization for supervised learning tasks like classification. In this work, we analyze the effect of smoothness enhancing formulations on domain adversarial training, the objective of which is a combination of task loss (eg. classification, regression, etc.) and adversarial terms. We find that converging to a smooth minima with respect to (w.r.t.) task loss stabilizes the adversarial training leading to better performance on target domain. In contrast to task loss, our analysis shows that converging to smooth minima w.r.t. adversarial loss leads to sub-optimal generalization on the target domain. Based on the analysis, we introduce the Smooth Domain Adversarial Training (SDAT) procedure, which effectively enhances the performance of existing domain adversarial methods for both classification and object detection tasks. Our analysis also provides insight into the extensive usage of SGD over Adam in the community for domain adversarial training.

cs.LG↗

arXiv2022

Variational Learning for Unsupervised Knowledge Grounded Dialogs

Recent methods for knowledge grounded dialogs generate responses by incorporating information from an external textual document. These methods do not require the exact document to be known during training and rely on the use of a retrieval system to fetch relevant documents from a large index. The documents used to generate the responses are modeled as latent variables whose prior probabilities need to be estimated. Models such as RAG and REALM, marginalize the document probabilities over the documents retrieved from the index to define the log likelihood loss function which is optimized end-to-end. In this paper, we develop a variational approach to the above technique wherein, we instead maximize the Evidence Lower bound (ELBO). Using a collection of three publicly available open-conversation datasets, we demonstrate how the posterior distribution, that has information from the ground-truth response, allows for a better approximation of the objective function during training. To overcome the challenges associated with sampling over a large knowledge collection, we develop an efficient approach to approximate the ELBO. To the best of our knowledge we are the first to apply variational training for open-scale unsupervised knowledge grounded dialog systems.

cs.CL↗

arXiv2021

Accelerating Gradient-based Meta Learner

Meta Learning has been in focus in recent years due to the meta-learner model's ability to adapt well and generalize to new tasks, thus, reducing both the time and data requirements for learning. However, a major drawback of meta learner is that, to reach to a state from where learning new tasks becomes feasible with less data, it requires a large number of iterations and a lot of time. We address this issue by proposing various acceleration techniques to speed up meta learning algorithms such as MAML (Model Agnostic Meta Learning). We present 3.73X acceleration on a well known RNN optimizer based meta learner proposed in literature [11]. We introduce a novel method of training tasks in clusters, which not only accelerates the meta learning process but also improves model accuracy performance. Keywords: Meta learning, RNN optimizer, AGI, Performance optimization

cs.LG↗

arXiv2021

Lambda Numbers of Finite $p$-Groups

An $L(2,1)$-labelling of a finite graph $Γ$ is a function that assigns integer values to the vertices $V(Γ)$ of $Γ$ (colouring of $V(Γ)$ by ${\mathbb{Z}}$) so that the absolute difference of two such values is at least $2$ for adjacent vertices and is at least $1$ for vertices which are precisely distance $2$ apart. The lambda number $λ(Γ)$ of $Γ$ measures the least number of integers needed for such a labelling (colouring). A power graph $Γ_G$ of a finite group $G$ is a graph with vertex set as the elements of $G$ and two vertices are joined by an edge if and only if one of them is a positive integer power of the other. It is known that $λ(Γ_G) \geq |G|$ for any finite group. In this paper we show that if $G$ is a finite group of a prime power order, then $λ(Γ_G) = |G|$ if and only if $G$ is neither cyclic nor a generalized quaternion $2$-group. This settles a partial classification of finite groups achieving the lower bound of lambda number.

math.GR↗

arXiv2020

Increasing distillable key rate from bound entangled states by using local filtration

We show the enhancement of distillable key rate for quantum key distribution(QKD), by local filtering, for several bound entangled states. Through our work it becomes evident that the local filtration operations, while transforming one bound entangled state to another, have the potential to increase the utility of the new state for QKD. We demonstrate three examples of `one way distillable key rate' enhancement by local filtering and in this process, discover new bound entangled states which are key distillable.

quant-ph↗

arXiv2019

Adversarial Approximate Inference for Speech to Electroglottograph Conversion

Speech produced by human vocal apparatus conveys substantial non-semantic information including the gender of the speaker, voice quality, affective state, abnormalities in the vocal apparatus etc. Such information is attributed to the properties of the voice source signal, which is usually estimated from the speech signal. However, most of the source estimation techniques depend heavily on the goodness of the model assumptions and are prone to noise. A popular alternative is to indirectly obtain the source information through the Electroglottographic (EGG) signal that measures the electrical admittance around the vocal folds using dedicated hardware. In this paper, we address the problem of estimating the EGG signal directly from the speech signal, devoid of any hardware. Sampling from the intractable conditional distribution of the EGG signal given the speech signal is accomplished through optimization of an evidence lower bound. This is constructed via minimization of the KL-divergence between the true and the approximated posteriors of a latent variable learned using a deep neural auto-encoder that serves an informative prior. We demonstrate the efficacy of the method at generating the EGG signal by conducting several experiments on datasets comprising multiple speakers, voice qualities, noise settings and speech pathologies. The proposed method is evaluated on many benchmark metrics and is found to agree with the gold standard while proving better than the state-of-the-art algorithms on a few tasks such as epoch extraction.

eess.AS↗

arXiv2019

Variational Inference with Latent Space Quantization for Adversarial Resilience

Despite their tremendous success in modelling high-dimensional data manifolds, deep neural networks suffer from the threat of adversarial attacks - Existence of perceptually valid input-like samples obtained through careful perturbation that lead to degradation in the performance of the underlying model. Major concerns with existing defense mechanisms include non-generalizability across different attacks, models and large inference time. In this paper, we propose a generalized defense mechanism capitalizing on the expressive power of regularized latent space based generative models. We design an adversarial filter, devoid of access to classifier and adversaries, which makes it usable in tandem with any classifier. The basic idea is to learn a Lipschitz constrained mapping from the data manifold, incorporating adversarial perturbations, to a quantized latent space and re-map it to the true data manifold. Specifically, we simultaneously auto-encode the data manifold and its perturbations implicitly through the perturbations of the regularized and quantized generative latent space, realized using variational inference. We demonstrate the efficacy of the proposed formulation in providing resilience against multiple attack types (black and white box) and methods, while being almost real-time. Our experiments show that the proposed method surpasses the state-of-the-art techniques in several cases.

cs.CV↗

arXiv2017

On-Disk Data Processing: Issues and Future Directions

In this paper, we present a survey of "on-disk" data processing (ODDP). ODDP, which is a form of near-data processing, refers to the computing arrangement where the secondary storage drives have the data processing capability. Proposed ODDP schemes vary widely in terms of the data processing capability, target applications, architecture and the kind of storage drive employed. Some ODDP schemes provide only a specific but heavily used operation like sort whereas some provide a full range of operations. Recently, with the advent of Solid State Drives, powerful and extensive ODDP solutions have been proposed. In this paper, we present a thorough review of architectures developed for different on-disk processing approaches along with current and future challenges and also identify the future directions which ODDP can take.

cs.DC↗

arXiv2015

Population inversion in two-level systems possessing permanent dipoles

Bare-state population inversion is demonstrated in a two-level system with all dipole matrix elements nonzero. A laser field is resonantly driving the sample whereas a second weaker and lower frequency coherent field additionally pumps it near resonance with the dynamically-Stark-splitted states. Due to existence of differing permanent dipole moments in the excited and ground bare states, quantum coherences among the involved dressed-states are induced leading to inversion in the steady-state. Furthermore, large refractive indices are feasible as well as the determination of the diagonal matrix elements via the absorption or emission spectra. The results apply to available biomolecular, spin or asymmetric quantum dot systems.

quant-ph↗

arXiv2015

De-Fragmenting the Cloud

Existing VM placement schemes have measured their effectiveness solely by looking either Physical Machine's resources(CPU, memory) or network resource. However, real applications use all resource types to varying degrees. The result of applying existing placement schemes to VMs running real applications is a fragmented data center where resources along one dimension become unusable even though they are available because of the unavailability of resources along other dimensions. An example of this fragmentation is unusable CPU because of a bottlenecked network link from the physical machine which has available CPU. To date, evaluations of the efficacy of VM placement schemes has not recognized this fragmentation and it's ill effects, let alone try to measure it and avoid it. In this paper, we first define the notion of what we term "relative resource fragmentation" and illustrate how it can be measured in a data center. The metric we put forth for capturing the degree of fragmentation is comprehensive and includes all key data center resource types. We then propose a scheme of minimizing this fragmentation so as to maximize the availability of existing set of data center resources. Results of empirical evaluations of our placement scheme compared to existing network based placement schemes show a reduction of fragmentation by as much as 15% and increase in number of successfully placed applications by upto 20%.

cs.DC↗

arXiv2011

Non-Standard Probabilistic Teleportation through Conventionally Non-Teleporting Channels

A non-standard teleportation scheme is proposed, wherein probabilistic teleportation is achieved in conventionally non-teleporting channels. We make use of entanglement monogamy to incorporate an unknown state in a multipartite entangled channel, such that the receiver partially gets disentangled from the network. Subsequently, the sender performs local measurement based teleportation protocol in an appropriate measurement basis, which results with the receiver in the possession of an unknown state, connected by local unitary transformation with the state to be teleported. This procedure succeeds in a number of cases, like that of W and other non-maximally entangled four qubit states, where the conventional measurement based approach has failed. It is also found that in certain four particle channels, the present procedure does not succeed, although the conventional one works well.

quant-ph↗