Searcharxiv⌕ Search

arXiv subjects

Sourav Dutta

Publications and source records attributed to Sourav Dutta.

At least 55 records · Page 3Linked to original sources
arXiv2022

AX-MABSA: A Framework for Extremely Weakly Supervised Multi-label Aspect Based Sentiment Analysis

Aspect Based Sentiment Analysis is a dominant research area with potential applications in social media analytics, business, finance, and health. Prior works in this area are primarily based on supervised methods, with a few techniques using weak supervision limited to predicting a single aspect category per review sentence. In this paper, we present an extremely weakly supervised multi-label Aspect Category Sentiment Analysis framework which does not use any labelled data. We only rely on a single word per class as an initial indicative information. We further propose an automatic word selection technique to choose these seed categories and sentiment words. We explore unsupervised language model post-training to improve the overall performance, and propose a multi-label generator model to generate multiple aspect category-sentiment pairs per review sentence. Experiments conducted on four benchmark datasets showcase our method to outperform other weakly supervised baselines by a significant margin.

cs.CL↗
arXiv2022

High precision measurement of the hyperfine splitting and ac Stark shift of the $7d$ $^{2}D_{3/2}$ state in atomic cesium

We report the measurement of hyperfine splitting in the $7d$ $^{2}D_{3/2}$ state of $^{133}$Cs using high resolution Doppler-free two-photon spectroscopy in a Cs vapor cell. We determine the hyperfine coupling constants $A = 7.3509(9)$ MHz and $B = -0.041(8)$ MHz, which represent an order of magnitude improvement in the precision. We also obtain bounds on the magnitude of the nuclear magnetic octupole coupling constant $C$. Additionally, we measure the ac Stark shift of the $6s$ $^{2}S_{1/2} \rightarrow 7d$ $^{2}D_{3/2}$ transition at 767.8 nm to be $-49 \pm 5$ Hz/(W/cm$^2$), in agreement with theoretical calculations. We further report the measurement of collisional shift [$-32.6 \pm 2.0$ kHz/mTorr] and pressure broadening for the individual hyperfine levels of the $6s$ $^{2}S_{1/2} \rightarrow 7d$ $^{2}D_{3/2}$ transition. These measurements provide valuable inputs for analysis of systematic effects in optical frequency standards based on the cesium $6s$ $^{2}S_{1/2} \rightarrow 7d$ $^{2}D_{3/2}$ two-photon transition

physics.atom-ph↗
arXiv2022

Hyperfine coupling constants of the cesium $7D_{5/2}$ state measured up to the octupole term

We report the measurement of the hyperfine splitting in the $7D_{5/2}$ state of $^{133}$Cs using high resolution Doppler-free two-photon spectroscopy enabled by precise frequency scans using an acousto-optic modulator. All the six hyperfine levels are resolved in our spectra. We determine the hyperfine coupling constants A = -1.70867(62) MHz and B = 0.050(14) MHz which represent over 20-times improvement in the precision of both A and B. Moreover, our measurement is sufficiently precise to put bounds on the value of the magnetic octupole coupling constant C = 0.4(1.4) kHz for the $7D_{5/2}$ state. We additionally report the measurement of ac Stark shift [-46 $\pm$ 4 Hz/(W/cm$^2$)], collisional shift and pressure broadening which are important for optical frequency standards based on the $6S_{1/2} \rightarrow 7D_{5/2}$ two-photon transition.

physics.atom-ph↗
arXiv2022

ACO based Adaptive RBFN Control for Robot Manipulators

This paper describes a new approach for approximating the inverse kinematics of a manipulator using an Ant Colony Optimization (ACO) based RBFN (Radial Basis Function Network). In this paper, a training solution using the ACO and the LMS (Least Mean Square) algorithm is presented in a two-phase training procedure. To settle the problem that the cluster results of k-mean clustering Radial Basis Function (RBF) are easy to be influenced by the selection of initial characters and converge to a local minimum, Ant Colony Optimization (ACO) for the RBF neural networks which will optimize the center of RBF neural networks and reduce the number of the hidden layer neurons nodes is presented. The result demonstrates that the accuracy of Ant Colony Optimization for the Radial Basis Function (RBF) neural networks is higher, and the extent of fitting has been improved.

cs.RO↗
arXiv2022

Aligned Weight Regularizers for Pruning Pretrained Neural Networks

While various avenues of research have been explored for iterative pruning, little is known what effect pruning has on zero-shot test performance and its potential implications on the choice of pruning criteria. This pruning setup is particularly important for cross-lingual models that implicitly learn alignment between language representations during pretraining, which if distorted via pruning, not only leads to poorer performance on language data used for retraining but also on zero-shot languages that are evaluated. In this work, we show that there is a clear performance discrepancy in magnitude-based pruning when comparing standard supervised learning to the zero-shot setting. From this finding, we propose two weight regularizers that aim to maximize the alignment between units of pruned and unpruned networks to mitigate alignment distortion in pruned cross-lingual models and perform well for both non zero-shot and zero-shot settings. We provide experimental results on cross-lingual tasks for the zero-shot setting using XLM-RoBERTa$_{\mathrm{Base}}$, where we also find that pruning has varying degrees of representational degradation depending on the language corresponding to the zero-shot test set. This is also the first study that focuses on cross-lingual language model compression.

cs.CL↗
arXiv2021

Deep Neural Compression Via Concurrent Pruning and Self-Distillation

Pruning aims to reduce the number of parameters while maintaining performance close to the original network. This work proposes a novel \emph{self-distillation} based pruning strategy, whereby the representational similarity between the pruned and unpruned versions of the same network is maximized. Unlike previous approaches that treat distillation and pruning separately, we use distillation to inform the pruning criteria, without requiring a separate student network as in knowledge distillation. We show that the proposed {\em cross-correlation objective for self-distilled pruning} implicitly encourages sparse solutions, naturally complementing magnitude-based pruning criteria. Experiments on the GLUE and XGLUE benchmarks show that self-distilled pruning increases mono- and cross-lingual language model performance. Self-distilled pruned models also outperform smaller Transformers with an equal number of parameters and are competitive against (6 times) larger distilled networks. We also observe that self-distillation (1) maximizes class separability, (2) increases the signal-to-noise ratio, and (3) converges faster after pruning steps, providing further insights into why self-distilled pruning improves generalization.

cs.LG↗
arXiv2021

EdinSaar@WMT21: North-Germanic Low-Resource Multilingual NMT

We describe the EdinSaar submission to the shared task of Multilingual Low-Resource Translation for North Germanic Languages at the Sixth Conference on Machine Translation (WMT2021). We submit multilingual translation models for translations to/from Icelandic (is), Norwegian-Bokmal (nb), and Swedish (sv). We employ various experimental approaches, including multilingual pre-training, back-translation, fine-tuning, and ensembling. In most translation directions, our models outperform other submitted systems.

cs.CL↗
arXiv2021

Sequence-to-Sequence Learning on Keywords for Efficient FAQ Retrieval

Frequently-Asked-Question (FAQ) retrieval provides an effective procedure for responding to user's natural language based queries. Such platforms are becoming common in enterprise chatbots, product question answering, and preliminary technical support for customers. However, the challenge in such scenarios lies in bridging the lexical and semantic gap between varied query formulations and the corresponding answers, both of which typically have a very short span. This paper proposes TI-S2S, a novel learning framework combining TF-IDF based keyword extraction and Word2Vec embeddings for training a Sequence-to-Sequence (Seq2Seq) architecture. It achieves high precision for FAQ retrieval by better understanding the underlying intent of a user question captured via the representative keywords. We further propose a variant with an additional neural network module for guiding retrieval via relevant candidate identification based on similarity features. Experiments on publicly available dataset depict our approaches to provide around 92% precision-at-rank-5, exhibiting nearly 13% improvement over existing approaches.

cs.IR↗
arXiv2021

VerSaChI: Finding Statistically Significant Subgraph Matches using Chebyshev's Inequality

Approximate subgraph matching, which is an important primitive for many applications like question answering, community detection, and motif discovery, often involves large labeled graphs such as knowledge graphs, social networks, and protein sequences. Effective methods for extracting matching subgraphs, in terms of label and structural similarities to a query, should depict accuracy, computational efficiency, and robustness to noise. In this paper, we propose VerSaChI for finding the top-k most similar subgraphs based on 2-hop label and structural overlap similarity with the query. The similarity is characterized using Chebyshev's inequality to compute the chi-square statistical significance for measuring the degree of matching of the subgraphs. Experiments on real-life graph datasets showcase significant improvements in terms of accuracy compared to state-of-the-art methods, as well as robustness to noise.

cs.DB↗
arXiv2021

Data-driven reduced order modeling of environmental hydrodynamics using deep autoencoders and neural ODEs

Model reduction for fluid flow simulation continues to be of great interest across a number of scientific and engineering fields. In a previous work [arXiv:2104.13962], we explored the use of Neural Ordinary Differential Equations (NODE) as a non-intrusive method for propagating the latent-space dynamics in reduced order models. Here, we investigate employing deep autoencoders for discovering the reduced basis representation, the dynamics of which are then approximated by NODE. The ability of deep autoencoders to represent the latent-space is compared to the traditional proper orthogonal decomposition (POD) approach, again in conjunction with NODE for capturing the dynamics. Additionally, we compare their behavior with two classical non-intrusive methods based on POD and radial basis function interpolation as well as dynamic mode decomposition. The test problems we consider include incompressible flow around a cylinder as well as a real-world application of shallow water hydrodynamics in an estuarine system. Our findings indicate that deep autoencoders can leverage nonlinear manifold learning to achieve a highly efficient compression of spatial information and define a latent-space that appears to be more suitable for capturing the temporal dynamics through the NODE framework.

cs.LG↗
arXiv2021

Logic Compatible High-Performance Ferroelectric Transistor Memory

Silicon ferroelectric field-effect transistors (FeFETs) with low-k interfacial layer (IL) between ferroelectric gate stack and silicon channel suffers from high write voltage, limited write endurance and large read-after-write latency due to early IL breakdown and charge trapping and detrapping at the interface. We demonstrate low voltage, high speed memory operation with high write endurance using an IL-free back-end-of-line (BEOL) compatible FeFET. We fabricate IL-free FeFETs with 28nm channel length and 126nm width under a thermal budget <400C by integrating 5nm thick Hf0.5Zr0.5O2 gate stack with amorphous Indium Tungsten Oxide (IWO) semiconductor channel. We report 1.2V memory window and read current window of 10^5 for program and erase, write latency of 20ns with +/-2V write pulses, read-after-write latency <200ns, write endurance cycles exceeding 5x10^10 and 2-bit/cell programming capability. Array-level analysis establishes IL-free BEOL FeFET as a promising candidate for logic-compatible high-performance on-chip buffer memory and multi-bit weight cell for compute-in-memory accelerators.

cond-mat.mes-hall↗
arXiv2021

A greedy non-intrusive reduced order model for shallow water equations

In this work, we develop Non-Intrusive Reduced Order Models (NIROMs) that combine Proper Orthogonal Decomposition (POD) with a Radial Basis Function (RBF) interpolation method to construct efficient reduced order models for time-dependent problems arising in large scale environmental flow applications. The performance of the POD-RBF NIROM is compared with a traditional nonlinear POD (NPOD) model by evaluating the accuracy and robustness for test problems representative of riverine flows. Different greedy algorithms are studied in order to determine a near-optimal distribution of interpolation points for the RBF approximation. A new power-scaled residual greedy (psr-greedy) algorithm is proposed to address some of the primary drawbacks of the existing greedy approaches. The relative performances of these greedy algorithms are studied with numerical experiments using realistic two-dimensional (2D) shallow water flow applications involving coastal and riverine dynamics.

physics.comp-ph↗
arXiv2021

Neural Ordinary Differential Equations for Data-Driven Reduced Order Modeling of Environmental Hydrodynamics

Model reduction for fluid flow simulation continues to be of great interest across a number of scientific and engineering fields. Here, we explore the use of Neural Ordinary Differential Equations, a recently introduced family of continuous-depth, differentiable networks (Chen et al 2018), as a way to propagate latent-space dynamics in reduced order models. We compare their behavior with two classical non-intrusive methods based on proper orthogonal decomposition and radial basis function interpolation as well as dynamic mode decomposition. The test problems we consider include incompressible flow around a cylinder as well as real-world applications of shallow water hydrodynamics in riverine and estuarine systems. Our findings indicate that Neural ODEs provide an elegant framework for stable and accurate evolution of latent-space dynamics with a promising potential of extrapolatory predictions. However, in order to facilitate their widespread adoption for large-scale systems, significant effort needs to be directed at accelerating their training times. This will enable a more comprehensive exploration of the hyperparameter space for building generalizable Neural ODE approximations over a wide range of system dynamics.

cs.LG↗
arXiv2021

Quantum cosmology with symmetry analysis for quintom dark energy model

Quantum cosmology with quintom dark energy model has been investigated in the present work using symmetry analysis of the underlying physical system. In the background of the flat FLRW model quintom cosmological model has been studied using Noether symmetry and appropriate conserved charge is obtained. The Wheeler-DeWitt (WD) equation is constructed on the minisuperspace and solutions are obtained using conserved charge.

physics.gen-ph↗
arXiv2021

An Ising Hamiltonian Solver using Stochastic Phase-Transition Nano- Oscillators

Computationally hard problems, including combinatorial optimization, can be mapped into the problem of finding the ground-state of an Ising Hamiltonian. Building physical systems with collective computational ability and distributed parallel processing capability can accelerate the ground-state search. Here, we present a continuous-time dynamical system (CTDS) approach where the ground-state solution appears as stable points or attractor states of the CTDS. We harness the emergent dynamics of a network of phase-transition nano-oscillators (PTNO) to build an Ising Hamiltonian solver. The hardware fabric comprises of electrically coupled injection-locked stochastic PTNOs with bi-stable phases emulating artificial Ising spins. We demonstrate the ability of the stochastic PTNO-CTDS to progressively find more optimal solution by increasing the strength of the injection-locking signal - akin to performing classical annealing. We demonstrate in silico that the PTNO-CTDS prototype solves a benchmark non-deterministic polynomial time (NP)-hard Max-Cut problem with high probability of success. Using experimentally calibrated numerical simulations and incorporating non-idealities, we investigate the performance of our Ising Hamiltonian solver on dense Max-Cut problems with increasing graph size. We report a high energy-efficiency of 1.3x10^7 solutions/sec/Watt for 100-node dense Max-cut problems which translates to a 5x improvement over the recently demonstrated memristor-based Hopfield network and several orders of magnitude improvement over other candidates such as CPU and GPU, quantum annealer and photonic Ising solver approaches. Such an energy efficient hardware exhibiting high solution-throughput/Watt can find applications in industrial planning and manufacturing, defense and cyber-security, bioinformatics and drug discovery.

cond-mat.mes-hall↗
arXiv2021

Neural Sampling Machine with Stochastic Synapse allows Brain-like Learning and Inference

Many real-world mission-critical applications require continual online learning from noisy data and real-time decision making with a defined confidence level. Probabilistic models and stochastic neural networks can explicitly handle uncertainty in data and allow adaptive learning-on-the-fly, but their implementation in a low-power substrate remains a challenge. Here, we introduce a novel hardware fabric that implements a new class of stochastic NN called Neural-Sampling-Machine that exploits stochasticity in synaptic connections for approximate Bayesian inference. Harnessing the inherent non-linearities and stochasticity occurring at the atomic level in emerging materials and devices allows us to capture the synaptic stochasticity occurring at the molecular level in biological synapses. We experimentally demonstrate in-silico hybrid stochastic synapse by pairing a ferroelectric field-effect transistor -based analog weight cell with a two-terminal stochastic selector element. Such a stochastic synapse can be integrated within the well-established crossbar array architecture for compute-in-memory. We experimentally show that the inherent stochastic switching of the selector element between the insulator and metallic state introduces a multiplicative stochastic noise within the synapses of NSM that samples the conductance states of the FeFET, both during learning and inference. We perform network-level simulations to highlight the salient automatic weight normalization feature introduced by the stochastic synapses of the NSM that paves the way for continual online learning without any offline Batch Normalization. We also showcase the Bayesian inferencing capability introduced by the stochastic synapse during inference mode, thus accounting for uncertainty in data. We report 98.25%accuracy on standard image classification task as well as estimation of data uncertainty in rotated samples.

cond-mat.dis-nn↗
arXiv2020

Unsupervised Word Translation Pairing using Refinement based Point Set Registration

Cross-lingual alignment of word embeddings play an important role in knowledge transfer across languages, for improving machine translation and other multi-lingual applications. Current unsupervised approaches rely on similarities in geometric structure of word embedding spaces across languages, to learn structure-preserving linear transformations using adversarial networks and refinement strategies. However, such techniques, in practice, tend to suffer from instability and convergence issues, requiring tedious fine-tuning for precise parameter setting. This paper proposes BioSpere, a novel framework for unsupervised mapping of bi-lingual word embeddings onto a shared vector space, by combining adversarial initialization and refinement procedure with point set registration algorithm used in image processing. We show that our framework alleviates the shortcomings of existing methodologies, and is relatively invariant to variable adversarial learning performance, depicting robustness in terms of parameter choices and training losses. Experimental evaluation on parallel dictionary induction task demonstrates state-of-the-art results for our framework on diverse language pairs.

cs.CL↗