arXiv subjects

Abhishek Jha

Publications and source records attributed to Abhishek Jha.

27 records · Page 2Linked to original sources

arXiv2022

Barlow constrained optimization for Visual Question Answering

Visual question answering is a vision-and-language multimodal task, that aims at predicting answers given samples from the question and image modalities. Most recent methods focus on learning a good joint embedding space of images and questions, either by improving the interaction between these two modalities, or by making it a more discriminant space. However, how informative this joint space is, has not been well explored. In this paper, we propose a novel regularization for VQA models, Constrained Optimization using Barlow's theory (COB), that improves the information content of the joint space by minimizing the redundancy. It reduces the correlation between the learned feature components and thereby disentangles semantic concepts. Our model also aligns the joint space with the answer embedding space, where we consider the answer and image+question as two different `views' of what in essence is the same semantic information. We propose a constrained optimization policy to balance the categorical and redundancy minimization forces. When built on the state-of-the-art GGE model, the resulting model improves VQA accuracy by 1.4% and 4% on the VQA-CP v2 and VQA v2 datasets respectively. The model also exhibits better interpretability.

cs.CV↗

arXiv2021

On the Least Common Multiple of Polynomial Sequences at Prime Arguments

Cilleruelo conjectured that if $f\in\mathbb{Z}[x]$ is an irreducible polynomial of degree $d\ge 2$ then, $\log \operatorname{lcm} \{f(n)\mid n<x\} \sim (d-1)x\log x.$ In this article, we investigate the analogue of prime arguments, namely, $\operatorname{lcm} \{f(p)\mid p<x\}$ where $p$ denotes a prime and obtain non-trivial lower bounds on it. Further, we also show some results regarding the greatest prime divisor of $f(p).$

math.NT↗

arXiv2021

Exploring Low-Cost Transformer Model Compression for Large-Scale Commercial Reply Suggestions

Fine-tuning pre-trained language models improves the quality of commercial reply suggestion systems, but at the cost of unsustainable training times. Popular training time reduction approaches are resource intensive, thus we explore low-cost model compression techniques like Layer Dropping and Layer Freezing. We demonstrate the efficacy of these techniques in large-data scenarios, enabling the training time reduction for a commercial email reply suggestion system by 42%, without affecting the model relevance or user engagement. We further study the robustness of these techniques to pre-trained model and dataset size ablation, and share several insights and recommendations for commercial applications.

cs.CL↗

arXiv2021

On Quotients of Values of Euler's Function on Factorials

Recently, there has been some interest in values of arithmetical functions on members of special sequences, such as Euler's totient function $ϕ$ on factorials, linear recurrences, etc. In this article, we investigate, for given positive integers $a$ and $b$, the least positive integer $c=c(a,b)$ such that the quotient $ϕ(c!)/ϕ(a!)ϕ(b!)$ is an integer. We derive results on the limit of the ratio $c(a,b)/(a+b)$ as $a$ and $b$ tend to infinity. Furthermore, we show that $c(a,b)>a+b$ for all pairs of positive integers $(a,b)$ with an exception of a set of density zero.

math.NT↗

arXiv2021

Glimpse-Attend-and-Explore: Self-Attention for Active Visual Exploration

Active visual exploration aims to assist an agent with a limited field of view to understand its environment based on partial observations made by choosing the best viewing directions in the scene. Recent methods have tried to address this problem either by using reinforcement learning, which is difficult to train, or by uncertainty maps, which are task-specific and can only be implemented for dense prediction tasks. In this paper, we propose the Glimpse-Attend-and-Explore model which: (a) employs self-attention to guide the visual exploration instead of task-specific uncertainty maps; (b) can be used for both dense and sparse prediction tasks; and (c) uses a contrastive stream to further improve the representations learned. Unlike previous works, we show the application of our model on multiple tasks like reconstruction, segmentation and classification. Our model provides encouraging results while being less dependent on dataset bias in driving the exploration. We further perform an ablation study to investigate the features and attention learned by our model. Finally, we show that our self-attention module learns to attend different regions of the scene by minimizing the loss on the downstream task. Code: https://github.com/soroushseifi/glimpse-attend-explore.

cs.CV↗

arXiv2020

Individual-level Modeling of COVID-19 Epidemic Risk

The ongoing COVID-19 pandemic calls for a multi-faceted public health response comprising complementary interventions to control the spread of the disease while vaccines and therapies are developed. Many of these interventions need to be informed by epidemic risk predictions given available data, including symptoms, contact patterns, and environmental factors. Here we propose a novel probabilistic formalism based on Individual-Level Models (ILMs) that offers rigorous formulas for the probability of infection of individuals, which can be parameterised via Maximum Likelihood Estimation (MLE) applied on compartmental models defined at the population level. We describe an approach where individual data collected in real-time is integrated with overall case counts to update the a predictor of the susceptibility of infection as a function of individual risk factors.

stat.AP↗

arXiv2020

Towards Automatic Face-to-Face Translation

In light of the recent breakthroughs in automatic machine translation systems, we propose a novel approach that we term as "Face-to-Face Translation". As today's digital communication becomes increasingly visual, we argue that there is a need for systems that can automatically translate a video of a person speaking in language A into a target language B with realistic lip synchronization. In this work, we create an automatic pipeline for this problem and demonstrate its impact on multiple real-world applications. First, we build a working speech-to-speech translation system by bringing together multiple existing modules from speech and language. We then move towards "Face-to-Face Translation" by incorporating a novel visual module, LipGAN for generating realistic talking faces from the translated audio. Quantitative evaluation of LipGAN on the standard LRW test set shows that it significantly outperforms existing approaches across all standard metrics. We also subject our Face-to-Face Translation pipeline, to multiple human evaluations and show that it can significantly improve the overall user experience for consuming and interacting with multimodal content across languages. Code, models and demo video are made publicly available. Demo video: https://www.youtube.com/watch?v=aHG6Oei8jF0 Code and models: https://github.com/Rudrabha/LipGAN

cs.CV↗

arXiv2018

Offline Extraction of Indic Regional Language from Natural Scene Image using Text Segmentation and Deep Convolutional Sequence

Regional language extraction from a natural scene image is always a challenging proposition due to its dependence on the text information extracted from Image. Text Extraction on the other hand varies on different lighting condition, arbitrary orientation, inadequate text information, heavy background influence over text and change of text appearance. This paper presents a novel unified method for tackling the above challenges. The proposed work uses an image correction and segmentation technique on the existing Text Detection Pipeline an Efficient and Accurate Scene Text Detector (EAST). EAST uses standard PVAnet architecture to select features and non maximal suppression to detect text from image. Text recognition is done using combined architecture of MaxOut convolution neural network (CNN) and Bidirectional long short term memory (LSTM) network. After recognizing text using the Deep Learning based approach, the native Languages are translated to English and tokenized using standard Text Tokenizers. The tokens that very likely represent a location is used to find the Global Positioning System (GPS) coordinates of the location and subsequently the regional languages spoken in that location is extracted. The proposed method is tested on a self generated dataset collected from Government of India dataset and experimented on Standard Dataset to evaluate the performance of the proposed technique. Comparative study with a few state-of-the-art methods on text detection, recognition and extraction of regional language from images shows that the proposed method outperforms the existing methods.

cs.CV↗

arXiv2015

A study on the effect of age, gender and paralysis on sEMG signals

Surface Electromyography (sEMG) is a technology to measure the bio-potentials across the muscles. The true prospective of this technology is yet to be explored. In this paper, a simple and economic construction of a sEMG sensor is proposed. These sensors are used to determine the differences in the Electromyography (EMG) signal patterns of different individuals. Signals of several volunteers from different age groups, gender and individual having paralysis have been obtained. The sEMG data acquisition is done using the soundcard of a computer, hence reducing the need of additional hardware. Finally, the data is used to analyse the relationship between electromyography and factors like age, gender and health condition i.e. paralysis.

physics.med-ph↗