arXiv subjects

Subhankar Ghosh

Publications and source records attributed to Subhankar Ghosh.

49 records · Page 3Linked to original sources

arXiv2023

VANI: Very-lightweight Accent-controllable TTS for Native and Non-native speakers with Identity Preservation

We introduce VANI, a very lightweight multi-lingual accent controllable speech synthesis system. Our model builds upon disentanglement strategies proposed in RADMMM and supports explicit control of accent, language, speaker and fine-grained $F_0$ and energy features for speech synthesis. We utilize the Indic languages dataset, released for LIMMITS 2023 as part of ICASSP Signal Processing Grand Challenge, to synthesize speech in 3 different languages. Our model supports transferring the language of a speaker while retaining their voice and the native accent of the target language. We utilize the large-parameter RADMMM model for Track $1$ and lightweight VANI model for Track $2$ and $3$ of the competition.

cs.SD↗

arXiv2022

Adapter-Based Extension of Multi-Speaker Text-to-Speech Model for New Speakers

Fine-tuning is a popular method for adapting text-to-speech (TTS) models to new speakers. However this approach has some challenges. Usually fine-tuning requires several hours of high quality speech per speaker. There is also that fine-tuning will negatively affect the quality of speech synthesis for previously learnt speakers. In this paper we propose an alternative approach for TTS adaptation based on using parameter-efficient adapter modules. In the proposed approach, a few small adapter modules are added to the original network. The original weights are frozen, and only the adapters are fine-tuned on speech for new speaker. The parameter-efficient fine-tuning approach will produce a new model with high level of parameter sharing with original model. Our experiments on LibriTTS, HiFi-TTS and VCTK datasets validate the effectiveness of adapter-based method through objective and subjective metrics.

eess.AS↗

arXiv2022

TIC: Text-Guided Image Colorization

Image colorization is a well-known problem in computer vision. However, due to the ill-posed nature of the task, image colorization is inherently challenging. Though several attempts have been made by researchers to make the colorization pipeline automatic, these processes often produce unrealistic results due to a lack of conditioning. In this work, we attempt to integrate textual descriptions as an auxiliary condition, along with the grayscale image that is to be colorized, to improve the fidelity of the colorization process. To the best of our knowledge, this is one of the first attempts to incorporate textual conditioning in the colorization pipeline. To do so, we have proposed a novel deep network that takes two inputs (the grayscale image and the respective encoded text description) and tries to predict the relevant color gamut. As the respective textual descriptions contain color information of the objects present in the scene, the text encoding helps to improve the overall quality of the predicted colors. We have evaluated our proposed model using different metrics and found that it outperforms the state-of-the-art colorization algorithms both qualitatively and quantitatively.

cs.CV↗

arXiv2022

Towards a Tighter Bound on Possible-Rendezvous Areas: Preliminary Results

Given trajectories with gaps, we investigate methods to tighten spatial bounds on areas (e.g., nodes in a spatial network) where possible rendezvous activity could have occurred. The problem is important for reducing the onerous amount of manual effort to post-process possible rendezvous areas using satellite imagery and has many societal applications to improve public safety, security, and health. The problem of rendezvous detection is challenging due to the difficulty of interpreting missing data within a trajectory gap and the very high cost of detecting gaps in such a large volume of location data. Most recent literature presents formal models, namely space-time prism, to track an object's rendezvous patterns within trajectory gaps on a spatial network. However, the bounds derived from the space-time prism are rather loose, resulting in unnecessarily extensive post-processing manual effort. To address these limitations, we propose a Time Slicing-based Gap-Aware Rendezvous Detection (TGARD) algorithm to tighten the spatial bounds in spatial networks. We propose a Dual Convergence TGARD (DC-TGARD) algorithm to improve computational efficiency using a bi-directional pruning approach. Theoretical results show the proposed spatial bounds on the area of possible rendezvous are tighter than that from related work (space-time prism). Experimental results on synthetic and real-world spatial networks (e.g., road networks) show that the proposed DC-TGARD is more scalable than the TGARD algorithm.

cs.DB↗

arXiv2021

Shared and Private VAEs with Generative Replay for Continual Learning

Continual learning tries to learn new tasks without forgetting previously learned ones. In reality, most of the existing artificial neural network(ANN) models fail, while humans do the same by remembering previous works throughout their life. Although simply storing all past data can alleviate the problem, it needs large memory and often infeasible in real-world applications where last data access is limited. We hypothesize that the model that learns to solve each task continually has some task-specific properties and some task-invariant characteristics. We propose a hybrid continual learning model that is more suitable in real case scenarios to address the issues that has a task-invariant shared variational autoencoder and T task-specific variational autoencoders. Our model combines generative replay and architectural growth to prevent catastrophic forgetting. We show our hybrid model effectively avoids forgetting and achieves state-of-the-art results on visual continual learning benchmarks such as MNIST, Permuted MNIST(QMNIST), CIFAR100, and miniImageNet datasets. We discuss results on a few more datasets, such as SVHN, Fashion-MNIST, EMNIST, and CIFAR10.

cs.CV↗

arXiv2021

Dynamic VAEs with Generative Replay for Continual Zero-shot Learning

Continual zero-shot learning(CZSL) is a new domain to classify objects sequentially the model has not seen during training. It is more suitable than zero-shot and continual learning approaches in real-case scenarios when data may come continually with only attributes for a few classes and attributes and features for other classes. Continual learning(CL) suffers from catastrophic forgetting, and zero-shot learning(ZSL) models cannot classify objects like state-of-the-art supervised classifiers due to lack of actual data(or features) during training. This paper proposes a novel continual zero-shot learning (DVGR-CZSL) model that grows in size with each task and uses generative replay to update itself with previously learned classes to avoid forgetting. We demonstrate our hybrid model(DVGR-CZSL) outperforms the baselines and is effective on several datasets, i.e., CUB, AWA1, AWA2, and aPY. We show our method is superior in task sequentially learning with ZSL(Zero-Shot Learning). We also discuss our results on the SUN dataset.

cs.CV↗

arXiv2021

Adversarial Training of Variational Auto-encoders for Continual Zero-shot Learning(A-CZSL)

Most of the existing artificial neural networks(ANNs) fail to learn continually due to catastrophic forgetting, while humans can do the same by maintaining previous tasks' performances. Although storing all the previous data can alleviate the problem, it takes a large memory, infeasible in real-world utilization. We propose a continual zero-shot learning model(A-CZSL) that is more suitable in real-case scenarios to address the issue that can learn sequentially and distinguish classes the model has not seen during training. Further, to enhance the reliability, we develop A-CZSL for a single head continual learning setting where task identity is revealed during the training process but not during the testing. We present a hybrid network that consists of a shared VAE module to hold information of all tasks and task-specific private VAE modules for each task. The model's size grows with each task to prevent catastrophic forgetting of task-specific skills, and it includes a replay approach to preserve shared skills. We demonstrate our hybrid model outperforms the baselines and is effective on several datasets, i.e., CUB, AWA1, AWA2, and aPY. We show our method is superior in class sequentially learning with ZSL(Zero-Shot Learning) and GZSL(Generalized Zero-Shot Learning).

cs.CV↗

arXiv2014

Online Stroke and Akshara Recognition GUI in Assamese Language Using Hidden Markov Model

The work describes the development of Online Assamese Stroke & Akshara Recognizer based on a set of language rules. In handwriting literature strokes are composed of two coordinate trace in between pen down and pen up labels. The Assamese aksharas are combination of a number of strokes, the maximum number of strokes taken to make a combination being eight. Based on these combinations eight language rule models have been made which are used to test if a set of strokes form a valid akshara. A Hidden Markov Model is used to train 181 different stroke patterns which generates a model used during stroke level testing. Akshara level testing is performed by integrating a GUI (provided by CDAC-Pune) with the Binaries of HTK toolkit classifier, HMM train model and the language rules using a dynamic linked library (dll). We have got a stroke level performance of 94.14% and akshara level performance of 84.2%.

cs.CV↗

arXiv2013

Multivariate Concentration Inequalities with Size Biased Couplings

Let $\mathbf{W}=(W_1,W_2,...,W_k)$ be a random vector with nonnegative coordinates having nonzero and finite variances. We prove concentration inequalities for $\mathbf{W}$ using size biased couplings that generalize the previous univariate results. Two applications on local dependence and counting patterns are provided.

math.PR↗

arXiv2011

Concentration of measures via size biased couplings

Let $Y$ be a nonnegative random variable with mean $μ$ and finite positive variance $σ^2$, and let $Y^s$, defined on the same space as $Y$, have the $Y$ size biased distribution, that is, the distribution characterized by E[Yf(Y)]=μE f(Y^s) for all functions $f$ for which these expectations exist. Under a variety of conditions on the coupling of Y and $Y^s$, including combinations of boundedness and monotonicity, concentration of measure inequalities hold. Examples include the number of relatively ordered subsequences of a random permutation, sliding window statistics including the number of m-runs in a sequence of coin tosses, the number of local maximum of a random function on a lattice, the number of urns containing exactly one ball in an urn allocation model, the volume covered by the union of $n$ balls placed uniformly over a volume n subset of d dimensional Euclidean space, the number of bulbs switched on at the terminal time in the so called lightbulb process, and the infinitely divisible and compound Poisson distributions that satisfy a bounded moment generating function condition.

math.PR↗

arXiv2011

Concentration of measure for the number of isolated vertices in the Erdős-Rényi random graph by size bias couplings

A concentration of measure result is proved for the number of isolated vertices $Y$ in the Erdős-Rényi random graph model on $n$ edges with edge probability $p$. When $μ$ and $σ^2$ denote the mean and variance of $Y$ respectively, $P((Y-μ)/σ\ge t)$ admits a bound of the form $e^{-kt^2}$ for some constant positive $k$ under the assumption $p \in (0,1)$ and $np\rightarrow c \in (0,\infty)$ as $n \rightarrow \infty$. The left tail inequality $$ P(\frac{Y-μ}σ\le -t)&\le& \exp(-\frac{t^2σ^2}{4μ}) $$ holds for all $n \in {2,3,...},p \in (0,1)$ and $t \ge 0$. The results are shown by coupling $Y$ to a random variable $Y^s$ having the $Y$-size biased distribution, that is, the distribution characterized by $E[Yf(Y)]=μE[f(Y^s)] $ for all functions $f$ for which these expectations exist.

math.PR↗

arXiv2010

$L^p$ bounds for a central limit theorem with involutions

Let $E=((e_{ij}))_{n\times n}$ be a fixed array of real numbers such that $e_{ij}=e_{ji}, e_{ii}=0$ for $1\le i,j \le n$. Let the permutation group be denoted by $S_n$ and the collection of involutions with no fixed points by $Π_n$, that is, $Π_n=\{π\in S_n: π^2= id, π(i)\neq i\,\forall i\}$ with id denoting the identity permutation. For $π$ uniformly chosen from $Π_n$, let $Y_E=\sum_{i=1}^n e_{iπ(i)}$ and $W=(Y_E-μ_E)/σ_E$ where $μ_E=E(Y_E)$ and $σ_E^2= Var(Y_E)$. Denoting by $F_W$ and $Φ$ the distribution functions of $W$ and a $\mathcal{N}(0,1)$ variate respectively, we bound $||F_W-Φ||_p$ for $ 1\le p\le \infty$ using Stein's method and the zero bias transformation. Optimal Berry-Esseen or $L^\infty$ bounds for the classical problem where $π$ is chosen uniformly from $S_n$ were obtained by Bolthausen using Stein's method. Although in our case $π\in Π_n$ uniformly, the $L^p$ bounds we obtain are of similar form as Bolthausen's bound which holds for $p=\infty$. The difficulty in extending Bolthausen's method from $S_n$ to $Π_n$ arising due to the involution restriction is tackled by the use of zero bias transformations.

math.PR↗

arXiv2010

Multivariate concentration of measure type results using exchangeable pairs and size biasing

Let $(\mathbf{W,W'})$ be an exchangeable pair of vectors in $\mathbb{R}^k$. Suppose this pair satisfies \beas E(\mathbf{W}'|\mathbf{W})=(I_k-Λ)\mathbf{W}+\mathbf{R(W)}. \enas If $||\mathbf{W-W'}||_2\le K$ and $\mathbf{R(W)}=0$, then concentration of measure results of following form is proved for all $\mathbf{w}\succeq 0$ when the moment generating function of $\mathbf{W}$ is finite. \beas P(\mathbf{W}\succeq\mathbf{w}),P(\mathbf{W}\preceq -\mathbf{w})\le \exp(-\frac{||\mathbf{w}||_2^2}{2K^2ν_1}), \enas for an explicit constant $ν_1$, where $\succeq$ stands for coordinate wise $\ge$ ordering. This result is applied to examples like complete non degenerate U-statistics. Also, we deal with the example of doubly indexed permutation statistics where $\mathbf{R(W)}\neq 0$ and obtain similar concentration of measure inequalities. Practical examples from doubly indexed permutation statistics include Mann-Whitney-Wilcoxon statistic and random intersection of two graphs. Both these two examples are used in nonparametric statistical testing. We conclude the paper with a multivariate generalization of a recent concentration result due to Ghosh and Goldstein \cite{cnm} involving bounded size bias couplings.

math.PR↗