arXiv subjects

Payel Das

Publications and source records attributed to Payel Das.

At least 109 records · Page 6Linked to original sources

arXiv2020

DualTKB: A Dual Learning Bridge between Text and Knowledge Base

In this work, we present a dual learning approach for unsupervised text to path and path to text transfers in Commonsense Knowledge Bases (KBs). We investigate the impact of weak supervision by creating a weakly supervised dataset and show that even a slight amount of supervision can significantly improve the model performance and enable better-quality transfers. We examine different model architectures, and evaluation metrics, proposing a novel Commonsense KB completion metric tailored for generative models. Extensive experimental results show that the proposed method compares very favorably to the existing baselines. This approach is a viable step towards a more advanced system for automatic KB construction/expansion and the reverse operation of KB conversion to coherent textual descriptions.

cs.CL↗

arXiv2020

A Decentralized Parallel Algorithm for Training Generative Adversarial Nets

Generative Adversarial Networks (GANs) are a powerful class of generative models in the deep learning community. Current practice on large-scale GAN training utilizes large models and distributed large-batch training strategies, and is implemented on deep learning frameworks (e.g., TensorFlow, PyTorch, etc.) designed in a centralized manner. In the centralized network topology, every worker needs to either directly communicate with the central node or indirectly communicate with all other workers in every iteration. However, when the network bandwidth is low or network latency is high, the performance would be significantly degraded. Despite recent progress on decentralized algorithms for training deep neural networks, it remains unclear whether it is possible to train GANs in a decentralized manner. The main difficulty lies at handling the nonconvex-nonconcave min-max optimization and the decentralized communication simultaneously. In this paper, we address this difficulty by designing the \textbf{first gradient-based decentralized parallel algorithm} which allows workers to have multiple rounds of communications in one iteration and to update the discriminator and generator simultaneously, and this design makes it amenable for the convergence analysis of the proposed decentralized algorithm. Theoretically, our proposed decentralized algorithm is able to solve a class of non-convex non-concave min-max problems with provable non-asymptotic convergence to first-order stationary point. Experimental results on GANs demonstrate the effectiveness of the proposed algorithm.

math.OC↗

arXiv2020

Combinatorial Black-Box Optimization with Expert Advice

We consider the problem of black-box function optimization over the boolean hypercube. Despite the vast literature on black-box function optimization over continuous domains, not much attention has been paid to learning models for optimization over combinatorial domains until recently. However, the computational complexity of the recently devised algorithms are prohibitive even for moderate numbers of variables; drawing one sample using the existing algorithms is more expensive than a function evaluation for many black-box functions of interest. To address this problem, we propose a computationally efficient model learning algorithm based on multilinear polynomials and exponential weight updates. In the proposed algorithm, we alternate between simulated annealing with respect to the current polynomial representation and updating the weights using monomial experts' advice. Numerical experiments on various datasets in both unconstrained and sum-constrained boolean optimization indicate the competitive performance of the proposed algorithm, while improving the computational time up to several orders of magnitude compared to state-of-the-art algorithms in the literature.

cs.LG↗

arXiv2020

Explaining Chemical Toxicity using Missing Features

Chemical toxicity prediction using machine learning is important in drug development to reduce repeated animal and human testing, thus saving cost and time. It is highly recommended that the predictions of computational toxicology models are mechanistically explainable. Current state of the art machine learning classifiers are based on deep neural networks, which tend to be complex and harder to interpret. In this paper, we apply a recently developed method named contrastive explanations method (CEM) to explain why a chemical or molecule is predicted to be toxic or not. In contrast to popular methods that provide explanations based on what features are present in the molecule, the CEM provides additional explanation on what features are missing from the molecule that is crucial for the prediction, known as the pertinent negative. The CEM does this by optimizing for the minimum perturbation to the model using a projected fast iterative shrinkage-thresholding algorithm (FISTA). We verified that the explanation from CEM matches known toxicophores and findings from other work.

q-bio.QM↗

arXiv2020

Active learning of deep surrogates for PDEs: Application to metasurface design

Surrogate models for partial-differential equations are widely used in the design of meta-materials to rapidly evaluate the behavior of composable components. However, the training cost of accurate surrogates by machine learning can rapidly increase with the number of variables. For photonic-device models, we find that this training becomes especially challenging as design regions grow larger than the optical wavelength. We present an active learning algorithm that reduces the number of training points by more than an order of magnitude for a neural-network surrogate model of optical-surface components compared to random samples. Results show that the surrogate evaluation is over two orders of magnitude faster than a direct solve, and we demonstrate how this can be exploited to accelerate large-scale engineering optimization.

cs.LG↗

arXiv2020

Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness

Mode connectivity provides novel geometric insights on analyzing loss landscapes and enables building high-accuracy pathways between well-trained neural networks. In this work, we propose to employ mode connectivity in loss landscapes to study the adversarial robustness of deep neural networks, and provide novel methods for improving this robustness. Our experiments cover various types of adversarial attacks applied to different network architectures and datasets. When network models are tampered with backdoor or error-injection attacks, our results demonstrate that the path connection learned using limited amount of bonafide data can effectively mitigate adversarial effects while maintaining the original accuracy on clean data. Therefore, mode connectivity provides users with the power to repair backdoored or error-injected models. We also use mode connectivity to investigate the loss landscapes of regular and robust models against evasion attacks. Experiments show that there exists a barrier in adversarial robustness loss on the path connecting regular and adversarially-trained models. A high correlation is observed between the adversarial robustness loss and the largest eigenvalue of the input Hessian matrix, for which theoretical justifications are provided. Our results suggest that mode connectivity offers a holistic tool and practical means for evaluating and improving adversarial robustness.

cs.LG↗

arXiv2020

CogMol: Target-Specific and Selective Drug Design for COVID-19 Using Deep Generative Models

The novel nature of SARS-CoV-2 calls for the development of efficient de novo drug design approaches. In this study, we propose an end-to-end framework, named CogMol (Controlled Generation of Molecules), for designing new drug-like small molecules targeting novel viral proteins with high affinity and off-target selectivity. CogMol combines adaptive pre-training of a molecular SMILES Variational Autoencoder (VAE) and an efficient multi-attribute controlled sampling scheme that uses guidance from attribute predictors trained on latent features. To generate novel and optimal drug-like molecules for unseen viral targets, CogMol leverages a protein-molecule binding affinity predictor that is trained using SMILES VAE embeddings and protein sequence embeddings learned unsupervised from a large corpus. CogMol framework is applied to three SARS-CoV-2 target proteins: main protease, receptor-binding domain of the spike protein, and non-structural protein 9 replicase. The generated candidates are novel at both molecular and chemical scaffold levels when compared to the training data. CogMol also includes insilico screening for assessing toxicity of parent molecules and their metabolites with a multi-task toxicity classifier, synthetic feasibility with a chemical retrosynthesis predictor, and target structure binding with docking simulations. Docking reveals favorable binding of generated molecules to the target protein structure, where 87-95 % of high affinity molecules showed docking free energy < -6 kcal/mol. When compared to approved drugs, the majority of designed compounds show low parent molecule and metabolite toxicity and high synthetic feasibility. In summary, CogMol handles multi-constraint design of synthesizable, low-toxic, drug-like molecules with high target specificity and selectivity, and does not need target-dependent fine-tuning of the framework or target structure information.

cs.LG↗

arXiv2020

Learning Implicit Text Generation via Feature Matching

Generative feature matching network (GFMN) is an approach for training implicit generative models for images by performing moment matching on features from pre-trained neural networks. In this paper, we present new GFMN formulations that are effective for sequential data. Our experimental results show the effectiveness of the proposed method, SeqGFMN, for three distinct generation tasks in English: unconditional text generation, class-conditional text generation, and unsupervised text style transfer. SeqGFMN is stable to train and outperforms various adversarial approaches for text generation and text style transfer.

cs.CL↗

arXiv2020

Toward A Neuro-inspired Creative Decoder

Creativity, a process that generates novel and meaningful ideas, involves increased association between task-positive (control) and task-negative (default) networks in the human brain. Inspired by this seminal finding, in this study we propose a creative decoder within a deep generative framework, which involves direct modulation of the neuronal activation pattern after sampling from the learned latent space. The proposed approach is fully unsupervised and can be used off-the-shelf. Several novelty metrics and human evaluation were used to evaluate the creative capacity of the deep decoder. Our experiments on different image datasets (MNIST, FMNIST, MNIST+FMNIST, WikiArt and CelebA) reveal that atypical co-activation of highly activated and weakly activated neurons in a deep decoder promotes generation of novel and meaningful artifacts.

cs.AI↗

arXiv2020

seestar: Selection functions for spectroscopic surveys of the Milky Way

Selection functions are vital for understanding the observational biases of spectroscopic surveys. With the wide variety of multi-object spectrographs currently in operation and becoming available soon, we require easily generalisable methods for determining the selection functions of these surveys. Previous work, however, has largely been focused on generating individual, tailored selection functions for every data release of each survey. Moreover, no methods for combining these selection functions to be used for joint catalogues have been developed. We have developed a Poisson likelihood estimation method for calculating selection functions in a Bayesian framework, which can be generalised to any multi-object spectrograph. We include a robust treatment of overlapping fields within a survey as well as selection functions for combined samples with overlapping footprints. We also provide a method for transforming the selection function that depends on the sky positions, colour, and apparent magnitude of a star to one that depends on the galactic location, metallicity, mass, and age of a star. This `intrinsic' selection function is invaluable for chemodynamical models of the Milky Way. We demonstrate that our method is successful at recreating synthetic spectroscopic samples selected from a mock galaxy catalogue.

astro-ph.GA↗

arXiv2020

Improving Efficiency in Large-Scale Decentralized Distributed Training

Decentralized Parallel SGD (D-PSGD) and its asynchronous variant Asynchronous Parallel SGD (AD-PSGD) is a family of distributed learning algorithms that have been demonstrated to perform well for large-scale deep learning tasks. One drawback of (A)D-PSGD is that the spectral gap of the mixing matrix decreases when the number of learners in the system increases, which hampers convergence. In this paper, we investigate techniques to accelerate (A)D-PSGD based training by improving the spectral gap while minimizing the communication cost. We demonstrate the effectiveness of our proposed techniques by running experiments on the 2000-hour Switchboard speech recognition task and the ImageNet computer vision task. On an IBM P9 supercomputer, our system is able to train an LSTM acoustic model in 2.28 hours with 7.5% WER on the Hub5-2000 Switchboard (SWB) test set and 13.3% WER on the CallHome (CH) test set using 64 V100 GPUs and in 1.98 hours with 7.7% WER on SWB and 13.3% WER on CH using 128 V100 GPUs, the fastest training time reported to date.

cs.LG↗

arXiv2019

Ages and kinematics of chemically selected, accreted Milky Way halo stars

We exploit the [Mg/Mn]-[Al/Fe] chemical abundance plane to help identify nearby halo stars in the 14th data release from the APOGEE survey that have been accreted on to the Milky Way. Applying a Gaussian Mixture Model, we find a `blob' of 856 likely accreted stars, with a low disc contamination rate of ~7%. Cross-matching the sample with the second data release from Gaia gives us access to parallaxes and apparent magnitudes, which place constraints on distances and intrinsic luminosities. Using a Bayesian isochrone pipeline, this enables us to estimate new ages for the accreted stars, with typical uncertainties of ~20%. Our new catalogue is further supplemented with estimates of orbital parameters. The blob stars span a metallicities between -0.5 to -2.5, and [Mg/Fe] between -0.1 to 0.5. They constitute ~30% of the metal-poor ([Fe/H] < -0.8) halo at metallicities of ~-1.4. Our new ages are mainly range between 8 to 13 Gyr, with the oldest stars the metal-poorest, and with the highest [Mg/Fe] abundance. If the blob stars are assumed to belong to a single progenitor, the ages imply that the system merged with our Milky Way around 8 Gyr ago and that star formation proceeded for ~5 Gyr. Dynamical arguments suggest that such a single progenitor would have a total mass of ~1011Msun, similar to that found by other authors using chemical evolution models and simulations. Comparing the scatter in the [Mg/Fe]-[Fe/H] plane of the blob stars to that measured for stars belonging to the Large Magellanic Cloud suggests that the blob does indeed contain stars from only one progenitor.

astro-ph.GA↗

arXiv2018

PepCVAE: Semi-Supervised Targeted Design of Antimicrobial Peptide Sequences

Given the emerging global threat of antimicrobial resistance, new methods for next-generation antimicrobial design are urgently needed. We report a peptide generation framework PepCVAE, based on a semi-supervised variational autoencoder (VAE) model, for designing novel antimicrobial peptide (AMP) sequences. Our model learns a rich latent space of the biological peptide context by taking advantage of abundant, unlabeled peptide sequences. The model further learns a disentangled antimicrobial attribute space by using the feedback from a jointly trained AMP classifier that uses limited labeled instances. The disentangled representation allows for controllable generation of AMPs. Extensive analysis of the PepCVAE-generated sequences reveals superior performance of our model in comparison to a plain VAE, as PepCVAE generates novel AMP sequences with higher long-range diversity, while being closer to the training distribution of biological peptides. These features are highly desired in next-generation antimicrobial design.

q-bio.QM↗

arXiv2018

Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives

In this paper we propose a novel method that provides contrastive explanations justifying the classification of an input by a black box classifier such as a deep neural network. Given an input we find what should be %necessarily and minimally and sufficiently present (viz. important object pixels in an image) to justify its classification and analogously what should be minimally and necessarily \emph{absent} (viz. certain background pixels). We argue that such explanations are natural for humans and are used commonly in domains such as health care and criminology. What is minimally but critically \emph{absent} is an important part of an explanation, which to the best of our knowledge, has not been explicitly identified by current explanation methods that explain predictions of neural networks. We validate our approach on three real datasets obtained from diverse domains; namely, a handwritten digits dataset MNIST, a large procurement fraud dataset and a brain activity strength dataset. In all three cases, we witness the power of our approach in generating precise explanations that are also easy for human experts to understand and evaluate.

cs.AI↗

arXiv2018

MADE: A spectroscopic Mass, Age, and Distance Estimator for red giant stars with Bayesian machine learning

We present a new approach (MADE) that generates mass, age, and distance estimates of red giant stars from a combination of astrometric, photometric, and spectroscopic data. The core of the approach is a Bayesian artificial neural network (ANN) that learns from and completely replaces stellar isochrones. The ANN is trained using a sample of red giant stars with mass estimates from asteroseismology. A Bayesian isochrone pipeline uses the astrometric, photometric, spectroscopic, and asteroseismology data to determine posterior distributions for the training outputs: mass, age, and distance. Given new inputs, posterior predictive distributions for the outputs are computed, taking into account both input uncertainties, and uncertainties in the ANN parameters. We apply MADE to $\sim10\,000$ red giants in the overlap between the 14$^{\mathrm{th}}$ data release from the APO Galactic Evolution Experiment (APOGEE, Abolfathi et al. 2018) and the Tycho-Gaia astrometric solution (TGAS, Michalik et al. 2015). The ANN is able to reduce the uncertainty on mass, age, and distance estimates for training-set stars with high output uncertainties allocated through the Bayesian isochrone pipeline. The fractional uncertainties on mass are $<10\%$ and on age are between $10$ to $25\%$. Moreover, the time taken for our ANN to predict masses, ages, and distances for the entire catalogue of APOGEE-TGAS stars is of a similar order of the time taken by the Bayesian isochrone pipeline to run on a handful of stars. Our resulting catalogue clearly demonstrates the expected thick and thin disc components in the [M/H]-[$α$/M] plane, when examined by age.

astro-ph.GA↗

arXiv2018

Isochrone ages for ~3 million stars with the second Gaia data release

We present a catalogue of distances, masses and ages for $\sim3$ million stars in the second Gaia data release with spectroscopic parameters available from the large spectroscopic surveys: APOGEE, Gaia-ESO, GALAH, LAMOST, RAVE and SEGUE. We use a Bayesian framework to characterise the probability density functions of distance, mass and age using photometric, spectroscopic and astrometric information, supplemented with spectroscopic masses where available for giant stars. Furthermore, we provide posterior extinction estimates ($A_V$) to every star using published extinction maps as a prior input. We provide an appendix with extinction coefficients for Gaia photometry derived from stellar models, which account for variation with intrinsic colour and total extinction. Our pipeline provides output estimates of the spectroscopic parameters, which can be used to inform improved spectroscopic analysis. We complement our catalogues with Galactocentric coordinates and actions with associated uncertainties. As a demonstration of the power of our catalogue, we produce velocity dispersion profiles of the disc separated by age and Galactocentric radius (between $3$ and $15\,\mathrm{kpc}$ from the Galactic centre). This suggests that the velocity dispersion profiles flatten with radius in the outer Galaxy ($>8\,\mathrm{kpc}$) and that at all radii the velocity dispersion follows the smooth power law with age observed in the solar neighbourhood.

astro-ph.GA↗

arXiv2017

Autism Classification Using Brain Functional Connectivity Dynamics and Machine Learning

The goal of the present study is to identify autism using machine learning techniques and resting-state brain imaging data, leveraging the temporal variability of the functional connections (FC) as the only information. We estimated and compared the FC variability across brain regions between typical, healthy subjects and autistic population by analyzing brain imaging data from a world-wide multi-site database known as ABIDE (Autism Brain Imaging Data Exchange). Our analysis revealed that patients diagnosed with autism spectrum disorder (ASD) show increased FC variability in several brain regions that are associated with low FC variability in the typical brain. We then used the enhanced FC variability of brain regions as features for training machine learning models for ASD classification and achieved 65% accuracy in identification of ASD versus control subjects within the dataset. We also used node strength estimated from number of functional connections per node averaged over the whole scan as features for ASD classification.The results reveal that the dynamic FC measures outperform or are comparable with the static FC measures in predicting ASD.

q-bio.NC↗

arXiv2017

Neurology-as-a-Service for the Developing World

Electroencephalography (EEG) is an extensively-used and well-studied technique in the field of medical diagnostics and treatment for brain disorders, including epilepsy, migraines, and tumors. The analysis and interpretation of EEGs require physicians to have specialized training, which is not common even among most doctors in the developed world, let alone the developing world where physician shortages plague society. This problem can be addressed by teleEEG that uses remote EEG analysis by experts or by local computer processing of EEGs. However, both of these options are prohibitively expensive and the second option requires abundant computing resources and infrastructure, which is another concern in developing countries where there are resource constraints on capital and computing infrastructure. In this work, we present a cloud-based deep neural network approach to provide decision support for non-specialist physicians in EEG analysis and interpretation. Named `neurology-as-a-service,' the approach requires almost no manual intervention in feature engineering and in the selection of an optimal architecture and hyperparameters of the neural network. In this study, we deploy a pipeline that includes moving EEG data to the cloud and getting optimal models for various classification tasks. Our initial prototype has been tested only in developed world environments to-date, but our intention is to test it in developing world environments in future work. We demonstrate the performance of our proposed approach using the BCI2000 EEG MMI dataset, on which our service attains 63.4% accuracy for the task of classifying real vs. imaginary activity performed by the subject, which is significantly higher than what is obtained with a shallow approach such as support vector machines.

stat.ML↗