arXiv subjects

Saurabh Singh

Publications and source records attributed to Saurabh Singh.

At least 91 records · Page 5Linked to original sources

arXiv2020

Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks

Batch Normalization (BN) uses mini-batch statistics to normalize the activations during training, introducing dependence between mini-batch elements. This dependency can hurt the performance if the mini-batch size is too small, or if the elements are correlated. Several alternatives, such as Batch Renormalization and Group Normalization (GN), have been proposed to address this issue. However, they either do not match the performance of BN for large batches, or still exhibit degradation in performance for smaller batches, or introduce artificial constraints on the model architecture. In this paper we propose the Filter Response Normalization (FRN) layer, a novel combination of a normalization and an activation function, that can be used as a replacement for other normalizations and activations. Our method operates on each activation channel of each batch element independently, eliminating the dependency on other batch elements. Our method outperforms BN and other alternatives in a variety of settings for all batch sizes. FRN layer performs $\approx 0.7-1.0\%$ better than BN on top-1 validation accuracy with large mini-batch sizes for Imagenet classification using InceptionV3 and ResnetV2-50 architectures. Further, it performs $>1\%$ better than GN on the same problem in the small mini-batch size regime. For object detection problem on COCO dataset, FRN layer outperforms all other methods by at least $0.3-0.5\%$ in all batch size regimes.

cs.LG↗

arXiv2020

Scalable Model Compression by Entropy Penalized Reparameterization

We describe a simple and general neural network weight compression approach, in which the network parameters (weights and biases) are represented in a "latent" space, amounting to a reparameterization. This space is equipped with a learned probability model, which is used to impose an entropy penalty on the parameter representation during training, and to compress the representation using a simple arithmetic coder after training. Classification accuracy and model compressibility is maximized jointly, with the bitrate--accuracy trade-off specified by a hyperparameter. We evaluate the method on the MNIST, CIFAR-10 and ImageNet classification benchmarks using six distinct model architectures. Our results show that state-of-the-art model compression can be achieved in a scalable and general way without requiring complex procedures such as multi-stage training.

cs.LG↗

arXiv2019

Investigation of Thermoelectric properties of Magnetic Insulator FeRuTiSi Using First Principle Calculation

In this work, we have investigated the electronic structure and thermoelectric properties of quaternary heusler alloy, FeRuTiSi, using first principle DFT tools implemented in WIEN2k and BoltzTraP code. Electronic structure calculations using TB-mBJ potential shows appearance of flat band at the conduction band edge, thus electron in conduction band have the large effective mass (me*), and therefore mainly contribute for negatively large value of Seebeck coefficient (S). This alloy has indirect band gap of 0.59 eV, and shows the n-type transport behavior. Under the constant relaxation time approximation (tau = 10 -14 s), temperature dependent Seebeck coefficient, electrical conductivity (sigma), and electronic thermal conductivity (ke) were also estimated. The maximum figure-of-merit (ZT), for the FeRuTiSi compound is found to be ~0.86 at 840 K, with n-type doping, which suggests that this quaternary alloy can be a good candidate among the n-type material for thermoelectric applications in high-temperature reg

cond-mat.str-el↗

arXiv2019

Thermoelectric properties of Half-Metallic FeMnScGa Using First Principle Calculation

Here, we have investigated the thermoelectric properties of FeMnScGa alloy by combined use of full potential linearized augmented plane-wave (FP-LAPW) method and Boltzmann transport theory implemented in Wien2K and BoltzTraP code, respectively. Using the TB-mBJ potential, half-metallic nature is clearly observed with energy band gap of ~0.41 eV for down spin channel. Transport coefficients in 200-1000 K temperature range are investigated under the constant relaxation time approximations (tau=10-14 s). Total contributions to the Seebeck coefficients from up and dn-spin channels are estimated by using two-current model. The calculated values of Seebeck coefficients are found to be negative in the entire temperature region under present study, which suggests the n-type characteristic of this alloy. The magnitude of Seebeck coefficient is found to be increasing with temperature and exhibit the large value ~ -87 micro-V/K in 700-1000 K temperature range. The maximum in magnitude of total electrical conductivity value is found to be ~2.0 x 105 ohm-1m-1 at 1000 K. The present study shows, with half-metallic electronic structure, FeMnScGa have very good thermoelectric behavior, and can be suitable for thermoelectric application in high temperature region.

cond-mat.str-el↗

arXiv2019

Canada and the SKA from 2020-2030

This white paper submitted for the 2020 Canadian Long-Range Planning process (LRP2020) presents the prospects for Canada and the Square Kilometre Array (SKA) from 2020-2030, focussing on the first phase of the project (SKA1) scheduled to begin construction early in the next decade. SKA1 will make transformational advances in our understanding of the Universe across a wide range of fields, and Canadians are poised to play leadership roles in several. Canadian key SKA technologies will ensure a good return on capital investment in addition to strong scientific returns, positioning Canadian astronomy for future opportunities well beyond 2030. We therefore advocate for Canada's continued scientific and technological engagement in the SKA from 2020-2030 through participation in the construction and operations phases of SKA1.

astro-ph.IM↗

arXiv2019

Distinctive Thermoelectric Properties of Supersaturated Si-Ge-P Compounds: Achieving Figure of Merit ZT > 3.6

The efficiency of energy conversion in thermoelectric generators (TEGs) is directly proportional to electrical conductivity and Seebeck coefficient while inversely to thermal conductivity. The challenge is to optimize these interdependent parameters simultaneously. In this work, the problem is addressed with a novel approach of nanostructuring and constructive electronic structure modification to achieve a very high value of dimensionless figure of merit ZT greater than 3.6 at 1000 K with negative Seebeck coefficient. Supersaturated solid-solutions of Si-Ge containing 1 atomic percent Fe and 10 atomic percent P are prepared by high-energy ball milling. The bulk samples consisting of ultra-fine nano-crystallites 9.7 nm are obtained by the sophisticated low-temperature & high-pressure sintering process. Despite that the electrical resistivity is slightly high due to the localization of electrons is associated with the highly disordered structure and low electrical density of states near the chemical potential, a very low thermal conductivity \k{appa} less than 1 W m-1K-1 and very large magnitude of Seebeck coefficient exceeding 470 uV K-1 are achieved in association with the nanostructuring and the Fe 3d impurity states, respectively, to realize a very large magnitude of ZT.

cond-mat.mtrl-sci↗

arXiv2019

EvalNorm: Estimating Batch Normalization Statistics for Evaluation

Batch normalization (BN) has been very effective for deep learning and is widely used. However, when training with small minibatches, models using BN exhibit a significant degradation in performance. In this paper we study this peculiar behavior of BN to gain a better understanding of the problem, and identify a cause. We propose 'EvalNorm' to address the issue by estimating corrected normalization statistics to use for BN during evaluation. EvalNorm supports online estimation of the corrected statistics while the model is being trained, and does not affect the training scheme of the model. As a result, EvalNorm can also be used with existing pre-trained models allowing them to benefit from our method. EvalNorm yields large gains for models trained with smaller batches. Our experiments show that EvalNorm performs 6.18% (absolute) better than vanilla BN for a batchsize of 2 on ImageNet validation set and from 1.5 to 7.0 points (absolute) gain on the COCO object detection benchmark across a variety of setups.

cs.CV↗

arXiv2019

Apparatus for Seebeck coefficient measurement of wire, thin film and bulk materials in the wide temperature range (80-650K)

A Seebeck coefficient measurement apparatus has been designed and developed, which is very effective for accurate characterization of different type of samples in a wide temperature range (80 - 650K) simultaneously covering low as well as the high-temperature regime. Reducing the complexity of the technical design of sample holder and data collections has always been challenging to implement in a single instrument when samples are in different geometrical shape and electronic structure. Our unique design of sample holder with pressure probes covers measurements of different samples shapes (wires, thin films and pellets) as well as different resistivity ranges (metals, semiconductors and insulators). It is suitable for characterization of different samples sizes (3-12 mm). A double heater configuration powered by a dual channel source meter is employed for maintaining a desired constant temperature difference across the sample for the whole temperature range. Two K-type thermocouples are used for simultaneously reading of temperatures and Seebeck voltages by utilizing different channels of a multichannel digital multimeter. Calibration of the system has been carried out using constantan, chromel and alumel materials and recorded data is found to be very accurate and consistent with earlier reports. The Seebeck coefficients of standard samples of constantan (wire) and GaN (thin film) have been reported, which shows the measurement capability of designed setup with versatile samples.

physics.app-ph↗

arXiv2019

The redshifted 21-cm signal in the EDGES low-band spectrum

The EDGES collaboration reported the finding of an unexpectedly deep absorption in the radio background at 78 MHz and interpreted the dip as a first detection of redshifted 21-cm from Cosmic Dawn. We have attempted an alternate analysis, adopting a maximally smooth function approach to model the foreground. A joint fit to the spectrum using such a function together with a flattened absorption profile yields a best fit absorption amplitude of $921 \pm 35$ mK. The depth of the 21-cm absorption inferred by the EDGES analysis required invoking non-standard cosmology or new physics or new sources at Cosmic Dawn and this tension with accepted models is compounded by our analysis that suggests absorption of greater depth. Alternatively, the measured spectrum may be equally-well fit assuming that there exists a residual unmodeled systematic sinusoidal feature and we explore this possibility further by examining for any additional 21-cm signal. The data then favors an absorption with Gaussian model parameters of amplitude $133 \pm 60$ mK, best width at half-power $9 \pm 3$ MHz and center frequency $72.5 \pm 0.8$ MHz. We also examine the consistency of the measured spectrum with plausible redshifted 21-cm models: a set of 3 of the 265 profiles in the global 21-cm atlas of Cohen et al. 2017 are favored by the spectrum. We conclude that the EDGES data may be consistent with standard cosmology and astrophysics, without invoking excess radio backgrounds or baryon-dark matter interactions.

astro-ph.CO↗

arXiv2018

Image-Dependent Local Entropy Models for Learned Image Compression

The leading approach for image compression with artificial neural networks (ANNs) is to learn a nonlinear transform and a fixed entropy model that are optimized for rate-distortion performance. We show that this approach can be significantly improved by incorporating spatially local, image-dependent entropy models. The key insight is that existing ANN-based methods learn an entropy model that is shared between the encoder and decoder, but they do not transmit any side information that would allow the model to adapt to the structure of a specific image. We present a method for augmenting ANN-based image coders with image-dependent side information that leads to a 17.8% rate reduction over a state-of-the-art ANN-based baseline model on a standard evaluation set, and 70-98% reductions on images with low visual complexity that are poorly captured by a fixed, global entropy model.

cs.CV↗

arXiv2018

Variational image compression with a scale hyperprior

We describe an end-to-end trainable model for image compression based on variational autoencoders. The model incorporates a hyperprior to effectively capture spatial dependencies in the latent representation. This hyperprior relates to side information, a concept universal to virtually all modern image codecs, but largely unexplored in image compression using artificial neural networks (ANNs). Unlike existing autoencoder compression methods, our model trains a complex prior jointly with the underlying autoencoder. We demonstrate that this model leads to state-of-the-art image compression when measuring visual quality using the popular MS-SSIM index, and yields rate-distortion performance surpassing published ANN-based methods when evaluated using a more traditional metric based on squared error (PSNR). Furthermore, we provide a qualitative comparison of models trained for different distortion metrics.

eess.IV↗

arXiv2018

Spatially adaptive image compression using a tiled deep network

Deep neural networks represent a powerful class of function approximators that can learn to compress and reconstruct images. Existing image compression algorithms based on neural networks learn quantized representations with a constant spatial bit rate across each image. While entropy coding introduces some spatial variation, traditional codecs have benefited significantly by explicitly adapting the bit rate based on local image complexity and visual saliency. This paper introduces an algorithm that combines deep neural networks with quality-sensitive bit rate adaptation using a tiled network. We demonstrate the importance of spatial context prediction and show improved quantitative (PSNR) and qualitative (subjective rater assessment) results compared to a non-adaptive baseline and a recently published image compression model based on fully-convolutional neural networks.

cs.CV↗

arXiv2017

SARAS 2 constraints on global 21-cm signals from the Epoch of Reionization

Spectral distortions in the cosmic microwave background over the 40--200~MHz band are imprinted by neutral hydrogen in the intergalactic medium prior to the end of reionization. This signal, produced in the redshift range $z = 6-34$ at the rest frame wavelength of 21 cm, has not been detected yet; and poor understanding of high redshift astrophysics results in a large uncertainty in the expected spectrum. The SARAS~2 radiometer was purposely designed to detect the sky-averaged 21-cm signal. The instrument, deployed at the Timbaktu Collective (Southern India) in April--June 2017, collected 63~hr of science data, which were examined for the presence of the cosmological 21-cm signal. In our previous work the first-light data from SARAS~2 radiometer were analyzed with Bayesian likelihood-ratio tests using $264$ plausible astrophysical scenarios. In this paper we re-examine the data using an improved analysis based on the frequentist approach and forward modeling. We show that SARAS~2 data rejects 27 models, out of which 25 are rejected at a significance $>5σ$. All the rejected models share the scenario of inefficient heating of the primordial gas by the first population of X-ray sources along with rapid reionization.

astro-ph.CO↗

arXiv2017

Aligned Image-Word Representations Improve Inductive Transfer Across Vision-Language Tasks

An important goal of computer vision is to build systems that learn visual representations over time that can be applied to many tasks. In this paper, we investigate a vision-language embedding as a core representation and show that it leads to better cross-task transfer than standard multi-task learning. In particular, the task of visual recognition is aligned to the task of visual question answering by forcing each to use the same word-region embeddings. We show this leads to greater inductive transfer from recognition to VQA than standard multitask learning. Visual recognition also improves, especially for categories that have relatively few recognition training labels but appear often in the VQA setting. Thus, our paper takes a small step towards creating more general vision systems by showing the benefit of interpretable, flexible, and trainable core representations.

cs.CV↗

arXiv2017

Effect of nanostructure on thermoelectric properties of La$_{0.7}$Sr$_{0.3}$MnO$_{3}$ in 300-600 K range

In oxide materials, nanostructuring effect has been found very promising approach for the enhancement of \textit{figure-of-merit}, \textit{ZT}. In the present work, we have synthesized La$_{0.7}$Sr$_{0.3}$MnO$_{3}$ (LSMO) compound using sol-gel method and samples of crystallite size of 34, 41, and 49 nm were obtained by giving different heat treatment. Seebeck coefficient ($α$), electrical resistivity ($ρ$), and thermal conductivity ($κ$) measurements were carried out in 300-600 K temperature range. The systematic change in the values of $α$ from $\sim$ -19 $μ$V/K to $\sim$ -24 $μ$V/K and drastic reduction in the values of $κ$ from $\sim$0.88 W/mK to $\sim$0.23 W/mK are observed as crystallite size is reduced from 49 nm to 34 nm at $\sim$600 K. Also, fall in the values of $ρ$ in the paramagnetic (PM) insulator phase (400-600 K) are effectively responsible for the increasing trend in the values of \textit{ZT} at high temperature. For the crystallite size of 41 nm, value of \textit{ZT} at 600 K was found to be $\sim$0.017, which can be further increased up to $\sim$0.045 around 650 K temperature. The predicted value of \textit{ZT} suggests that LSMO can be suitable oxide material for thermoelectric applications at high temperature.

physics.app-ph↗

arXiv2017

SARAS 2: A Spectral Radiometer for probing Cosmic Dawn and the Epoch of Reionization through detection of the global 21 cm signal

The global 21 cm signal from Cosmic Dawn (CD) and the Epoch of Reionization (EoR), at redshifts $z \sim 6-30$, probes the nature of first sources of radiation as well as physics of the Inter-Galactic Medium (IGM). Given that the signal is predicted to be extremely weak, of wide fractional bandwidth, and lies in a frequency range that is dominated by Galactic and Extragalactic foregrounds as well as Radio Frequency Interference, detection of the signal is a daunting task. Critical to the experiment is the manner in which the sky signal is represented through the instrument. It is of utmost importance to design a system whose spectral bandpass and additive spurious can be well calibrated and any calibration residual does not mimic the signal. SARAS is an ongoing experiment that aims to detect the global 21 cm signal. Here we present the design philosophy of the SARAS 2 system and discuss its performance and limitations based on laboratory and field measurements. Laboratory tests with the antenna replaced with a variety of terminations, including a network model for the antenna impedance, show that the gain calibration and modeling of internal additives leave no residuals with Fourier amplitudes exceeding 2~mK, or residual Gaussians of 25 MHz width with amplitudes exceeding 2~mK. Thus, even accounting for reflection and radiation efficiency losses in the antenna, the SARAS~2 system is capable of detection of complex 21-cm profiles at the level predicted by currently favoured models for thermal baryon evolution.

astro-ph.IM↗

arXiv2017

Theoretical Study of Thermopower Behavior of LaFeO$_{3}$ Compound in High Temperature Region

The electronic structure and thermopower ($α$) behavior of LaFeO$_{3}$ compound were investigated by combining the ab-initio electronic structures and Boltzmann transport calculations. LSDA plus Hubbard U (U = 5 eV) calculation on G-type anti-ferromagnetic (AFM) configuration gives an energy gap of $\sim$2 eV, which is very close to the experimentally reported energy gap. The calculated values of effective mass of holes (m$^{*}$$_{h}$) in valance band (VB) are found $\sim$4 times that of the effective mass of electrons (m$^{*}$$_{e}$) in conduction band (CB). The large effective masses of holes are responsible for the large and positive thermopower exhibited by this compound. The calculated values of $α$ using BoltzTraP code are found to be large and positive in the 300-1200 K temperature range, which is in agreement with the experimentally reported data.

cond-mat.str-el↗

arXiv2017

Revisiting Unreasonable Effectiveness of Data in Deep Learning Era

The success of deep learning in vision can be attributed to: (a) models with high capacity; (b) increased computational power; and (c) availability of large-scale labeled data. Since 2012, there have been significant advances in representation capabilities of the models and computational capabilities of GPUs. But the size of the biggest dataset has surprisingly remained constant. What will happen if we increase the dataset size by 10x or 100x? This paper takes a step towards clearing the clouds of mystery surrounding the relationship between `enormous data' and visual deep learning. By exploiting the JFT-300M dataset which has more than 375M noisy labels for 300M images, we investigate how the performance of current vision tasks would change if this data was used for representation learning. Our paper delivers some surprising (and some expected) findings. First, we find that the performance on vision tasks increases logarithmically based on volume of training data size. Second, we show that representation learning (or pre-training) still holds a lot of promise. One can improve performance on many vision tasks by just training a better base model. Finally, as expected, we present new state-of-the-art results for different vision tasks including image classification, object detection, semantic segmentation and human pose estimation. Our sincere hope is that this inspires vision community to not undervalue the data and develop collective efforts in building larger datasets.

cs.CV↗