arXiv subjects

Gaurav Sharma

Publications and source records attributed to Gaurav Sharma.

At least 55 records · Page 3Linked to original sources

arXiv2019

Learning 2D to 3D Lifting for Object Detection in 3D for Autonomous Vehicles

We address the problem of 3D object detection from 2D monocular images in autonomous driving scenarios. We propose to lift the 2D images to 3D representations using learned neural networks and leverage existing networks working directly on 3D data to perform 3D object detection and localization. We show that, with carefully designed training mechanism and automatically selected minimally noisy data, such a method is not only feasible, but gives higher results than many methods working on actual 3D inputs acquired from physical sensors. On the challenging KITTI benchmark, we show that our 2D to 3D lifted method outperforms many recent competitive 3D networks while significantly outperforming previous state-of-the-art for 3D detection from monocular images. We also show that a late fusion of the output of the network trained on generated 3D images, with that trained on real 3D images, improves performance. We find the results very interesting and argue that such a method could serve as a highly reliable backup in case of malfunction of expensive 3D sensors, if not potentially making them redundant, at least in the case of low human injury risk autonomous navigation scenarios like warehouse automation.

cs.CV↗

arXiv2019

Spin-lattice Coupling and Magneto-dielectricity in Ca3Mn2O7 Layered Perovskite

Dielectric study on Ca3Mn2O7 features relaxor-like segmented dynamics below the antiferromagnetic ordering. Dipolar relaxations of different origin are spectrally resolved exhibiting distinct H-field alterations. This identifies their allegiance to different magnetic sub-phases and establishes dual coupling of electrical, magnetic, and structural degrees of freedom. Further, strong spin-lattice coupling has been affirmed with Raman spectroscopy across the magnetic ordering. Short-range electrical correlations collaterally cause measurable harmonic dielectric response in the system. The \c{hi}_3^e-susceptibility signal yields genuine harmonic magneto-dielectricity, consistent with but exhibiting two orders of magnitude larger H-field effect, vis-à-vis that obtained in the fundamental dielectric constant ε'.

cond-mat.mtrl-sci↗

arXiv2019

Raman spectroscopic investigation of relaxor behavior in Pr doped SrTiO3 and Origin of Fano resonance

Detailed Raman spectroscopy studies on polycrystalline Sr1-xPrxTiO3 (x=0.01, 0.025, 0.05, 0.075, 0.09, 0.13, 0.15, 0.17) samples are reported elucidating the microscopic mechanism of relaxor ferroelectrics. The polar mode was observed upto very high temperature ~1000 K suggesting that the dipoles exists at temperatures well above the characteristic relaxor temperatures and they develop a short range correlation at 505 K leading to formation of PNRs. The TO2 polar mode showed anomalous softening in cooling below T~505 K supporting the growth of PNRs. Fano resonance is reported in the Pr doped compounds which decreases with increasing doping. Our work on Pr doped SrTiO3 reveals that the local TiO6 octahedral tilt scales with the intensity of the polar mode and hence can be used as an order parameter for relaxor transition. The study showed that the paraelectric to relaxor ferroelectric phase transition in this compound is random polarizability instability driven phenomenon which is correlated with the local octahedral tilt angle. The study supports competition and cancelation between lattice and polar instabilities at global length scale while cooperation between the two at local length scale. The modulation in local structure of the material in temperature interval related to dielectric anomaly has been observed which is not been investigated previously by any structural tool.

cond-mat.mtrl-sci↗

arXiv2019

Phonon Invisibility Driven by Robust Magneto-Elastic Coupling in AlFeO3 Thin Film

The thin films of lead free magneto-electric compound AlFeO3 have been deposited using pulsed laser deposition technique. X-ray diffraction, X-ray absorption spectroscopy and reflectivity measurements established the orthorhombic structure and material density of 4.5g/cc which is comparable with bulk AlFeO3. The Raman mode corresponding to AlFeO3 was found to vanish when magnetic field of 800 Oe was applied at room temperature. Additionally, it was observed that the Raman phonon mode present at the room temperature becomes invisible in the temperature window of 280K-236K and reappears below it. The detailed analysis of magnetization showed a change in magnetic order in this temperature interval. The invisibility of Raman phonon mode corresponding to AlFeO3 have been attributed to the lattice deformation caused by the magnetoelastic effect. The presence of strong spin-lattice coupling is also validated by the renormalization of phonon frequencies below 200 K.

cond-mat.mtrl-sci↗

arXiv2019

Strain Healing of Spin-Orbit Coupling: A Cause for Enhanced Magnetic Moment in Epitaxial SrRuO3 Thin Films

Enhanced magnetic moment and coercivity in SrRuO3 thin films are significant issues for advanced technological usages and hence are researched extensively in recent times. Most of the previous reports on thin films with enhanced magnetic moment attributed the high spin state for the enhancement. Our magnetization results show high magnetic moment of 3.3 Bohr-magnetron/Ru ion in the epitaxial thin films grown on LSAT substrate against 1.2 Bohr-magnetron/Ru ion observed in bulk compound. Contrary to the expectation the Ru ions are found to be in low spin state and the orbital moment is shown to be contributing significantly in the enhancement of magnetic moment. We employed x-ray absorption spectroscopy and resonant valance band spectroscopy to probe the spin state and orbital contributions in these films. The existence of strong spin-orbit coupling responsible for the de-quenching of the 4d orbitals is confirmed by the observation of the non-statistical large branching ratio at the Ru M2,3 absorption edges. The relaxation of orbital quenching by strain engineering provides a new tool for enhancing magnetic moment. Strain disorder is shown to be an efficient mean to control the spin-orbit coupling.

cond-mat.str-el↗

arXiv2018

A Generative Model for Dynamic Networks with Applications

Networks observed in real world like social networks, collaboration networks etc., exhibit temporal dynamics, i.e. nodes and edges appear and/or disappear over time. In this paper, we propose a generative, latent space based, statistical model for such networks (called dynamic networks). We consider the case where the number of nodes is fixed, but the presence of edges can vary over time. Our model allows the number of communities in the network to be different at different time steps. We use a neural network based methodology to perform approximate inference in the proposed model and its simplified version. Experiments done on synthetic and real world networks for the task of community detection and link prediction demonstrate the utility and effectiveness of our model as compared to other similar existing approaches.

cs.SI↗

arXiv2018

Spin-lattice coupling mediated giant magnetodielectricity across the spin reorientation in Ca2FeCoO5

The structural, phonon, magnetic, dielectric, and magneto dielectric responses of the pure bulk Brownmillerite compound Ca2FeCoO5 are reported. This compound showed giant magneto dielectric response (10%-24%) induced by strong spin-lattice coupling across its spin reorientation transition (150-250 K). The role of two Debye temperatures pertaining to differently coordinated sites in the dielectric relaxations is established. The positive giant magneto-dielectricity is shown to be a direct consequence of the modulations in the lattice degrees of freedom through applied external field across the spin reorientation transition. Our study illustrates novel control of magneto-dielectricity by tuning the spin reorientation transition in a material that possess strong spin lattice coupling.

cond-mat.mtrl-sci↗

arXiv2018

Creating A Multi-track Classical Musical Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications

We introduce a dataset for facilitating audio-visual analysis of music performances. The dataset comprises 44 simple multi-instrument classical music pieces assembled from coordinated but separately recorded performances of individual tracks. For each piece, we provide the musical score in MIDI format, the audio recordings of the individual tracks, the audio and video recording of the assembled mixture, and ground-truth annotation files including frame-level and note-level transcriptions. We describe our methodology for the creation of the dataset, particularly highlighting our approaches for addressing the challenges involved in maintaining synchronization and expressiveness. We demonstrate the high quality of synchronization achieved with our proposed approach by comparing the dataset with existing widely-used music audio datasets. We anticipate that the dataset will be useful for the development and evaluation of existing music information retrieval (MIR) tasks, as well as for novel multi-modal tasks. We benchmark two existing MIR tasks (multi-pitch analysis and score-informed source separation) on the dataset and compare with other existing music audio datasets. Additionally, we consider two novel multi-modal MIR tasks (visually informed multi-pitch analysis and polyphonic vibrato analysis) enabled by the dataset and provide evaluation measures and baseline systems for future comparisons (from our recent work). Finally, we propose several emerging research directions that the dataset enables.

cs.MM↗

arXiv2018

Zero-Shot Object Detection

We introduce and tackle the problem of zero-shot object detection (ZSD), which aims to detect object classes which are not observed during training. We work with a challenging set of object classes, not restricting ourselves to similar and/or fine-grained categories as in prior works on zero-shot classification. We present a principled approach by first adapting visual-semantic embeddings for ZSD. We then discuss the problems associated with selecting a background class and motivate two background-aware approaches for learning robust detectors. One of these models uses a fixed background class and the other is based on iterative latent assignments. We also outline the challenge associated with using a limited number of training classes and propose a solution based on dense sampling of the semantic label space using auxiliary data with a large number of categories. We propose novel splits of two standard detection datasets - MSCOCO and VisualGenome, and present extensive empirical results in both the traditional and generalized zero-shot settings to highlight the benefits of the proposed methods. We provide useful insights into the algorithm and conclude by posing some open questions to encourage further research.

cs.CV↗

arXiv2018

Unsupervised Learning of Face Representations

We present an approach for unsupervised training of CNNs in order to learn discriminative face representations. We mine supervised training data by noting that multiple faces in the same video frame must belong to different persons and the same face tracked across multiple frames must belong to the same person. We obtain millions of face pairs from hundreds of videos without using any manual supervision. Although faces extracted from videos have a lower spatial resolution than those which are available as part of standard supervised face datasets such as LFW and CASIA-WebFace, the former represent a much more realistic setting, e.g. in surveillance scenarios where most of the faces detected are very small. We train our CNNs with the relatively low resolution faces extracted from video frames collected, and achieve a higher verification accuracy on the benchmark LFW dataset cf. hand-crafted features such as LBPs, and even surpasses the performance of state-of-the-art deep networks such as VGG-Face, when they are made to work with low resolution input images.

cs.CV↗

arXiv2018

Large Scale Novel Object Discovery in 3D

We present a method for discovering never-seen-before objects in 3D point clouds obtained from sensors like Microsoft Kinect. We generate supervoxels directly from the point cloud data and use them with a Siamese network, built on a recently proposed 3D convolutional neural network architecture. We use known objects to train a non-linear embedding of supervoxels, by optimizing the criteria that supervoxels which fall on the same object should be closer than those which fall on different objects, in the embedding space. We test on unknown objects, which were not seen during training, and perform clustering in the learned embedding space of supervoxels to effectively perform novel object discovery. We validate the method with extensive experiments, quantitatively showing that it can discover numerous unseen objects while being trained on only a few dense 3D models. We also show very good qualitative results of object discovery in point cloud data when the test objects, either specific instances or even categories, were never seen during training.

cs.CV↗

arXiv2017

Vehicle Tracking in Wide Area Motion Imagery via Stochastic Progressive Association Across Multiple Frames (SPAAM)

Vehicle tracking in Wide Area Motion Imagery (WAMI) relies on associating vehicle detections across multiple WAMI frames to form tracks corresponding to individual vehicles. The temporal window length, i.e., the number $M$ of sequential frames, over which associations are collectively estimated poses a trade-off between accuracy and computational complexity. A larger $M$ improves performance because the increased temporal context enables the use of motion models and allows occlusions and spurious detections to be handled better. The number of total hypotheses tracks, on the other hand, grows exponentially with increasing $M$, making larger values of $M$ computationally challenging to tackle. In this paper, we introduce SPAAM an iterative approach that progressively grows $M$ with each iteration to improve estimated tracks by exploiting the enlarged temporal context while keeping computation manageable through two novel approaches for pruning association hypotheses. First, guided by a road network, accurately co-registered to the WAMI frames, we disregard unlikely associations that do not agree with the road network. Second, as $M$ is progressively enlarged at each iteration, the related increase in association hypotheses is limited by revisiting only the subset of association possibilities rendered open by stochastically determined dis-associations for the previous iteration. The stochastic dis-association at each iteration maintains each estimated association according to an estimated probability for confidence, obtained via a probabilistic model. Associations at each iteration are then estimated globally over the $M$ frames by (approximately) solving a binary integer programming problem for selecting a set of compatible tracks. Vehicle tracking results obtained over test WAMI datasets indicate that our proposed approach provides significant performance improvements over 3 alternatives.

cs.CV↗

arXiv2017

Discriminatively Trained Latent Ordinal Model for Video Classification

We study the problem of video classification for facial analysis and human action recognition. We propose a novel weakly supervised learning method that models the video as a sequence of automatically mined, discriminative sub-events (eg. onset and offset phase for "smile", running and jumping for "highjump"). The proposed model is inspired by the recent works on Multiple Instance Learning and latent SVM/HCRF -- it extends such frameworks to model the ordinal aspect in the videos, approximately. We obtain consistent improvements over relevant competitive baselines on four challenging and publicly available video based facial analysis datasets for prediction of expression, clinical pain and intent in dyadic conversations and on three challenging human action datasets. We also validate the method with qualitative results and show that they largely support the intuitions behind the method.

cs.CV↗

arXiv2017

Multichannel Attention Network for Analyzing Visual Behavior in Public Speaking

Public speaking is an important aspect of human communication and interaction. The majority of computational work on public speaking concentrates on analyzing the spoken content, and the verbal behavior of the speakers. While the success of public speaking largely depends on the content of the talk, and the verbal behavior, non-verbal (visual) cues, such as gestures and physical appearance also play a significant role. This paper investigates the importance of visual cues by estimating their contribution towards predicting the popularity of a public lecture. For this purpose, we constructed a large database of more than $1800$ TED talk videos. As a measure of popularity of the TED talks, we leverage the corresponding (online) viewers' ratings from YouTube. Visual cues related to facial and physical appearance, facial expressions, and pose variations are extracted from the video frames using convolutional neural network (CNN) models. Thereafter, an attention-based long short-term memory (LSTM) network is proposed to predict the video popularity from the sequence of visual features. The proposed network achieves state-of-the-art prediction accuracy indicating that visual cues alone contain highly predictive information about the popularity of a talk. Furthermore, our network learns a human-like attention mechanism, which is particularly useful for interpretability, i.e. how attention varies with time, and across different visual cues by indicating their relative importance.

cs.MM↗

arXiv2017

AdaScan: Adaptive Scan Pooling in Deep Convolutional Neural Networks for Human Action Recognition in Videos

We propose a novel method for temporally pooling frames in a video for the task of human action recognition. The method is motivated by the observation that there are only a small number of frames which, together, contain sufficient information to discriminate an action class present in a video, from the rest. The proposed method learns to pool such discriminative and informative frames, while discarding a majority of the non-informative frames in a single temporal scan of the video. Our algorithm does so by continuously predicting the discriminative importance of each video frame and subsequently pooling them in a deep learning framework. We show the effectiveness of our proposed pooling method on standard benchmarks where it consistently improves on baseline pooling methods, with both RGB and optical flow based Convolutional networks. Further, in combination with complementary video representations, we show results that are competitive with respect to the state-of-the-art results on two challenging and publicly available benchmark datasets.

cs.CV↗

arXiv2017

An Empirical Evaluation of Visual Question Answering for Novel Objects

We study the problem of answering questions about images in the harder setting, where the test questions and corresponding images contain novel objects, which were not queried about in the training data. Such setting is inevitable in real world-owing to the heavy tailed distribution of the visual categories, there would be some objects which would not be annotated in the train set. We show that the performance of two popular existing methods drop significantly (up to 28%) when evaluated on novel objects cf. known objects. We propose methods which use large existing external corpora of (i) unlabeled text, i.e. books, and (ii) images tagged with classes, to achieve novel object based visual question answering. We do systematic empirical studies, for both an oracle case where the novel objects are known textually, as well as a fully automatic case without any explicit knowledge of the novel objects, but with the minimal assumption that the novel objects are semantically related to the existing objects in training. The proposed methods for novel object based visual question answering are modular and can potentially be used with many visual question answering architectures. We show consistent improvements with the two popular architectures and give qualitative analysis of the cases where the model does well and of those where it fails to bring improvements.

cs.CV↗

arXiv2016

Deep fusion of visual signatures for client-server facial analysis

Facial analysis is a key technology for enabling human-machine interaction. In this context, we present a client-server framework, where a client transmits the signature of a face to be analyzed to the server, and, in return, the server sends back various information describing the face e.g. is the person male or female, is she/he bald, does he have a mustache, etc. We assume that a client can compute one (or a combination) of visual features; from very simple and efficient features, like Local Binary Patterns, to more complex and computationally heavy, like Fisher Vectors and CNN based, depending on the computing resources available. The challenge addressed in this paper is to design a common universal representation such that a single merged signature is transmitted to the server, whatever be the type and number of features computed by the client, ensuring nonetheless an optimal performance. Our solution is based on learning of a common optimal subspace for aligning the different face features and merging them into a universal signature. We have validated the proposed method on the challenging CelebA dataset, on which our method outperforms existing state-of-the-art methods when rich representation is available at test time, while giving competitive performance when only simple signatures (like LBP) are available at test time due to resource constraints on the client.

cs.CV↗

arXiv2016

CP-mtML: Coupled Projection multi-task Metric Learning for Large Scale Face Retrieval

We propose a novel Coupled Projection multi-task Metric Learning (CP-mtML) method for large scale face retrieval. In contrast to previous works which were limited to low dimensional features and small datasets, the proposed method scales to large datasets with high dimensional face descriptors. It utilises pairwise (dis-)similarity constraints as supervision and hence does not require exhaustive class annotation for every training image. While, traditionally, multi-task learning methods have been validated on same dataset but different tasks, we work on the more challenging setting with heterogeneous datasets and different tasks. We show empirical validation on multiple face image datasets of different facial traits, e.g. identity, age and expression. We use classic Local Binary Pattern (LBP) descriptors along with the recent Deep Convolutional Neural Network (CNN) features. The experiments clearly demonstrate the scalability and improved performance of the proposed method on the tasks of identity and age based face image retrieval compared to competitive existing methods, on the standard datasets and with the presence of a million distractor face images.

cs.CV↗