Searcharxiv⌕ Search

arXiv subjects

Avinash Sharma

Publications and source records attributed to Avinash Sharma.

35 records · Page 2Linked to original sources
arXiv2021

Attention based Occlusion Removal for Hybrid Telepresence Systems

Traditionally, video conferencing is a widely adopted solution for telecommunication, but a lack of immersiveness comes inherently due to the 2D nature of facial representation. The integration of Virtual Reality (VR) in a communication/telepresence system through Head Mounted Displays (HMDs) promises to provide users a much better immersive experience. However, HMDs cause hindrance by blocking the facial appearance and expressions of the user. To overcome these issues, we propose a novel attention-enabled encoder-decoder architecture for HMD de-occlusion. We also propose to train our person-specific model using short videos (1-2 minutes) of the user, captured in varying appearances, and demonstrated generalization to unseen poses and appearances of the user. We report superior qualitative and quantitative results over state-of-the-art methods. We also present applications of this approach to hybrid video teleconferencing using existing animation and 3D face reconstruction pipelines.

cs.CV↗
arXiv2021

Robust 3D Garment Digitization from Monocular 2D Images for 3D Virtual Try-On Systems

In this paper, we develop a robust 3D garment digitization solution that can generalize well on real-world fashion catalog images with cloth texture occlusions and large body pose variations. We assumed fixed topology parametric template mesh models for known types of garments (e.g., T-shirts, Trousers) and perform mapping of high-quality texture from an input catalog image to UV map panels corresponding to the parametric mesh model of the garment. We achieve this by first predicting a sparse set of 2D landmarks on the boundary of the garments. Subsequently, we use these landmarks to perform Thin-Plate-Spline-based texture transfer on UV map panels. Subsequently, we employ a deep texture inpainting network to fill the large holes (due to view variations & self-occlusions) in TPS output to generate consistent UV maps. Furthermore, to train the supervised deep networks for landmark prediction & texture inpainting tasks, we generated a large set of synthetic data with varying texture and lighting imaged from various views with the human present in a wide variety of poses. Additionally, we manually annotated a small set of fashion catalog images crawled from online fashion e-commerce platforms to finetune. We conduct thorough empirical evaluations and show impressive qualitative results of our proposed 3D garment texture solution on fashion catalog images. Such 3D garment digitization helps us solve the challenging task of enabling 3D Virtual Try-on.

cs.CV↗
arXiv2021

Reconstruction of People in Loose Clothing

3D human body reconstruction from monocular images is an interesting and ill-posed problem in computer vision with wider applications in multiple domains. In this paper, we propose SHARP, a novel end-to-end trainable network that accurately recovers the detailed geometry and appearance of 3D people in loose clothing from a monocular image. We propose a sparse and efficient fusion of a parametric body prior with a non-parametric peeled depth map representation of clothed models. The parametric body prior constraints our model in two ways: first, the network retains geometrically consistent body parts that are not occluded by clothing, and second, it provides a body shape context that improves prediction of the peeled depth maps. This enables SHARP to recover fine-grained 3D geometrical details with just L1 losses on the 2D maps, given an input image. We evaluate SHARP on publicly available Cloth3D and THuman datasets and report superior performance to state-of-the-art approaches.

cs.CV↗
arXiv2021

3D Shape Registration Using Spectral Graph Embedding and Probabilistic Matching

We address the problem of 3D shape registration and we propose a novel technique based on spectral graph theory and probabilistic matching. The task of 3D shape analysis involves tracking, recognition, registration, etc. Analyzing 3D data in a single framework is still a challenging task considering the large variability of the data gathered with different acquisition devices. 3D shape registration is one such challenging shape analysis task. The main contribution of this chapter is to extend the spectral graph matching methods to very large graphs by combining spectral graph matching with Laplacian embedding. Since the embedded representation of a graph is obtained by dimensionality reduction we claim that the existing spectral-based methods are not easily applicable. We discuss solutions for the exact and inexact graph isomorphism problems and recall the main spectral properties of the combinatorial graph Laplacian; We provide a novel analysis of the commute-time embedding that allows us to interpret the latter in terms of the PCA of a graph, and to select the appropriate dimension of the associated embedded metric space; We derive a unit hyper-sphere normalization for the commute-time embedding that allows us to register two shapes with different samplings; We propose a novel method to find the eigenvalue-eigenvector ordering and the eigenvector signs using the eigensignature (histogram) which is invariant to the isometric shape deformations and fits well in the spectral graph matching framework, and we present a probabilistic shape matching formulation using an expectation maximization point registration algorithm which alternates between aligning the eigenbases and finding a vertex-to-vertex assignment.

cs.CV↗
arXiv2020

GlocalNet: Class-aware Long-term Human Motion Synthesis

Synthesis of long-term human motion skeleton sequences is essential to aid human-centric video generation with potential applications in Augmented Reality, 3D character animations, pedestrian trajectory prediction, etc. Long-term human motion synthesis is a challenging task due to multiple factors like, long-term temporal dependencies among poses, cyclic repetition across poses, bi-directional and multi-scale dependencies among poses, variable speed of actions, and a large as well as partially overlapping space of temporal pose variations across multiple class/types of human activities. This paper aims to address these challenges to synthesize a long-term (> 6000 ms) human motion trajectory across a large variety of human activity classes (>50). We propose a two-stage activity generation method to achieve this goal, where the first stage deals with learning the long-term global pose dependencies in activity sequences by learning to synthesize a sparse motion trajectory while the second stage addresses the generation of dense motion trajectories taking the output of the first stage. We demonstrate the superiority of the proposed method over SOTA methods using various quantitative evaluation metrics on publicly available datasets.

cs.CV↗
arXiv2020

PeeledHuman: Robust Shape Representation for Textured 3D Human Body Reconstruction

We introduce PeeledHuman - a novel shape representation of the human body that is robust to self-occlusions. PeeledHuman encodes the human body as a set of Peeled Depth and RGB maps in 2D, obtained by performing ray-tracing on the 3D body model and extending each ray beyond its first intersection. This formulation allows us to handle self-occlusions efficiently compared to other representations. Given a monocular RGB image, we learn these Peeled maps in an end-to-end generative adversarial fashion using our novel framework - PeelGAN. We train PeelGAN using a 3D Chamfer loss and other 2D losses to generate multiple depth values per-pixel and a corresponding RGB field per-vertex in a dual-branch setup. In our simple non-parametric solution, the generated Peeled Depth maps are back-projected to 3D space to obtain a complete textured 3D shape. The corresponding RGB maps provide vertex-level texture details. We compare our method with current parametric and non-parametric methods in 3D reconstruction and find that we achieve state-of-the-art-results. We demonstrate the effectiveness of our representation on publicly available BUFF and MonoPerfCap datasets as well as loose clothing data collected by our calibrated multi-Kinect setup.

cs.CV↗
arXiv2020

AFN: Attentional Feedback Network based 3D Terrain Super-Resolution

Terrain, representing features of an earth surface, plays a crucial role in many applications such as simulations, route planning, analysis of surface dynamics, computer graphics-based games, entertainment, films, to name a few. With recent advancements in digital technology, these applications demand the presence of high-resolution details in the terrain. In this paper, we propose a novel fully convolutional neural network-based super-resolution architecture to increase the resolution of low-resolution Digital Elevation Model (LRDEM) with the help of information extracted from the corresponding aerial image as a complementary modality. We perform the super-resolution of LRDEM using an attention-based feedback mechanism named 'Attentional Feedback Network' (AFN), which selectively fuses the information from LRDEM and aerial image to enhance and infuse the high-frequency features and to produce the terrain realistically. We compare the proposed architecture with existing state-of-the-art DEM super-resolution methods and show that the proposed architecture outperforms enhancing the resolution of input LRDEM accurately and in a realistic manner.

eess.IV↗
arXiv2020

Feedback Neural Network based Super-resolution of DEM for generating high fidelity features

High resolution Digital Elevation Models(DEMs) are an important requirement for many applications like modelling water flow, landslides, avalanches etc. Yet publicly available DEMs have low resolution for most parts of the world. Despite tremendous success in image super resolution task using deep learning solutions, there are very few works that have used these powerful systems on DEMs to generate HRDEMs. Motivated from feedback neural networks, we propose a novel neural network architecture that learns to add high frequency details iteratively to low resolution DEM, turning it into a high resolution DEM without compromising its fidelity. Our experiments confirm that without any additional modality such as aerial images(RGB), our network DSRFB achieves RMSEs of 0.59 to 1.27 across 4 different datasets.

eess.IV↗
arXiv2020

Action Quality Assessment using Siamese Network-Based Deep Metric Learning

Automated vision-based score estimation models can be used as an alternate opinion to avoid judgment bias. In the past works the score estimation models were learned by regressing the video representations to the ground truth score provided by the judges. However such regression-based solutions lack interpretability in terms of giving reasons for the awarded score. One solution to make the scores more explicable is to compare the given action video with a reference video. This would capture the temporal variations w.r.t. the reference video and map those variations to the final score. In this work, we propose a new action scoring system as a two-phase system: (1) A Deep Metric Learning Module that learns similarity between any two action videos based on their ground truth scores given by the judges; (2) A Score Estimation Module that uses the first module to find the resemblance of a video to a reference video in order to give the assessment score. The proposed scoring model has been tested for Olympics Diving and Gymnastic vaults and the model outperforms the existing state-of-the-art scoring models.

cs.CV↗
arXiv2019

DeepHuMS: Deep Human Motion Signature for 3D Skeletal Sequences

3D Human Motion Indexing and Retrieval is an interesting problem due to the rise of several data-driven applications aimed at analyzing and/or re-utilizing 3D human skeletal data, such as data-driven animation, analysis of sports bio-mechanics, human surveillance etc. Spatio-temporal articulations of humans, noisy/missing data, different speeds of the same motion etc. make it challenging and several of the existing state of the art methods use hand-craft features along with optimization based or histogram based comparison in order to perform retrieval. Further, they demonstrate it only for very small datasets and few classes. We make a case for using a learned representation that should recognize the motion as well as enforce a discriminative ranking. To that end, we propose, a 3D human motion descriptor learned using a deep network. Our learned embedding is generalizable and applicable to real-world data - addressing the aforementioned challenges and further enables sub-motion searching in its embedding space using another network. Our model exploits the inter-class similarity using trajectory cues, and performs far superior in a self-supervised setting. State of the art results on all these fronts is shown on two large scale 3D human motion datasets - NTU RGB+D and HDM05.

cs.CV↗
arXiv2019

HumanMeshNet: Polygonal Mesh Recovery of Humans

3D Human Body Reconstruction from a monocular image is an important problem in computer vision with applications in virtual and augmented reality platforms, animation industry, en-commerce domain, etc. While several of the existing works formulate it as a volumetric or parametric learning with complex and indirect reliance on re-projections of the mesh, we would like to focus on implicitly learning the mesh representation. To that end, we propose a novel model, HumanMeshNet, that regresses a template mesh's vertices, as well as receives a regularization by the 3D skeletal locations in a multi-branch, multi-task setup. The image to mesh vertex regression is further regularized by the neighborhood constraint imposed by mesh topology ensuring smooth surface reconstruction. The proposed paradigm can theoretically learn local surface deformations induced by body shape variations and can therefore learn high-resolution meshes going ahead. We show comparable performance with SoA (in terms of surface and joint error) with far lesser computational complexity, modeling cost and therefore real-time reconstructions on three publicly available datasets. We also show the generalizability of the proposed paradigm for a similar task of predicting hand mesh models. Given these initial results, we would like to exploit the mesh topology in an explicit manner going ahead.

cs.CV↗
arXiv2019

Augmented Reality Prosthesis Training Setup for Motor Skill Enhancement

Adjusting to amputation can often time be difficult for the body. Post-surgery, amputees have to wait for up to several months before receiving a properly fitted prosthesis. In recent years, there has been a trend toward quantitative outcome measures. In this paper, we developed the augmented reality (AR) version of one such measure, the Prosthetic Hand Assessment Measure (PHAM). The AR version of the PHAM - HoloPHAM, offers amputees the advantage to train with pattern recognition, at their own time and convenience, pre- and post-prosthesis fitting. We provide a rigorous analysis of our system, focusing on its ability to simulate reach, grasp, and touch in AR. Similarity of motion joint dynamics for reach in physical and AR space were compared, with experiments conducted to illustrate how depth in AR is perceived. To show the effectiveness and validity of our system for prosthesis training, we conducted a 10-day study with able-bodied subjects (N = 3) to see the effect that training on the HoloPHAM had on other established functional outcome measures. A washout phase of 5 days was incorporated to observe the effect without training. Comparisons were made with standardized outcome metrics, along with the progression of kinematic variability over time. Statistically significant (p<0.05) improvements were observed between pre- and post-training stages. Our results show that AR can be an effective tool for prosthesis training with pattern recognition systems, fostering motor learning for reaching movement tasks, and paving the possibility of replacing physical training.

cs.HC↗
arXiv2018

Deep Textured 3D Reconstruction of Human Bodies

Recovering textured 3D models of non-rigid human body shapes is challenging due to self-occlusions caused by complex body poses and shapes, clothing obstructions, lack of surface texture, background clutter, sparse set of cameras with non-overlapping fields of view, etc. Further, a calibration-free environment adds additional complexity to both - reconstruction and texture recovery. In this paper, we propose a deep learning based solution for textured 3D reconstruction of human body shapes from a single view RGB image. This is achieved by first recovering the volumetric grid of the non-rigid human body given a single view RGB image followed by orthographic texture view synthesis using the respective depth projection of the reconstructed (volumetric) shape and input RGB image. We propose to co-learn the depth information readily available with affordable RGBD sensors (e.g., Kinect) while showing multiple views of the same object during the training phase. We show superior reconstruction performance in terms of quantitative and qualitative results, on both, publicly available datasets (by simulating the depth channel with virtual Kinect) as well as real RGBD data collected with our calibrated multi Kinect setup.

cs.CV↗
arXiv2013

An Approach Of Substitution Method Based On ASCII Codes In Encryption Technique

In poly alphabetic substitution the plain texts letters are enciphered differently according to their position. The name poly alphabetic suggests that there are more than one key so we have used two keys combination instead of just one, in order to produce the cipher text. We can also use three or more keys to make the enciphering process more complicated. In this paper have produced ASCII Codes of the plain text and then we have reversed it said reverse ASCII Codes and then we have generated two keys K1 is generated by addition of reverse ASCII Codes and K2 is generated by addition of ASCII Codes. Then these K1 and K2 Keys are alternatively applied on Reverse ASCII codes in order to produce cipher text. On the Destination hand Deciphering is used to produce the plain text again. Our technique generates random cipher text for the same plain text and this is the major advantage of our technique.

cs.CR↗
arXiv2012

Effects of Flavor Dependence on Weak Decays of J/psi and Υ

We give the detailed analysis of effects of flavor dependence of average transverse quark momentum inside a meson on J/psi --> P and Υ--> Bc transition form factors and two-body weak hadronic decays of J/psi and Υemploying the factorization scheme. We predict the branching ratios of semileptonic and nonleptonic weak decays of J/psi and Υmesons in Cabibbo-angle-enhanced and Cabibbo-angle-suppressed modes.

hep-ph↗
arXiv2008

Isospectral Potentials and Quarkonia Low Energy Spectra: A Possibility of simultaneous fit to Masses and leptonic decay widths"

Some of the non-relativistic QQ/bar interaction potentials and its isospectral partner-potentials have been employed to evaluate the masses and leptonic decay widths of Charmonium and Upsilon states. An attempt has been made to develop a prescription to provide a simultaneous fit to Masses and leptonic decay widths of Quarkonia. Preliminary calculation shows that the class of isospectral-potentials corresponding to: V(r)= -(alphas/r)+ar^n appears to yield the desired results. The various associated theoretical and phenomenological issues are discussed in light of recent data. The possible relation with the 'Wrinkled potentials' is also discussed.

hep-ph↗