arXiv subjects

Mohit Gupta

Publications and source records attributed to Mohit Gupta.

At least 55 records · Page 3Linked to original sources

arXiv2023

Unlocking the Performance of Proximity Sensors by Utilizing Transient Histograms

We provide methods which recover planar scene geometry by utilizing the transient histograms captured by a class of close-range time-of-flight (ToF) distance sensor. A transient histogram is a one dimensional temporal waveform which encodes the arrival time of photons incident on the ToF sensor. Typically, a sensor processes the transient histogram using a proprietary algorithm to produce distance estimates, which are commonly used in several robotics applications. Our methods utilize the transient histogram directly to enable recovery of planar geometry more accurately than is possible using only proprietary distance estimates, and consistent recovery of the albedo of the planar surface, which is not possible with proprietary distance estimates alone. This is accomplished via a differentiable rendering pipeline, which simulates the transient imaging process, allowing direct optimization of scene geometry to match observations. To validate our methods, we capture 3,800 measurements of eight planar surfaces from a wide range of viewpoints, and show that our method outperforms the proprietary-distance-estimate baseline by an order of magnitude in most scenarios. We demonstrate a simple robotics application which uses our method to sense the distance to and slope of a planar surface from a sensor mounted on the end effector of a robot arm.

cs.RO↗

arXiv2023

Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformers

Vision Transformers achieve impressive accuracy across a range of visual recognition tasks. Unfortunately, their accuracy frequently comes with high computational costs. This is a particular issue in video recognition, where models are often applied repeatedly across frames or temporal chunks. In this work, we exploit temporal redundancy between subsequent inputs to reduce the cost of Transformers for video processing. We describe a method for identifying and re-processing only those tokens that have changed significantly over time. Our proposed family of models, Eventful Transformers, can be converted from existing Transformers (often without any re-training) and give adaptive control over the compute cost at runtime. We evaluate our method on large-scale datasets for video object detection (ImageNet VID) and action recognition (EPIC-Kitchens 100). Our approach leads to significant computational savings (on the order of 2-4x) with only minor reductions in accuracy.

cs.CV↗

arXiv2023

Gate-tunable Superconducting Diode Effect in a Three-terminal Josephson Device

The phenomenon of non-reciprocal critical current in a Josephson device, termed the Josephson diode effect, has garnered much recent interest. Realization of the diode effect requires inversion symmetry breaking, typically obtained by spin-orbit interactions. Here we report observation of the Josephson diode effect in a three-terminal Josephson device based upon an InAs quantum well two-dimensional electron gas proximitized by an epitaxial aluminum superconducting layer. We demonstrate that the diode efficiency in our devices can be tuned by a small out-of-plane magnetic field or by electrostatic gating. We show that the Josephson diode effect in these devices is a consequence of the artificial realization of a current-phase relation that contains higher harmonics. We also show nonlinear DC intermodulation and simultaneous two-signal rectification, enabled by the multi-terminal nature of the devices. Furthermore, we show that the diode effect is an inherent property of multi-terminal Josephson devices, establishing an immediately scalable approach by which potential applications of the Josephson diode effect can be realized, agnostic to the underlying material platform. These Josephson devices may also serve as gate-tunable building blocks in designing topologically protected qubits.

cond-mat.mes-hall↗

arXiv2023

Diffusive and Ballistic Transport in Ultra-thin InSb Nanowire Devices Using a Few-layer-Graphene-AlOx Gate

Quantum devices based on InSb nanowires (NWs) are a prime candidate system for realizing and exploring topologically-protected quantum states and for electrically-controlled spin-based qubits. The influence of disorder on achieving reliable topological regimes has been studied theoretically, highlighting the importance of optimizing both growth and nanofabrication. In this work we investigate both aspects. We developed InSb nanowires with ultra-thin diameters, as well as a new gating approach, involving few-layer graphene (FLG) and Atomic Layer Deposition (ALD)-grown AlOx. Low-temperature electronic transport measurements of these devices reveal conductance plateaus and Fabry-Pérot interference, evidencing phase-coherent transport in the regime of few quantum modes. The approaches developed in this work could help mitigate the role of material and fabrication-induced disorder in semiconductor-based quantum devices.

cond-mat.mes-hall↗

arXiv2023

Field-Free Spin-Orbit Torque driven Switching of Perpendicular Magnetic Tunnel Junction through Bending Current

Current-induced spin-orbit torques (SOTs) enable fast and efficient manipulation of the magnetic state of magnetic tunnel junctions (MTJs), making it attractive for memory, in-memory computing, and logic applications. However, the requirement of the external magnetic field to achieve deterministic switching in perpendicular magnetized SOT-MTJs limits its implementation for practical applications. Here, we introduce a field-free switching (FFS) solution for the SOT-MTJ device by shaping the SOT channel to create a "bend" in the SOT current. The resulting bend in the charge current creates a spatially non-uniform spin current, which translates into inhomogeneous SOT on an adjacent magnetic free layer enabling deterministic switching. We demonstrate FFS experimentally on scaled SOT-MTJs at nanosecond time scales. This proposed scheme is scalable, material-agnostic, and readily compatible with wafer-scale manufacturing, thus creating a pathway for developing purely current-driven SOT systems.

cond-mat.mes-hall↗

arXiv2022

A Soft SIMD Based Energy Efficient Computing Microarchitecture

The ever-increasing size and computational complexity of today's machine-learning algorithms pose an increasing strain on the underlying hardware. In this light, novel and dedicated architectural solutions are required to optimize energy efficiency by leveraging opportunities (such as intrinsic parallelism and robustness to quantization errors) exposed by algorithms. We herein address this challenge by introducing a flexible two-stages computing pipeline. The pipeline can support fine-grained operand quantization through software-supported Single Instruction Multiple Data (SIMD) operations. Moreover, it can efficiently execute sequential multiplications over SIMD sub-words thanks to zero-skipping and Canonical Signed Digit (CSD) coding. Finally, a lightweight repacking unit allows changing the bitwidth of sub-words at run-time dynamically. These features are implemented within a tight energy and area budget. Indeed, experimental results showcase that our approach greatly outperforms traditional hardware SIMD ones both in terms of area and energy requirements. In particular, our pipeline occupies up to 53.1% smaller than a hardware SIMD one supporting the same sub-word widths, while performing multiplication up to 88.8% more efficiently.

cs.AR↗

arXiv2022

Turbulent route to two-dimensional soft crystals

We investigate the effects of a two-dimensional, incompressible, turbulent flow on mono-disperse soft granular particles and show the emergence of a crystalline phase due to the interplay of Stokesian drag (measured through the Stokes number) and short-range inter-particle interactions. We quantify this phase through the bond order parameter and local density fluctuations and find a sharp transition between the crystalline and non-crystalline phase as a function of the Stokes number. Furthermore, the nature of preferential concentration, as characterised by the radial distribution function and the correlation dimension $D_2$, is significantly different from that of particle-laden flows in the absence of repulsive potentials.

physics.flu-dyn↗

arXiv2022

3D Scene Inference from Transient Histograms

Time-resolved image sensors that capture light at pico-to-nanosecond timescales were once limited to niche applications but are now rapidly becoming mainstream in consumer devices. We propose low-cost and low-power imaging modalities that capture scene information from minimal time-resolved image sensors with as few as one pixel. The key idea is to flood illuminate large scene patches (or the entire scene) with a pulsed light source and measure the time-resolved reflected light by integrating over the entire illuminated area. The one-dimensional measured temporal waveform, called \emph{transient}, encodes both distances and albedoes at all visible scene points and as such is an aggregate proxy for the scene's 3D geometry. We explore the viability and limitations of the transient waveforms by themselves for recovering scene information, and also when combined with traditional RGB cameras. We show that plane estimation can be performed from a single transient and that using only a few more it is possible to recover a depth map of the whole scene. We also show two proof-of-concept hardware prototypes that demonstrate the feasibility of our approach for compact, mobile, and budget-limited applications.

cs.CV↗

arXiv2022

Selective Control of Conductance Modes in Multi-terminal Josephson Junctions

The Andreev bound state spectra of multi-terminal Josephson junctions form an artificial band structure, which is predicted to host tunable topological phases under certain conditions. However, the number of conductance modes between the terminals of multi-terminal Josephson junction must be few in order for this spectrum to be experimentally accessible. In this work we employ a quantum point contact geometry in three-terminal Josephson devices. We demonstrate independent control of conductance modes between each pair of terminals and access to the single-mode regime coexistent with the presence of superconducting coupling. These results establish a full platform on which to realize tunable Andreev bound state spectra in multi-terminal Josephson junctions.

cond-mat.mes-hall↗

arXiv2022

Event Neural Networks

Video data is often repetitive; for example, the contents of adjacent frames are usually strongly correlated. Such redundancy occurs at multiple levels of complexity, from low-level pixel values to textures and high-level semantics. We propose Event Neural Networks (EvNets), which leverage this redundancy to achieve considerable computation savings during video inference. A defining characteristic of EvNets is that each neuron has state variables that provide it with long-term memory, which allows low-cost, high-accuracy inference even in the presence of significant camera motion. We show that it is possible to transform a wide range of neural networks into EvNets without re-training. We demonstrate our method on state-of-the-art architectures for both high- and low-level visual processing, including pose recognition, object detection, optical flow, and image enhancement. We observe roughly an order-of-magnitude reduction in computational costs compared to conventional networks, with minimal reductions in model accuracy.

cs.CV↗

arXiv2022

Robust Scene Inference under Noise-Blur Dual Corruptions

Scene inference under low-light is a challenging problem due to severe noise in the captured images. One way to reduce noise is to use longer exposure during the capture. However, in the presence of motion (scene or camera motion), longer exposures lead to motion blur, resulting in loss of image information. This creates a trade-off between these two kinds of image degradations: motion blur (due to long exposure) vs. noise (due to short exposure), also referred as a dual image corruption pair in this paper. With the rise of cameras capable of capturing multiple exposures of the same scene simultaneously, it is possible to overcome this trade-off. Our key observation is that although the amount and nature of degradation varies for these different image captures, the semantic content remains the same across all images. To this end, we propose a method to leverage these multi exposure captures for robust inference under low-light and motion. Our method builds on a feature consistency loss to encourage similar results from these individual captures, and uses the ensemble of their final predictions for robust visual recognition. We demonstrate the effectiveness of our approach on simulated images as well as real captures with multiple exposures, and across the tasks of object detection and image classification.

cs.CV↗

arXiv2022

Single-Photon Structured Light

We present a novel structured light technique that uses Single Photon Avalanche Diode (SPAD) arrays to enable 3D scanning at high-frame rates and low-light levels. This technique, called "Single-Photon Structured Light", works by sensing binary images that indicates the presence or absence of photon arrivals during each exposure; the SPAD array is used in conjunction with a high-speed binary projector, with both devices operated at speeds as high as 20~kHz. The binary images that we acquire are heavily influenced by photon noise and are easily corrupted by ambient sources of light. To address this, we develop novel temporal sequences using error correction codes that are designed to be robust to short-range effects like projector and camera defocus as well as resolution mismatch between the two devices. Our lab prototype is capable of 3D imaging in challenging scenarios involving objects with extremely low albedo or undergoing fast motion, as well as scenes under strong ambient illumination.

cs.CV↗

arXiv2022

Banana Sub-Family Classification and Quality Prediction using Computer Vision

India is the second largest producer of fruits and vegetables in the world, and one of the largest consumers of fruits like Banana, Papaya and Mangoes through retail and ecommerce giants like BigBasket, Grofers and Amazon Fresh. However, adoption of technology in supply chain and retail stores is still low and there is a great potential to adopt computer-vision based technology for identification and classification of fruits. We have chosen banana fruit to build a computer vision based model to carry out the following three use-cases (a) Identify Banana from a given image (b) Determine sub-family or variety of Banana (c) Determine the quality of Banana. Successful execution of these use-cases using computer-vision model would greatly help with overall inventory management automation, quality control, quick and efficient weighing and billing which all are manual labor intensive currently. In this work, we suggest a machine learning pipeline that combines the ideas of CNNs, transfer learning, and data augmentation towards improving Banana fruit sub family and quality image classification. We have built a basic CNN and then went on to tune a MobileNet Banana classification model using a combination of self-curated and publicly-available dataset of 3064 images. The results show an overall 93.4% and 100% accuracy for sub-family/variety and for quality test classifications respectively.

cs.CV↗

arXiv2022

Elastic wave manipulation in geodesic lenses

We investigate wave propagation in curved, thin elastic waveguides, where curvature is shown to be equivalent to a spatially modulated refractive index. We establish this relationship within a theoretical framework that leverages non-Euclidean transformations. Theoretical predictions illustrate how curved profiles can be designed to manipulate wave trajectories in analogy with geometrical optics principles. Experiments on 3D printed curved membranes confirm the theoretical predictions, and suggest curvature as a viable approach for guiding, focusing and steering of elastic waves.

physics.app-ph↗

arXiv2021

iToF2dToF: A Robust and Flexible Representation for Data-Driven Time-of-Flight Imaging

Indirect Time-of-Flight (iToF) cameras are a promising depth sensing technology. However, they are prone to errors caused by multi-path interference (MPI) and low signal-to-noise ratio (SNR). Traditional methods, after denoising, mitigate MPI by estimating a transient image that encodes depths. Recently, data-driven methods that jointly denoise and mitigate MPI have become state-of-the-art without using the intermediate transient representation. In this paper, we propose to revisit the transient representation. Using data-driven priors, we interpolate/extrapolate iToF frequencies and use them to estimate the transient image. Given direct ToF (dToF) sensors capture transient images, we name our method iToF2dToF. The transient representation is flexible. It can be integrated with different rule-based depth sensing algorithms that are robust to low SNR and can deal with ambiguous scenarios that arise in practice (e.g., specular MPI, optical cross-talk). We demonstrate the benefits of iToF2dToF over previous methods in real depth sensing scenarios.

cs.CV↗

arXiv2021

Photon-Starved Scene Inference using Single Photon Cameras

Scene understanding under low-light conditions is a challenging problem. This is due to the small number of photons captured by the camera and the resulting low signal-to-noise ratio (SNR). Single-photon cameras (SPCs) are an emerging sensing modality that are capable of capturing images with high sensitivity. Despite having minimal read-noise, images captured by SPCs in photon-starved conditions still suffer from strong shot noise, preventing reliable scene inference. We propose photon scale-space a collection of high-SNR images spanning a wide range of photons-per-pixel (PPP) levels (but same scene content) as guides to train inference model on low photon flux images. We develop training techniques that push images with different illumination levels closer to each other in feature representation space. The key idea is that having a spectrum of different brightness levels during training enables effective guidance, and increases robustness to shot noise even in extreme noise cases. Based on the proposed approach, we demonstrate, via simulations and real experiments with a SPAD camera, high-performance on various inference tasks such as image classification and monocular depth estimation under ultra low-light, down to < 1 PPP.

eess.IV↗

arXiv2021

Music Generation using Three-layered LSTM

This paper explores the idea of utilising Long Short-Term Memory neural networks (LSTMNN) for the generation of musical sequences in ABC notation. The proposed approach takes ABC notations from the Nottingham dataset and encodes it to be fed as input for the neural networks. The primary objective is to input the neural networks with an arbitrary note, let the network process and augment a sequence based on the note until a good piece of music is produced. Multiple calibrations have been done to amend the parameters of the network for optimal generation. The output is assessed on the basis of rhythm, harmony, and grammar accuracy.

cs.SD↗

arXiv2021

Invisible Perturbations: Physical Adversarial Examples Exploiting the Rolling Shutter Effect

Physical adversarial examples for camera-based computer vision have so far been achieved through visible artifacts -- a sticker on a Stop sign, colorful borders around eyeglasses or a 3D printed object with a colorful texture. An implicit assumption here is that the perturbations must be visible so that a camera can sense them. By contrast, we contribute a procedure to generate, for the first time, physical adversarial examples that are invisible to human eyes. Rather than modifying the victim object with visible artifacts, we modify light that illuminates the object. We demonstrate how an attacker can craft a modulated light signal that adversarially illuminates a scene and causes targeted misclassifications on a state-of-the-art ImageNet deep learning model. Concretely, we exploit the radiometric rolling shutter effect in commodity cameras to create precise striping patterns that appear on images. To human eyes, it appears like the object is illuminated, but the camera creates an image with stripes that will cause ML models to output the attacker-desired classification. We conduct a range of simulation and physical experiments with LEDs, demonstrating targeted attack rates up to 84%.

cs.CV↗