arXiv subjects

Udayan Ganguly

Publications and source records attributed to Udayan Ganguly.

36 records · Page 2Linked to original sources

arXiv2021

Simplified Klinokinesis using Spiking Neural Networks for Resource-Constrained Navigation on the Neuromorphic Processor Loihi

C. elegans shows chemotaxis using klinokinesis where the worm senses the concentration based on a single concentration sensor to compute the concentration gradient to perform foraging through gradient ascent/descent towards the target concentration followed by contour tracking. The biomimetic implementation requires complex neurons with multiple ion channel dynamics as well as interneurons for control. While this is a key capability of autonomous robots, its implementation on energy-efficient neuromorphic hardware like Intel's Loihi requires adaptation of the network to hardware-specific constraints, which has not been achieved. In this paper, we demonstrate the adaptation of chemotaxis based on klinokinesis to Loihi by implementing necessary neuronal dynamics with only LIF neurons as well as a complete spike-based implementation of all functions e.g. Heaviside function and subtractions. Our results show that Loihi implementation is equivalent to the software counterpart on Python in terms of performance - both during foraging and contour tracking. The Loihi results are also resilient in noisy environments. Thus, we demonstrate a successful adaptation of chemotaxis on Loihi - which can now be combined with the rich array of SNN blocks for SNN based complex robotic control.

q-bio.NC↗

arXiv2021

Hardware-Friendly Synaptic Orders and Timescales in Liquid State Machines for Speech Classification

Liquid State Machines are brain inspired spiking neural networks (SNNs) with random reservoir connectivity and bio-mimetic neuronal and synaptic models. Reservoir computing networks are proposed as an alternative to deep neural networks to solve temporal classification problems. Previous studies suggest 2nd order (double exponential) synaptic waveform to be crucial for achieving high accuracy for TI-46 spoken digits recognition. The proposal of long-time range (ms) bio-mimetic synaptic waveforms is a challenge to compact and power efficient neuromorphic hardware. In this work, we analyze the role of synaptic orders namely: δ (high output for single time step), 0th (rectangular with a finite pulse width), 1st (exponential fall) and 2nd order (exponential rise and fall) and synaptic timescales on the reservoir output response and on the TI-46 spoken digits classification accuracy under a more comprehensive parameter sweep. We find the optimal operating point to be correlated to an optimal range of spiking activity in the reservoir. Further, the proposed 0th order synapses perform at par with the biologically plausible 2nd order synapses. This is substantial relaxation for circuit designers as synapses are the most abundant components in an in-memory implementation for SNNs. The circuit benefits for both analog and mixed-signal realizations of 0th order synapse are highlighted demonstrating 2-3 orders of savings in area and power consumptions by eliminating Op-Amps and Digital to Analog Converter circuits. This has major implications on a complete neural network implementation with focus on peripheral limitations and algorithmic simplifications to overcome them.

eess.AS↗

arXiv2020

India's Rise in Nanoelectronics Research

Modern semiconductors innovation has a strong relation to scale and skill. While India has a significant demand for semiconductors, it has a daunting challenge to create a semiconductor ecosystem. Yet, India has quietly come a long way. Starting with Centers of Excellence in Nanoelectronics (CENs) initiated in 2006 and broad science and technology funding, India has transformed its nanoelectronics research ecosystem. From negligible contributions as late as 2011, India has risen to be a top contributor to IEEE Electron Devices journals today. Our study presents important observations in terms of ecosystem development. First, there is a 6 year incubation time from infrastructure initiation to first papers. Then, 4 more years to become globally competitive. Second, growth in experimental research is essential along with modeling & simulations. Finally, the aspirational goals of translational research to contribute to the global technology roadmap requires cutting-edge manufacturing infrastructure & ecosystem access, which still needs development. The learning informs a call to action for the research ecosystem i.e. academia, industry, and policy-makers. First, sustain and amplify successful strategies of national research infrastructure & funding growth. Second, enhance international collaborations to add further scale & infrastructure to R&D. Finally, strengthen the industry-academia-policy consortium approach to transform to an innovation-based economy. Ultimately, the electron devices community is entering an exciting phase where Beyond Moore offers open opportunities in materials, devices to systems, and algorithms. India must build on its success to play a significant role in this new world of disruptive innovation.

eess.SY↗

arXiv2020

Adaptive Chemotaxis for improved Contour Tracking using Spiking Neural Networks

In this paper we present a Spiking Neural Network (SNN) for autonomous navigation, inspired by the chemotaxis network of the worm Caenorhabditis elegans. In particular, we focus on the problem of contour tracking, wherein the bot must reach and subsequently follow a desired concentration setpoint. Past schemes that used only klinokinesis can follow the contour efficiently but take excessive time to reach the setpoint. We address this shortcoming by proposing a novel adaptive klinotaxis mechanism that builds upon a previously proposed gradient climbing circuit. We demonstrate how our klinotaxis circuit can autonomously be configured to perform gradient ascent, gradient descent and subsequently be disabled to seamlessly integrate with the aforementioned klinokinesis circuit. We also incorporate speed regulation (orthokinesis) to further improve contour tracking performance. Thus for the first time, we present a model that successfully integrates klinokinesis, klinotaxis and orthokinesis. We demonstrate via contour tracking simulations that our proposed scheme achieves an 2.4x reduction in the time to reach the setpoint, along with a simultaneous 8.7x reduction in average deviation from the setpoint.

q-bio.NC↗

arXiv2020

Reaction-Drift Model for Switching Transients in Pr$_{0.7}$Ca$_{0.3}$MnO$_3$-Based Resistive RAM

Pr$_{0.7}$Ca$_{0.3}$MnO$_3$ (PCMO) based RRAM shows promising memory properties like non-volatility, low variability, multiple resistance states and scalability. From a modeling perspective, the charge carrier DC current modeling of PCMO RRAM by drift diffusion (DD) in the presence of fixed oxygen ion vacancy traps and self-heating (SH) in Technology Computer Aided Design (TCAD) (but without oxygen ionic transport) was able to explain the experimentally observed space charge limited conduction (SCLC) characteristics, prior to resistive switching. Further, transient analysis using DD+SH model was able to reproduce the experimentally observed fast current increase at ~100 ns timescale, prior to resistive switching. However, a complete quantitative transient current transport plus resistive switching model requires the inclusion of ionic transport. We propose a Reaction-Drift (RD) model for oxygen ion vacancy related trap density variation, which is combined with the DD+SH model. Earlier we have shown that the Set transient consists of 3 stages and Reset transient consists of 4 stages experimentally. In this work, the DD+SH+RD model is able to reproduce the entire transient behavior over 10 ns - 1 s range in timescale for both the Set and Reset operations for different applied biases and ambient temperatures. Remarkably, a universal Reset experimental behavior, log(I) is proportional to (m X log(t)) where m~-1/10 is reproduced in simulations. This model is the first model for PCMO RRAMs to significantly reproduce transient Set/Reset behavior. This model establishes the presence of self-heating and ionic-drift limited resistive switching as primary physical phenomena in these RRAMs.

physics.app-ph↗

arXiv2020

Understanding the Location of Resistance Change in the Pr0.7Ca0.3MnO3 RRAM

Pr1-xCaxMnO3 (PCMO) based resistance random access memory (RRAM) is attractive in large scale memory and neuromorphic applications as it is non-filamentary, area scalable and has multiple resistance states along with excellent endurance and retention. The PCMO RRAM exhibit area scalable resistive switching when in contact with the reactive electrode. The interface redox reaction based resistance switching is observed electrically. Yet, whether resistance change occurs through partial (close to interface) or entire bulk is largely debated. Essentially, a two-terminal device is unable to provide direct evidence of the resistance change location in the PCMO RRAM. In this paper, we propose and experimentally demonstrate a novel three-terminal RRAM device in which a thin third terminal (~20nm) is inserted laterally in a typical vertical 2 terminal RRAM device of PCMO thickness of ~80nm. Using the 3T-RRAM method, we show that resistance change occurs largely at the upper bulk (near reactive electrode interface) - which is highly asymmetric. Yet it produces SCLC based resistance change with symmetric IV characteristics. It is the first time that an interface redox and bulk SCLC based resistance change has been experimentally shown as correlated and consistent - enabled by the 3rd terminal of the RRAM. Such a study enables a critical understanding of the device which enables the design and development of PCMO RRAM for memory and neuromorphic computing applications.

physics.app-ph↗

arXiv2020

Software-Level Accuracy Using Stochastic Computing With Charge-Trap-Flash Based Weight Matrix

The in-memory computing paradigm with emerging memory devices has been recently shown to be a promising way to accelerate deep learning. Resistive processing unit (RPU) has been proposed to enable the vector-vector outer product in a crossbar array using a stochastic train of identical pulses to enable one-shot weight update, promising intense speed-up in matrix multiplication operations, which form the bulk of training neural networks. However, the performance of the system suffers if the device does not satisfy the condition of linear conductance change over around 1,000 conductance levels. This is a challenge for nanoscale memories. Recently, Charge Trap Flash (CTF) memory was shown to have a large number of levels before saturation, but variable non-linearity. In this paper, we explore the trade-off between the range of conductance change and linearity. We show, through simulations, that at an optimum choice of the range, our system performs nearly as well as the models trained using exact floating point operations, with less than 1% reduction in the performance. Our system reaches an accuracy of 97.9% on MNIST dataset, 89.1% and 70.5% accuracy on CIFAR-10 and CIFAR-100 datasets (using pre-extracted features). We also show its use in reinforcement learning, where it is used for value function approximation in Q-Learning, and learns to complete an episode the mountain car control problem in around 146 steps. Benchmarked to state-of-the-art, the CTF based RPU shows best in class performance to enable software equivalent performance.

cs.LG↗

arXiv2019

Structured Mean-field Variational Inference and Learning in Winner-take-all Spiking Neural Networks

The Bayesian view of the brain hypothesizes that the brain constructs a generative model of the world, and uses it to make inferences via Bayes' rule. Although many types of approximate inference schemes have been proposed for hierarchical Bayesian models of the brain, the questions of how these distinct inference procedures can be realized by hierarchical networks of spiking neurons remains largely unresolved. Based on a previously proposed multi-compartment neuron model in which dendrites perform logarithmic compression, and stochastic spiking winner-take-all (WTA) circuits in which firing probability of each neuron is normalized by activities of other neurons, here we construct Spiking Neural Networks that perform \emph{structured} mean-field variational inference and learning, on hierarchical directed probabilistic graphical models with discrete random variables. In these models, we do away with symmetric synaptic weights previously assumed for \emph{unstructured} mean-field variational inference by learning both the feedback and feedforward weights separately. The resulting online learning rules take the form of an error-modulated local Spike-Timing-Dependent Plasticity rule. Importantly, we consider two types of WTA circuits in which only one neuron is allowed to fire at a time (hard WTA) or neurons can fire independently (soft WTA), which makes neurons in these circuits operate in regimes of temporal and rate coding respectively. We show how the hard WTA circuits can be used to perform Gibbs sampling whereas the soft WTA circuits can be used to implement a message passing algorithm that computes the marginals approximately. Notably, a simple change in the amount of lateral inhibition realizes switching between the hard and soft WTA spiking regimes. Hence the proposed network provides a unified view of the two previously disparate modes of inference and coding by spiking neurons.

q-bio.NC↗

arXiv2019

Ultra-low Energy charge trap flash based synapse enabled by parasitic leakage mitigation

Brain-inspired computation promises complex cognitive tasks at biological energy efficiencies. The brain contains $10^4$ synapses per neuron. Hence, ultra-low energy, high-density synapses are needed for spiking neural networks (SNN). In this paper, we use tunneling enabled CTF (Charge Trap Flash) stack for ultra-low-energy operation (1F); Further, CTF on an SOI platform and back-to-back connected pn diode and Zener diode (2D) prevent parasitic leakage to preserve energy advantage in array operation. A bulk $100 μm $ x $100 μm$ CTF operation offers tunable, gradual conductance change $(ΔG) i.e. 10^4 $levels, which gives $100$x improvement over literature. SPICE simulations of 1F2D synapse shows ultra-low energy $(\leqslant 3 fJ/pulse)$ at 180 nm node for long-term potentiation (LTP) and depression (LTD), at 180nm node for long-term potentiation (LTP) and depression (LTD), which is comparable to energy estimate in biological synapses (10 fJ). A record low learning rate (i.e., maximum $ΔG< 1%$ of G-range) is observed - which is tunable. Excellent reliability ($>10^6 endurance cycles at full conductance swing) is observed. Such a highly energy efficient synapse with tunable learning rate on the CMOS platform is a key enabler for the human-brain-scale systems. Keywords: Spiking Neural Network; Charge trap flash, SONAS, Fowler-Nordheim Tunneling, Synapse

cs.ET↗

arXiv2019

Band-to-Band Tunneling based Ultra-Energy Efficient Silicon Neuron

The human brain comprises about a hundred billion neurons connected through quadrillion synapses. Spiking Neural Networks (SNNs) take inspiration from the brain to model complex cognitive and learning tasks. Neuromorphic engineering implements SNNs in hardware, aspiring to mimic the brain at scale (i.e., 100 billion neurons) with biological area and energy efficiency. The design of ultra-energy efficient and compact neurons is essential for the large-scale implementation of SNNs in hardware. In this work, we have experimentally demonstrated a Partially Depleted (PD) Silicon-On-Insulator (SOI) MOSFET based Leaky-Integrate & Fire (LIF) neuron where energy-and area-efficiency is enabled by two elements of design - first tunneling based operation and second compact sub-threshold SOI control circuit design. Band-to-Band Tunneling (BTBT) induced hole storage in the body is used for the "Integrate" function of the neuron. A compact control circuit "Fires" a spike when the body potential exceeds the firing threshold. The neuron then "Resets" by removing the stored holes from the body contact of the device. Additionally, the control circuit provides "Leakiness" in the neuron which is an essential property of biological neurons. The proposed neuron provides 10x higher area efficiency compared to CMOS design with equivalent energy/spike. Alternatively, it has 10^4x higher energy efficiency at area-equivalent neuron technologies. Biologically comparable energy- and area-efficiency along with CMOS compatibility make the proposed device attractive for large-scale hardware implementation of SNNs.

cs.NE↗

arXiv2019

Predicting Performance using Approximate State Space Model for Liquid State Machines

Liquid State Machine (LSM) is a brain-inspired architecture used for solving problems like speech recognition and time series prediction. LSM comprises of a randomly connected recurrent network of spiking neurons. This network propagates the non-linear neuronal and synaptic dynamics. Maass et al. have argued that the non-linear dynamics of LSMs is essential for its performance as a universal computer. Lyapunov exponent (mu), used to characterize the "non-linearity" of the network, correlates well with LSM performance. We propose a complementary approach of approximating the LSM dynamics with a linear state space representation. The spike rates from this model are well correlated to the spike rates from LSM. Such equivalence allows the extraction of a "memory" metric (tau_M) from the state transition matrix. tau_M displays high correlation with performance. Further, high tau_M system require lesser epochs to achieve a given accuracy. Being computationally cheap (1800x time efficient compared to LSM), the tau_M metric enables exploration of the vast parameter design space. We observe that the performance correlation of the tau_M surpasses the Lyapunov exponent (mu), (2-4x improvement) in the high-performance regime over multiple datasets. In fact, while mu increases monotonically with network activity, the performance reaches a maxima at a specific activity described in literature as the "edge of chaos". On the other hand, tau_M remains correlated with LSM performance even as mu increases monotonically. Hence, tau_M captures the useful memory of network activity that enables LSM performance. It also enables rapid design space exploration and fine-tuning of LSM parameters for high performance.

cs.NE↗

arXiv2018

Transient Phenomena in Sub-Band Gap Impact Ionization in Si NIPIN Diode

Sub-band-gap (SBG) impact ionization (II) enables steep subthreshold slope that enables devices to overcome the thermal limit of 60mV/decade. This phenomenon at low voltage enables various applications in logic, memory and neuromorphic engineering. Recently, we have demonstrated sub-0.2V II in NIPIN diode experimentally primarily based on the steady-state analysis. In this paper, we present the detailed experimental transient behavior of SBG-II in NIPIN. The SBG-II generated holes are stored in the p-well. First, we extract the leakage mechanism from the p-well to show two mechanisms (i) recombination-generation (RG) and (ii) over the barrier (OTB) where OTB dominates when barrier height $ phy_b<0.59eV $. Second, we analytically extract the SBG II current (Iii) at 300K from experimental results. The drain current (Id), the electric field (E-field), and Iii are plotted in time. We observe that Iii increase as E-field reduces which indicates that E-field does not primarily contribute to Iii. Further, the Id shows two distinct behaviors (i) Iii (Id) is constant at the beginning and (ii) eventually universal Iii (Id) is linear, i.e. Iii=k X Id where $ k=10^-3$; We also show that the electrons primarily contributing to Id are directly incapable of II due to insufficient energy $(<Eg)$. Fischetti's model showed that SBG-II is primarily caused by hot electrons that accept energy in an Auger-like process from cold drain electrons to enable SBG-II. We speculate that if the Id electrons heat-up the cold drain electrons, which would further energize the hot electrons to produce the observed Iii (Id) universal dependence.

physics.app-ph↗

arXiv2018

A case for multiple and parallel RRAMs as synaptic model for training SNNs

To enable a dense integration of model synapses in a spiking neural networks hardware, various nano-scale devices are being considered. Such a device, besides exhibiting spike-time dependent plasticity (STDP), needs to be highly scalable, have a large endurance and require low energy for transitioning between states. In this work, we first introduce and empirically determine two new specifications for an synapse in SNNs: number of conductance levels per synapse and maximum learning-rate. To the best of our knowledge, there are no RRAMs that meet the latter specification. As a solution, we propose the use of multiple PCMO-RRAMs in parallel within a synapse. While synaptic reading, all PCMO-RRAMs are simultaneously read and for each synaptic conductance-change event, the mechanism for conductance STDP is initiated for only one RRAM, randomly picked from the set. Second, to validate our solution, we experimentally demonstrate STDP of conductance of a PCMO-RRAM and then show that due to a large learning-rate, a single PCMO-RRAM fails to model a synapse in the training of an SNN. As anticipated, network training improves as more PCMO-RRAMs are added to the synapse. Fourth, we discuss the circuit-requirements for implementing such a scheme, to conclude that the requirements are within bounds. Thus, our work presents specifications for synaptic devices in trainable SNNs, indicates the shortcomings of state-of-art synaptic contenders, and provides a solution to extrinsically meet the specifications and discusses the peripheral circuitry that implements the solution.

cs.ET↗

arXiv2018

Analytical Modeling of Metal Gate Granularity based Threshold Voltage Variability in NWFET

Estimation of threshold voltage V T variability for NWFETs has been compu- tationally expensive due to lack of analytical models. Variability estimation of NWFET is essential to design the next generation logic circuits. Compared to any other process induced variabilities, Metal Gate Granularity (MGG) is of paramount importance due to its large impact on V T variability. Here, an analytical model is proposed to estimate V T variability caused by MGG. We extend our earlier FinFET based MGG model to a cylindrical NWFET by sat- isfying three additional requirements. First, the gate dielectric layer is replaced by Silicon of electro-statically equivalent thickness using long cylinder approxi- mation; Second, metal grains in NWFETs satisfy periodic boundary condition in azimuthal direction; Third, electrostatics is analytically solved in cylindri- cal polar coordinates with gate boundary condition defined by MGG. We show that quantum effects only shift the mean of the V T distribution without sig- nificant impact on the variability estimated by our electrostatics-based model. The V T distribution estimated by our model matches TCAD simulations. The model quantitatively captures grain size dependence with σ(V T ) with excellent accuracy (6%error) compared to stochastic 3D TCAD simulations, which is a significant improvement over the state-of- the-art model with fails to produce even a qualitative agreement. The proposed model is 63 times faster compared to commercial TCAD simulations.

physics.app-ph↗

arXiv2017

An On-chip Trainable and Clock-less Spiking Neural Network with 1R Memristive Synapses

Spiking neural networks (SNNs) are being explored in an attempt to mimic brain's capability to learn and recognize at low power. Crossbar architecture with highly scalable Resistive RAM or RRAM array serving as synaptic weights and neuronal drivers in the periphery is an attractive option for SNN. Recognition (akin to reading the synaptic weight) requires small amplitude bias applied across the RRAM to minimize conductance change. Learning (akin to writing or updating the synaptic weight) requires large amplitude bias pulses to produce a conductance change. The contradictory bias amplitude requirement to perform reading and writing simultaneously and asynchronously, akin to biology, is a major challenge. Solutions suggested in the literature rely on time-division-multiplexing of read and write operations based on clocks, or approximations ignoring the reading when coincidental with writing. In this work, we overcome this challenge and present a clock-less approach wherein reading and writing are performed in different frequency domains. This enables learning and recognition simultaneously on an SNN. We validate our scheme in SPICE circuit simulator by translating a two-layered feed-forward Iris classifying SNN to demonstrate software-equivalent performance. The system performance is not adversely affected by a voltage dependence of conductance in realistic RRAMs, despite departing from linearity. Overall, our approach enables direct implementation of biological SNN algorithms in hardware.

cs.NE↗

arXiv2017

A Software-equivalent SNN Hardware using RRAM-array for Asynchronous Real-time Learning

Spiking Neural Network (SNN) naturally inspires hardware implementation as it is based on biology. For learning, spike time dependent plasticity (STDP) may be implemented using an energy efficient waveform superposition on memristor based synapse. However, system level implementation has three challenges. First, a classic dilemma is that recognition requires current reading for short voltage$-$spikes which is disturbed by large voltage$-$waveforms that are simultaneously applied on the same memristor for real$-$time learning i.e. the simultaneous read$-$write dilemma. Second, the hardware needs to exactly replicate software implementation for easy adaptation of algorithm to hardware. Third, the devices used in hardware simulations must be realistic. In this paper, we present an approach to address the above concerns. First, the learning and recognition occurs in separate arrays simultaneously in real$-$time, asynchronously $-$ avoiding non$-$biomimetic clocking based complex signal management. Second, we show that the hardware emulates software at every stage by comparison of SPICE (circuit$-$simulator) with MATLAB (mathematical SNN algorithm implementation in software) implementations. As an example, the hardware shows 97.5 per cent accuracy in classification which is equivalent to software for a Fisher$-$Iris dataset. Third, the STDP is implemented using a model of synaptic device implemented using HfO2 memristor. We show that an increasingly realistic memristor model slightly reduces the hardware performance (85 per cent), which highlights the need to engineer RRAM characteristics specifically for SNN.

cs.NE↗

arXiv2016

A simple and efficient SNN and its performance & robustness evaluation method to enable hardware implementation

Spiking Neural Networks (SNN) are more closely related to brain-like computation and inspire hardware implementation. This is enabled by small networks that give high performance on standard classification problems. In literature, typical SNNs are deep and complex in terms of network structure, weight update rules and learning algorithms. This makes it difficult to translate them into hardware. In this paper, we first develop a simple 2-layered network in software which compares with the state of the art on four different standard data-sets within SNNs and has improved efficiency. For example, it uses lower number of neurons (3 x), synapses (3.5 x) and epochs for training (30 x) for the Fisher Iris classification problem. The efficient network is based on effective population coding and synapse-neuron co-design. Second, we develop a computationally efficient (15000 x) and accurate (correlation of 0.98) method to evaluate the performance of the network without standard recognition tests. Third, we show that the method produces a robustness metric that can be used to evaluate noise tolerance.

cs.NE↗

arXiv2016

Enhanced Circuit Densities in Epitaxially Defined FinFETs (EDFinFETs) over FinFETs

FinFET technology is prone to suffer from Line Edge Roughness (LER) based VT variation with scaling. To address this, we proposed an Epitaxially Defined (ED) FinFET (EDFinFET) as an alternate to FinFET architecture for 10 nm node and beyond. We showed by statistical simulations that EDFinFET reduces LER based VT variability by 90% and overall variability by 59%. However, EDFinFET consists of wider fins as the fin widths are not constrained by electrostatics and variability (cf. FinFETs have fin width ~ LG/3 where LG is gate-length). This indicates that EDFinFET based circuits may be less dense. In this study we show that wide fins enable taller fin heights. The ability to engineer multiple STI levels on tall fins enables different transistor widths (i.e. various W/Ls e.g. 1-10) in a single fin. This capability ensures that even though individual EDFinFET devices have ~2x larger footprints than FinFETs, EDFinFET may produce equal or higher circuit density for basic building blocks like inverters or NAND gates for W/Ls of 2 and higher.

cs.ET↗