arXiv subjects

Kaushik Roy

Publications and source records attributed to Kaushik Roy.

At least 253 records · Page 14Linked to original sources

arXiv2018

Toward Fast Neural Computing using All-Photonic Phase Change Spiking Neurons

The rapid growth of brain-inspired computing coupled with the inefficiencies in the CMOS implementations of neuromrphic systems has led to intense exploration of efficient hardware implementations of the functional units of the brain, namely, neurons and synapses. However, efforts have largely been invested in implementations in the electrical domain with potential limitations of switching speed, packing density of large integrated systems and interconnect losses. As an alternative, neuromorphic engineering in the photonic domain has recently gained attention. In this work, we demonstrate a purely photonic operation of an Integrate-and-Fire Spiking neuron, based on the phase change dynamics of Ge$_2$Sb$_2$Te$_5$ (GST) embedded on top of a microring resonator, which alleviates the energy constraints of PCMs in electrical domain. We also show that such a neuron can be potentially integrated with on-chip synapses into an all-Photonic Spiking Neural network inferencing framework which promises to be ultrafast and can potentially offer a large operating bandwidth.

cs.ET↗

arXiv2018

SPARE: Spiking Networks Acceleration Using CMOS ROM-Embedded RAM as an In-Memory-Computation Primitive

Despite huge success of artificial intelligence, hardware systems running these algorithms consume orders of magnitude higher energy compared to the human brain, mainly due to heavy data movements between the memory unit and the computation cores. Spiking neural networks (SNNs) built using bio-plausible neuron and synaptic models have emerged as the power-efficient choice for designing cognitive applications. These algorithms involve several lookup-table (LUT) based function evaluations such as high-order polynomials and transcendental functions for solving complex neuro-synaptic models, that typically require additional storage. To that effect, we propose `SPARE' - an in-memory, distributed processing architecture built on ROM-embedded RAM technology, for accelerating SNNs. ROM-embedded RAMs allow storage of LUTs, embedded within a typical memory array, without additional area overhead. Our proposed architecture consists of a 2-D array of Processing Elements (PEs). Since most of the computations are done locally within each PE, unnecessary data transfers are restricted, thereby alleviating the von-Neumann bottleneck. We evaluate SPARE for two different ROM-Embedded RAM structures - CMOS based ROM-Embedded SRAMs (R-SRAMs) and STT-MRAM based ROM-Embedded MRAMs (R-MRAMs). Moreover, we analyze trade-offs in terms of energy, area and performance, for using the two technologies on a range of image classification benchmarks. Furthermore, we leverage the additional storage density to implement complex neuro-synaptic functionalities. This enhances the utility of the proposed architecture by provisioning implementation of any neuron/synaptic behavior as necessitated by the application. Our results show up-to 1.75x, 1.95x and 1.95x improvement in energy, iso-storage area, and iso-area performance, respectively, by using neural network accelerators built on ROM-embedded RAM primitives.

cs.ET↗

arXiv2018

X-SRAM: Enabling In-Memory Boolean Computations in CMOS Static Random Access Memories

Silicon-based Static Random Access Memories (SRAM) and digital Boolean logic have been the workhorse of the state-of-art computing platforms. Despite tremendous strides in scaling the ubiquitous metal-oxide-semiconductor transistor, the underlying \textit{von-Neumann} computing architecture has remained unchanged. The limited throughput and energy-efficiency of the state-of-art computing systems, to a large extent, results from the well-known \textit{von-Neumann bottleneck}. The energy and throughput inefficiency of the von-Neumann machines have been accentuated in recent times due to the present emphasis on data-intensive applications like artificial intelligence, machine learning \textit{etc}. A possible approach towards mitigating the overhead associated with the von-Neumann bottleneck is to enable \textit{in-memory} Boolean computations. In this manuscript, we present an augmented version of the conventional SRAM bit-cells, called \textit{the X-SRAM}, with the ability to perform in-memory, vector Boolean computations, in addition to the usual memory storage operations. We propose at least six different schemes for enabling in-memory vector computations including NAND, NOR, IMP (implication), XOR logic gates with respect to different bit-cell topologies $-$ the 8T cell and the 8$^+$T Differential cell. In addition, we also present a novel \textit{`read-compute-store'} scheme, wherein the computed Boolean function can be directly stored in the memory without the need of latching the data and carrying out a subsequent write operation. The feasibility of the proposed schemes has been verified using predictive transistor models and Monte-Carlo variation analysis.

cs.ET↗

arXiv2018

Exploiting Inherent Error-Resiliency of Neuromorphic Computing to achieve Extreme Energy-Efficiency through Mixed-Signal Neurons

Neuromorphic computing, inspired by the brain, promises extreme efficiency for certain classes of learning tasks, such as classification and pattern recognition. The performance and power consumption of neuromorphic computing depends heavily on the choice of the neuron architecture. Digital neurons (Dig-N) are conventionally known to be accurate and efficient at high speed, while suffering from high leakage currents from a large number of transistors in a large design. On the other hand, analog/mixed-signal neurons are prone to noise, variability and mismatch, but can lead to extremely low-power designs. In this work, we will analyze, compare and contrast existing neuron architectures with a proposed mixed-signal neuron (MS-N) in terms of performance, power and noise, thereby demonstrating the applicability of the proposed mixed-signal neuron for achieving extreme energy-efficiency in neuromorphic computing. The proposed MS-N is implemented in 65 nm CMOS technology and exhibits > 100X better energy-efficiency across all frequencies over two traditional digital neurons synthesized in the same technology node. We also demonstrate that the inherent error-resiliency of a fully connected or even convolutional neural network (CNN) can handle the noise as well as the manufacturing non-idealities of the MS-N up to certain degrees. Notably, a system-level implementation on MNIST datasets exhibits a worst-case increase in classification error by 2.1% when the integrated noise power in the bandwidth is ~ 0.1 uV2, along with +-3σ amount of variation and mismatch introduced in the transistor parameters for the proposed neuron with 8-bit precision.

cs.ET↗

arXiv2018

ASP: Learning to Forget with Adaptive Synaptic Plasticity in Spiking Neural Networks

A fundamental feature of learning in animals is the "ability to forget" that allows an organism to perceive, model and make decisions from disparate streams of information and adapt to changing environments. Against this backdrop, we present a novel unsupervised learning mechanism ASP (Adaptive Synaptic Plasticity) for improved recognition with Spiking Neural Networks (SNNs) for real time on-line learning in a dynamic environment. We incorporate an adaptive weight decay mechanism with the traditional Spike Timing Dependent Plasticity (STDP) learning to model adaptivity in SNNs. The leak rate of the synaptic weights is modulated based on the temporal correlation between the spiking patterns of the pre- and post-synaptic neurons. This mechanism helps in gradual forgetting of insignificant data while retaining significant, yet old, information. ASP, thus, maintains a balance between forgetting and immediate learning to construct a stable-plastic self-adaptive SNN for continuously changing inputs. We demonstrate that the proposed learning methodology addresses catastrophic forgetting while yielding significantly improved accuracy over the conventional STDP learning method for digit recognition applications. Additionally, we observe that the proposed learning model automatically encodes selective attention towards relevant features in the input data while eliminating the influence of background noise (or denoising) further improving the robustness of the ASP learning.

cs.NE↗

arXiv2018

Design of a Low Voltage Analog-to-Digital Converter using Voltage Controlled Stochastic Switching of Low Barrier Nanomagnets

The inherent stochasticity in many nano-scale devices makes them prospective candidates for low-power computations. Such devices have been demonstrated to exhibit probabilistic switching between two stable states to achieve stochastic behavior. Recently, superparamagnetic nanomagnets (having low energy barrier EB $\sim$ 1kT) have shown promise of achieving stochastic switching at GHz rates, with very low currents. On the other hand, voltage-controlled switching of nanomagnets through the Magneto-electric (ME) effect has shown further improvements in energy efficiency. In this simulation paper, we first analyze the stochastic switching characteristics of such super-paramagnetic nanomagnets in a voltage-controlled spintronic device. We study the influence of external bias on the switching behavior. Subsequently, we show that our proposed device leverages the voltage controlled stochasticity in performing low-voltage 8-bit analog to digital conversions. This eliminates the need for comparators, unlike the Complementary Metal-Oxide Semiconductor (CMOS)-based flash Analog-to-Digital converters (ADC). This device allows for a simple and compact design which can potentially be applied in implementing sensors which desire low voltage conversions.

cs.ET↗

arXiv2018

An All-Memristor Deep Spiking Neural Computing System: A Step Towards Realizing the Low Power,Stochastic Brain

Deep 'Analog Artificial Neural Networks' (ANNs) perform complex classification problems with remarkably high accuracy. However, they rely on humongous amount of power to perform the calculations, veiling the accuracy benefits. The biological brain on the other hand is significantly more powerful than such networks and consumes orders of magnitude less power, indicating us about some conceptual mismatch. Given that the biological neurons communicate using energy efficient trains of spikes, and the behavior is non-deterministic, incorporating these effects in Deep Artificial Neural Networks may drive us few steps towards a more realistic neuron. In this work, we propose how the inherent stochasticity of nano-scale resistive devices can be harnessed to emulate the functionality of a spiking neuron that can be incorporated in deep stochastic Spiking Neural Networks (SNN). At the algorithmic level, we propose how the training can be modified to convert an ANN to an SNN while supporting the stochastic activation function offered by these devices. We devise circuit architectures to incorporate stochastic memristive neurons along with memristive crossbars which perform the functionality of the synaptic weights. We tested the proposed All Memristor deep stochastic SNN for image classification and observed only about 1% degradation in accuracy with the ANN baseline after incorporating the circuit and device related non-idealities. We witnessed that the network is robust to certain variations and consumes ~ 6.4x less energy than its CMOS counterpart.

cs.ET↗

arXiv2018

TraNNsformer: Neural network transformation for memristive crossbar based neuromorphic system design

Implementation of Neuromorphic Systems using post Complementary Metal-Oxide-Semiconductor (CMOS) technology based Memristive Crossbar Array (MCA) has emerged as a promising solution to enable low-power acceleration of neural networks. However, the recent trend to design Deep Neural Networks (DNNs) for achieving human-like cognitive abilities poses significant challenges towards the scalable design of neuromorphic systems (due to the increase in computation/storage demands). Network pruning [7] is a powerful technique to remove redundant connections for designing optimally connected (maximally sparse) DNNs. However, such pruning techniques induce irregular connections that are incoherent to the crossbar structure. Eventually they produce DNNs with highly inefficient hardware realizations (in terms of area and energy). In this work, we propose TraNNsformer - an integrated training framework that transforms DNNs to enable their efficient realization on MCA-based systems. TraNNsformer first prunes the connectivity matrix while forming clusters with the remaining connections. Subsequently, it retrains the network to fine tune the connections and reinforce the clusters. This is done iteratively to transform the original connectivity into an optimally pruned and maximally clustered mapping. Without accuracy loss, TraNNsformer reduces the area (energy) consumption by 28% - 55% (49% - 67%) with respect to the original network. Compared to network pruning, TraNNsformer achieves 28% - 49% (15% - 29%) area (energy) savings. Furthermore, TraNNsformer is a technology-aware framework that allows mapping a given DNN to any MCA size permissible by the memristive technology for reliable operations.

cs.ET↗

arXiv2018

Capacitively Driven Global Interconnect with Magnetoelectric Switching Based Receiver for Higher Energy Efficiency

We propose capacitively driven low-swing global interconnect circuit using a receiver that utilizes magnetoelectric (ME) effect induced magnetization switching to reduce the energy consumption. Capacitively driven wire has recently been shown to be effective in improving the performance of global interconnects. Such techniques can reduce the signal swing in the interconnect by using a capacitive divider network and does not require an additional voltage supply. However, the large reduction in signal swing makes it necessary to use differential signaling and amplification for successful regeneration at the receiver, which add area and static power. ME effect induced magnetization reversal has recently been proposed which shows the possibility of using a low voltage to switch a nanomagnet adjacent to a multi-ferroic oxide. Here, we propose an ME effect based receiver that uses the low voltage at the receiving end of the global wire to switch a nanomagnet. The nanomagnet is also used as the free layer of a magnetic tunnel junction (MTJ), the resistance of which is tuned through the ME effect. This change in MTJ resistance is converted to full swing binary signals by using simple digital components. This process allows capacitive low swing interconnection without differential signaling or amplification, which leads to significant energy efficiency. Our simulation results indicate that for 5-10 mm long global wires in IBM 45 nm technology, capacitive ME design consumes 3x lower energy compared to full-swing CMOS design and 2x lower energy compared to differential amplifier based low-swing capacitive CMOS design.

cs.ET↗

arXiv2018

Inclusive prompt photon production in electron-nucleus scattering at small x

We compute the differential cross-section for inclusive prompt photon production in deeply inelastic scattering (DIS) of electrons on nuclei at small $x$ in the framework of the Color Glass Condensate (CGC) effective theory. The leading order (LO) computation in this framework resums leading logarithms in $x$ as well as power corrections to all orders in $Q_{s,A}^2/Q^2$, where $Q_{s,A}(x)$ is the nuclear saturation scale. This LO result is proportional to universal dipole and quadrupole Wilson line correlators in the nucleus. In the soft photon limit, the Low-Burnett-Kroll theorem allows us to recover existing results on inclusive DIS dijet production. The $k_{\perp}$ and collinearly factorized expressions for prompt photon production in DIS are also recovered in a leading twist approximation to our result. In the latter case, our result corresponds to the dominant next-to-leading order (NLO) perturbative QCD contribution at small $x$. We next discuss the computation of the NLO corrections to inclusive prompt photon production in the CGC framework. In particular, we emphasize the advantages for higher order computations in inclusive photon production, and for fully inclusive DIS, arising from the simple momentum space structure of the dressed quark and gluon "shock wave" propagators in the "wrong" light cone gauge $A^-=0$ for a nucleus moving with $P^{+}_{N} \rightarrow \infty$.

hep-ph↗

arXiv2018

Optical Receiver with Helicity Dependent Switching of Magnetization

In this work, we propose helicity-dependent switching (HDS) of magnetization of Co/Pt for energy efficient optical receiver. Designing a low power optical receiver for optical-to-electrical signal conversion has proven to be very challenging. Current day receivers use a photodiode that produces a photocurrent in response to input optical signals, and power hungry trans-impedance amplifiers are required to amplify the small photocurrents. Here, we propose light helicity induced switching of magnetization to overcome the requirement of photodiodes and subsequent trans-impedance amplification by sensing the change in magnetization with a magnetic tunnel junction (MTJ). Magnetization switching of a thin ferromagnet layer using circularly polarized laser pulses have recently been demonstrated which shows one-to-one correspondence between light helicity and the magnetization state. We propose to utilize this phenomena by using digital input dependent circularly polarized laser pulses to directly switch the magnetization state of a thin Co/Pt ferromagnet layer at the receiver. The Co/Pt layer is used as the free layer of an MTJ, the resistance of which is modified by the laser pulses. With the one-to-one dependence between input data and output magnetization state, the MTJ resistance is directly converted to digital output signal. Our device to circuit level simulation results indicate that, HDS based optical receiver consumes only 0.124 pJ/bit energy, which is much lower than existing techniques.

physics.app-ph↗

arXiv2018

Chaos-guided Input Structuring for Improved Learning in Recurrent Neural Networks

Anatomical studies demonstrate that brain reformats input information to generate reliable responses for performing computations. However, it remains unclear how neural circuits encode complex spatio-temporal patterns. We show that neural dynamics are strongly influenced by the phase alignment between the input and the spontaneous chaotic activity. Input structuring along the dominant chaotic projections causes the chaotic trajectories to become stable channels (or attractors), hence, improving the computational capability of a recurrent network. Using mean field analysis, we derive the impact of input structuring on the overall stability of attractors formed. Our results indicate that input alignment determines the extent of intrinsic noise suppression and hence, alters the attractor state stability, thereby controlling the network's inference ability.

q-bio.NC↗

arXiv2018

Stochastic Spiking Neural Networks Enabled by Magnetic Tunnel Junctions: From Nontelegraphic to Telegraphic Switching Regimes

Stochastic spiking neural networks based on nanoelectronic spin devices can be a possible pathway to achieving "brainlike" compact and energy-effcient cognitive intelligence. The computational model attempt to exploit the intrinsic device stochasticity of nanoelectronic synaptic or neural components to perform learning or inference. However, there has been limited analysis on the scaling effect of stochastic spin devices and its impact on the operation of such stochastic networks at the system level. This work attempts to explore the design space and analyze the performance of nanomagnet-based stochastic neuromorphic computing architectures for magnets with different barrier heights. We illustrate how the underlying network architecture must be modified to account for the random telegraphic switching behavior displayed by magnets with low barrier heights as they are scaled into the superparamagnetic regime. We perform a device-to-system-level analysis on a deep neural-network architecture for a digit-recognition problem on the MNIST data set.

cs.ET↗

arXiv2018

Deep learning for word-level handwritten Indic script identification

We propose a novel method that uses convolutional neural networks (CNNs) for feature extraction. Not just limited to conventional spatial domain representation, we use multilevel 2D discrete Haar wavelet transform, where image representations are scaled to a variety of different sizes. These are then used to train different CNNs to select features. To be precise, we use 10 different CNNs that select a set of 10240 features, i.e. 1024/CNN. With this, 11 different handwritten scripts are identified, where 1K words per script are used. In our test, we have achieved the maximum script identification rate of 94.73% using multi-layer perceptron (MLP). Our results outperform the state-of-the-art techniques.

cs.CV↗

arXiv2017

Encoding Neural and Synaptic Functionalities in Electron Spin: A Pathway to Efficient Neuromorphic Computing

Present day computers expend orders of magnitude more computational resources to perform various cognitive and perception related tasks that humans routinely perform everyday. This has recently resulted in a seismic shift in the field of computation where research efforts are being directed to develop a neurocomputer that attempts to mimic the human brain by nanoelectronic components and thereby harness its efficiency in recognition problems. Bridging the gap between neuroscience and nanoelectronics, this paper attempts to provide a review of the recent developments in the field of spintronic device based neuromorphic computing. Description of various spin-transfer torque mechanisms that can be potentially utilized for realizing device structures mimicking neural and synaptic functionalities is provided. A cross-layer perspective extending from the device to the circuit and system level is presented to envision the design of an All-Spin neuromorphic processor enabled with on-chip learning functionalities. Device-circuit-algorithm co-simulation framework calibrated to experimental results suggest that such All-Spin neuromorphic systems can potentially achieve almost two orders of magnitude energy improvement in comparison to state-of-the-art CMOS implementations.

cs.ET↗

arXiv2017

Magnetic Tunnel Junction Mimics Stochastic Cortical Spiking Neurons

Brain-inspired computing architectures attempt to mimic the computations performed in the neurons and the synapses in the human brain in order to achieve its efficiency in learning and cognitive tasks. In this work, we demonstrate the mapping of the probabilistic spiking nature of pyramidal neurons in the cortex to the stochastic switching behavior of a Magnetic Tunnel Junction in presence of thermal noise. We present results to illustrate the efficiency of neuromorphic systems based on such probabilistic neurons for pattern recognition tasks in presence of lateral inhibition and homeostasis. Such stochastic MTJ neurons can also potentially provide a direct mapping to the probabilistic computing elements in Belief Networks for performing regenerative tasks.

cs.ET↗

arXiv2017

Short-Term Plasticity and Long-Term Potentiation in Magnetic Tunnel Junctions: Towards Volatile Synapses

Synaptic memory is considered to be the main element responsible for learning and cognition in humans. Although traditionally non-volatile long-term plasticity changes have been implemented in nanoelectronic synapses for neuromorphic applications, recent studies in neuroscience have revealed that biological synapses undergo meta-stable volatile strengthening followed by a long-term strengthening provided that the frequency of the input stimulus is sufficiently high. Such "memory strengthening" and "memory decay" functionalities can potentially lead to adaptive neuromorphic architectures. In this paper, we demonstrate the close resemblance of the magnetization dynamics of a Magnetic Tunnel Junction (MTJ) to short-term plasticity and long-term potentiation observed in biological synapses. We illustrate that, in addition to the magnitude and duration of the input stimulus, frequency of the stimulus plays a critical role in determining long-term potentiation of the MTJ. Such MTJ synaptic memory arrays can be utilized to create compact, ultra-fast and low power intelligent neural systems.

cs.ET↗

arXiv2017

Technology Aware Training in Memristive Neuromorphic Systems based on non-ideal Synaptic Crossbars

The advances in the field of machine learning using neuromorphic systems have paved the pathway for extensive research on possibilities of hardware implementations of neural networks. Various memristive technologies such as oxide-based devices, spintronics and phase change materials have been explored to implement the core functional units of neuromorphic systems, namely the synaptic network, and the neuronal functionality, in a fast and energy efficient manner. However, various non-idealities in the crossbar implementations of the synaptic arrays can significantly degrade performance of neural networks and hence, impose restrictions on feasible crossbar sizes. In this work, we build mathematical models of various non-idealities that occur in crossbar implementations such as source resistance, neuron resistance and chip-to-chip device variations and analyze their impact on the classification accuracy of a fully connected network (FCN) and convolutional neural network (CNN) trained with standard training algorithm. We show that a network trained under ideal conditions can suffer accuracy degradation as large as 59.84% for FCNs and 62.4% for CNNs when implemented on non-ideal crossbars for relevant non-ideality ranges. This severely constrains the sizes for crossbars. As a solution, we propose a technology aware training algorithm which incorporates the mathematical models of the non-idealities in the standard training algorithm. We demonstrate that our proposed methodology achieves significant recovery of testing accuracy within 1.9% of the ideal accuracy for FCNs and 1.5% for CNNs. We further show that our proposed training algorithm can potentially allow the use of significantly larger crossbar arrays of sizes 784$\times$500 for FCNs and 4096$\times$512 for CNNs with a minor or no trade-off in accuracy

cs.ET↗