arXiv subjects

Shibani Santurkar

Publications and source records attributed to Shibani Santurkar.

24 records · Page 2Linked to original sources

arXiv2018

Adversarially Robust Generalization Requires More Data

Machine learning models are often susceptible to adversarial perturbations of their inputs. Even small perturbations can cause state-of-the-art classifiers with high "standard" accuracy to produce an incorrect prediction with high confidence. To better understand this phenomenon, we study adversarially robust learning from the viewpoint of generalization. We show that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of "standard" learning. This gap is information theoretic and holds irrespective of the training algorithm or the model family. We complement our theoretical results with experiments on popular image classification datasets and show that a similar gap exists here as well. We postulate that the difficulty of training robust classifiers stems, at least partially, from this inherently larger sample complexity.

cs.LG↗

arXiv2017

Deep Tensor Convolution on Multicores

Deep convolutional neural networks (ConvNets) of 3-dimensional kernels allow joint modeling of spatiotemporal features. These networks have improved performance of video and volumetric image analysis, but have been limited in size due to the low memory ceiling of GPU hardware. Existing CPU implementations overcome this constraint but are impractically slow. Here we extend and optimize the faster Winograd-class of convolutional algorithms to the $N$-dimensional case and specifically for CPU hardware. First, we remove the need to manually hand-craft algorithms by exploiting the relaxed constraints and cheap sparse access of CPU memory. Second, we maximize CPU utilization and multicore scalability by transforming data matrices to be cache-aware, integer multiples of AVX vector widths. Treating 2-dimensional ConvNets as a special (and the least beneficial) case of our approach, we demonstrate a 5 to 25-fold improvement in throughput compared to previous state-of-the-art.

cs.CV↗

arXiv2017

Generative Compression

Traditional image and video compression algorithms rely on hand-crafted encoder/decoder pairs (codecs) that lack adaptability and are agnostic to the data being compressed. Here we describe the concept of generative compression, the compression of data using generative models, and suggest that it is a direction worth pursuing to produce more accurate and visually pleasing reconstructions at much deeper compression levels for both image and video data. We also demonstrate that generative compression is orders-of-magnitude more resilient to bit error rates (e.g. from noisy wireless channels) than traditional variable-length coding schemes.

cs.CV↗

arXiv2017

Toward Streaming Synapse Detection with Compositional ConvNets

Connectomics is an emerging field in neuroscience that aims to reconstruct the 3-dimensional morphology of neurons from electron microscopy (EM) images. Recent studies have successfully demonstrated the use of convolutional neural networks (ConvNets) for segmenting cell membranes to individuate neurons. However, there has been comparatively little success in high-throughput identification of the intercellular synaptic connections required for deriving connectivity graphs. In this study, we take a compositional approach to segmenting synapses, modeling them explicitly as an intercellular cleft co-located with an asymmetric vesicle density along a cell membrane. Instead of requiring a deep network to learn all natural combinations of this compositionality, we train lighter networks to model the simpler marginal distributions of membranes, clefts and vesicles from just 100 electron microscopy samples. These feature maps are then combined with simple rules-based heuristics derived from prior biological knowledge. Our approach to synapse detection is both more accurate than previous state-of-the-art (7% higher recall and 5% higher F1-score) and yields a 20-fold speed-up compared to the previous fastest implementations. We demonstrate by reconstructing the first complete, directed connectome from the largest available anisotropic microscopy dataset (245 GB) of mouse somatosensory cortex (S1) in just 9.7 hours on a single shared-memory CPU system. We believe that this work marks an important step toward the goal of a microscope-pace streaming connectomics pipeline.

cs.CV↗

arXiv2014

A neural circuit for navigation inspired by C. elegans Chemotaxis

We develop an artificial neural circuit for contour tracking and navigation inspired by the chemotaxis of the nematode Caenorhabditis elegans. In order to harness the computational advantages spiking neural networks promise over their non-spiking counterparts, we develop a network comprising 7-spiking neurons with non-plastic synapses which we show is extremely robust in tracking a range of concentrations. Our worm uses information regarding local temporal gradients in sodium chloride concentration to decide the instantaneous path for foraging, exploration and tracking. A key neuron pair in the C. elegans chemotaxis network is the ASEL & ASER neuron pair, which capture the gradient of concentration sensed by the worm in their graded membrane potentials. The primary sensory neurons for our network are a pair of artificial spiking neurons that function as gradient detectors whose design is adapted from a computational model of the ASE neuron pair in C. elegans. Simulations show that our worm is able to detect the set-point with approximately four times higher probability than the optimal memoryless Levy foraging model. We also show that our spiking neural network is much more efficient and noise-resilient while navigating and tracking a contour, as compared to an equivalent non-spiking network. We demonstrate that our model is extremely robust to noise and with slight modifications can be used for other practical applications such as obstacle avoidance. Our network model could also be extended for use in three-dimensional contour tracking or obstacle avoidance.

cs.NE↗

arXiv2014

Sub-threshold CMOS Spiking Neuron Circuit Design for Navigation Inspired by C. elegans Chemotaxis

We demonstrate a spiking neural network for navigation motivated by the chemotaxis network of Caenorhabditis elegans. Our network uses information regarding temporal gradients in the tracking variable's concentration to make navigational decisions. The gradient information is determined by mimicking the underlying mechanisms of the ASE neurons of C. elegans. Simulations show that our model is able to forage and track a target set-point in extremely noisy environments. We develop a VLSI implementation for the main gradient detector neurons, which could be integrated with standard comparator circuitry to develop a robust circuit for navigation and contour tracking.

cs.NE↗