arXiv subjects

Aran Nayebi

Publications and source records attributed to Aran Nayebi.

32 records · Page 2Linked to original sources

arXiv2019

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction

Recently, deep feedforward neural networks have achieved considerable success in modeling biological sensory processing, in terms of reproducing the input-output map of sensory neurons. However, such models raise profound questions about the very nature of explanation in neuroscience. Are we simply replacing one complex system (a biological circuit) with another (a deep network), without understanding either? Moreover, beyond neural representations, are the deep network's computational mechanisms for generating neural responses the same as those in the brain? Without a systematic approach to extracting and understanding computational mechanisms from deep neural network models, it can be difficult both to assess the degree of utility of deep learning approaches in neuroscience, and to extract experimentally testable hypotheses from deep networks. We develop such a systematic approach by combining dimensionality reduction and modern attribution methods for determining the relative importance of interneurons for specific visual computations. We apply this approach to deep network models of the retina, revealing a conceptual understanding of how the retina acts as a predictive feature extractor that signals deviations from expectations for diverse spatiotemporal stimuli. For each stimulus, our extracted computational mechanisms are consistent with prior scientific literature, and in one case yields a new mechanistic hypothesis. Thus overall, this work not only yields insights into the computational mechanisms underlying the striking predictive capabilities of the retina, but also places the framework of deep networks as neuroscientific models on firmer theoretical foundations, by providing a new roadmap to go beyond comparing neural representations to extracting and understand computational mechanisms.

q-bio.NC↗

arXiv2019

Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs

Deep convolutional artificial neural networks (ANNs) are the leading class of candidate models of the mechanisms of visual processing in the primate ventral stream. While initially inspired by brain anatomy, over the past years, these ANNs have evolved from a simple eight-layer architecture in AlexNet to extremely deep and branching architectures, demonstrating increasingly better object categorization performance, yet bringing into question how brain-like they still are. In particular, typical deep models from the machine learning community are often hard to map onto the brain's anatomy due to their vast number of layers and missing biologically-important connections, such as recurrence. Here we demonstrate that better anatomical alignment to the brain and high performance on machine learning as well as neuroscience measures do not have to be in contradiction. We developed CORnet-S, a shallow ANN with four anatomically mapped areas and recurrent connectivity, guided by Brain-Score, a new large-scale composite of neural and behavioral benchmarks for quantifying the functional fidelity of models of the primate ventral visual stream. Despite being significantly shallower than most models, CORnet-S is the top model on Brain-Score and outperforms similarly compact models on ImageNet. Moreover, our extensive analyses of CORnet-S circuitry variants reveal that recurrence is the main predictive factor of both Brain-Score and ImageNet top-1 performance. Finally, we report that the temporal evolution of the CORnet-S "IT" neural population resembles the actual monkey IT population dynamics. Taken together, these results establish CORnet-S, a compact, recurrent ANN, as the current best model of the primate ventral visual stream.

cs.CV↗

arXiv2018

Task-Driven Convolutional Recurrent Models of the Visual System

Feed-forward convolutional neural networks (CNNs) are currently state-of-the-art for object classification tasks such as ImageNet. Further, they are quantitatively accurate models of temporally-averaged responses of neurons in the primate brain's visual system. However, biological visual systems have two ubiquitous architectural features not shared with typical CNNs: local recurrence within cortical areas, and long-range feedback from downstream areas to upstream areas. Here we explored the role of recurrence in improving classification performance. We found that standard forms of recurrence (vanilla RNNs and LSTMs) do not perform well within deep CNNs on the ImageNet task. In contrast, novel cells that incorporated two structural features, bypassing and gating, were able to boost task accuracy substantially. We extended these design principles in an automated search over thousands of model architectures, which identified novel local recurrent cells and long-range feedback connections useful for object recognition. Moreover, these task-optimized ConvRNNs matched the dynamics of neural activity in the primate visual system better than feedforward networks, suggesting a role for the brain's recurrent connections in performing difficult visual behaviors.

q-bio.NC↗

arXiv2017

Biologically inspired protection of deep networks from adversarial attacks

Inspired by biophysical principles underlying nonlinear dendritic computation in neural circuits, we develop a scheme to train deep neural networks to make them robust to adversarial attacks. Our scheme generates highly nonlinear, saturated neural networks that achieve state of the art performance on gradient based adversarial examples on MNIST, despite never being exposed to adversarially chosen examples during training. Moreover, these networks exhibit unprecedented robustness to targeted, iterative schemes for generating adversarial examples, including second-order methods. We further identify principles governing how these networks achieve their robustness, drawing on methods from information geometry. We find these networks progressively create highly flat and compressed internal representations that are sensitive to very few input dimensions, while still solving the task. Moreover, they employ highly kurtotic weight distributions, also found in the brain, and we demonstrate how such kurtosis can protect even linear classifiers from adversarial attack.

stat.ML↗

arXiv2017

Deep Learning Models of the Retinal Response to Natural Scenes

A central challenge in neuroscience is to understand neural computations and circuit mechanisms that underlie the encoding of ethologically relevant, natural stimuli. In multilayered neural circuits, nonlinear processes such as synaptic transmission and spiking dynamics present a significant obstacle to the creation of accurate computational models of responses to natural stimuli. Here we demonstrate that deep convolutional neural networks (CNNs) capture retinal responses to natural scenes nearly to within the variability of a cell's response, and are markedly more accurate than linear-nonlinear (LN) models and Generalized Linear Models (GLMs). Moreover, we find two additional surprising properties of CNNs: they are less susceptible to overfitting than their LN counterparts when trained on small amounts of data, and generalize better when tested on stimuli drawn from a different distribution (e.g. between natural scenes and white noise). Examination of trained CNNs reveals several properties. First, a richer set of feature maps is necessary for predicting the responses to natural scenes compared to white noise. Second, temporally precise responses to slowly varying inputs originate from feedforward inhibition, similar to known retinal mechanisms. Third, the injection of latent noise sources in intermediate layers enables our model to capture the sub-Poisson spiking variability observed in retinal ganglion cells. Fourth, augmenting our CNNs with recurrent lateral connections enables them to capture contrast adaptation as an emergent property of accurately describing retinal responses to natural scenes. These methods can be readily generalized to other sensory modalities and stimulus ensembles. Overall, this work demonstrates that CNNs not only accurately capture sensory circuit responses to natural scenes, but also yield information about the circuit's internal structure and function.

q-bio.NC↗

arXiv2015

Quantum lower bound for inverting a permutation with advice

Given a random permutation $f: [N] \to [N]$ as a black box and $y \in [N]$, we want to output $x = f^{-1}(y)$. Supplementary to our input, we are given classical advice in the form of a pre-computed data structure; this advice can depend on the permutation but \emph{not} on the input $y$. Classically, there is a data structure of size $\tilde{O}(S)$ and an algorithm that with the help of the data structure, given $f(x)$, can invert $f$ in time $\tilde{O}(T)$, for every choice of parameters $S$, $T$, such that $S\cdot T \ge N$. We prove a quantum lower bound of $T^2\cdot S \ge \tildeΩ(εN)$ for quantum algorithms that invert a random permutation $f$ on an $ε$ fraction of inputs, where $T$ is the number of queries to $f$ and $S$ is the amount of advice. This answers an open question of De et al. We also give a $Ω(\sqrt{N/m})$ quantum lower bound for the simpler but related Yao's box problem, which is the problem of recovering a bit $x_j$, given the ability to query an $N$-bit string $x$ at any index except the $j$-th, and also given $m$ bits of advice that depend on $x$ but not on $j$.

quant-ph↗

arXiv2014

Quantum algorithms for shortest paths problems in structured instances

We consider the quantum time complexity of the all pairs shortest paths (APSP) problem and some of its variants. The trivial classical algorithm for APSP and most all pairs path problems runs in $O(n^3)$ time, while the trivial algorithm in the quantum setting runs in $\tilde{O}(n^{2.5})$ time, using Grover search. A major open problem in classical algorithms is to obtain a truly subcubic time algorithm for APSP, i.e. an algorithm running in $O(n^{3-\varepsilon})$ time for constant $\varepsilon>0$. To approach this problem, many truly subcubic time classical algorithms have been devised for APSP and its variants for structured inputs. Some examples of such problems are APSP in geometrically weighted graphs, graphs with small integer edge weights or a small number of weights incident to each vertex, and the all pairs earliest arrivals problem. In this paper we revisit these problems in the quantum setting and obtain the first nontrivial (i.e. $O(n^{2.5-\varepsilon})$ time) quantum algorithms for the problems.

quant-ph↗

arXiv2013

On the distribution of Carmichael numbers

Erdős conjectured in 1956 that there are $x^{1-o(1)}$ Carmichael numbers up to $x$. Pomerance made this conjecture more precise and proposed that there are $x^{1-{\frac{\{1+o(1)\}\log\log\log x}{\log\log x}}}$ Carmichael numbers up to $x$. At the time, his data tables up to $25 \cdot 10^{9}$ appeared to support his conjecture. However, Pinch extended this data and showed that up to $10^{21}$, Pomerance's conjecture did not appear well-supported. Thus, the purpose of this paper is two-fold. First, we build upon the work of Pomerance and others to present an alternate conjecture regarding the distribution of Carmichael numbers that fits proven bounds and is better supported by Pinch's new data. Second, we provide another conjecture concerning the distribution of Carmichael numbers that sharpens Pomerance's heuristic arguments. We also extend and update counts pertaining to pseudoprimes and Carmichael numbers, and discuss the distribution of One-Parameter Quadratic-Base Test pseudoprimes.

math.NT↗

arXiv2013

Practical intractability: a critique of the hypercomputation movement

For over a decade, the hypercomputation movement has produced computational models that in theory solve the algorithmically unsolvable, but they are not physically realizable according to currently accepted physical theories. While opponents to the hypercomputation movement provide arguments against the physical realizability of specific models in order to demonstrate this, these arguments lack the generality to be a satisfactory justification against the construction of \emph{any} information-processing machine that computes beyond the universal Turing machine. To this end, I present a more mathematically concrete challenge to hypercomputability, and will show that one is immediately led into physical impossibilities, thereby demonstrating the infeasibility of hypercomputers more generally. This gives impetus to propose and justify a more plausible starting point for an extension to the classical paradigm that is physically possible, at least in principle. Instead of attempting to rely on infinities such as idealized limits of infinite time or numerical precision, or some other physically unattainable source, one should focus on extending the classical paradigm to better encapsulate modern computational problems that are not well-expressed/modeled by the closed-system paradigm of the Turing machine. I present the first steps toward this goal by considering contemporary computational problems dealing with intractability and issues surrounding cyber-physical systems, and argue that a reasonable extension to the classical paradigm should focus on these issues in order to be practically viable.

math.LO↗

arXiv2013

Exponential prefixed polynomial equations

A prefixed polynomial equation is an equation of the form $P(t_1,\ldots,t_n) = 0$, where $P$ is a polynomial whose variables $t_1,\ldots,t_n$ range over the natural numbers, preceded by quantifiers over some, or all, of its variables. Here, we consider exponential prefixed polynomial equations (EPPEs), where variables can also occur as exponents. We obtain a relatively concise EPPE equivalent to the combinatorial principle of the Paris-Harrington theorem for pairs (which is independent of primitive recursive arithmetic), as well as an EPPE equivalent to Goodstein's theorem (which is independent of Peano arithmetic). Some new devices are used in addition to known methods for the elimination of bounded universal quantifiers for Diophantine predicates.

math.LO↗

arXiv2012

Upper bounds on the solutions to $n = p+m^2$

Hardy and Littlewood conjectured that every large integer $n$ that is not a square is the sum of a prime and a square. They believed that the number $\mathcal{R}(n)$ of such representations for $n = p+m^2$ is asymptotically given by \mathcal{R}(n) \sim \frac{\sqrt{n}}{\log n}\prod_{p=3}^{\infty}(1-\frac{1}{p-1}(\frac{n}{p})), where $p$ is a prime, $m$ is an integer, and $(\frac{n}{p})$ denotes the Legendre symbol. Unfortunately, as we will later point out, this conjecture is difficult to prove and not \emph{all} integers that are nonsquares can be represented as the sum of a prime and a square. Instead in this paper we prove two upper bounds for $\mathcal{R}(n)$ for $n \le N$. The first upper bound applies to \emph{all} $n \le N$. The second upper bound depends on the possible existence of the Siegel zero, and assumes its existence, and applies to all $N/2 < n \le N$ but at most $\ll N^{1-δ_1}$ of these integers, where $N$ is a sufficiently large positive integer and $0< δ_1 \le 0.000025$.

math.NT↗

arXiv2011

Fast matrix multiplication techniques based on the Adleman-Lipton model

On distributed memory electronic computers, the implementation and association of fast parallel matrix multiplication algorithms has yielded astounding results and insights. In this discourse, we use the tools of molecular biology to demonstrate the theoretical encoding of Strassen's fast matrix multiplication algorithm with DNA based on an $n$-moduli set in the residue number system, thereby demonstrating the viability of computational mathematics with DNA. As a result, a general scalable implementation of this model in the DNA computing paradigm is presented and can be generalized to the application of \emph{all} fast matrix multiplication algorithms on a DNA computer. We also discuss the practical capabilities and issues of this scalable implementation. Fast methods of matrix computations with DNA are important because they also allow for the efficient implementation of other algorithms (i.e. inversion, computing determinants, and graph theory) with DNA.

q-bio.QM↗

arXiv2011

On integers as the sum of a prime and a $k$-th power

Let $\mathcal{R}_k(n)$ be the number of representations of an integer $n$ as the sum of a prime and a $k$-th power. Define E_k(X) := |\{n \le X, n \in I_k, n\text{not a sum of a prime and a $k$-th power}\}|. Hardy and Littlewood conjectured that for $k = 2$ and $k=3$, E_k(X) \ll_{k} 1. In this note we present an alternative approach grounded in the theory of Diophantine equations towards a proof of the conjecture for all $k \ge 2$.

math.NT↗

arXiv2011

A Note on the Inverse Laplace Transformation of $f(t)$

Let $\mathcal{L}\{f(t)\} = \int_{0}^{\infty}e^{-st}f(t)dt$ denote the Laplace transform of $f$. It is well-known that if $f(t)$ is a piecewise continuous function on the interval $t:[0,\infty)$ and of exponential order for $t > N$; then $\lim_{s\to\infty}F(s) = 0$, where $F(s) = \mathcal{L}\{f(t)\}$. In this paper we prove that the lesser known converse does not hold true; namely, if $F(s)$ is a continuous function in terms of $s$ for which $\lim_{s\to\infty}F(s) = 0$, then it does not follow that $F(s)$ is the Laplace transform of a piecewise continuous function of exponential order.

math.CA↗