Zoe Tosi | Searcharxiv

arXiv2021

Is Disaggregation possible for HPC Cognitive Simulation?

Cognitive simulation (CogSim) is an important and emerging workflow for HPC scientific exploration and scientific machine learning (SciML). One challenging workload for CogSim is the replacement of one component in a complex physical simulation with a fast, learned, surrogate model that is "inside" of the computational loop. The execution of this in-the-loop inference is particularly challenging because it requires frequent inference across multiple possible target models, can be on the simulation's critical path (latency bound), is subject to requests from multiple MPI ranks, and typically contains a small number of samples per request. In this paper we explore the use of large, dedicated Deep Learning / AI accelerators that are disaggregated from compute nodes for this CogSim workload. We compare the trade-offs of using these accelerators versus the node-local GPU accelerators on leadership-class HPC systems.

cs.DC↗

arXiv2018

Cortical Circuits from Scratch: A Metaplastic Architecture for the Emergence of Lognormal Firing Rates and Realistic Topology

Our current understanding of neuroplasticity paints a picture of a complex interconnected system of dependent processes which shape cortical structure so as to produce an efficient information processing system. Indeed, the cooperation of these processes is associated with robust, stable, adaptable networks with characteristic features of activity and synaptic topology. However, combining the actions of these mechanisms in models has proven exceptionally difficult and to date no model has been able to do so without significant hand-tuning. Until such a model exists that can successfully combine these mechanisms to form a stable circuit with realistic features, our ability to study neuroplasticity in the context of (more realistic) dynamic networks and potentially reap whatever rewards these features and mechanisms imbue biological networks with is hindered. We introduce a model which combines five known plasticity mechanisms that act on the network as well as a unique metaplastic mechanism which acts on other plasticity mechanisms, to produce a neural circuit model which is both stable and capable of broadly reproducing many characteristic features of cortical networks. The MANA (metaplastic artificial neural architecture) represents the first model of its kind in that it is able to self-organize realistic, nonrandom features of cortical networks, from a null initial state (no synaptic connectivity or neuronal differentiation). In the same vein as models like the SORN (self-organizing recurrent network) MANA represents further progress toward the reverse engineering of the brain at the network level.

q-bio.NC↗