arXiv subjects

Feifei Zhao

Publications and source records attributed to Feifei Zhao.

47 records · Page 3Linked to original sources

arXiv2023

Brain-inspired Evolutionary Architectures for Spiking Neural Networks

The complex and unique neural network topology of the human brain formed through natural evolution enables it to perform multiple cognitive functions simultaneously. Automated evolutionary mechanisms of biological network structure inspire us to explore efficient architectural optimization for Spiking Neural Networks (SNNs). Instead of manually designed fixed architectures or hierarchical Network Architecture Search (NAS), this paper evolves SNNs architecture by incorporating brain-inspired local modular structure and global cross-module connectivity. Locally, the brain region-inspired module consists of multiple neural motifs with excitatory and inhibitory connections; Globally, we evolve free connections among modules, including long-term cross-module feedforward and feedback connections. We further introduce an efficient multi-objective evolutionary algorithm based on a few-shot performance predictor, endowing SNNs with high performance, efficiency and low energy consumption. Extensive experiments on static datasets (CIFAR10, CIFAR100) and neuromorphic datasets (CIFAR10-DVS, DVS128-Gesture) demonstrate that our proposed model boosts energy efficiency, archiving consistent and remarkable performance. This work explores brain-inspired neural architectures suitable for SNNs and also provides preliminary insights into the evolutionary mechanisms of biological neural networks in the human brain.

cs.NE↗

arXiv2023

Enhancing Efficient Continual Learning with Dynamic Structure Development of Spiking Neural Networks

Children possess the ability to learn multiple cognitive tasks sequentially, which is a major challenge toward the long-term goal of artificial general intelligence. Existing continual learning frameworks are usually applicable to Deep Neural Networks (DNNs) and lack the exploration on more brain-inspired, energy-efficient Spiking Neural Networks (SNNs). Drawing on continual learning mechanisms during child growth and development, we propose Dynamic Structure Development of Spiking Neural Networks (DSD-SNN) for efficient and adaptive continual learning. When learning a sequence of tasks, the DSD-SNN dynamically assigns and grows new neurons to new tasks and prunes redundant neurons, thereby increasing memory capacity and reducing computational overhead. In addition, the overlapping shared structure helps to quickly leverage all acquired knowledge to new tasks, empowering a single network capable of supporting multiple incremental tasks (without the separate sub-network mask for each task). We validate the effectiveness of the proposed model on multiple class incremental learning and task incremental learning benchmarks. Extensive experiments demonstrated that our model could significantly improve performance, learning speed and memory capacity, and reduce computational overhead. Besides, our DSD-SNN model achieves comparable performance with the DNNs-based methods, and significantly outperforms the state-of-the-art (SOTA) performance for existing SNNs-based continual learning methods.

cs.AI↗

arXiv2023

BrainCog: A Spiking Neural Network based Brain-inspired Cognitive Intelligence Engine for Brain-inspired AI and Brain Simulation

Spiking neural networks (SNNs) have attracted extensive attentions in Brain-inspired Artificial Intelligence and computational neuroscience. They can be used to simulate biological information processing in the brain at multiple scales. More importantly, SNNs serve as an appropriate level of abstraction to bring inspirations from brain and cognition to Artificial Intelligence. In this paper, we present the Brain-inspired Cognitive Intelligence Engine (BrainCog) for creating brain-inspired AI and brain simulation models. BrainCog incorporates different types of spiking neuron models, learning rules, brain areas, etc., as essential modules provided by the platform. Based on these easy-to-use modules, BrainCog supports various brain-inspired cognitive functions, including Perception and Learning, Decision Making, Knowledge Representation and Reasoning, Motor Control, and Social Cognition. These brain-inspired AI models have been effectively validated on various supervised, unsupervised, and reinforcement learning tasks, and they can be used to enable AI models to be with multiple brain-inspired cognitive functions. For brain simulation, BrainCog realizes the function simulation of decision-making, working memory, the structure simulation of the Neural Circuit, and whole brain structure simulation of Mouse brain, Macaque brain, and Human brain. An AI engine named BORN is developed based on BrainCog, and it demonstrates how the components of BrainCog can be integrated and used to build AI models and applications. To enable the scientific quest to decode the nature of biological intelligence and create AI, BrainCog aims to provide essential and easy-to-use building blocks, and infrastructural support to develop brain-inspired spiking neural network based AI, and to simulate the cognitive brains at multiple scales. The online repository of BrainCog can be found at https://github.com/braincog-x.

cs.NE↗

arXiv2023

Adaptive structure evolution and biologically plausible synaptic plasticity for recurrent spiking neural networks

The architecture design and multi-scale learning principles of the human brain that evolved over hundreds of millions of years are crucial to realizing human-like intelligence. Spiking Neural Network (SNN) based Liquid State Machine (LSM) serves as a suitable architecture to study brain-inspired intelligence because of its brain-inspired structure and the potential for integrating multiple biological principles. Existing researches on LSM focus on different certain perspectives, including high-dimensional encoding or optimization of the liquid layer, network architecture search, and application to hardware devices. There is still a lack of in-depth inspiration from the learning and structural evolution mechanism of the brain. Considering these limitations, this paper presents a novel LSM learning model that integrates adaptive structural evolution and multi-scale biological learning rules. For structural evolution, an adaptive evolvable LSM model is developed to optimize the neural architecture design of liquid layer with separation property. For brain-inspired learning of LSM, we propose a dopamine-modulated Bienenstock-Cooper-Munros (DA-BCM) method that incorporates global long-term dopamine regulation and local trace-based BCM synaptic plasticity. Comparative experimental results on different decision-making tasks show that introducing structural evolution of the liquid layer, and the DA-BCM regulation of the liquid layer and the readout layer could improve the decision-making ability of LSM and flexibly adapt to rule reversal. This work is committed to exploring how evolution can help to design more appropriate network architectures and how multi-scale neuroplasticity principles coordinated to enable the optimization and learning of LSMs for relatively complex decision-making tasks.

cs.NE↗

arXiv2023

Adaptive Sparse Structure Development with Pruning and Regeneration for Spiking Neural Networks

Spiking Neural Networks (SNNs) are more biologically plausible and computationally efficient. Therefore, SNNs have the natural advantage of drawing the sparse structural plasticity of brain development to alleviate the energy problems of deep neural networks caused by their complex and fixed structures. However, previous SNNs compression works are lack of in-depth inspiration from the brain development plasticity mechanism. This paper proposed a novel method for the adaptive structural development of SNN (SD-SNN), introducing dendritic spine plasticity-based synaptic constraint, neuronal pruning and synaptic regeneration. We found that synaptic constraint and neuronal pruning can detect and remove a large amount of redundancy in SNNs, coupled with synaptic regeneration can effectively prevent and repair over-pruning. Moreover, inspired by the neurotrophic hypothesis, neuronal pruning rate and synaptic regeneration rate were adaptively adjusted during the learning-while-pruning process, which eventually led to the structural stability of SNNs. Experimental results on spatial (MNIST, CIFAR-10) and temporal neuromorphic (N-MNIST, DVS-Gesture) datasets demonstrate that our method can flexibly learn appropriate compression rate for various tasks and effectively achieve superior performance while massively reducing the network energy consumption. Specifically, for the spatial MNIST dataset, our SD-SNN achieves 99.51\% accuracy at the pruning rate 49.83\%, which has a 0.05\% accuracy improvement compared to the baseline without compression. For the neuromorphic DVS-Gesture dataset, 98.20\% accuracy with 1.09\% improvement is achieved by our method when the compression rate reaches 55.50\%.

cs.NE↗

arXiv2018

A note on Li-Yau type gradient estimate

In this paper, we obtain Li-Yau type gradient estimates with time dependent parameter for positive solutions of the heat equation that are different with the estimates by Li-Xu \cite{LX} and Qian \cite{Qi}. As an application of the estimate, we also obtained improvements of Davies' Li-Yau type gradient estimate.

math.DG↗

arXiv2018

Li-Yau multiplier set and optimal Li-Yau gradient estimate on hyperbolic spaces

In this paper, motivated by finding sharp Li-Yau type gradient estimate for positive solution of heat equations on complete Riemannian manifolds with negative Ricci curvature lower bound, we first introduce the notion of Li-Yau multiplier set and show that it can be computed by heat kernel of the manifold. Then, an optimal Li-Yau type gradient estimate is obtained on hyperbolic spaces by using recurrence relations of heat kernels on hyperbolic spaces. Finally, as an application, we obtain sharp Harnack inequalities on hyperbolic spaces.

math.DG↗

arXiv2018

Sharp Li-Yau type gradient estimates on hyperbolic spaces

In this paper, motivated by the works of Bakry et. al in finding sharp Li-Yau type gradient estimate for positive solutions of the heat equation on complete Riemannian manifolds with nonzero Ricci curvature lower bound, we first introduce a general form of Li-Yau type gradient estimate and show that the validity of such an estimate for any positive solutions of the heat equation reduces to the validity of the estimate for the heat kernel of the Riemannian manifold. Then, a sharp Li-Yau type gradient estimate on the three dimensional hyperbolic space is obtained by using the explicit expression of the heat kernel and some optimal Li-Yau type gradient estimates on general hyperbolic spaces are obtained.

math.DG↗

arXiv2018

Heat kernel recurrence on space forms and applications

In this paper, we first give a direct proof for two recurrence relations of the heat kernels for hyperbolic spaces in \cite{DM}. Then, by similar computation, we give two similar recurrence relations of the heat kernels for spheres. Finally, as an application, we compute the diagonal of heat kernels for odd dimensional hyperbolic spaces and the heat trace asymptotic expansions for odd dimensional spheres.

math.DG↗

arXiv2017

Affine Self-similar solutions of the affine curve shortening flow I: The degenerate case

In this paper, we consider affine self-similar solutions for the affine curve shortening flow in the Euclidean plane. We obtain the equations of all affine self-similar solutions up to affine transformations and solve the equations or give descriptions of the solutions for the degenerate case. Some new special solutions for the affine curve shortening flow are found.

math.DG↗

arXiv2017

On the Li-Yau type gradient estimate of Li and Xu

In this paper, we obtain a Li-Yau type gradient estimate with time dependent parameter for positive solutions of the heat equation, so that the Li-Yau type gradient estimate of Li-Xu are special cases of the estimate. We also obtain improvements of Davies' Li-Yau type gradient estimate. The argument is different with those of Li-Xu and Qian.

math.DG↗