A computational model of behavioral adaptation to solve the credit assignment problem

Abstract

The adaptive fitness of an organism in its ecological niche is highly reliant upon its ability to associate an environmental or internal stimulus with a behavior response through reinforcement. This simple but powerful observation has been successfully applied in a number of contexts within computational neuroscience and reinforcement learning to model both human and animal behaviors. However, a critical challenge faced by these models is the credit assignment problem which asks how past behavior comes to be associated with a delayed reinforcement signal. In this paper we reformulate the credit assignment problem to ask how past stimuli come to be linked to adaptive behavioral responses in the context of a simple neuronal circuit. We propose a biologically plausible variant of a spiking neural network which can model a wide variety of behavioral, learning, and evolutionary phenomena. Our model suggests one fundamental mechanism, potentially in use in the brains of both simple and complex organisms, that would allow it to associate a behavior with an adaptive response. We present results that showcase the model's versatility and biological plausibility in a number of tasks related to classical and operant conditioning including behavioral chaining. We then provide further simulations to demonstrate how adaptive behaviors such as reflexes and simple category detection may have evolved using our model. Our results indicate the potential for further modifications and extensions of our model to replicate more sophisticated and biologically plausible behavioral, learning, and intelligence phenomena found throughout the animal kingdom.