Product of Tensors and Description of Networks

Abstract

Any kind of network can be naturally represented by a Directed Acyclic Graph (DAG); additionally, activation functions can determine the reaction of each node of the network with respect to the signal(s) incoming. We study the characterization of the signal distribution in a network under the lens of tensor algebra. More specifically, we describe every activation function as tensor distributions with respect to the nodes, called \textit{activation tensors}. The distribution of the signal is encoded in the \textit{total tensor} of the network. We formally prove that the total tensor can be obtained by computing the \textit{Batthacharya-Mesner Product} (BMP), an $n$-ary operation for tensors of order $n$, on the set of the activation tensors properly ordered and processed via two basic operations, that we call \textit{blow} and \textit{forget}. Our theoretical framework can be validated through the related code developed in Python.