Free Energy Minimization Using the 2-D Cluster Variation Method: Initial Code Verification and Validation

Abstract

A new approach for general artificial intelligence (GAI), building on neural network deep learning architectures, can make use of one or more hidden layers that have the ability to continuously reach a free energy minimum even after input stimulus is removed, allowing for a variety of possible behaviors. One reason that this approach has not been developed until now has been the lack of a suitable free energy equation. The Cluster Variation Method (CVM) offers a means for characterizing 2-D local pattern distributions, or configuration variables, and provides a free energy formalism in terms of these configuration variables. The equilibrium distribution of these configuration variables is defined in terms of a single interaction enthalpy parameter, h, for the case of equiprobable distribution of bistate units. For non-equiprobable distributions, the equilibrium distribution can be characterized by providing a fixed value for the fraction of units in the active state (x1), corresponding to the influence of a per-unit activation enthalpy, together with the pairwise interaction enthalpy parameter h. This paper provides verification and validation (V&V) for code that computes the configuration variable and thermodynamic values for 2-D CVM grids characterized by different interaction enthalpy parameters, or h-values. This work provides a foundation for experimenting with a 2-D CVM-based hidden layer that can, as an alternative to responding strictly to inputs, also now independently come to its own free energy minimum and also return to a free energy-minimized state after perturbations, which will enable a range of input-independent behaviors. A further use of this 2-D CVM grid is that by characterizing local patterns in terms of their corresponding h-values (together with their x1 values), we have a means for quantitatively characterizing different kinds of neural topographies.