arXiv subjects

Alianna J. Maren

Publications and source records attributed to Alianna J. Maren.

4 recordsLinked to original sources

arXiv2024

Derivation of the Variational Bayes Equations

The derivation of key equations for the variational Bayes approach is well-known in certain circles. However, translating the fundamental derivations (e.g., as found in Beal's work) to Friston's notation is somewhat delicate. Further, the notion of using variational Bayes in the context of a system with a Markov blanket requires special attention. This Technical Report presents the derivation in detail. It further illustrates how the variational Bayes method provides a framework for a new computational engine, incorporating the 2-D cluster variation method (CVM), which provides a necessary free energy equation that can be minimized across both the external and representational systems' states, respectively.

cs.NE↗

arXiv2022

A Variational Approach to Parameter Estimation for Characterizing 2-D Cluster Variation Method Topographies

One of the biggest challenges in characterizing 2-D topographies is succinctly communicating the dominant nature of local configurations. In a 2-D grid composed of bistate units, this could be expressed as finding the characteristic configuration variables such as nearest-neighbor pairs and triplet combinations. The 2-D cluster variation method (CVM) provides a theoretical framework for associating a set of configuration variables with only two parameters, for a system that is at free energy equilibrium. This work presents a method for determining which of many possible two-parameter sets provides the ``most suitable'' match for a given 2-D topography, drawing from methods used for variational inference. This particular work focuses exclusively on topographies for which the activation enthalpy parameter (epsilon_0) is zero, so that the distribution between two states is equiprobable. This condition is used since, when the two states are equiprobable, there is an analytic solution giving the configuration variable values as functions of the h-value, where we define h in terms of the interaction enthalpy parameter (epsilon_1) as h = exp(2*epsilon_1). This allows the computationally-achieved configuration variable values to be compared with the analytically-predicted values for a given h-value. The method is illustrated using four patterns derived from three different naturally-occurring black-and-white topographies, where each pattern meets the equiprobability criterion. We achieve expected results, that is, as the patterns progress from having relatively low numbers of like-near-like nodes to increasing like-near-like masses, the h-values for each corresponding free energy-minimized model also increase. Further, the corresponding configuration variable values for the (free energy-minimized) model patterns are in approximate alignment with the analytically-predicted values.

cs.NE↗

arXiv2019

2-D Cluster Variation Method Free Energy: Fundamentals and Pragmatics

Despite being invented in 1951 by R. Kikuchi, the 2-D Cluster Variation Method (CVM), has not yet received attention. Nevertheless, this method can usefully characterize 2-D topographies using just two parameters; the activation enthalpy and the interaction enthalpy. This Technical Report presents 2-D CVM details, including the dependence of the various configuration variables on the enthalpy parameters, as well as illustrations of various topographies (ranging from scale-free-like to rich club-like) that result from different parameter selection. The complete derivation for the analytic solution, originally presented simply as a result in Kikuchi and Brush (1967) is given here, along with careful comparison of the analytically-predicted configuration variables versus those obtained when performing computational free energy minimization on a 2-D grid. The 2-D CVM can potentially function as a secondary free energy minimization within the hidden layer of a neural network, providing a basis for extending node activations over time and allowing temporal correlation of patterns.

cs.NE↗

arXiv2019

Free Energy Minimization Using the 2-D Cluster Variation Method: Initial Code Verification and Validation

A new approach for general artificial intelligence (GAI), building on neural network deep learning architectures, can make use of one or more hidden layers that have the ability to continuously reach a free energy minimum even after input stimulus is removed, allowing for a variety of possible behaviors. One reason that this approach has not been developed until now has been the lack of a suitable free energy equation. The Cluster Variation Method (CVM) offers a means for characterizing 2-D local pattern distributions, or configuration variables, and provides a free energy formalism in terms of these configuration variables. The equilibrium distribution of these configuration variables is defined in terms of a single interaction enthalpy parameter, h, for the case of equiprobable distribution of bistate units. For non-equiprobable distributions, the equilibrium distribution can be characterized by providing a fixed value for the fraction of units in the active state (x1), corresponding to the influence of a per-unit activation enthalpy, together with the pairwise interaction enthalpy parameter h. This paper provides verification and validation (V&V) for code that computes the configuration variable and thermodynamic values for 2-D CVM grids characterized by different interaction enthalpy parameters, or h-values. This work provides a foundation for experimenting with a 2-D CVM-based hidden layer that can, as an alternative to responding strictly to inputs, also now independently come to its own free energy minimum and also return to a free energy-minimized state after perturbations, which will enable a range of input-independent behaviors. A further use of this 2-D CVM grid is that by characterizing local patterns in terms of their corresponding h-values (together with their x1 values), we have a means for quantitatively characterizing different kinds of neural topographies.

cs.NE↗