arXiv subjects

Kenric P. Nelson

Publications and source records attributed to Kenric P. Nelson.

At least 19 recordsLinked to original sources

arXiv2026

The unique, universal entropy for complex systems

An axiomatic foundation regarding the entropy for complex systems is established. Missing from decades of research was the requirement that entropy must measure the uncertainty at the informational scale of the maximizing distribution, where the log-log slope equals $-1$. Additionally, entropy must be extensive across the full universality scaling classes defined by Hanel-Thurner. The coupled entropy, maximized by the coupled stretched exponential distributions, is proven to be the unique, universal entropy that satisfies these requirements. The non-additivity of the entropy is equal to the long-range dependence or nonlinear statistical coupling. The entropy-matched extensivity is a function of the coupling, stretching parameter, and dimensions. Evidence is provided that the Tsallis $q$-statistics creates misalignment in the physical modeling of complex systems. Information thermodynamic applications are reviewed, including measuring complexity, a zeroth law of temperature, the thermodynamic consistency of the coupled free energy, and a model of intelligence in non-equilibrium.

cond-mat.stat-mech↗

arXiv2026

On the uniqueness of the coupled entropy

The coupled entropy, $H_κ,$ is proven to uniquely satisfy the requirement that a generalized entropy be a measure of the uncertainty at the scale, $σ,$ for a class of non-exponential distributions. The coupled stretched exponential distributions, including the generalized Pareto and Student's t distributions, are uniquely parameterized to quantify linear uncertainty with the scale and nonlinear uncertainty with the tail shape for a broad class of complex systems. Thereby, the coupled entropy optimizes the representation of the uncertainty due to linear sources. Lemmas for the composability and extensivity of the coupled entropy are proven. The uniqueness of the coupled entropy is further supported by demonstrating consistent thermodynamic relationships, which correspond to a model used for the momentum of high-energy particle collisions. Applications of the coupled entropy in measuring statistical complexity, training variational inference algorithms, and designing communication channels are reviewed.

cond-mat.stat-mech↗

arXiv2025

Coupled Entropy: A Goldilocks Generalization for Complex Systems

The coupled entropy is proven to correct a flaw in the derivation of the Tsallis entropy and thereby solidify the theoretical foundations for analyzing the uncertainty of complex systems. The Tsallis entropy originated from considering power probabilities $p_i^q$ in which \textit{q} independent, identically-distributed random variables share the same state. The maximum entropy distribution was derived to be a \textit{q}-exponential, which is a member of the shape ($κ$), scale ($σ$) distributions. Unfortunately, the $q$-exponential parameters were treated as though valid substitutes for the shape and scale. This flaw causes a misinterpretation of the generalized temperature and an imprecise derivation of the generalized entropy. The coupled entropy is derived from the generalized Pareto distribution (GPD) and the Student's t distribution, whose shape derives from nonlinear sources and scale derives from linear sources of uncertainty. The Tsallis entropy of the GPD converges to one as $κ\rightarrow\infty$, which makes it too cold. The normalized Tsallis entropy (NTE) introduces a nonlinear term multiplying the scale and the coupling, making it too hot. The coupled entropy provides perfect balance, ranging from $\ln σ$ for $κ=0$ to $σ$ as $κ\rightarrow\infty$. One could say, the coupled entropy allows scientists, engineers, and analysts to eat their porridge, confident that its measure of uncertainty reflects the mathematical physics of the scale of non-exponential distributions while minimizing the dependence on the shape or nonlinear coupling. Examples of complex systems design including a coupled variation inference algorithm are reviewed.

stat.ML↗

arXiv2025

Independent Approximates provide a maximum likelihood estimate for heavy-tailed distributions

Heavy-tailed distributions are infamously difficult to estimate because their moments tend to infinity as the shape of the tail decay increases. Nevertheless, this study shows the utilization of a modified group of moments for estimating a heavy-tailed distribution. These modified moments are determined from powers of the original distribution. The nth-power distribution is guaranteed to have finite moments up to n-1. Samples from the nth-power distribution are drawn from n-tuple Independent Approximates, which are the set of independent samples grouped into n-tuples and sub-selected to be approximately equal to each other. We show that Independent Approximates are a maximum likelihood estimator for the generalized Pareto and the Student's t distributions, which are members of the family of coupled exponential distributions. We use the first (original), second, and third power distributions to estimate their zeroth (geometric mean), first, and second power-moments respectively. In turn, these power-moments are used to estimate the scale and shape of the distributions. A least absolute deviation criteria is used to select the optimal set of Independent Approximates. Estimates using higher powers and moments are possible though the number of n-tuples that are approximately equal may be limited.

stat.ME↗

arXiv2024

Three-state Opinion Dynamics for Financial Markets on Complex Networks

This work investigates the effects of complex networks on the collective behavior of a three-state opinion formation model in economic systems. Our model considers two distinct types of investors in financial markets: noise traders and fundamentalists. Financial states evolve via probabilistic dynamics that include economic strategies with local and global influences. The local majoritarian opinion drives noise traders' market behavior, while the market index influences the financial decisions of fundamentalist agents. We introduce a level of market anxiety $q$ present in the decision-making process that influences financial action. In our investigation, nodes of a complex network represent market agents, whereas the links represent their financial interactions. We investigate the stochastic dynamics of the model on three distinct network topologies, including scale-free networks, small-world networks and Erd{ö}s-Rényi random graphs. Our model mirrors various traits observed in real-world financial return series, such as heavy-tailed return distributions, volatility clustering, and short-term memory correlation of returns. The histograms of returns are fitted by coupled Gaussian distributions, quantitatively revealing transitions from a leptokurtic to a mesokurtic regime under specific economic heterogeneity. We show that the market dynamics depend mainly on the average agent connectivity, anxiety level, and market composition rather than on specific features of network topology.

physics.soc-ph↗

arXiv2024

Open Problems within Nonextensive Statistical Mechanics

Nonextensive Statistical Mechanics has developed into an important framework for modeling the thermodynamics of complex systems and the information of complex signals. Upon the 80th birthday of the field's founder, Constantino Tsallis, a review of open problems that can stimulate future research is provided. Over the thirty-year development of NSM a variety of criticisms have been published ranging from questions about the justification for generalizing the entropy function to interpretation of the generalizing parameter q. While these criticisms have been addressed in the past and the breadth of applications has demonstrated the utility of the NSM methodologies, this review provides insights on how the field can continue to improve the understanding and application of complex system models. The review starts by grounding q-statistics within scale-shape distributions and then frames a series of open problems for investigation. The open problems include using the degree of freedom to quantify the difference between entropy and its generalization; clarifying the physical interpretation of the parameter q; improving the definition of the generalized product using multidimensional analysis; defining a generalized Fourier transform applicable to signal processing applications; and re-examination of the normalization of nonextensive entropy. The review concludes with a proposal that the shape parameter is a candidate for defining the statistical complexity of a system.

cond-mat.stat-mech↗

arXiv2022

Independent Approximates enable closed-form estimation of heavy-tailed distributions

A new statistical estimation method, Independent Approximates (IAs), is defined and proven to enable closed-form estimation of the parameters of heavy-tailed distributions. Given independent, identically distributed samples from a one-dimensional distribution, IAs are formed by partitioning samples into pairs, triplets, or nth-order groupings and retaining the median of those groupings that are approximately equal. The pdf of the IAs is proven to be the normalized n^th power of the original density. From this property, heavy-tailed distributions are proven to have well-defined means for their IA pairs, finite second moments for their IA triplets, and a finite, well-defined (n-1)^th moment for the nth grouping. Estimation of the location, scale, and shape (inverse of degree of freedom) of the generalized Pareto and Student's t distributions are possible via a system of three equations. Performance analysis of the IA estimation methodology for the Student's t distribution demonstrates that the method converges to the maximum likelihood estimate. Closed-form estimates of the location and scale are determined from the mean of the IA pairs and the second moment of the IA triplets, respectively. For the Student's t distribution, the geometric mean of the original samples provides a third equation to determine the shape, though its nonlinear solution requires an iterative solver. With 10,000 samples the relative bias of the parameter estimates is less than 0.01 and the relative precision is less than +/- 0.1. Statistical physics applications are carried out for both a small sample (331) astrophysics dataset and a large sample (2 x 10^8) standard map simulation.

stat.ME↗

arXiv2022

Opinion Dynamics in Financial Markets via Random Networks

We investigate the financial market dynamics by introducing a heterogeneous agent-based opinion formation model. In this work, we organize the individuals in a financial market by their trading strategy, namely noise traders and fundamentalists. The opinion of a local majority compels the market exchanging behavior of noise traders, whereas the global behavior of the market influences the fundamentalist agents' decisions. We introduce a noise parameter $q$ to represent a level of anxiety and perceived uncertainty regarding the market behavior, enabling the possibility for an adrift financial action. We place the individuals as nodes in an Erdös-Rényi random graph, where the links represent their social interaction. At a given time, they assume one of two possible opinion states $\pm 1$ regarding buying or selling an asset. The model exhibits such fundamental qualitative and quantitative real-world market features as the distribution of logarithmic returns with fat-tails, clustered volatility, and long-term correlation of returns. We use Student's t distributions to fit the histograms of logarithmic returns, showing the gradual shift from a leptokurtic to a mesokurtic regime, depending on the fraction of fundamentalist agents. We also compare our results with the distribution of logarithmic returns of several real-world financial indices.

q-fin.ST↗

arXiv2021

Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder

We present a coupled Variational Auto-Encoder (VAE) method that improves the accuracy and robustness of the probabilistic inferences on represented data. The new method models the dependency between input feature vectors (images) and weighs the outliers with a higher penalty by generalizing the original loss function to the coupled entropy function, using the principles of nonlinear statistical coupling. We evaluate the performance of the coupled VAE model using the MNIST dataset. Compared with the traditional VAE algorithm, the output images generated by the coupled VAE method are clearer and less blurry. The visualization of the input images embedded in 2D latent variable space provides a deeper insight into the structure of new model with coupled loss function: the latent variable has a smaller deviation and a more compact latent space generates the output values. We analyze the histogram of the likelihoods of the input images using the generalized mean, which measures the model's accuracy as a function of the relative risk. The neutral accuracy, which is the geometric mean and is consistent with a measure of the Shannon cross-entropy, is improved. The robust accuracy, measured by the -2/3 generalized mean, is also improved.

cs.LG↗

arXiv2020

Use of Student's t-Distribution for the Latent Layer in a Coupled Variational Autoencoder

A Coupled Variational Autoencoder, which incorporates both a generalized loss function and latent layer distribution, shows improvement in the accuracy and robustness of generated replicas of MNIST numerals. The latent layer uses a Student's t-distribution to incorporate heavy-tail decay. The loss function uses a coupled logarithm, which increases the penalty on images with outlier likelihood. The generalized mean of the generated image's likelihood is used to measure the performance of the algorithm's decisiveness, accuracy, and robustness.

cs.LG↗

arXiv2020

Applying the Decisiveness and Robustness Metrics to Convolutional Neural Networks

We review three recently-proposed classifier quality metrics and consider their suitability for large-scale classification challenges such as applying convolutional neural networks to the 1000-class ImageNet dataset. These metrics, referred to as the "geometric accuracy," "decisiveness," and "robustness," are based on the generalized mean ($ρ$ equals 0, 1, and -2/3, respectively) of the classifier's self-reported and measured probabilities of correct classification. We also propose some minor clarifications to standardize the metric definitions. With these updates, we show some examples of calculating the metrics using deep convolutional neural networks (AlexNet and DenseNet) acting on large datasets (the German Traffic Sign Recognition Benchmark and ImageNet).

cs.LG↗

arXiv2019

A Three-state Opinion Formation Model for Financial Markets

We propose a three-state microscopic opinion formation model for the purpose of simulating the dynamics of financial markets. In order to mimic the heterogeneous composition of the mass of investors in a market, the agent-based model considers two different types of traders: noise traders and contrarians. Agents are represented as nodes in a network of interactions and they can assume any of three distinct possible states (e.g. buy, sell or remain inactive). The time evolution of the state of an agent is dictated by probabilistic dynamics that include both local and global influences. A noise trader is subject to local interactions, tending to assume the majority state of its nearest neighbors, whilst a contrarian is subject to a global interaction with the behavior of the market as a whole, tending to assume the state of the global minority of the market. The model exhibits the typical qualitative and quantitative features of real financial time series, including distributions of returns with heavy tails, volatility clustering and long-time memory for the absolute values of the returns. The distributions of returns are fitted by means of coupled Gaussian distributions, quantitatively revealing transitions between leptokurtic, mesokurtic and platykurtic regimes in terms of a non-linear statistical coupling which describes the complexity of the system.

physics.soc-ph↗

arXiv2018

Reduced Perplexity: A simplified perspective on assessing probabilistic forecasts

A simple, intuitive approach to the assessment of probabilistic inferences is introduced. The Shannon information metrics are translated to the probability domain. The translation shows that the negative logarithmic score and the geometric mean are equivalent measures of the accuracy of a probabilistic inference. Thus there is both a quantitative reduction in perplexity, which is the inverse of the geometric mean of the probabilities, as good inference algorithms reduce the uncertainty and a qualitative reduction due to the increased clarity between the original set of probabilistic forecasts and their central tendency, the geometric mean. Further insight is provided by showing that the Rényi and Tsallis entropy functions translated to the probability domain are both the weighted generalized mean of the distribution. The generalized mean of probabilistic forecasts forms a spectrum of performance metrics referred to as a Risk Profile. The arithmetic mean is used to measure the decisiveness, while the -2/3 mean is used to measure the robustness.

stat.OT↗

arXiv2018

Use of the geometric mean as a statistic for the scale of the coupled Gaussian distributions

The geometric mean is shown to be an appropriate statistic for the scale of a heavy-tailed coupled Gaussian distribution or equivalently the Student's t distribution. The coupled Gaussian is a member of a family of distributions parameterized by the nonlinear statistical coupling which is the reciprocal of the degree of freedom and is proportional to fluctuations in the inverse scale of the Gaussian. Existing estimators of the scale of the coupled Gaussian have relied on estimates of the full distribution, and they suffer from problems related to outliers in heavy-tailed distributions. In this paper, the scale of a coupled Gaussian is proven to be equal to the product of the generalized mean and the square root of the coupling. From our numerical computations of the scales of coupled Gaussians using the generalized mean of random samples, it is indicated that only samples from a Cauchy distribution (with coupling parameter one) form an unbiased estimate with diminishing variance for large samples. Nevertheless, we also prove that the scale is a function of the geometric mean, the coupling term and a harmonic number. Numerical experiments show that this estimator is unbiased with diminishing variance for large samples for a broad range of coupling values.

stat.ME↗

arXiv2016

On the average uncertainty for systems with nonlinear coupling

The increased uncertainty and complexity of nonlinear systems have motivated investigators to consider generalized approaches to defining an entropy function. New insights are achieved by defining the average uncertainty in the probability domain as a transformation of entropy functions. The Shannon entropy when transformed to the probability domain is the weighted geometric mean of the probabilities. For the exponential and Gaussian distributions, we show that the weighted geometric mean of the distribution is equal to the density of the distribution at the location plus the scale, i.e. at the width of the distribution. The average uncertainty is generalized via the weighted generalized mean, in which the moment is a function of the nonlinear source. Both the Renyi and Tsallis entropies transform to this definition of the generalized average uncertainty in the probability domain. For the generalized Pareto and Student's t-distributions, which are the maximum entropy distributions for these generalized entropies, the appropriate weighted generalized mean also equals the density of the distribution at the location plus scale. A coupled entropy function is proposed, which is equal to the normalized Tsallis entropy divided by one plus the coupling.

math.ST↗

arXiv2015

A definition of the coupled-product for multivariate coupled-exponentials

The coupled-product and coupled-exponential of the generalized calculus of nonextensive statistical mechanics are defined for multivariate functions. The nonlinear statistical coupling is indexed such that k_d = k/(1+dk), where d is the dimensions of the argument of the multivariate coupled-exponential. The coupled-Gaussian distribution is defined such that the argument of the coupled-exponential depends on the coupled-moments but not the coupling parameter. The multivariate version of the coupled-product is defined such that the output dimensions are the sum of the input dimensions. This enables construction of the multivariate coupled-Gaussian from univariate coupled-Gaussians. The resulting construction forms a model of coupling between distributions, generalizing the product of independent Gaussians.

cond-mat.stat-mech↗

arXiv2014

Probabilistic graphs using coupled random variables

Neural network design has utilized flexible nonlinear processes which can mimic biological systems, but has suffered from a lack of traceability in the resulting network. Graphical probabilistic models ground network design in probabilistic reasoning, but the restrictions reduce the expressive capability of each node making network designs complex. The ability to model coupled random variables using the calculus of nonextensive statistical mechanics provides a neural node design incorporating nonlinear coupling between input states while maintaining the rigor of probabilistic reasoning. A generalization of Bayes rule using the coupled product enables a single node to model correlation between hundreds of random variables. A coupled Markov random field is designed for the inferencing and classification of UCI's MLR 'Multiple Features Data Set' such that thousands of linear correlation parameters can be replaced with a single coupling parameter with just a (3%, 4%) percent reduction in (classification, inference) performance.

cs.LG↗

arXiv2011

A risk profile for information fusion algorithms

E.T. Jaynes, originator of the maximum entropy interpretation of statistical mechanics, emphasized that there is an inevitable trade-off between the conflicting requirements of robustness and accuracy for any inferencing algorithm. This is because robustness requires discarding of information in order to reduce the sensitivity to outliers. The principal of nonlinear statistical coupling, which is an interpretation of the Tsallis entropy generalization, can be used to quantify this trade-off. The coupled-surprisal, -ln_k (p)=-(p^k-1)/k, is a generalization of Shannon surprisal or the logarithmic scoring rule, given a forecast p of a true event by an inferencing algorithm. The coupling parameter k=1-q, where q is the Tsallis entropy index, is the degree of nonlinear coupling between statistical states. Positive (negative) values of nonlinear coupling decrease (increase) the surprisal information metric and thereby biases the risk in favor of decisive (robust) algorithms relative to the Shannon surprisal (k=0). We show that translating the average coupled-surprisal to an effective probability is equivalent to using the generalized mean of the true event probabilities as a scoring rule. The metric is used to assess the robustness, accuracy, and decisiveness of a fusion algorithm. We use a two-parameter fusion algorithm to combine input probabilities from N sources. The generalized mean parameter 'alpha' varies the degree of smoothing and raising to a power N^beta with beta between 0 and 1 provides a model of correlation.

cs.IT↗