arXiv subjects

Kevin Vanslette

Publications and source records attributed to Kevin Vanslette.

15 recordsLinked to original sources

arXiv2023

Inferential Moments of Uncertain Multivariable Systems

This article expands the framework of Bayesian inference and provides direct probabilistic methods for approaching inference tasks that are typically handled with information theory. We treat Bayesian probability updating as a random process and uncover intrinsic quantitative features of joint probability distributions called inferential moments. Inferential moments quantify shape information about how a prior distribution is expected to update in response to yet to be obtained information. Further, we quantify the unique probability distribution whose statistical moments are the inferential moments in question. We find a power series expansion of the mutual information in terms of inferential moments, which implies a connection between inferential theoretic logic and elements of information theory. Of particular interest is the inferential deviation, which is the expected variation of the probability of one variable in response to an inferential update of another. We explore two applications that analyze the inferential deviations of a Bayesian network to improve decision-making. We implement simple greedy algorithms for exploring sensor tasking using inferential deviations that generally outperform similar greedy mutual information algorithms in terms of root mean squared error between epistemic probability estimates and the ground truth probabilities they are estimating.

physics.data-an↗

arXiv2021

Reliable Neural Networks for Regression Uncertainty Estimation

While deep neural networks are highly performant and successful in a wide range of real-world problems, estimating their predictive uncertainty remains a challenging task. To address this challenge, we propose and implement a loss function for regression uncertainty estimation based on the Bayesian Validation Metric (BVM) framework while using ensemble learning. The proposed loss reproduces maximum likelihood estimation in the limiting case. A series of experiments on in-distribution data show that the proposed method is competitive with existing state-of-the-art methods. Experiments on out-of-distribution data show that the proposed method is robust to statistical change and exhibits superior predictive capability.

cs.LG↗

arXiv2020

A Generalized Bayesian Approach to Model Calibration

In model development, model calibration and validation play complementary roles toward learning reliable models. In this article, we expand the Bayesian Validation Metric framework to a general calibration and validation framework by inverting the validation mathematics into a generalized Bayesian method for model calibration and regression. We perform Bayesian regression based on a user's definition of model-data agreement. This allows for model selection on any type of data distribution, unlike Bayesian and standard regression techniques, that "fail" in some cases. We show that our tool is capable of representing and combining least squares, likelihood-based, and Bayesian calibration techniques in a single framework while being able to generalize aspects of these methods. This tool also offers new insights into the interpretation of the predictive envelopes (also known as confidence bands) while giving the analyst more control over these envelopes. We demonstrate the validity of our method by providing three numerical examples to calibrate different models, including a model for energy dissipation in lap joints under impact loading. By calibrating models with respect to the validation metrics one desires a model to ultimately pass, reliability and safety metrics may be integrated into and automatically adopted by the model in the calibration phase.

stat.ME↗

arXiv2020

The Design of Global Correlation Quantifiers and Continuous Notions of Statistical Sufficiency

Using first principles from inference, we design a set of functionals for the purposes of \textit{ranking} joint probability distributions with respect to their correlations. Starting with a general functional, we impose its desired behaviour through the \textit{Principle of Constant Correlations} (PCC), which constrains the correlation functional to behave in a consistent way under statistically independent inferential transformations. The PCC guides us in choosing the appropriate design criteria for constructing the desired functionals. Since the derivations depend on a choice of partitioning the variable space into $n$ disjoint subspaces, the general functional we design is the $n$-partite information (NPI), of which the \textit{total correlation} and \textit{mutual information} are special cases. Thus, these functionals are found to be uniquely capable of determining whether a certain class of inferential transformations, $ρ\xrightarrow{*}ρ'$, preserve, destroy or create correlations. This provides conceptual clarity by ruling out other possible global correlation quantifiers. Finally, the derivation and results allow us to quantify non-binary notions of statistical sufficency. Our results express what percentage of the correlations are preserved under a given inferential transformation or variable mapping.

cs.IT↗

arXiv2020

Why Simple Quadrature is just as good as Monte Carlo

We motive and calculate Newton--Cotes quadrature integration variance and compare it directly with Monte Carlo (MC) integration variance. We find an equivalence between deterministic quadrature sampling and random MC sampling by noting that MC random sampling is statistically indistinguishable from a method that uses deterministic sampling on a randomly shuffled (permuted) function. We use this statistical equivalence to regularize the form of permissible Bayesian quadrature integration priors such that they are guaranteed to be objectively comparable with MC. This leads to the proof that simple quadrature methods have expected variances that are less than or equal to their corresponding theoretical MC integration variances. Separately, using Bayesian probability theory, we find that the theoretical standard deviations of the unbiased errors of simple Newton--Cotes composite quadrature integrations improve over their worst case errors by an extra dimension independent factor $\propto N^{-1/2}$. This dimension independent factor is validated in our simulations.

math.ST↗

arXiv2019

A General Model Validation and Testing Tool

We construct and propose the "Bayesian Validation Metric" (BVM) as a general model validation and testing tool. We find the BVM to be capable of representing all of the standard validation metrics (square error, reliability, probability of agreement, frequentist, area, probability density comparison, statistical hypothesis testing, and Bayesian model testing) as special cases and find that it can be used to improve, generalize, or further quantify their uncertainties. Thus, the BVM allows us to assess the similarities and differences between existing validation metrics in a new light. The BVM has the capacity to allow users to invent and select models according to novel validation requirements. We formulate and test a few novel compound validation metrics that improve upon other validation metrics in the literature. Further, we construct the BVM Ratio for the purpose of quantifying model selection under user defined definitions of agreement in the presence or absence of uncertainty. This construction generalizes the Bayesian model testing framework.

stat.ME↗

arXiv2019

Vectorized Uncertainty Propagation and Input Probability Sensitivity Analysis

In this article we construct a theoretical and computational process for assessing Input Probability Sensitivity Analysis (IPSA) using a Graphics Processing Unit (GPU) enabled technique called Vectorized Uncertainty Propagation (VUP). VUP propagates probability distributions through a parametric computational model in a way that's computational time complexity grows sublinearly in the number of distinct propagated input probability distributions. VUP can therefore be used to efficiently implement IPSA, which estimates a model's probabilistic sensitivity to measurement and parametric uncertainty over each relevant measurement location. Theory and simulation illustrate the effectiveness of these methods.

stat.CO↗

arXiv2018

The Inferential Design of Entropy and its Application to Quantum Measurements

This thesis synthesizes probability and entropic inference with Quantum Mechanics (QM) and quantum measurement [1-6]. It is shown that the standard and quantum relative entropies are tools designed for the purpose of updating probability distributions and density matrices, respectively [1]. The derivation of the standard and quantum relative entropies are completed in tandem and follow from the same inferential principle - the principle of minimal updating [21,66]. As the quantum maximum entropy method is derived using the standard quantum mechanical formalism, the quantum maximum entropy method may be appended to the standard quantum mechanical formalism and remove collapse as a required postulate, in agreement with [11]. The quantum maximum entropy method is found to be a "universal method of density matrix inference" as it can process information about data and moments simultaneously (giving joint generalized quantum inference solutions), which when processed separately gives the Quantum Bayes Rule [2,39] or a canonical quantum (von Neumann) maximum entropy solution [10], respectively, as special cases. The second part of this thesis revolves around a foundational theory of QM called Entropic Dynamics (ED) [13]. Rather than appending an interpretation to QM, ED states its interpretation, "that particles have definite, yet unknown, positions and that entropic probability updating works" - only then does ED derive QM as an application of inference consistent with these assumptions. This shift in interpretation allows one to solve the quantum measurement problem [3,14] and avoid being ruled out by quantum no-go theorems [4]. Observables are divvied-up into two classes in ED: they are the ontic "beables" [15] (particle position), and the epistemic "inferables" [3], which are not predisposed to be part of the ontology as they are inferred in general from position detections.

quant-ph↗

arXiv2017

A Multiple Observer Probability Analysis for Bell Scenarios in Special Relativity

Here we present a Multiple Observer Probability Analysis (MOPA) for the purpose of clarifying topics in experimental Bell scenarios. Because Bell scenarios are interested in quantum effects between nonlocal measurement devices, we assign an observer to each device: Alice and Bob. Given that the observers are stationary and space-like separated, each observer is privy to different information along their shared equi-temporal lines due to permutations in the order they observe events. Therefore, each observer is inclined to assign different probability distributions to the same set of propositions due to these informational differences. The observers are obligated to update their probability distributions on the basis of locally observed events, and in this sense, factuality is informational locality. In this framework, only local variables or detections may be factual, but nothing prevents an observer from inquiring or making if-then inferences on the counterfactual basis of a nonlocal proposition being true. Indeed the objects pertaining to these nonlocal counterfactual propositions may be far outside an observer's light cone. The MOPA arrives at the conclusion that the CHSH inequality is only nonlocally violated "counterfactually" by each observer whereas local violations of the CHSH may be factual or counterfactual. We believe the MOPA to better gel probability theory (and thus QM) with Special Relativity than does the standard locality conditions imposed in the Bell and CHSH inequalities. The no-signaling condition is reinterpreted, and perhaps further clarified, in the MOPA and statements about counterfactuality and observer dependent QM are made.

quant-ph↗

arXiv2017

The Quantum Bayes Rule and Generalizations from the Quantum Maximum Entropy Method

The recent article "Entropic Updating of Probability and Density Matrices" [1] derives and demonstrates the inferential origins of both the standard and quantum relative entropies in unison. Operationally, the standard and quantum relative entropies are shown to be designed for the purpose of inferentially updating probability distributions and density matrices, respectively, when faced with incomplete information. We call the inferential updating procedure for density matrices the "quantum maximum entropy method". Standard inference techniques in probability theory can be criticized for lacking concrete physical consequences in physics; but here, because we are updating quantum mechanical density matrices, the quantum maximum entropy method has direct physical and experimental consequences. The present article gives a new derivation of the Quantum Bayes Rule, and some generalizations, using the quantum maximum entropy method while discuss some of the limitations the quantum maximum entropy method puts on the measurement process in Quantum Mechanics.

quant-ph↗

arXiv2017

Entropic Updating of Probability and Density Matrices

We find that the standard relative entropy and the Umegaki entropy are designed for the purpose of inferentially updating probability and density matrices respectively. From the same set of inferentially guided design criteria, both of the previously stated entropies are derived in parallel. This formulates a quantum maximum entropy method for the purpose of inferring density matrices in the absence of complete information in Quantum Mechanics.

quant-ph↗

arXiv2017

Entropic Dynamics: A hybrid-contextual theory of Quantum Mechanics

The Bell-KS theorem and the more recent $ψ$-epistemic \emph{no-go} theorems of QM are discussed in the context of Entropic Dynamics. In doing so we find that the Bell-KS theorem allows for, a perhaps overlooked, hybrid-contextual model of QM in which one set of commuting observables (position in this case) is non-contextual and all other observables are contextual. Entropic Dynamics is in a unique position as compared to other foundational theories of QM because it derives QM using standard techniques in Bayesian probability theory. In this formalism, position is the preferred basis from which inferences about other contextual operators are made. This leads to the interpretation that Entropic Dynamics is a hybrid-contextual model of QM, which we show to be consistent with the Bell-KS theorem and QM.

quant-ph↗

arXiv2017

Quantum Measurement and Weak Values in Entropic Dynamics

The problem of measurement in quantum mechanics is studied within the Entropic Dynamics framework. We discuss von Neumann and Weak measurements, wavefunction collapse, and Weak Values as examples of bayesian and entropic inference.

quant-ph↗

arXiv2016

How Kirkwood and Probability Distributions Differ: A Coxian Perspective

Kolmogorov's first axiom of probability is probability takes values between 0 and 1; however, in Cox's derivation of probability having a maximum value of unity is arbitrary since he derives probability as a tool to rank degrees of plausibility. Probability can then be used to make inferences in instances of incomplete information, which is the foundation of Baysian probability theory. This article formulates a rule, which if obeyed, allows probability to take complex values and still be consistent with the interpretation of probability theory as being a tool to rank plausibility. It is then shown that Kirkwood distributions and the conditional complex probability distributions proposed by Hofmann do not obey this rule and therefore cannot rank plausibility. Not only do these quasiprobability distributions relax Kolmogorov's first axiom of probability, they also are void of the defining property of a probability distribution from a Coxian and Baysian perspective - they lack the ability to rank plausibility.

quant-ph↗

arXiv2016

Theoretical Study of Variable Measurement Uncertainty $h_I$ and Infinite Unobservable Entropy

This paper examines the statistical mechanical and thermodynamical consequences of variable phase-space volume element $h_I=\bigtriangleup x_i\bigtriangleup p_i$. Varying $h_I$ leads to variations in the amount of measured information of a system but the maximum entropy remains constant due to the uncertainty principle. By taking $h_u\rightarrow 0^+$ an infinite unobservable entropy is attained leading to an infinite unobservable energy per particle and an unobservable chemical equilibrium between all particles. The amount of heat fluxing though measurement apparatus is formulated as a function of $h_I$ for systems in steady state equilibrium as well as the number of measured particles or sub-particles so any system can be described as unitary or composite in number. Some example systems are given using variable $h_I$.

physics.gen-ph↗