arXiv subjects

M. Grendar

Publications and source records attributed to M. Grendar.

10 recordsLinked to original sources

arXiv2011

Is the p-value a good measure of evidence? An asymptotic consistency criterion

What are the criteria that a measure of statistical evidence should satisfy? It is argued that a measure of evidence should be consistent. Consistency is an asymptotic criterion: the probability that if a measure of evidence in data strongly testifies against a hypothesis H, then H is indeed not true, should go to one, as more and more data appear. The p-value is not consistent, while the ratio of likelihoods is.

math.ST↗

arXiv2008

Maximum Probability and Relative Entropy Maximization. Bayesian Maximum Probability and Empirical Likelihood

Works, briefly surveyed here, are concerned with two basic methods: Maximum Probability and Bayesian Maximum Probability; as well as with their asymptotic instances: Relative Entropy Maximization and Maximum Non-parametric Likelihood. Parametric and empirical extensions of the latter methods - Empirical Maximum Maximum Entropy and Empirical Likelihood - are also mentioned. The methods are viewed as tools for solving certain ill-posed inverse problems, called Pi-problem, Phi-problem, respectively. Within the two classes of problems, probabilistic justification and interpretation of the respective methods are discussed.

math.ST↗

arXiv2007

L-Divergence Consistency for a Discrete Prior

Posterior distribution over a countable set M of continuous data-sampling distributions piles up at L-projection of the true distribution r on M, provided that the L-projection is unique. If there are several L-projections of r on M, then the posterior probability splits among them equally.

math.PR↗

arXiv2007

Conditioning by rare sources

In this paper we study the exponential decay of posterior probability of a set of sources and conditioning by rare sources for both uniform and general prior distributions of sources. The decay rate is determined by $L$-divergence and rare sources from a convex, closed set asymptotically conditionally concentrate on an $L$-projection. $L$-projection on a linear family of sources belongs to $\varLambda$-family of distributions. The results parallel those of Large Deviations for Empirical Measures (Sanov's Theorem and Conditional Limit Theorem).

math.ST↗

arXiv2006

Effective support size

Notion of effective size of support (Ess) of a random variable is introduced. A small set of natural requirements that a measure of Ess should satisfy is presented. The measure with prescribed properties is in a direct (exp-) relationship to the family of Renyi's $α$-entropies. Question of selecting the value of $α$ most appropriate for the purpose of measuring Ess is briefly addressed.

math.ST↗

arXiv2005

Maximum Probability and Maximum Entropy methods: Bayesian interpretation

(Jaynes') Method of (Shannon-Kullback's) Relative Entropy Maximization (REM or MaxEnt) can be - at least in the discrete case - according to the Maximum Probability Theorem (MPT) viewed as an asymptotic instance of the Maximum Probability method (MaxProb). A simple bayesian interpretation of MaxProb is given here. MPT carries the interpretation over into REM.

physics.data-an↗

arXiv2003

Maximum Entropy method with non-linear moment constraints: challenges

Traditionally, the Method of (Shannon-Kullback's) Relative Entropy Maximization (REM) is considered with linear moment constraints. In this work, the method is studied under frequency moment constraints which are non-linear in probabilities. The constraints challenge some justifications of REM since a) axiomatic systems are developed for classical linear moment constraints, b) the feasible set of distributions which is defined by frequency moment constraints admits several entropy maximizing distributions (I-projections), hence probabilistic justification of REM via Conditioned Weak Law of Large Numbers cannot be invoked. However, REM is not left completely unjustified in this setting, since Entropy Concentration Theorem and Maximum Probability Theorem can be applied. Maximum Renyi-Tsallis' entropy method (maxTent) enters this work because of non-linearity of X-frequency moment constraints which are used in Non-extensive Thermodynamics. It is shown here that under X-frequency moment constraints maxTent distribution can be unique and different than the I-projection. This implies that maxTent does not choose the most probable distribution and that the maxTent distribution is asymptotically conditionally improbable. What are adherents of maxTent accomplishing when they maximize Renyi's or Tsallis' entropy?

physics.data-an↗

arXiv2003

Chernoff's bound forms

Chernoff's bound binds a tail probability (ie. $Pr(X \ge a)$, where $a \ge EX$). Assuming that the distribution of $X$ is $Q$, the logarithm of the bound is known to be equal to the value of relative entropy (or minus Kullback-Leibler distance) for $I$-projection $\hat P$ of $Q$ on a set $\mathcal{H} \triangleq \{P: E_PX = a\}$. Here, Chernoff's bound is related to Maximum Likelihood on exponential form and consequently implications for the notion of complementarity are discussed. Moreover, a novel form of the bound is proposed, which expresses the value of the Chernoff's bound directly in terms of the $I$-projection (or generalized $I$-projection).

math.PR↗

arXiv2002

Why Maximum Entropy? A Non-axiomatic Approach

Ill-posed inverse problems of the form y = X p where y is J-dimensional vector of a data, p is m-dimensional probability vector which cannot be measured directly and matrix X of observable variables is a known J,m matrix, J < m, are frequently solved by Shannon's entropy maximization (MaxEnt). Several axiomatizations were proposed to justify the MaxEnt method (also) in this context. The main aim of the presented work is two-fold: 1) to view the concept of complementarity of MaxEnt and Maximum Likelihood (ML) tasks from a geometric perspective, and consequently 2) to provide an intuitive and non-axiomatic answer to the 'Why MaxEnt?' question.

math-ph↗

arXiv2002

Randomness as an Equilibrium. Potential and Probability Density

Randomness is viewed through an analogy between a physical quantity, density of gas, and a mathematical construct -- probability density. Boltzmann's deduction of equilibrium distribution of ideal gas placed in an external potential field than provides a way of viewing probability density from a perspective of forces/potentials, hidden behind it.

math.PR↗