Searcharxiv⌕ Search

arXiv subjects

Vladimir Vovk

Publications and source records attributed to Vladimir Vovk.

At least 37 records · Page 2Linked to original sources
arXiv2024

The diachronic Bayesian

It is well known that a Bayesian probability forecast for all future observations should be a probability measure in order to satisfy a natural condition of coherence. The main topics of this paper are the evolution of the Bayesian probability measure and ways of testing its adequacy as it evolves over time. The process of testing evolving Bayesian beliefs is modelled in terms of betting, similarly to the standard Dutch book treatment of coherence. The resulting picture is adapted to forecasting several steps ahead and making almost optimal decisions.

stat.ME↗
arXiv2024

Multiple testing in game-theoretic probability: pictures and questions

The usual way of testing probability forecasts in game-theoretic probability is via construction of test martingales. The standard assumption is that all forecasts are output by the same forecaster. In this paper I will discuss possible extensions of this picture to testing probability forecasts output by several forecasters. This corresponds to multiple hypothesis testing in statistics. One interesting phenomenon is that even a slight relaxation of the requirement of family-wise validity leads to a very significant increase in the efficiency of testing procedures. The main goal of this paper is to report results of preliminary simulation studies and list some directions of further research.

stat.ME↗
arXiv2024

Merging sequential e-values via martingales

We study the problem of merging sequential or independent e-values into one e-value or e-process. We describe a class of e-value merging functions via martingales and show that it dominates all merging methods for sequential e-values. All admissible methods for constructing e-processes can also be obtained in this way. In the case of merging independent e-values, the situation becomes much more complicated, and we provide a general class of such merging functions based on martingales applied to reordered data.

math.ST↗
arXiv2024

The power of forgetting in statistical hypothesis testing

This paper places conformal testing in a general framework of statistical hypothesis testing. A standard approach to testing a composite null hypothesis $H$ is to test each of its elements and to reject $H$ when each of its elements is rejected. It turns out that we can fully cover conformal testing using this approach only if we allow forgetting some of the data. However, we will see that the standard approach covers conformal testing in a weak asymptotic sense and under restrictive assumptions. I will also list several possible directions of further research, including developing a general scheme of online testing.

math.ST↗
arXiv2024

Efficiency of nonparametric e-tests

The notion of an e-value has been recently proposed as a possible alternative to critical regions and p-values in statistical hypothesis testing. In this paper we consider testing the nonparametric hypothesis of symmetry, introduce analogues for e-values of three popular nonparametric tests, define an analogue for e-values of Pitman's asymptotic relative efficiency, and apply it to the three nonparametric tests. We discuss limitations of our simple definition of asymptotic relative efficiency and list directions of further research.

math.ST↗
arXiv2023

A Conversation with A. Philip Dawid

Beginning in the 1970s, Alexander Philip Dawid has been a leading contributor to the foundations of statistics and especially to the development and application of Bayesian statistics. He is also known for his work on causality, especially his notation for conditional independence and his critique of the overuse of counterfactuals, and for his contributions to forensic statistics. Dawid was born in Lancashire, England, on February 1, 1946. His family moved to London soon afterwards, and he attended the City of London School from 1956 to 1963. He studied mathematics at Cambridge, earning a BA (Bachelor of Arts) degree in 1966. After earning a Diploma in Mathematical Statistics in the academic year 1966-1967, he studied for a PhD at Imperial, then at UCL, where he became a Lecturer in Statistics in 1969. In 1978, he left UCL for a position as Professor of Statistics in the Department of Mathematics, The City University, London, where he served as Head of Statistics Section and Director of the Statistical Laboratory. He returned to the Department of Statistics at UCL in 1981, serving as Head of Department from 1983 to 1993. He moved to the University of Cambridge in 2007, becoming Professor of Statistics and Fellow of Darwin College. He has continued his work in mathematical statistics after retiring from Cambridge in 2013 and was elected Fellow of the Royal Society in 2018.

stat.OT↗
arXiv2023

Logic of subjective probability

In this paper I discuss both syntax and semantics of subjective probability. The semantics determines ways of testing probability statements. Among important varieties of subjective probabilities are intersubjective probabilities and impersonal probabilities, and I will argue that well-tested impersonal probabilities acquire features of objective probabilities. Jeffreys's law, my next topic, states that two successful probability forecasters must issue forecasts that are close to each other, thus supporting the idea of objective probabilities. Finally, I will discuss connections between subjective and frequentist probability.

cs.AI↗
arXiv2023

Game-theoretic statistics and safe anytime-valid inference

Safe anytime-valid inference (SAVI) provides measures of statistical evidence and certainty -- e-processes for testing and confidence sequences for estimation -- that remain valid at all stopping times, accommodating continuous monitoring and analysis of accumulating data and optional stopping or continuation for any reason. These measures crucially rely on test martingales, which are nonnegative martingales starting at one. Since a test martingale is the wealth process of a player in a betting game, SAVI centrally employs game-theoretic intuition, language and mathematics. We summarize the SAVI goals and philosophy, and report recent advances in testing composite hypotheses and estimating functionals in nonparametric settings.

math.ST↗
arXiv2023

Testing exchangeability in the batch mode with e-values and Markov alternatives

The topic of this paper is testing exchangeability using e-values in the batch mode, with the Markov model as alternative. The null hypothesis of exchangeability is formalized as a Kolmogorov-type compression model, and the Bayes mixture of the Markov model w.r. to the uniform prior is taken as simple alternative hypothesis. Using e-values instead of p-values leads to a computationally efficient testing procedure. In the appendixes I explain connections with the algorithmic theory of randomness and with the traditional theory of testing statistical hypotheses. In the standard statistical terminology, this paper proposes a new permutation test. This test can also be interpreted as a poor man's version of Kolmogorov's deficiency of randomness.

stat.ME↗
arXiv2022

Confidence and discoveries with e-values

We discuss systematically two versions of confidence regions: those based on p-values and those based on e-values, a recent alternative to p-values. Both versions can be applied to multiple hypothesis testing, and in this paper we are interested in procedures that control the number of false discoveries under arbitrary dependence between the base p- or e-values. We introduce a procedure that is based on e-values and show that it is efficient both computationally and statistically using simulated and real-world datasets. Comparison with the corresponding standard procedure based on p-values is not straightforward, but there are indications that the new one performs significantly better in some situations.

math.ST↗
arXiv2021

Conformal testing: binary case with Markov alternatives

We continue study of conformal testing in binary model situations. In this note we consider Markov alternatives to the null hypothesis of exchangeability. We propose two new classes of conformal test martingales; one class is statistically efficient in our experiments, and the other class partially sacrifices statistical efficiency to gain computational efficiency.

math.ST↗
arXiv2021

Protected probabilistic classification

This paper proposes a way of protecting probabilistic prediction models against changes in the data distribution, concentrating on the case of classification and paying particular attention to binary classification. This is important in applications of machine learning, where the quality of a trained prediction algorithm may drop significantly in the process of its exploitation. Our techniques are based on recent work on conformal test martingales and older work on prediction with expert advice, namely tracking the best expert.

cs.LG↗
arXiv2021

Enhancement of prediction algorithms by betting

This note proposes a procedure for enhancing the quality of probabilistic prediction algorithms via betting against their predictions. It is inspired by the success of the conformal test martingales that have been developed recently.

cs.LG↗
arXiv2021

Conformal testing in a binary model situation

Conformal testing is a way of testing the IID assumption based on conformal prediction. The topic of this note is computational evaluation of the performance of conformal testing in a model situation in which IID binary observations generated from a Bernoulli distribution are followed by IID binary observations generated from another Bernoulli distribution, with the parameters of the distributions and changepoint unknown. Existing conformal test martingales can be used for this task and work well in simple cases, but their efficiency can be improved greatly.

cs.LG↗
arXiv2021

Admissible ways of merging p-values under arbitrary dependence

Methods of merging several p-values into a single p-value are important in their own right and widely used in multiple hypothesis testing. This paper is the first to systematically study the admissibility (in Wald's sense) of p-merging functions and their domination structure, without any information on the dependence structure of the input p-values. As a technical tool we use the notion of e-values, which are alternatives to p-values recently promoted by several authors. We obtain several results on the representation of admissible p-merging functions via e-values and on (in)admissibility of existing p-merging functions. By introducing new admissible p-merging functions, we show that some classic merging methods can be strictly improved to enhance power without compromising validity under arbitrary dependence.

math.ST↗
arXiv2021

Retrain or not retrain: Conformal test martingales for change-point detection

We argue for supplementing the process of training a prediction algorithm by setting up a scheme for detecting the moment when the distribution of the data changes and the algorithm needs to be retrained. Our proposed schemes are based on exchangeability martingales, i.e., processes that are martingales under any exchangeable distribution for the data. Our method, based on conformal prediction, is general and can be applied on top of any modern prediction algorithm. Its validity is guaranteed, and in this paper we make first steps in exploring its efficiency.

cs.LG↗