arXiv subjects

Michael J. Lew

Publications and source records attributed to Michael J. Lew.

3 recordsLinked to original sources

arXiv2019

A reckless guide to P-values: local evidence, global errors

This chapter demystifies P-values, hypothesis tests and significance tests, and introduces the concepts of local evidence and global error rates. The local evidence is embodied in \textit{this} data and concerns the hypotheses of interest for \textit{this} experiment, whereas the global error rate is a property of the statistical analysis and sampling procedure. It is shown using simple examples that local evidence and global error rates can be, and should be, considered together when making inferences. Power analysis for experimental design for hypothesis testing are explained, along with the more locally focussed expected P-values. Issues relating to multiple testing, HARKing, and P-hacking are explained, and it is shown that, in many situation, their effects on local evidence and global error rates are in conflict, a conflict that can always be overcome by a fresh dataset from replication of key experiments. Statistics is complicated, and so is science. There is no singular right way to do either, and universally acceptable compromises may not exist. Statistics offers a wide array of tools for assisting with scientific inference by calibrating uncertainty, but statistical inference is not a substitute for scientific inference. P-values are useful indices of evidence and deserve their place in the statistical toolbox of basic pharmacologists.

stat.OT↗

arXiv2015

The likelihood principle does not entail a `sure thing', `evil demon' or `determinist' hypothesis

The likelihood principle makes strong claims about the nature of statistical evidence but is controversial. Its claims are undermined by the existence of several examples that are assumed to show that it allows, with unity probability, domination of all other hypotheses by the uninteresting, determinist hypothesis that whatever happened had to happen. Such examples are generally assumed to be important obstacles to the application of the likelihood principle: they are counter-examples to the principle. A re-analysis of Birnbaum's 1969 `counter-example', demonstrates that the standardly reported analyses of such examples involves an inappropriate treatment of a nuisance parameter and that, when the nuisance parameter is adequately considered, there is no conflict between the evidential consequences of the likelihood principle and the intuitive evidential account of the problem. It also shows that the conclusion that the likelihood principle allows the determinist hypothesis to dominate with unity probability requires a misconception about the scope of the likelihood principle or an inappropriately specified statistical model. Whatever happened did \textit{not} have to happen.

math.ST↗

arXiv2013

To P or not to P: on the evidential nature of P-values and their place in scientific inference

The customary use of P-values in scientific research has been attacked as being ill-conceived, and the utility of P-values has been derided. This paper reviews common misconceptions about P-values and their alleged deficits as indices of experimental evidence and, using an empirical exploration of the properties of P-values, documents the intimate relationship between P-values and likelihood functions. It is shown that P-values quantify experimental evidence not by their numerical value, but through the likelihood functions that they index. Many arguments against the utility of P-values are refuted and the conclusion is drawn that P-values are useful indices of experimental evidence. The widespread use of P-values in scientific research is well justified by the actual properties of P-values, but those properties need to be more widely understood.

stat.ME↗