MLE's bias pathology, Model Updated Maximum Likelihood Estimates and Wallace's Minimum Message Length method

Abstract

The inherent bias pathology of the maximum likelihood (ML) estimation method is confirmed for models with unknown parameters $\theta$ and $\psi$ when MLE $\hat \psi$ is function of MLE $\hat \theta.$ To reduce $\hat \psi$'s bias the likelihood equation to be solved for $\psi$ is updated using the model for the data $Y$ in it. Model updated (MU) MLE, $\hat \psi_{MU},$ often reduces either totally or partially $\hat \psi$'s bias when estimating shape parameter $\psi.$ For the Pareto model $\hat \psi_{MU}$ reduces also $\hat \psi$'s variance. The results explain the difference that puzzled R. A. Fisher, between biased $\hat \psi$ and the unbiased estimate he obtained for two models with the "2-stage procedure". MUMLE's implementation is equivalent to the abandoned 2-stage procedure thus justifying its use. MUMLE and Firth's bias correcting likelihood are also obtained with the Minimum Message Length method thus motivating its use in frequentist inference and, more generally, model updating with a prior distribution.