Monte Carlo Sampling Bias in the Microwave Uncertainty Framework

Abstract

Uncertainty propagation software can have unknown, inadvertent biases introduced by various means. This work is a case study in bias identification and reduction in one such software package, the Microwave Uncertainty Framework (MUF). The general purpose of the MUF is to provide automated multivariate statistical uncertainty propagation and analysis on a Monte Carlo (MC) basis. Combine is a key module in the MUF, responsible for merging data, raw or transformed, to accurately reflect the variability in the data and in its central tendency. In this work the performance of Combine's MC replicates is analytically compared against its stated design goals. An alternative construction is proposed for Combine's MC replicates and its performance is compared, too, against Combine's design goals. These comparisons are made within an archetypal two-stage scenario in which received data are first transformed in conjunction with shared systematic error and then combined to produce summary information. These comparisons reveal the limited conditions under which Combine's uncertainty results are unbiased and the extent of these biases when these conditions are not met. For small MC sample sizes neither construction, current or alternative, fully meets Combine's design goals, nor does either construction consistently outperform the other. However, for large MC sample sizes the bias in the proposed alternative construction is asymptotically zero, and this construction is recommended.