The Propagation and Reduction of Uncertainty Left Unquantified by Confidence Intervals, p-Values, Neural Network Predictions, Posterior Distributions, and Other Statistical Results

Abstract

In the use of statistical models to analyze data, there is not only the uncertainty quantified by the models but also uncertainty about which models are adequate for some purpose, such as weighing the evidence for or against a hypothesis of scientific interest. This paper provides methods for propagating such unquantified uncertainty to the results under a unified framework of adequate model averaging. Specifically, the weight of each model used in the average is the probability that it is the most useful model. To allow for the case that none of the models considered would be useful, a catch-all model is included in the model average at a different level of the hierarchy. The catch-all model is the vacuous model in imprecise probability theory, the model that puts no restrictions on the probabilities of statements about the unknown values of interest. That enables defining the proportion of the uncertainty left unquantified by a model as the probability that it is inadequate in the sense of being less useful than the catch-all model. A lower bound for the proportion of unquantified uncertainty of the averaged model decreases as more models are added to the average.