Bayesian Probability and Scalar Performance Measures in Gaussian Models

Abstract

The transformation of a real, continuous variable into an event probability is reviewed from the Bayesian point of view, after which a Gaussian model is employed to derive an explicit expression for the probability. In turn, several scalar (one-dimensional) measures of performance quality and reliability diagrams are computed. It is shown that if the optimization of scalar measures is of concern, then prior probabilities must be treated carefully, whereas no special care is required for reliability diagrams. Specifically, since a scalar measure gauges only one component of performance quality?a multidimensional entity?it is possible to find the critical value of prior probability that optimizes that scalar measure; this value of ?prior probability? is often not equal to the ?true? value as estimated from group sample sizes. Optimum reliability, however, is obtained when prior probability is equal to the estimate based on group sample sizes. Exact results are presented for the critical value of ?prior probability? that optimize the fraction correct, the true skill statistic, and the reliability diagram, but the critical success index and the Heidke skill statistic are treated only graphically. Finally, an example based on surface air pressure data is employed to illustrate the results in regard to precipitation forecasting.