Verification of Probabilistic Predictions: A Brief Review

Abstract

The evaluation process is considered in some detail with particular reference to probabilistic predictions. The process consists of several ordered steps at each of which elements (of the process) are identified. Consideration of the purposes leads to the identification of two distinct forms of evaluation: operational evaluation concerned with the value of predictions to the user and empirical evaluation, or verification, concerned with the perfection of predictions, i.e., the association between predictions and observations. Attributes, i.e., desirable properties, of predictions are defined with reference to these purposes, and a number of measures of the attributes for empirical evaluation are considered. An artificial example of comparative verification in which different measures appear to yield contradictory results is used to demonstrate the importance of, and need for, a careful analysis of the evaluation process.