Assessing the Usefulness of Probabilistic Forecasts

Abstract

The errors in both the initialization and simulated evolution of weather and climate models create significant uncertainties in forecasts at lead times beyond a few days. Modern prediction systems sample the sources of these uncertainties to produce a probability distribution function of future meteorological conditions to help end users in their risk assessment and decision-making processes. The performance of prediction systems is assessed using data from a set of historical forecasts and the corresponding observations. There are many aspects to the correspondence between forecasts and observations, and various summary scores have been created to measure the different features of forecast quality. The main concern for end users is the usefulness of forecasts. There are two independent and sufficient aspects for the assessment of the usefulness of forecasts to end users: 1) the statistical consistency of forecast statements with observations and 2) the extra information contained in the forecast relative to the situation in which such predictions are unavailable. In this paper two new scores, the full-pdf-reliability Rpdf and information quantity IQ, are proposed to measure these two independent aspects of usefulness. In contrast to all existing summary scores, both Rpdf and IQ depend upon all moments of the forecast pdf. When taken together, the values of Rpdf and IQ offer a general measure of the usefulness of ensemble predictions.