Decomposition of a New Proper Score for Verification of Ensemble Forecasts

Abstract

new proper score, the error-spread score (ES), has recently been proposed for evaluation of ensemble forecasts of continuous variables. The ES is formulated with respect to the moments of the ensemble forecast. It is particularly sensitive to evaluating how well an ensemble forecast represents uncertainty: is the probabilistic forecast well calibrated? In this paper, it is shown that the ES can be decomposed into its reliability, resolution, and uncertainty components in a similar way to the Brier score. The first term evaluates the reliability of the forecast standard deviation and skewness, rewarding systems where the forecast moments reliably indicate the properties of the verification. The second term evaluates the resolution of the forecast standard deviation and skewness, and rewards systems where the forecast moments vary from the climatological moments according to the predictability of the atmospheric flow. The uncertainty term depends only on the observed error distribution and is independent of the forecast standard deviation or skewness. The decomposition was demonstrated using forecasts made with the European Centre for Medium-Range Weather Forecasts ensemble prediction system, and was able to identify the source of the skill in the forecasts at different latitudes.