Measuring the Ensemble Spread–Error Relationship with a Probabilistic Approach: Stochastic Ensemble Results

Abstract

One widely accepted measure of the utility of ensemble prediction systems is the relationship between ensemble spread and deterministic forecast accuracy. Unfortunately, this relationship is often characterized by spread?error linear correlations, which oversimplify the true spread?error relationship and ignore the possibility that some end users have categorical sensitivities to forecast error. In the present paper, a simulation study is undertaken to estimate the idealized spread?error statistics for stochastic ensemble prediction systems of a finite size. Under a variety of spread?error metrics, the stochastic ensemble spread?error joint distributions are characterized by increasing scatter as the ensemble spread grows larger. A new method is introduced that recognizes the inherent nonlinearity of spread?error joint distributions and capitalizes on the fact that the probability of large forecast errors increases with ensemble spread. The ensemble spread?error relationship is measured by the skill of probability forecasts that are constructed from a history of ensemble-mean forecast errors using only cases with similar ensemble spread. Thus, the value of ensemble spread information is quantified by the ultimate benefit that is realized by end users of the probability forecasts based on these conditional-error climatologies. It is found that the skill of conditional-error-climatology forecasts based on stochastic ensemble spread is nearly equal to the skill of probability forecasts constructed directly from the raw ensemble statistics. The skill is largest for cases with anomalous spread and smallest for cases with near-normal spread. These results reinforce the findings of earlier studies and affirm that the temporal variability of ensemble spread controls its potential value as a predictor. Additionally, it is found that the skill of spread-based error probability forecasts is maximized when the chosen spread metric is consistent with the end user?s cost function.