Validation of the ECMWF Ensemble Prediction System Using Empirical Orthogonal Functions

Abstract

Empirical orthogonal function (EOF) analysis of deviations from the ensemble mean was used to validate the statistical properties of TL159 51-member ensemble forecasts run at the European Centre for Medium-Range Weather Forecasts (ECMWF) during the winter of 1996/97. The main purpose of the analysis was to verify the agreement between the amount of spread variance and error variance accounted for by different EOFs. A suitable score was defined to quantify the agreement between the variance spectra in a given EOF subspace. The agreement between spread and error distribution for individual principal components (PCs) was also tested using the nonparametric Mann?Whitney test. The analysis was applied at day 3, 5, and 7 forecasts of 500-hPa height over Europe and North America, and of 850-hPa temperature over Europe. The variance spectra indicate a better performance of the ECMWF Ensemble Prediction System (EPS) over Europe than over North America in the medium range. In the former area, the excess of error variance over spread variance tends to be confined to nonleading PCs, while for the first two PCs the error variance is smaller than spread at day 3 and in very close agreement at day 7. When averaged over a six-EOF subspace, the relative differences between spread and error PC variances are about 25% over Europe, with the smallest discrepancy (15%) for 850-hPa temperature at day 7. Overall, the distribution of variance between different EOFs produced by the EPS over Europe is in good agreement with the observed distribution, the differences being of comparable magnitude to the sampling errors of PC variances in individual seasons.