Objective Verification of the SAMEX ’98 Ensemble Forecasts

Abstract

During May 1998, the Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma coordinated a multi-institution numerical forecast project known as the Storm and Mesoscale Ensemble Experiment (SAMEX). SAMEX involved, for the first time, the real-time operation of four different ensembles of mesoscale models over the same region of the United States. The main purpose of this paper is the evaluation of the ensemble forecasts, performed at a relatively coarse resolution of 30 km. An additional SAMEX goal not discussed here is to compare the value of the ensemble forecasts against single forecasts made over smaller subregions of the Great Plains at both intermediate (10 km) and high (3 km) resolution. The SAMEX ?98 ensembles consisted of a single 36-h control forecast from the ARPS (at CAPS), the Penn State?NCAR fifth-generation Mesoscale Model (at NSSL), and the Eta Model and Regional Spectral Model (at NCEP), all with horizontal resolutions of approximately 30 km, and perturbed runs, resulting in a grand ensemble of 25 members. The forecasts of geopotential heights, temperatures, and moisture were verified against the Eta operational analyses, rather than observations. Unlike global ensembles, which tend to be useful in the medium range, the mesoscale SAMEX ensembles provided useful information in the short range. A major result is that the performance of the ensemble of multiple forecast systems is much better than that of each individual ensemble system, probably because it represents more realistically the current uncertainties in both models and initial conditions. A similar advantage from the use of multimodel, multianalysis systems has been observed with global ensembles. The SAMEX results also show that perturbations to model physics parameterizations, as well as the use of consistent perturbations in the boundary conditions, are important for regional ensemble forecasting. Efforts are now under way to compare the ensemble forecasts against those made using higher spatial resolution, and follow-on SAMEX experiments are anticipated in other geographical areas and weather regimes. Although the main results of this paper appear to be very robust, they were based on a small number of cases, and similar experiments carried out during other periods will help to test their significance.