Application of Object-Based Verification Techniques to Ensemble Precipitation Forecasts

Abstract

Both the Method for Object-based Diagnostic Evaluation (MODE) and contiguous rain area (CRA) object-based verification techniques have been used to analyze precipitation forecasts from two sets of ensembles to determine if spread-skill behavior observed using traditional measures can be seen in the object parameters. One set consisted of two eight-member Weather Research and Forecasting (WRF) model ensembles: one having mixed physics and dynamics with unperturbed initial and lateral boundary conditions (Phys) and another using common physics and a dynamic core but with perturbed initial and lateral boundary conditions (IC/LBC). Traditional measures found that spread grows much faster in IC/LBC than in Phys so that after roughly 24 h, better skill and spread are found in IC/LBC. These measures also reflected a strong diurnal signal of precipitation. The other set of ensembles included five members of a 4-km grid-spacing WRF ensemble (ENS4) and five members of a 20-km WRF ensemble (ENS20). Traditional measures suggested that the diurnal signal was better in ENS4 and spread increased more rapidly than in ENS20. Standard deviations (SDs) of four object parameters computed for the first set of ensembles using MODE and CRA showed the trend of enhanced spread growth in IC/LBC compared to Phys that had been observed in traditional measures, with the areal coverage of precipitation exhibiting the greatest growth in spread with time. The two techniques did not produce identical results; although, they did show the same general trends. A diurnal signal could be seen in the SDs of all parameters, especially rain rate, volume, and areal coverage. MODE results also found evidence of a diurnal signal and faster growth of spread in object parameters in ENS4 than in ENS20. Some forecasting approaches based on MODE and CRA output are also demonstrated. Forecasts based on averages of object parameters from each ensemble member were more skillful than forecasts based on MODE or CRA applied to an ensemble mean computed using the probability matching technique for areal coverage and volume, but differences in the two techniques were less pronounced for rain rate and displacement. The use of a probability threshold to define objects was also shown to be a valid forecasting approach with MODE.