Using Forecast Temporal Variability to Evaluate Model Behavior

Abstract

he statistics of model temporal variability ought to be the same as those of the filtered version of reality that the model is designed to represent. Here, simple diagnostics are introduced to quantify temporal variability on different time scales and are then applied to NCEP and CMC global ensemble forecasting systems. These diagnostics enable comparison of temporal variability in forecasts with temporal variability in the initial states from which the forecasts are produced. They also allow for an examination of how day-to-day variability in the forecast model changes as forecast integration time increases. Because the error in subsequent analyses will differ, it is shown that forecast temporal variability should lie between corresponding analysis variability and analysis variability minus 2 times the analysis error variance. This expectation is not always met and possible causes are discussed. The day-to-day variability in NCEP forecasts steadily decreases at a slow rate as forecast time increases. In contrast, temporal variability increases during the first few days in the CMC control forecasts, and then levels off, consistent with a spinup of the forecasts starting from overly smoothed analyses. The diagnostics successfully reflect a reduction in the temporal variability of the CMC perturbed forecasts after a system upgrade. The diagnostics also illustrate a shift in variability maxima from storm-track regions for 1-day variability to blocking regions for 10-day variability. While these patterns are consistent with previous studies examining temporal variability on different time scales, they have the advantage of being obtainable without the need for extended (e.g., multimonth) forecast integrations.