The Canadian Updateable Model Output Statistics (UMOS) System: Validation against Perfect Prog

Abstract

This paper describes validation tests of the Canadian Updateable Model Output Statistics (UMOS) system against the perfect prognosis forecast system and forecasts of weather elements from the operational numerical weather prediction model. Several update experiments were performed using 2-m temperature, 10-m wind direction and speed, and probability of precipitation as predictands. These experiments were designed to evaluate the ability of the UMOS system to provide improved forecasts during the period following a model change when the development samples contain data from two or more different model versions. Tests were run for about 200 Canadian stations for both summer and winter periods. Independent summer and winter samples were used in the evaluation, to compare UMOS forecast accuracy with the direct model output forecasts, the perfect prog forecasts, and MOS forecasts based only on data from the earlier model version. The authors were also able to compare the evaluation results of forecasts generated using the data from a 4-month summer ?parallel run? period for which two versions of the model were run concurrently. Results show that the UMOS forecasts are generally superior to both perfect prog and direct model output forecasts for all three weather elements. The UMOS forecasts are particularly responsive to bias changes; most forecast biases could be corrected with relatively little data from the newer model version. Although some of the improvement over perfect prog forecasts is apparently due solely to the use of MOS, the updating brings additional improvements even during the data blending period. The results also suggest that the higher-resolution predictions from the model bring advantages only for the first day of the forecast period. For the day-2 forecasts, the improvement over the much smoother perfect prog forecasts was smaller, especially for probability of precipitation.