Neighborhood-Based Verification of Precipitation Forecasts from Convection-Allowing NCAR WRF Model Simulations and the Operational NAMSource: Weather and Forecasting:;2010:;volume( 025 ):;issue: 005::page 1495DOI: 10.1175/2010WAF2222404.1Publisher: American Meteorological Society
Abstract: Since 2003 the National Center for Atmospheric Research (NCAR) has been running various experimental convection-allowing configurations of the Weather Research and Forecasting Model (WRF) for domains covering a large portion of the central United States during the warm season (April?July). In this study, the skill of 3-hourly accumulated precipitation forecasts from a large sample of these convection-allowing simulations conducted during 2004?05 and 2007?08 is compared to that from operational North American Mesoscale (NAM) model forecasts using a neighborhood-based equitable threat score (ETS). Separate analyses were conducted for simulations run before and after the implementation in 2007 of positive-definite (PD) moisture transport for the NCAR-WRF simulations. The neighborhood-based ETS (denoted ?ETS?r) relaxes the criteria for ?hits? (i.e., correct forecasts) by considering grid points within a specified radius r. It is shown that ?ETS?r is more useful than the traditional ETS because ?ETS?r can be used to diagnose differences in precipitation forecast skill between different models as a function of spatial scale, whereas the traditional ETS only considers the spatial scale of the verification grid. It was found that differences in ?ETS?r between NCAR-WRF and NAM generally increased with increasing r, with NCAR-WRF having higher scores. Examining time series of ?ETS?r for r = 100 and r = 0 km (which simply reduces to the ?traditional? ETS), statistically significant differences between NCAR-WRF and NAM were found at many forecast lead times for ?ETS?100 but only a few times for ?ETS?0. Larger and more statistically significant differences occurred with the 2007?08 cases relative to the 2004?05 cases. Because of differences in model configurations and dominant large-scale weather regimes, a more controlled experiment would have been needed to diagnose the reason for the larger differences that occurred with the 2007?08 cases. Finally, a compositing technique was used to diagnose the differences in the spatial distribution of the forecasts. This technique implied westward displacement errors for NAM model forecasts in both sets of cases and in NCAR-WRF model forecasts for the 2007?08 cases. Generally, the results are encouraging because they imply that advantages in convection-allowing relative to convection-parameterizing simulations noted in recent studies are reflected in an objective neighborhood-based metric.
|
Collections
Show full item record
| contributor author | Clark, Adam J. | |
| contributor author | Gallus, William A. | |
| contributor author | Weisman, Morris L. | |
| date accessioned | 2017-06-09T16:38:46Z | |
| date available | 2017-06-09T16:38:46Z | |
| date copyright | 2010/10/01 | |
| date issued | 2010 | |
| identifier issn | 0882-8156 | |
| identifier other | ams-71496.pdf | |
| identifier uri | http://onlinelibrary.yabesh.ir/handle/yetl/4213394 | |
| description abstract | Since 2003 the National Center for Atmospheric Research (NCAR) has been running various experimental convection-allowing configurations of the Weather Research and Forecasting Model (WRF) for domains covering a large portion of the central United States during the warm season (April?July). In this study, the skill of 3-hourly accumulated precipitation forecasts from a large sample of these convection-allowing simulations conducted during 2004?05 and 2007?08 is compared to that from operational North American Mesoscale (NAM) model forecasts using a neighborhood-based equitable threat score (ETS). Separate analyses were conducted for simulations run before and after the implementation in 2007 of positive-definite (PD) moisture transport for the NCAR-WRF simulations. The neighborhood-based ETS (denoted ?ETS?r) relaxes the criteria for ?hits? (i.e., correct forecasts) by considering grid points within a specified radius r. It is shown that ?ETS?r is more useful than the traditional ETS because ?ETS?r can be used to diagnose differences in precipitation forecast skill between different models as a function of spatial scale, whereas the traditional ETS only considers the spatial scale of the verification grid. It was found that differences in ?ETS?r between NCAR-WRF and NAM generally increased with increasing r, with NCAR-WRF having higher scores. Examining time series of ?ETS?r for r = 100 and r = 0 km (which simply reduces to the ?traditional? ETS), statistically significant differences between NCAR-WRF and NAM were found at many forecast lead times for ?ETS?100 but only a few times for ?ETS?0. Larger and more statistically significant differences occurred with the 2007?08 cases relative to the 2004?05 cases. Because of differences in model configurations and dominant large-scale weather regimes, a more controlled experiment would have been needed to diagnose the reason for the larger differences that occurred with the 2007?08 cases. Finally, a compositing technique was used to diagnose the differences in the spatial distribution of the forecasts. This technique implied westward displacement errors for NAM model forecasts in both sets of cases and in NCAR-WRF model forecasts for the 2007?08 cases. Generally, the results are encouraging because they imply that advantages in convection-allowing relative to convection-parameterizing simulations noted in recent studies are reflected in an objective neighborhood-based metric. | |
| publisher | American Meteorological Society | |
| title | Neighborhood-Based Verification of Precipitation Forecasts from Convection-Allowing NCAR WRF Model Simulations and the Operational NAM | |
| type | Journal Paper | |
| journal volume | 25 | |
| journal issue | 5 | |
| journal title | Weather and Forecasting | |
| identifier doi | 10.1175/2010WAF2222404.1 | |
| journal fristpage | 1495 | |
| journal lastpage | 1509 | |
| tree | Weather and Forecasting:;2010:;volume( 025 ):;issue: 005 | |
| contenttype | Fulltext |