Quantification of NSSL Warn-on-Forecast System Accuracy by Storm Age Using Object-Based Verification

Abstract

The National Severe Storm Laboratory’s Warn-on-Forecast System (WoFS) is a convection-allowing ensemble with rapidly cycled data assimilation (DA) of various satellite and radar datasets designed for prediction at 0–6-h lead time of hazardous weather. With the focus on short lead times, WoFS predictive accuracy is strongly dependent on its ability to accurately initialize and depict the evolution of ongoing storms. Since it takes multiple DA cycles to fully “spin up” ongoing storms, predictive skill is likely a function of storm age at the time of model initialization, meaning that older storms that have been through several DA cycles will be forecast with greater accuracy than newer storms that initiate just before model initialization or at any point after. To quantify this relationship, we apply an object-based spatial tracking and verification approach to map differences in the probability of detection (POD), in space–time, of predicted storm objects from WoFS with respect to Multi-Radar Multi-Sensor (MRMS) reflectivity objects. Object-tracking/matching statistics are computed for all suitable and available WoFS cases from 2017 to 2021. Our results indicate sharply increasing POD with increasing storm age for lead times within 3 h. PODs were about 0.3 for storm objects that emerge 2–3 h after model initialization, while for storm objects that were at least an hour old at the time of model initialization by DA, PODs ranged from around 0.7 to 0.9 depending on the lead time. These results should aid in forecaster interpretation of WoFS, as well as guide WoFS developers on improving the model and DA system.