Assessing the Impact of Collaborative Research Projects on NWS Warning Performance

Abstract

Since the inception of the Cooperative Program for Operational Meteorological Education and Training (COMET) Outreach Program in 1990, over 250 collaborative projects have been funded nationwide, involving approximately 90 National Weather Service (NWS) offices and over 70 different universities. However, until now there have not been any attempts to objectively assess the impact of these collaborative projects on NWS warning performance. A study was conducted to examine the impacts of COMET Cooperative and Partners collaborative research projects on NWS performance metrics for tornado, severe thunderstorm, flash flood, and winter storm warnings. The study evaluated relevant collaborative projects within the Eastern Region of the NWS completed between 1995 and 2001. In addition, the verification trends at the NWS Weather Forecast Office (WFO) in Raleigh, North Carolina, were examined to evaluate the influence of long-term collaborative activities on performance. WFO Raleigh has been continuously involved in collaborative projects with North Carolina State University since the 1980s, with the first COMET-funded project commencing in January 1991. There are many factors that influence warning program verification scores on the long term (e.g., technology infusion, implementation of applied research results, etc.) and the short term (e.g., weather ?regime? impacts on event types and frequencies, office staffing issues and experience levels, etc.). The study was designed to try to isolate, to the extent possible, the impacts of the collaborative projects from these other factors. The results indicate that warning program verification scores for offices involved in COMET collaborative research activities appear to improve at a greater rate than the overall performance of all NWS offices within the Eastern Region. The greatest impact was noted for warning lead times. Probabilities of detection (POD) also showed increased rates of improvement.