On the Calculation and Correction of Equitable Threat Score for Model Quantitative Precipitation Forecasts for Small Verification Areas: The Example of Taiwan

Abstract

s one of the most widely used skill scores for model quantitative precipitation forecast (QPF) verification and evaluation, the equitable threat score ETS differs from the threat score TS in that the random hits R are removed from its calculation. In practice, however, when applied to a set of verification points confined to small areas, the random assumption often becomes increasingly questionable or even invalid in larger events. As a result, the random hits are overestimated and the ETS becomes biased and not indicative of model skills. In this paper, such an issue is explored and demonstrated through the example of Taiwan with steep topography from Typhoon Morakot (2009) and mei-yu heavy-rainfall cases. It is found that the ETS is affected more seriously and scaled down by at least about 0.1 compared to the TS whenever the rain area occupies roughly 20% or more of the total verification area (if the random assumption of R is invalid). As such conditions often occur for small areas, it is vital to estimate R as correctly as possible for the ETS to work properly. A simple solution is offered by using all gridpoint values from the entire model domain, rather than just a small subset falling into the verification area, to estimate the random hit rate in the forecast. While the ETS remains unaltered in its definition, the proposed method yields the best estimates of R available by using the largest sample size from the model and subsequently better-behaved ETS values and is, therefore, recommended for all applications of ETS for small verification areas.