Diagnosing the Sensitivity of Binary Image Measures to Bias, Location, and Event Frequency within a Forecast Verification Framework

Abstract

hile the use of binary distance measures has a substantial history in the field of image processing, these techniques have only recently been applied in the area of forecast verification. Designed to quantify the distance between two images, these measures can easily be extended for use with paired forecast and observation fields. The behavior of traditional forecast verification metrics based on the dichotomous contingency table continues to be an area of active study, but the sensitivity of image metrics has not yet been analyzed within the framework of forecast verification. Four binary distance measures are presented and the response of each to changes in event frequency, bias, and displacement error is documented. The Hausdorff distance and its derivatives, the modified and partial Hausdorff distances, are shown only to be sensitive to changes in base rate, bias, and displacement between the forecast and observation. In addition to its sensitivity to these three parameters, the Baddeley image metric is also sensitive to additional aspects of the forecast situation. It is shown that the Baddeley metric is dependent not only on the spatial relationship between a forecast and observation but also the location of the events within the domain. This behavior may have considerable impact on the results obtained when using this measure for forecast verification. For ease of comparison, a hypothetical forecast event is presented to quantitatively analyze the various sensitivities of these distance measures.