Evaluating the Skill of Categorical Forecasts

Abstract

A generalized skill score is presented for evaluating forecasts in any number of categories. Each forecast in a sample is given a mark; the skill score for the sample is just the average mark. Each mark has an expected value of zero for an unskillful forecast and unity for a correct forecast. In the particular case of two categories the score corresponds to that of Hanssen and Kuipers (1965). For three or more categories an arbitrary penalty factor for incorrect forecasts, which may be as simple as the rank difference between forecast and observed categories, must be specified.