On the Uniqueness of Gandin and Murphy’s Equitable Performance Measures

Abstract

Gandin and Murphy (GM) have shown that if a skill score is linear in the scoring matrix, and if the scoring matrix is symmetric, then in the two-event case there exists a unique, ?equitable? skill score, namely, the True Skill Score (or Kuipers?s performance index). As such, this measure is treated as preferable to other measures because of its equitability. However, in most practical situations the scoring matrix is not symmetric due to the unequal costs associated with false alarms and misses. As a result, GM?s considerations must be reexamined without the assumption of a symmetric scoring matrix. In this note, it will be proven that if the scoring matrix is nonsymmetric, then there does not exist a unique performance measure, linear in the scoring matrix, that would satisfy any constraints of equitability. In short, there does not exist a unique, equitable skill score for two-category events that have unequal costs associated with a miss and a false alarm.