The Calibration Simplex: A Generalization of the Reliability Diagram for Three-Category Probability Forecasts

Abstract

ull exposition of the performance of a set of forecasts requires examination of the joint frequency distribution of those forecasts and their corresponding observations. In settings involving probability forecasts, this joint distribution has a high dimensionality, and communication of its information content is often best achieved graphically. This paper describes an extension of the well-known reliability diagram, which displays the joint distribution for probability forecasts of dichotomous events, to the case of probability forecasts for three disjoint events, such as ?below,? ?near,? and ?above normal.? The resulting diagram, called the calibration simplex, involves a discretization of the 2-simplex, which is an equilateral triangle. Characteristics and interpretation of the calibration simplex are illustrated using both idealized verification datasets, and the 6?10- and 8?14-day temperature and precipitation forecasts produced by the U.S. Climate Prediction Center.