On the Relationship between the Accuracy and Value of Forecasts in the Cost–Loss Ratio Situation

Abstract

This paper explores the relationship between the quality and value of imperfect forecasts. It is assumed that these forecasts are produced by a primitive probabilistic forecasting system and that the decision-making problem of concern is the cost-loss ratio situation. In this context, two parameters describing basic characteristics of the forecasts must be specified in order to determine forecast quality uniquely. As a result, a scalar measure of accuracy such as the Brier score cannot completely and unambiguously describe the quality of the imperfect forecasts. The relationship between forecast accuracy and forecast value is represented by a multivalued function?an accuracy/value envelope. Existence of this envelope implies that the Brier score is an imprecise measure of value and that forecast value can even decrease as forecast accuracy increases (and vice versa). The generality of these results and their implications for verification procedures and practices are discussed.