The Complex Relationship between Forecast Skill and Forecast Value: A Real-World Analysis

Abstract

For routine forecasts of temperature and precipitation, the relative skill advantage of human forecasters with respect to the numerical?statistical guidance is small (and diminishing). Since the relationship between forecast skill and the value of those forecasts is complex, the authors have examined their value across a range of real-world user contexts. It is found that although in most cases the meteorological information possessed considerable value to the users, human intervention in making those forecasts (as measured by National Weather Service forecasts) has generally led to minimal gains in value beyond that which is obtainable through direct use of numerical?statistical guidance. An important exception is the use of meteorological information by gas utilities during peak wintertime periods; in those circumstances, the value of human intervention was considerable. The presence of information in the National Weather Service forecasts independent of that contained in the numerical?statistical guidance was also established. Despite this, application of the additional information through a combined National Weather Service/guidance forecast provided only a small gain in value in most cases. In the most successful forecast context (the gas utility), the combined approach led to a loss of value relative to the unaltered National Weather Service forecasts. However, recent trends toward increased skill in probability of precipitation forecasts have led to some gains in the relative value of the National Weather Service forecasts, concurrent with a shift toward smaller optimal cost?loss ratio distributions, findings that are significant with respect to practical business considerations. Furthermore, all of the applications studied showed the potential for considerable further growth in forecast value with continued increases in forecast skill. The relevance of our findings to the future of public and private meteorological forecasting is briefly discussed.