On Subjective Probability Forecasting

Abstract

The subjective process of probability forecasting is analyzed. It is found to contain a sorling aspect, in which the forecaster distributes all instances into an ordered set of categories of likelihood of occurrence, and a laboling aspect, in which the forecaster assigns an anticipated relative frequency, or probability, of occurrence for each category. These two aspects are identified with the concepts of sharpness and validity, which have been introduced by other writers. The verification score proposed by Brier is shown to consist of the sum of measures of these two qualities. A satisfactory measure of synoptic skill is obtained by applying the Brier score to the synoptic probability forecast and to a control forecast of the climatological probability, and by expressing the difference as a percentage of the control score. In an analysis of a large number of short-range probability forecasts made by instructors and students in the synoptic laboratory of the Massachusetts Institute of Technology it is found that even inexperienced forecasters are capable of displaying validity and skill except when dealing with events which occur very rarely or nearly always. Skill for average or net conditions over 24-hr periods is found to he roughly twice the skill in forecasts for a particular instant and is found to vary with the directness with which the weather element can be inferred from prognostic charts. The average of the judgment of two or more forecasters with comparable experience is found to be a more skillful statement than the forecast of the most skilled individual.