Model Fidelity versus Skill in Seasonal Forecasting

Abstract

The relation between skill and fidelity of seasonal mean hindcasts of surface temperature by seven coupled atmosphere?ocean models is investigated. By definition, fidelity measures the agreement between model and observational climatological distributions, and skill measures the agreement between hindcasts and their corresponding verifications. While a relation between skill and fidelity seems intuitively plausible, it has not been checked systematically, nor is it mandated mathematically. New measures of skill and fidelity based on information theory are proposed. Specifically, fidelity is measured by the area average relative entropy between the climatological distributions of the model and observations, and skill is measured by the area averaged mutual information between forecast and verification. The fidelity measure is found to be dominated by the term measuring mean bias; that is, the discrepancy in climatological means is much larger than the discrepancy in climatological variances. Moreover, the mean bias is negatively correlated with skill at most initial months, lead times, and regions examined. Thus, models that more closely replicate the observed climatological mean tend to have better skill.