Information-Based Skill Scores for Probabilistic Forecasts

Abstract

The information content, that is, the predictive capability, of a forecast system is often quantified with skill scores. This paper introduces two ranked mutual information skill (RMIS) scores, RMISO and RMISY, for the evaluation of probabilistic forecasts. These scores are based on the concept of mutual information of random variables as developed in information theory. Like the ranked probability skill score (RPSS)?another and often applied skill score?the new scores compare cumulative probabilities for multiple event thresholds. The RMISO quantifies the fraction of information in the observational data that is explained by the forecasts. The RMISY quantifies the amount of useful information in the forecasts. Like the RPSS, the new scores are biased, but they can be debiased with a simple and robust method. This and additional promising characteristics of the scores are discussed with ensemble forecast assessment experiments.