Statistical Significance Testing in Numerical Weather Prediction

Abstract

Experiments are often performed with numerical forecast models to determine the response to a changed model formulation, initial conditions or boundary conditions. Such experiments are inherently subject to sampling error and it is not always easy to separate the signal (that part of the response due to the prescribed change) from the noise (due to the inherent variability of the model statistics). Climate modelers have also faced this problem and have developed a variety of techniques to determine the statistical significance of the response to a given model change. Conceptual frameworks will be developed for the practical use of statistical significance testing for both the climate drift problem and model sensitivity experiments. The statistical significance testing techniques used are taken largely from the climate modeling literature and include t?testing and Monte Carlo methods. It appears that statistical significance tests can be performed relatively inexpensively for model sensitivity experiments, but that testing the significance of climate drift experiments will require very large samples.