Numerical Model-Reality Intercomparison Tests Using Small-Sample Statistics

Abstract

When a numerical model's representation of a physical field is to be compared with a corresponding real observed field, it is usually the case that the numbers of realizations of model and observed field are relatively small, so that the natural procedure of producing histograms of pertinent statistics of the fields (e.g., means, variances) from the data sets themselves cannot be usually carried out. Also, it is not always safe to adopt assumptions of normality and independence of the data values. This prevents the confident use of classical statistical methods to make significance statements about the success or failure of the model's replication of the data. Here we suggest two techniques of determinable statistical power, in which small samples of spatially extensive physical fields can be made to blossom into workably large samples on which significance decisions can be based. We also introduce some new measures of location, spread and shape of multivariate data sets which may be used in conjunction with the two techniques. The result is a pair of new data intercomparison procedures which we illustrate using GCM simulations of the January sea-level pressure field and regional ocean model simulations of the new-shore velocity field of South America. We include with these procedures a method of determining the spatial and temporal locations of non-random errors between the model and data fields so that models can be improved accordingly.