Adaptive Tuning of Numerical Weather Prediction Models: Randomized GCV in Three- and Four-Dimensional Data Assimilation

Abstract

In variational data assimilation, optimal ingestion of the observational data, and optimal use of prior physical and statistical information involve the choice of numerous weighting, smoothing, and tuning parameters that control the filtering and merging of diverse sources of information. Generally these weights must be obtained from a partial and imperfect understanding of various sources of errors and are frequently chosen by a combination of historical information, physical reasoning, and trial and error. Generalized cross validation (GCV) has long been one of the methods of choice for choosing certain tuning, smoothing, regularization parameters in ill-posed inverse problems, smoothing, and filtering problems. In theory, it is well suited for the adaptive choice of certain parameters that occur in variational objective analysis and for data assimilation problems that are mathematically equivalent to variational problems. The main drawback of the use of GCV in data assimilation problems was that matrix decompositions were apparently needed to compute the GCV estirmtes. This limited the application of GCV to datasets of the order of less than about 1000. Recently, the randomized trace technique for computing the GCV estimates has been developed, and this makes the use of GCV feasible in essentially any variational problem that has an operating algorithm to produce estimates, given data. In this paper the authors demonstrate that the answers given by the randomized trace estimate are indistinguishable in a practical sense from those computed more exactly by traditional methods. Then the authors carry out an experiment to choose one of the main smoothing parameters (?) in the context of a variational objective analysis problem that is approximately solved by k iterations of a conjugate gradient algorithm. The authors show how the randomized trace technique can be used to obtain good values of both ? and k in this context. Finally, the authors describe how the method can be applied in operational-sized three- and four-dimensional variational data assimilation schemes, as well as in conjunction with a Kalman filter.