Potential Value of Expert Elicitation for Determining Differential Credibility of Regional Climate Change Simulations: An Exercise with the NARCCAP co-PIs for the Southwest Monsoon Region of North America

Abstract

n this brief article, we report the initial results of an expert elicitation with the co-PIs (regional climate modelers) of the North American Regional Climate Change Assessment Program regarding their evaluation of the relative quality of regional climate model simulations focusing on the subregion dominated by the North American monsoon (NAM). We assumed that an expert elicitation framework might reveal interesting beliefs and understanding that would be different from what would be obtained from calculating quantitative metrics associated with model quality.The simulations considered were of six regional climate models (RCMs) that used NCEP Reanalysis 2 as boundary conditions for the years 1980?2004. The domain covers most of North America and adjacent oceans. The seven participating regional modelers were asked to complete surveys on their background beliefs about model credibility and their judgments regarding the quality of the six models based on a series of plots of variables related to the NAM (e.g., temperature, winds, humidity, moisture flux, precipitation). The specific RCMs were not identified.We also compared the results of the expert elicitation with those obtained from using a series of metrics developed to evaluate a European collection of climate model simulations. The results proved to be quite different in the two cases.The results of this exercise proved very enlightening regarding regional modelers? perceptions of model quality and their beliefs about how this information should or should not be used. Based on these pilot study results, we believe a more complete study is warranted.