Significance Tests for Regression Model Hierarchies

Abstract

Methods of estimating the significance of optimal regression models selected from a model hierarchy proposed by Barnett and Hasselmann (1979) are reexamined allowing for the multiple-candidate nature of the selection criteria. It is found that the single-candidate models' significance value previously used can over- or underestimate the true multiple-candidate significance level of the selected model depending on the selection criteria used. A number of possible selection strategies to remove these problems are discussed and evaluated both theoretically and by Monte Carlo simulators.