A Monte Carlo Method for Testing the Statistical Significance of a Regression Equation

Abstract

A Monte Carlo technique for testing the statistical significance of an estimate by regression is described. It is illustrated on a medium-range prediction of precipitation. The purpose of the technique is to permit the use of all data available while developing a regression equation. This is especially important when the sample is small. The need for setting aside a fraction of the data to test the equation on an independent sample, or to estimate degrees of freedom before applying standard statistical tests, is eliminated. In the illustration, a stepwise regression procedure was used to select predictors and derive an equation to predict actually observed precipitation. Then the procedure was repeated 20 times, each time on a set of bogus values of precipitation drawn at random from the historical population of precipitation values. Each of the 20 equations resulting from bogus values of precipitation was used to estimate the precipitation and to give 20 values of per cent reduction of variance. A per cent reduction greater than the upper 5% would have led to a rejection of the null hypothesis and the inference that the prediction equation would yield skillful forecasts. In the example, the null hypothesis was not rejected.