Gaussian Process Regression With Interquartile Range Selection and Stacking Ensemble for Engineering Design

Abstract

Abstract. This study proposes a selection and stacking ensemble-based method to facilitate the determination of the covariance function type for Gaussian process regression. The proposed method operates at the model ensemble level and involves the use of multiple Gaussian process regression models with different types of covariance functions as base learners in a stacking ensemble with a final Gaussian process regression model as the meta-learner. First, the Pearson correlation coefficients between the leave-one-out predicted responses from each candidate base learner and the actual responses are computed and sorted in ascending order, after which the interquartile range (IQR) is calculated, and the candidate base learners that fall below the lower 1.5 × IQR are removed. Afterward, the adjacent gaps between the Pearson correlation coefficients that correspond to the remaining candidate base learners are calculated and sorted in ascending order, and some of the remaining candidate base learners are further removed according to the gaps that lie above the upper 1.5 × IQR. Finally, a newly constructed Gaussian process regression model with a linear covariance function is used as the meta-learner for final predictions. To validate the effectiveness of the proposed method, six analytical test functions, three engineering datasets, and one simulation case are used for a performance study along with three representative approaches. The results demonstrate that the proposed method achieves competitive accuracy and generalization ability. Furthermore, its performance is evaluated across four widely used open-source toolkits for Gaussian process regression, and the results confirm the robustness of the method.