Wrapper Methods for Inductive Learning: Example Application to Bridge Decks

Abstract

The decision tree algorithm is one of the most common techniques of inductive learning. This paper investigates the use of wrapper methods for bagging, boosting, and feature selection to improve the prediction accuracy of the decision tree algorithm. A set of concrete bridge decks is extracted from the Kansas bridge database, and the deterioration of the health index is selected as the decision/class value for induction. From the conducted experiments, the decision tree accuracy obtained is 67.7%, whereas bagging and the boosting gave 73.4% and 72.7%, respectively. Wrapping with a feature selection method gave an accuracy of 75.0%. If feature selection method is applied first, bagging and boosting do not provide any further improvement to the decision tree algorithm. A series of tests were conducted where the selected features were examined and manually eliminated for the data set. This revealed that the improvement obtained by the feature selection method can be misleading. For the problem at hand, the attributes selected were not the most important ones to the problem domain. Therefore, what may be an improvement from the machine learning or data mining viewpoint, can turn out to be a mistake from an engineering perspective. Automatically selected attributes should be checked carefully. Feature selection is not recommended in this case.