Model Induction with Support Vector Machines: Introduction and Applications

Abstract

The rapid advance in information processing systems in recent decades had directed engineering research towards the development of intelligent systems that can evolve models of natural phenomena automatically—“by themselves,” so to speak. In this respect, a wide range of machine learning techniques like decision trees, artificial neural networks (ANNs), Bayesian methods, fuzzy-rule based systems, and evolutionary algorithms have been successfully applied to model different civil engineering systems. In this study, the possibility of using yet another machine learning paradigm that is firmly based on the theory of statistical learning, namely that of the support vector machine (SVM), is investigated. An interesting property of this approach is that it is an approximate implementation of a structural risk minimization (SRM) induction principle that aims at minimizing a bound on the generalization error of a model, rather than minimizing only the mean square error over the data set. In this paper, the basic ideas underlying statistical learning theory and SVM are reviewed, and the potential of the SVM for feature classification and multiple regression (modeling) problems is demonstrated by applying the method to two different cases of model induction from empirical data. The relative performance of the SVM is then analyzed by comparing its results with that of ANNs on the same data sets.