Incremental learning with ensemble based SVM classifiers for non-stationary environments

Abstract

In this thesis, we evaluate the performance of support vector machines (SVM) ensemble, which is constructed by using Learn++ algorithm, on changing environment and propose incorporating forgetting mechanism to adapt this algorithm to changing environment. In most of the real world applications, the data is collected over an extended period of time and the distribution underlying the data is likely to change by time. These changes make the model built on old data inconsistent with the new data, and regular updating of the model is necessary. For effective learning in a changing environment, the algorithm should be able to detect context change and quickly adjust the hypothesis to the current context. This can be achieved by revising the model by incorporating new examples and eliminating the effect of outdated concepts. Learn++ is an ensemble based incremental learning algorithm that is able to learn new information. Therefore, it can be easily adapted to changing environments by using a forgetting mechanism to remove the redundant data from the ensemble. In this thesis, we propose using a forgetting strategy that is based on the performance of the base classifiers. Only the best K classifiers or the classifiers whose classification performance exceeds a threshold value are kept in the ensemble. Our results indicate that incorporating forgetting mechanism improves the classification performance of the proposed algorithm on a changing environment. The proposed algorithm can effectively handle the gradual changes.