Switching class labels to generate classification ensembles

Abstract

Ensembles that combine the decisions of classi ers generated by using perturbed versions of the training set where the classes of the training examples are randomly switched can produce a signi cant error reduction, provided that large numbers of units and high class switching rates are used. The classi ers generated by this procedure have statistically uncorrelated errors in the training set. Hence, the ensembles they form exhibit a similar dependence of the training error on ensemble size, independently of the classi cation problem. In particular, for binary classi cation problems, the classi cation performance of the ensemble on the training data can be analysed in terms of a Bernoulli process. Experiments on several UCI datasets demonstrate the improvements in classi cation accuracy that can be obtained using these class-switching ensembles.