Building heterogeneous ensembles by pooling homogeneous ensembles
EntityUAM. Departamento de Ingeniería Informática
10.1007/s13042-021-01442-1International Journal of Machine Learning and Cybernetics volume 13.2 (2022): 551-558
Funded byThe authors acknowledge financial support from PID2019-106827GB-I00/AEI/10.13039/501100011033
ProjectGobierno de España. PID2019-106827GB-I00/AEI/10.13039/501100011033
SubjectsEnsembles; Heterogeneous; Homogeneous; Optimal composition; Simplex; Informática
Rights© The Author(s) 2021, corrected publication 2021
Esta obra está bajo una Licencia Creative Commons Atribución 4.0 Internacional.
Heterogeneous ensembles consist of predictors of different types, which are likely to have different biases. If these biases are complementary, the combination of their decisions is beneficial and could be superior to homogeneous ensembles. In this paper, a family of heterogeneous ensembles is built by pooling classifiers from M homogeneous ensembles of different types of size T. Depending on the fraction of base classifiers of each type, a particular heterogeneous combination in this family is represented by a point in a regular simplex in M dimensions. The M vertices of this simplex represent the different homogeneous ensembles. A displacement away from one of these vertices effects a smooth transformation of the corresponding homogeneous ensemble into a heterogeneous one. The optimal composition of such heterogeneous ensemble can be determined using cross-validation or, if bootstrap samples are used to build the individual classifiers, out-of-bag data. The proposed heterogeneous ensemble building strategy, composed of neural networks, SVMs, and random trees (i.e. from a standard random forest), is analyzed in a comprehensive empirical analysis and compared to a benchmark of other heterogeneous and homogeneous ensembles. The achieved results illustrate the gains that can be achieved by the proposed ensemble creation method with respect to both homogeneous ensembles and to the tested heterogeneous building strategy at a fraction of the training cost.
This item appears in the following Collection(s)
Showing items related by title, author, creator and subject.