Feature selection on wide multiclass problems using OVA-RFE

Abstract

Feature selection is a pre–processing technique commonly used with high–dimensional datasets. It is aimed at reducing the dimensionality of the input space, discarding useless or redundant variables, in order to increase the performance and interpretability of models. For multiclass classification problems, recent works suggested that decomposing the multiclass problem in a set of binary ones, and doing the feature selection on the binary problems could be a sound strategy. In this work we combined the well–known Recursive Feature Elimination (RFE) algorithm with the simple One–Vs–All (OVA) technique for multiclass problems, to produce the new OVA–RFE selection method. We evaluated OVA–RFE using wide datasets from genomic and mass– spectrometry analysis, and several classifiers. In particular, we compared the new method with the traditional RFE (applied to a direct multiclass classifier) in terms of accuracy and stability. Our results show that OVA– RFE is no better than the traditional method, which is in opposition to previous results on similar methods. The opposite results are related to a different interpretation of the real number of variables in use by both methods.