The Generalization Complexity Measure for Continuous Input Data

Abstract

We introduce in this work an extension for the generalization complexity measure to continuous input data. The measure, originallydefined in Boolean space, quantifies the complexity of data in relationship to the prediction accuracy that can be expected whenusing a supervised classifier like a neural network, SVM, and so forth. We first extend the original measure for its use withcontinuous functions to later on, using an approach based on the use of the set of Walsh functions, consider the case of havinga finite number of data points (inputs/outputs pairs), that is, usually the practical case. Using a set of trigonometric functions amodel that gives a relationship between the size of the hidden layerof a neural network and the complexity is constructed. Finally,we demonstrate the application of the introduced complexity measure, by using the generated model, to the problem of estimatingan adequate neural network architecture for real-world data sets.