Sufficient reductions in regression with mixed predictors

Abstract

Most data sets comprise of measurements on continuous and categorical variables. Yet,modeling high-dimensional mixed predictors has received limited attention in the regressionand classication statistical literature. We study the general regression problem of inferringon a variable of interest based on high dimensional mixed continuous and binary predictors.The aim is to nd a lower dimensional function of the mixed predictor vector that containsall the modeling information in the mixed predictors for the response, which can be eithercontinuous or categorical. The approach we propose identies sucient reductions byreversing the regression and modeling the mixed predictors conditional on the response.We derive the maximum likelihood estimator of the sucient reductions, asymptotic testsfor dimension, and a regularized estimator, which simultaneously achieves variable (feature)selection and dimension reduction (feature extraction). We study the performance of theproposed method and compare it with other approaches through simulations and real dataexamples.