Sufficient Reductions in Regressions with Exponential Family Inverse Predictors

Abstract

We develop methodology for identifying and estimating sufficient reductions in regressions with predictors that, given the response, follow a multivariate exponential family distribution. This setup includes regressions where predictors are all continuous, all categorical, or mixtures of categorical and continuous. We derive the minimal sufficient reduction of the predictors and its maximum likelihood estimator by modeling the conditional distribution of the predictors given the response. Whereas nearly all extant estimators of sufficient reductions are linear and only partly capture the sufficient reduction, our method is not limited to linear reductions. It also provides the exact form of the sufficient reduction, which is exhaustive, its maximum likelihood (ML) estimates via an iterated reweighted least-square (IRLS) estimation algorithm, and asymptotic tests for the dimension of the regression. Supplementary materials for this article are available online.