Generalized discriminant analysis via kernel exponential families

Abstract

This paper introduces a novel supervised dimension reduction method for classification and regression problems using reproducing kernel Hilbert spaces. The proposed approach takes advantage of the modeling power of kernel exponential families to extract nonlinear summary statistics of the data that are sufficient to preserve information about the target response. For the special case of finite dimensional exponential family distributions, the proposed method is shown to simplify the known solutions for sufficient dimension reduction. A connection with support vector machines is shown and exploited to obtain efficient estimation procedures. Experiments with simulated and real data illustrate the potential of the proposed approach.