Non-linear generalization of principal component analysis: From a global to a local approach

Principal component analysis (PCA), also known as proper orthogonal decomposition or Karhunen-Loeve transform, is commonly used to reduce the dimensionality of a data set with a large number of interdependent variables. PCA is the optimal linear transformation with respect to minimizing the mean square reconstruction error but it only considers second-order statistics. If the data have non-linear dependencies, an important issue is to develop a technique which takes higher order statistics into account and which can eliminate dependencies not removed by PCA. Recognizing the shortcomings of PCA, researchers in the field of statistics and neural networks have developed non-linear extensions of PCA. The purpose of this paper is to present a non-linear generalization of PCA, called VQPCA. This algorithm builds local linear models by combining PCA with clustering of the input space. This paper concludes by observing from two illustrative examples that VQPCA is potentially a more effective tool than conventional PCA. (C) 2002 Elsevier Science Ltd. All rights reserved.