Predictable Component Analysis, Canonical Correlation Analysis, and Autoregressive Models

Abstract

The purpose of this paper is to clarify the relation between canonical correlation analysis, autoregressive models (also called linear inverse models), and a relatively new statistical technique known as predictable component analysis. Predictable component analysis is a procedure for decomposing forecast error into a complete set of uncorrelated patterns that optimize the normalized error variance. As such, the procedure determines the components of a forecast, out of all possible components, that are predicted the best and worst on average, in a normalized error variance sense. This procedure has been suggested previously in the context of information theory and of maximizing the variance of prewhitened forecast errors. It is shown that the most predictable components of a linear, first-order, autoregressive model are identical to the canonical patterns that optimize the temporal correlation between the original and time-lagged time series. Also, canonical patterns define a transformation that diagonalizes the propagator of the autoregressive model. Finally, the singular vectors of an autoregressive propagator have a one-to-one correspondence to the canonical patterns, provided that the singular vectors are computed with respect to a norm related to a prewhitening transformation. Forecasts based on canonical correlation analysis are shown to be identical to forecasts based on autoregressive models when all canonical patterns are superposed. The advantage of canonical correlation analysis is that it allows one to filter out the ?unpredictable components? of autoregressive models, defined as components in the forecast with skill below some statistical threshold of significance. Predictable component analysis allows a generalization of this filtering procedure to any forecast model.