MIMO Modeling by Learning Explicitly the Projection Space: The Maximum Correlation Ratio Cost Function

Abstract

Maximal correlation measures the statistical dependence between two random variables and has been broadly used by statisticians. In this paper, we present a novel regression framework based on a special case of the maximal correlation using the correlation ratio to train a Multiple-input Multiple-output (MIMO) model called the Bank of Wiener Models (BWM). Nonlinear time series modeling under maximal correlation unifies the cost and the mapping function under the same mathematical optimization framework. This provides direct optimality (maximal statistical dependence) between BWM model outputs and the desired response, without the restriction of constructing an error signal as conventionally done in regression. Based on the idea of correlation ratio, we propose the Maximal Correlation Algorithm (MCA), which approximates directly the correlation ratio between BWM outputs and the desired response. As an important consequence, MCA modularizes the training of models with hidden layers and avoids the end-to-end training of backpropagation (BP), while improving the equivalent mapping capability. We further propose several possible multi layer arrangements to create deep networks trained with MCA. We demonstrate experimentally that MCA performs at the same level of output error or better when compared to a single and multiple-hidden-layer MLP trained with BP and the Mean Squared Error (MSE). We also show that the system identification capabilities of MCA are superior to MLPs trained with BP. Hence, MCA has great promise in both nonlinear system identification and machine learning.