A Unifying Least Squares Perspective for Gappy Proper Orthogonal Decomposition and Probabilistic Principal Component Analysis

Abstract

In aerospace engineering, various problems such as restoring impaired experimental flow data can be handled by gappy proper orthogonal decomposition (POD). Similarly to gappy POD, probabilistic principal component analysis (PPCA) can approximate missing data with the help of an expectation-maximization (EM) algorithm, yielding an EM algorithm applied to PPCA (EM-PCA). Although both gappy POD and EM-PCA address the same missing date estimation problem, their antithetical formulation perspectives hinder their direct comparison; the development of the former is deterministic whereas that of the latter is probabilistic. In order to effectively differentiate both methods, this research provides a unifying least squares perspective to qualitatively dissect them within a unified least squares framework. By virtue of the unifying least squares perspective, gappy POD and the EM-PCA turn out to be similar in that they are twofold: basis and least squares coefficient evaluations. On the other hand, they are dissimilar because the EM-PCA, unlike gappy POD, dispenses with either a gappy norm or a POD basis. To illustrate the theoretical analysis of both methods, numerical experiments using simple and complex data sets quantitatively examine their performance in terms of convergence rates and computational cost. Finally, comprehensive comparisons, including theoretical and numerical aspects, establish that the EM-PCA is simpler and thereby more efficient than gappy POD.