Local Factor Analysis Research Articles

Learning local factor analysis versus mixture of factor analyzers with automatic model selection

Considering Factor Analysis (FA) for each component of Gaussian Mixture Model (GMM), clustering and local dimensionality reduction can be addressed simultaneously by Mixture of Factor Analyzers (MFA) and Local Factor Analysis (LFA), which correspond to two FA parameterizations, respectively. This paper investigates the performance of Variational Bayes (VB) and Bayesian Ying-Yang (BYY) harmony learning on MFA/LFA for the problem of automatically determining the component number and the local hidden dimensionalities (i.e., the number of factors of FA in each component). Similar to the existing VB learning algorithm on MFA, we develop an alternative VB algorithm on LFA with a similar conjugate Dirichlet–Normal–Gamma (DNG) prior on all parameters of LFA. Also, the corresponding BYY algorithms are developed for MFA and LFA. A wide range of synthetic experiments shows that LFA is superior to MFA in model selection under either VB or BYY, while BYY outperforms VB reliably on both MFA and LFA. These empirical findings are consistently observed from real applications on not only face and handwritten digit images clustering, but also unsupervised image segmentation.

Codimensional matrix pairing perspective of BYY harmony learning: hierarchy of bilinear systems, joint decomposition of data-covariance, and applications of network biology

One paper in a preceding issue of this journal has introduced the Bayesian Ying-Yang (BYY) harmony learning from a perspective of problem solving, parameter learning, and model selection. In a complementary role, the paper provides further insights from another perspective that a co-dimensional matrix pair (shortly co-dim matrix pair) forms a building unit and a hierarchy of such building units sets up the BYY system. The BYY harmony learning is re-examined via exploring the nature of a co-dim matrix pair, which leads to improved learning performance with refined model selection criteria and a modified mechanism that coordinates automatic model selection and sparse learning. Besides updating typical algorithms of factor analysis (FA), binary FA (BFA), binary matrix factorization (BMF), and nonnegative matrix factorization (NMF) to share such a mechanism, we are also led to (a) a new parametrization that embeds a de-noise nature to Gaussian mixture and local FA (LFA); (b) an alternative formulation of graph Laplacian based linear manifold learning; (c) a codecomposition of data and covariance for learning regularization and data integration; and (d) a co-dim matrix pair based generalization of temporal FA and state space model. Moreover, with help of a co-dim matrix pair in Hadamard product, we are led to a semi-supervised formation for regression analysis and a semi-blind learning formation for temporal FA and state space model. Furthermore, we address that these advances provide with new tools for network biology studies, including learning transcriptional regulatory, Protein-Protein Interaction network alignment, and network integration.

Radar HRRP Statistical Recognition With Local Factor Analysis by Automatic Bayesian Ying-Yang Harmony Learning

Radar high-resolution range profiles (HRRPs) are typical high-dimensional, non-Gaussian and interdimension dependently distributed data, the statistical modelling of which is a challenging task for HRRP based target recognition. Assuming the HRRP data follow interdimension dependent Gaussian distribution, factor analysis (FA) was recently applied to describe radar HRRPs and a two-phase procedure was used for model selection, showing promising recognition results. Besides the interdimensional dependence, this paper further models the non-Gaussianity of the radar HRRP data by local factor analysis (LFA). Moreover, since the two-phase procedure suffers from extensive computation and inaccurate evaluation on high-dimensional finite HRRPs, we adopt an automatic Bayesian Ying-Yang (BYY) harmony learning, which determines the component number and the hidden dimensionalities of LFA automatically during parameter learning. Experimental results show incremental improvements on recognition accuracy by three implementations, progressively from a two-phase FA, to a two-phase LFA, and then to an automatically learned LFA by BYY harmony learning.

Local factor analysis of rank-deficient reaction systems

The analysis of spectral measurement data sets using local factor analysis (LFA) requires the rank of the sub-matrix under study to be equal to the number of absorbing species present in the associated sub-system. However, because of mass balance or kinetic constraints, LFA will fail if local rank deficiency occurs. A local rank deficiency sub-system may be present in a global full-rank reaction system or a rank-deficient one. In this paper, the problems occurring when using window target-testing factor analysis (WTTFA), one type of the LFA methods, in a local rank-deficient situation are shown. A new augmented WTTFA (AWTTFA) is then proposed for the correct use of WTTFA when rank deficiency occurs. Principles of this new method have been demonstrated by a simulated kinetic system and an industrial batch data set.