Bayesian Ying-Yang Learning on Orthogonal Binary Factor Analysis

Abstract

Binary Factor Analysis (BFA) aims to discover latent binary structures in high dimensional data. Parameter learning in BFA suffers from exponential complexity and a large number of local optima. Model selection in BFA is therefore difficult. The traditional approach for model selection is implemented in a two phase procedure. On a prefixed range of model scales, maximum likelihood (ML) learning is performed for each candidate scale. After this enumeration, the optimum scale is selected according to some criterion. In contrast, the Bayesian Ying-Yang (BYY) learning starts from a high dimensional model and automatically deducts the dimension during parameter learning. The enumeration overhead in the two phase approach is saved. This paper investigates a subclass of BFA called Orthogonal Binary Factor Analysis (OBFA). A BYY machine for OBFA is constructed. The harmony measure, which serves as the objective function in the BYY harmony learning, is more accurately estimated by recovering a term that was missing in the previous studies on BYY learning based BFA. Comparison with traditional two phase implementations shows good performance of the proposed approach.KeywordsModel SelectionParameter LearningHigh Dimensional ModelLatent Trait ModelExternal ObservationThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.