Algorithm for orthogonal matrix nearness and its application to feature representation

Abstract

Various learning problems can be represented as certain canonical forms of orthogonal matrix nearness problems under the unitarily invariant norm. Since varying unitarily invariant norms favor different structured learning of input data, it is crucial to construct a unified scheme to learn encouraging distributions via any unitarily invariant norm. In this paper, we find that the orthogonal matrix nearness problem can be generalized to a common architecture of the unitarily invariant norm minimization, thus further exploit iterative and closed-form solutions. Firstly, we start with several special circumstances of orthogonal matrix nearness problems, where the unitarily invariant norm resorts to Frobenius norm. Secondly, a general scheme for orthogonal matrix nearness problems under any unitarily invariant norm is addressed and the corresponding iterative algorithm is proposed. Thirdly, feature representation problems are formulated as certain forms of orthogonal matrix nearness, indicating a joint framework for learning low-dimensional features with closed-form solutions. Finally, comprehensive experiments on real-world data sets demonstrate the effectiveness of the proposed method against compared state-of-the-art feature representation approaches. In particular, the proposed method gains about 15% improvements of clustering accuracy on BASEHOCK, CNAE and RCV1 data sets than benchmarks, and achieves at least 3% improvements on other data sets.