An Accelerated Symmetric Nonnegative Matrix Factorization Algorithm Using Extrapolation

Peitao Wang,Ji Tan,Taiheng Liu,Zhaoshui He,Yulei Bai,Zhijie Lin,Jun Lu,Beihai Tan

doi:10.3390/sym12071187

Peitao Wang, Ji Tan + Show 6 more

Open Access

https://doi.org/10.3390/sym12071187

Copy DOI

Publication Date: Jul 17, 2020
Citations: 2	License type: CC BY 4.0

Affiliation: Guangdong University of Technology, Peng Cheng Laboratory

Abstract

Symmetric nonnegative matrix factorization (SNMF) approximates a symmetric nonnegative matrix by the product of a nonnegative low-rank matrix and its transpose. SNMF has been successfully used in many real-world applications such as clustering. In this paper, we propose an accelerated variant of the multiplicative update (MU) algorithm of He et al. designed to solve the SNMF problem. The accelerated algorithm is derived by using the extrapolation scheme of Nesterov and a restart strategy. The extrapolation scheme plays a leading role in accelerating the MU algorithm of He et al. and the restart strategy ensures that the objective function of SNMF is monotonically decreasing. We apply the accelerated algorithm to clustering problems and symmetric nonnegative tensor factorization (SNTF). The experiment results on both synthetic and real-world data show that it is more than four times faster than the MU algorithm of He et al. and performs favorably compared to recent state-of-the-art algorithms.

Highlights

Given a symmetric nonnegative matrix A ∈ Rn+×n, symmetric nonnegative matrix factorization (SNMF) aims to find a nonnegative matrix G ∈ Rn+×r such that A ≈ GG T
The clustering quality is measured by clustering accuracy (CA) and normalized mutual information (NMI) [19]
NMI is a normalization of the Mutual information (MI) score to scale the results between 0 and 1

Summary

Introduction

Given a symmetric nonnegative matrix A ∈ Rn+×n , symmetric nonnegative matrix factorization (SNMF) aims to find a nonnegative matrix G ∈ Rn+×r (generally r n) such that A ≈ GG T. SNMF is a special but important class of symmetric nonnegative tensor factorization (SNTF) It can serve as a basic building block of SNTF algorithms for a higher order tensor [1]. In [8], Shi et al proposed two inexact BCD methods based on block successive upper-bounding minimization (BSUM), named scalar BSUM (sBSUM) and vector-wise BSUM (vBSUM). Both of them are guaranteed to converge to stationary solutions. We propose an accelerated variant of the basic multiplicative update (MU) algorithm of He et al [6]. We apply AMU-SNMF to some real-world clustering problems and use it for SNTF following the framework of the averaging approach [1].

Multiplicative Update Algorithm for SNMF

Nesterov’s Accelerated Gradient

Accelerated MU-SNMF Algorithm

Experiments and Results

Synthetic Data

Document Clustering

Object Clustering

Conclusions