An Asynchronously Alternative Stochastic Gradient Descent Algorithm for Efficiently Parallel Latent Feature Analysis on Shared-Memory

Abstract

A latent feature analysis (LFA) model is highly efficient in performing representation learning to high-dimensional incomplete (HDI) data like user-item interactions data from a recommender system. Stochastic gradient descent (SGD) is frequently adopted as the learning algorithm by an LFA model owing to its low computational complexity. However, a standard SGD algorithm is in nature a serial algorithm, which affects the resultant LFA model's scalability on mass HDI data. On the other hand, existing parallel SGD algorithms commonly suffer from low speedup when building an LFA model due to their frequent synchronizations during the training process. Motivated by this discovery, this paper proposes an Asynchronously Alternative Stochastic Gradient Descent (A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> SGD) algorithm to achieve an efficiently parallelized LFA model on shared-memory with two fold-ideas: a) adopting the principle of an alternative stochastic gradient descent algorithm to decouple the LFA process, thereby achieving two parallelizable subtasks with minimal learning information loss; b) designing a novel parallelization scheme by eliminating synchronizations from both the subtask and the thread perspectives, i.e., both sub-tasks are taken simultaneously, and their affiliated threads also executes without synchronizations. Rigorously theoretical convergence proof illustrates that the newly-proposed parallelization scheme guarantees the convergence of a resultant LFA model. Detailed experimental results on four real HDI datasets indicate that an A <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> SGD-based LFA model outperforms several state-of-the-art parallel SGD-based LFA models in terms of both missing data estimation accuracy and parallelization speedup.