Abstract

We investigated two commonly used momentum algorithms, Classical momentum (CM) and Nesterov momentum (NM). We found that, when used in Restricted Boltzmann machine (RBM), they have two main problems: The first one is their performances are not obvious and not as good as expected. The second one is they may lose accelerating ability in the later stage of training process. Aiming at these two problems, we proposed the Weight momentum algorithm and evaluated our approach on four datasets. It has been demonstrated that our methods can achieve better performance under both reconstruction error and classification rate criterions.

Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call