Abstract
The family of competitive activation models has recently attracted some interest. These models are a variation upon competitive neural networks where a local feedback process drives the competitive interaction rather than some form of lateral inhibition. However, this process can be viewed in terms of a generative model that reduces the generalized Kullback-Leibler divergence between the input distribution and the reconstruction distribution. From this insight we construct an online training method based on a stochastic gradient descent that reduces this measure while retaining the constraint of non-negativity inherent in the competitive neural network. We compare our results to non-negative matrix factorization (NMF), and show how the method results in a highly orthogonal, localized and parts-based representation of the data set, even when NMF does not, without the use of any explicit orthogonality or localization regularizers. Additionally, we show how the method leads to a basis better suited for discriminative tasks.
Published Version
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
