Thermodynamics energy for both supervised and unsupervised learning neural nets at a constant temperature.

Abstract

Unified Lyaponov function is given for the first time to prove the learning methodologies convergence of artificial neural network (ANN), both supervised and unsupervised, from the viewpoint of the minimization of the Helmholtz free energy at the constant temperature. Early in 1982, Hopfield has proven the supervised learning by the energy minimization principle. Recently in 1996, Bell & Sejnowski has algorithmically demonstrated Independent Component Analyses (ICA) generalizing the Principal Component Analyses (PCA) that the continuing reduction of early vision redundancy happens towards the "sparse edge maps" by maximization of the ANN output entropy. We explore the combination of both as Lyaponov function of which the proven convergence gives both learning methodologies. The unification is possible because of the thermodynamics Helmholtz free energy at a constant temperature. The blind de-mixing condition for more than two objects using two sensor measurement. We design two smart cameras with short term working memory to do better image de-mixing of more than two objects. We consider channel communication application that we can efficiently mix four images using matrices [AO] and [Al] to send through two channels.