A Novel Weights of Weights Approach for Efficient Transfer Learning in Artificial Neural Networks

Abstract

Deep learning is often reputed to be slow at training due to large number of layers. There are several approaches including used for improving training mechanism. Transfer learning sometimes referred as transfer of knowledge is one such approach. However, Transfer learning is often moving weights or layers as a whole including weights that might not be significant.This research tries to implement a novel Weights of Weights approach where significant weights are extracted from a training network. Only the significant weights are transferred to new untrained network instead of transferring all the layers or weights. Experiments are carried out using a 7-layered neural networks on multivariate wine dataset to evaluate the efficiency of the model.The experiment results show that the transferable model has provided better accuracy of over 40%-60% (based on layers) without retraining compared to traditional approach of copying all the layers with same and less number of nodes.