Multichannel concat-fusional convolutional neural networks

Abstract

Because of the advantages of deep learning and information fusion technology, it has drawn much attention for researchers to combine them to achieve target recognition, positioning, and tracking. However, when the existing neural network process multichannel images (e.g., color images), multiple channels as a whole input into neural networks, which makes it hard for networks to fully learn information in R, G, and B channels of images. Therefore, it is not conducive to the final learning effect of the networks. To solve the problem, using different combinations of R, G, and B channels of color images for feature-level fusion, this paper proposes three fusion types as “R/G/B”, “R+G/G+B/B+R”, and “R+G+B/R+G+B/R+G+B” multichannel concat-fusional convolutional neural networks. Experimental results show that multichannel concat-fusional convolutional neural networks with fusional types of “R+G/G+B/B+R” and “R+G+B/R+G+B/R+G+B” achieve better performance than the corresponding non-fusional convolutional neural networks on different datasets. It shows that networks with fusion types of “R+G/G+B/B+R” and “R+G+B/R+G+B/R+G+B” can learn more fully information of R, G, and B channels of color images and improve the learning performance of networks.