A Lightweight Block With Information Flow Enhancement for Convolutional Neural Networks

Abstract

Convolutional neural networks (CNNs) have demonstrated excellent capability in various visual recognition tasks but impose an excessive computational burden. The latter problem is commonly solved by utilizing lightweight sparse networks. However, such networks have a limited receptive field in a few layers, and the majority of these networks face a severe information barrage due to their sparse structures. Spurred by these deficiencies, this work proposes a Squeeze Convolution block with Information Flow Enhancement (SCIFE), comprising a Divide-and-Squeeze Convolution and an Information Flow Enhancement scheme. The former module constructs a multi-layer structure through multiple squeeze operations to increase the receptive field and reduce computation. The latter replaces the affine transformation with the point convolution and dynamically adjusts the activation function’s threshold, enhancing information flow in both channels and layers. Moreover, we reveal that the original affine transformation may harm the network’s generalization capability. To overcome this issue, we utilize a point convolution with a zero initial mean. SCIFE can serve as a plug-and-play replacement for vanilla convolution blocks in mainstream CNNs, while extensive experimental results demonstrate that CNNs equipped with SCIFE compress benchmark structures without sacrificing performance, outperforming their competitors.