Cubic-cross convolutional attention and count prior embedding for smoke segmentation

Abstract

It is very challenging to accurately segment smoke images because smoke has some adverse properties, such as semi-transparency and blurry boundary. Aiming at solving these problems, we first fuse convolutional results along different axes to equivalently produce a cubic-cross convolutional kernel, which enlarges receptive fields at affordable computational costs for capturing long-range dependency of smoke pixels, and then we propose a Cubic-cross Convolutional Attention (CCA). To embed global category information, we propose a count prior structure to model and supervise the count of smoke pixels. To ensure the network can correctly extract a count prior map, we impose a regression loss on the count prior map and corresponding ideal count map directly calculated from its ground truth. Then we multiply the reshaped input by the count prior map to produce a Count Prior Attention (CPA) map, which is upsampled to generate the final output. A cross entropy loss is used to supervise the final segmentation. Finally, we use ResNet50 for feature encoding, and stack CCA and CPA together to propose a Cubic-cross convolutional attention and Count prior Embedding Network (CCENet) for smoke segmentation. Experiments on both synthetic and real smoke datasets show that our method outperforms existing state-of-the-art methods.