PSANet: Automatic colourisation using position‐spatial attention for natural images

Abstract

AbstractDue to the richness of natural image semantics, natural image colourisation is a challenging problem. Existing methods often suffer from semantic confusion due to insufficient semantic understanding, resulting in unreasonable colour assignments, especially at the edges of objects. This phenomenon is referred to as colour bleeding. The authors have found that using the self‐attention mechanism benefits the model's understanding and recognition of object semantics. However, this leads to another problem in colourisation, namely dull colour. With this in mind, a Position‐Spatial Attention Network(PSANet) is proposed to address the colour bleeding and the dull colour. Firstly, a novel new attention module called position‐spatial attention module (PSAM) is introduced. Through the proposed PSAM module, the model enhances the semantic understanding of images while solving the dull colour problem caused by self‐attention. Then, in order to further prevent colour bleeding on object boundaries, a gradient‐aware loss is proposed. Lastly, the colour bleeding phenomenon is further improved by the combined effect of gradient‐aware loss and edge‐aware loss. Experimental results show that this method can reduce colour bleeding largely while maintaining good perceptual quality.