CosineTR: A dual-branch transformer-based network for semantic line detection

Abstract

Semantic line is a straight line based representation designed to well capture the spatial layout or structural shape of the scene in an image that is valuable as a high-level visual property. In this paper, we propose an efficient end-to-end trainable semantic line detection model named Complementary semantic line TRansformer (CosineTR), which is designed according to an old proverb “two heads are better than one”. CosineTR adopts a dual-branch framework to detect semantic lines with a coarse to fine strategy. These two branches are built based on well-designed attention modules to capture multi-scale line semantic features locally and globally, and are equipped with heatmap prediction head and parameter regression head respectively to perform semantic line detection from two different perspectives. In addition, we introduce bilateral region attention and Gaussian prior cross-attention modules to reinforce semantic contexts extracted by the two branches, and couple them to form complementary feature representations by leveraging a feature interaction method. Extensive experiments demonstrate that our approach is effective and achieves competitive semantic line detection performance on multiple datasets.