OIPNet: Multimodal Network with Orthogonal Information Processing for Semantic Segmentation in Indoor Scenes

Abstract

Semantic segmentation in indoor environments is a crucial task for artificial intelligence-driven visual robotics, enabling pixel-level classification results to facilitate robot path planning. Inspired by the success of multimodal models, we propose an end-to-end multimodal semantic segmentation model for image segmentation tasks in indoor scenes, which we call OIPNet. We design the OIP module to enhance the network’s ability to extract global information and enable information interaction in different directions. We have validated on NYUv2 and Sun RGB-D datasets, and the experiments show the generality and effectiveness of the proposed model. Our code is available at https://github.com/Mantee0810/OIP.