Structure‐aware multiple salient region detection and localization for autonomous robotic manipulation

Abstract

This paper proposes a multiple salient region detection and localization approach for unstructured industrial robot work environments with arbitrarily located and orientated objects. Different from the existing, the authors' novel technique to detect multiple salient regions performs locally adaptive center-surround operations on proto-object partitions obtained through color consistency and spatial proximity analysis. The multi-scale center-surround operations are done by masks that are local structure-aware yielding regions with precise and accurate boundaries as required for robotic manipulation. First, experiments to evaluate the multiple salient region detection performance are carried out using four standard databases having images with multiple salient objects. Quantitative result analysis using F-measure, shuffled F-measure, shuffled AUC and MAE, and subjective result inspection suggests that the proposed approach is in general better at collectively detecting multiple salient regions than the state-of-the-art, including those based on deep learning. Then, real-life experiments involving robotic manipulation are carried out to demonstrate the utility of the multiple salient region detection method. For robotic manipulation, object localization is improved after salient region detection by employing a fast shadow detection algorithm proposed based on hue analysis, and recognition through existing matching techniques is applied only at the localized salient regions. The benefit of the novel multiple salient region detection approach in the robotic manipulation system is shown using localization and pose estimation accuracy, rates of detection and recognition, positional and angular errors, and processing speed.