Road marking extraction in UAV imagery using attentive capsule feature pyramid network

Abstract

Accurately and precisely delineating road-markings from very high spatial resolution unmanned aerial vehicle (UAV) images face many challenges, such as complex scenarios, diverse road marking sizes and shapes, and absent and occluded road markings. To address these issues, we formulate an attentive capsule feature pyramid network (ACapsFPN) by integrating capsule representations with attention mechanisms into the feature pyramid network (FPN), aiming at improving road marking extraction accuracy. Different from the current convolutional neural network (CNN) models based on scalar neuron representations, capsule networks characterize entity features by leveraging vectorial capsule neurons, whose lengths and instantiation parameters contribute to the identification of features and their variants. By constructing a capsule FPN, the ACapsFPN is capable of extracting and integrating multi-level and multi-scale capsule features to provide high-quality and semantically-strong feature abstractions. By formulating a multi-scale context feature descriptor and the ternary feature attention modules, the ACapsFPN can emphasize informative features to generate a class-specific feature representation. Quantitative and qualitative evaluations show the ACapsFPN provides a valuable means for extracting road markings in UAV images under different kinds of complex conditions. In addition, comparative analyses with existing alternatives also demonstrate the superiority and robustness of the ACapsFPN in UAV road marking extraction.