Message-Passing-Driven Triplet Representation for Geo-Object Relational Inference in HRSI

Abstract

A high-resolution remote sensing image (HRSI) scene typically contains multiple geo-objects, and geospatial relations among these geo-objects are obvious. As the important information conveyed by HRSI, the intelligent expression of geospatial relation is helpful in understanding HRSI scenes. Previous HRSI semantic understanding was mainly based on image captions that only generate one sentence to describe image content, thereby resulting in insufficient understanding of the scene. Thus, the present letter proposes an approach to represent geospatial relations in an HRSI scene with structured form of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\langle $ </tex-math></inline-formula> subject, geospatial relation, object <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\rangle $ </tex-math></inline-formula> . A geospatial relation triplet representation data set that contains visual and semantic information, such as category, location, and geospatial relations of the geo-objects, is constructed first. An “object-relation” message-passing mechanism is adopted to enhance the information exchange between the geo-objects and geospatial relations to predict triplets accurately. The experimental results show that the proposed method can effectively predict the geospatial relation in a HRSI scene.