Cloud Image Retrieval for Sea Fog Recognition (CIR-SFR) Using Double Branch Residual Neural Network

Abstract

Sea fog is a common weather phenomenon at sea, which reduces visibility and causes tremendous hazards to marine transportation, marine fishing and other maritime operations. Traditional sea fog monitoring methods have enormous difficulties in characterizing the diversity of sea fog and distinguishing sea fog from low-level clouds. Thus, we propose a cloud image retrieval method for sea fog recognition (CIR-SFR) in a deep learning framework by combining the advantages of metric learning. CIR-SFR includes the feature extraction module and the retrieval-based sea fog recognition module. The feature extraction module adopts the Double Branch Residual Neural Network (DBRNN) to comprehensively extract the global and local features of cloud images. By introducing local branches and using activation masks, DBRNN can focus on regions of interest in cloud images. Moreover, cloud image features are projected into the semantic space by introducing Multi-Similarity Loss, which effectively improves the discrimination ability of sea fog and low-level clouds. For the retrieval-based sea fog recognition module, similar cloud images are retrieved from the cloud image dataset according to the distance in the feature space, and accurate sea fog recognition results are obtained by counting the percentage of various cloud image types in the retrieval results. To evaluate the sea fog recognition system, we establish a dataset of 2544 cloud images including clear sky, low-level cloud, medium high cloud and sea fog. Experimental results show that the proposed method outperforms the traditional methods in sea fog recognition, which provides a new way for sea fog recognition.