CycleGAN-Based SAR-Optical Image Fusion for Target Recognition

Abstract

The efficiency and accuracy of target recognition in synthetic aperture radar (SAR) imagery have seen significant progress lately, stemming from the encouraging advancements of automatic target recognition (ATR) technology based on deep learning. However, the development of a deep learning-based SAR ATR algorithm still faces two critical challenges: the difficulty of feature extraction caused by the unique nature of SAR imagery and the scarcity of datasets caused by the high acquisition cost. Due to its desirable image nature and extremely low acquisition cost, the simulated optical target imagery obtained through computer simulation is considered a valuable complement to SAR imagery. In this study, a CycleGAN-based SAR and simulated optical image fusion network (SOIF-CycleGAN) is designed and demonstrated to mitigate the adverse effects of both challenges simultaneously through SAR-optical image bidirectional translation. SAR-to-optical (S2O) image translation produces artificial optical images that are high-quality and rich in details, which are used as supplementary information for SAR images to assist ATR. Conversely, optical-to-SAR (O2S) image translation generates pattern-rich artificial SAR images and provides additional training data for SAR ATR algorithms. Meanwhile, a new dataset of SAR-optical image pairs containing eight different types of aircraft has been created for training and testing SOIF-CycleGAN. By combining image-quality assessment (IQA) methods and human vision, the evaluation verified that the proposed network possesses exceptional bidirectional translation capability. Finally, the results of the S2O and O2S image translations are simultaneously integrated into a SAR ATR network, resulting in an overall accuracy improvement of 6.33%. This demonstrates the effectiveness of SAR-optical image fusion in enhancing the performance of SAR ATR.