MCCGAN: An All-In-One Image Restoration Under Adverse Conditions Using Multidomain Contextual Conditional Gan

Abstract

Clear images are crucial for the optimal performance of various high-level vision-based tasks. However, some inevitable causes, such as bad weather and underwater conditions degrade scene visibility. The tiny particles present in the air absorb and scatter light, causing severe attenuation that results in unclear, low-brightness, and poor-contrast images. Several techniques have been introduced to restore the degradation. However, no model exists to date that can restore multiple degradations using a single model. Therefore, to improve the scene visibility, a unified model called a Multidomain Contextual Conditional Generative Adversarial Network (MCCGAN) is designed, which uses the same parameters across the domains to restore multiple degradations such as fog, haze, rain streaks, snowflakes, smoke, shadows, underwater, and muddy underwater. The proposed model has a novel addition of multiple [Formula: see text] convolutional context encoding bottleneck layers between a simple lightweight eight-block encoder and decoder with skip connections which learns the context of each input domain thoroughly, thus generating better-restored images. The MCCGAN is qualitatively and quantitatively compared to various state-of-the-art image-to-image translation models and tested on a few real unseen image domains such as smog, dust, and lightning, and the obtained results successfully improved scene visibility, proving the generalizability of MCCGAN. Moreover, the MS-COCO 2017 validation dataset is used for comparing the performance of object detection, instance segmentation, and image captioning on (1) weather-degraded images, (2) restored images by MCCGAN, and (3) ground truth images, and the results demonstrated the success of our model. An ablation study is also carried out to check the significance of the discriminator, skip connections, and bottleneck layers in MCCGAN, and the analysis suggests that MCCGAN performs better by adding a discriminator, skip connections, and four bottleneck layers in the generator architecture.