How pre-trained large model can help, when SAM meets image restoration

Abstract

In the process of image capture, degradation is inevitable due to noise, motion, down-sampling and so on. Therefore, image restoration is essential to improve the quality of images to enhance their visual effects and benefit downstream tasks. Adding prior knowledge can help the model better understand the image content and restoration requirements, thus improving the quality and efficiency. Semantic-level prior information can be generated by pre-trained large-scale models, such as segment anything models (SAM), and applied to a large number of downstream tasks. SAM has demonstrated powerful robustness and stability in restoration tasks, such as denoising, super-resolution, low-light enhancement, etc. Meanwhile, as an interactive component, SAM brings more control for users during the repair process. In this paper, we focus on the importance of SAM as prior information and systematically summarize a series of recent works combining SAM prior and low-level image restoration from three perspectives. In addition, we have summarized some potential problems and future directions of SAM.