Recaptured Screen Image Demoiréing in Raw Domain

Abstract

Capturing screen content by smart-phone cameras has become a daily routine to record or share instant information from display screens for convenience. However, these recaptured screen images are often degraded by moir&#x00E9; patterns and usually present color cast against the original screen source. We observe that performing demoir&#x00E9;ing in raw domain before feeding into the image signal processor (ISP) is more effective than demoir&#x00E9;ing in the sRGB domain as done in recent demoir&#x00E9;ing works. In this paper, we investigate the demoir&#x00E9;ing of raw images through a class-specific learning approach. To this end, we build the first well-aligned raw moir&#x00E9; image dataset by pixel-wise alignment between the recaptured images and source ones. Noting that document images occupy a large portion of screen contents and have different properties from generic images, we propose a class-specific learning strategy for textual images and natural color images. In addition, to deal with moir&#x00E9; patterns with various scales, a multi-scale encoder with multi-level feature fusion is proposed. The shared encoder enables us to extract rich representations for the two kinds of contents and the class-specific decoders benefit the specific content reconstruction by focusing on targeted representations. Experiment results demonstrate that our method achieves state-of-the-art demoir&#x00E9;ing performance. We have released the code and dataset in <uri>https://github.com/tju-chengyijia/RDNet</uri>.