LLISP: Low-Light Image Signal Processing Net via Two-Stage Network

Hongjin Zhu,Weiqiang Chen,Ronggang Wang,Xuesong Gao,Yang Zhao,Rongjie Wang

doi:10.1109/access.2021.3053607

Abstract

Images taken in extremely low light suffer from various problems such as heavy noise, blur, and color distortion. Assuming the low-light images contain a good representation of the scene content, current enhancement methods focus on finding a suitable illumination adjustment but often fail to deal with heavy noise and color distortion. Recently, some works try to suppress noise and reconstruct low-light images from raw data. But these works apply a network instead of an image signal processing pipeline (ISP) to map the raw data to enhanced results which leads to heavy learning burden for the network and get unsatisfactory results. In order to remove heavy noise, correct color bias and enhance details more effectively, we propose a two-stage Low Light Image Signal Processing Network named LLISP. The design of our network is inspired by the traditional ISP: processing the images in multiple stages according to the attributes of different tasks. In the first stage, a simple denoising module is introduced to reduce heavy noise. In the second stage, we propose a two-branch network to reconstruct the low-light images and enhance texture details. One branch aims at correcting color distortion and restoring image content, while another branch focuses on recovering realistic texture. Experimental results demonstrate that the proposed method can reconstruct high-quality images from low-light raw data and replace the traditional ISP.

Highlights

Typically, the raw sensor data we captured will be processed by an in-camera image signal processing pipeline (ISP) to generate JPEG-format images
1) DENOISING MODULE In this part, we show the importance of the Denoising Module (DNM) and compare the impact of different architectures and loss functions for this module
2) TEXTURE ENHANCING BRANCH In this part, we show the indispensability of the Texture Enhancing Branch (TEB) and compare different types of inputs for this branch

Summary

INTRODUCTION

The raw sensor data we captured will be processed by an in-camera image signal processing pipeline (ISP) to generate JPEG-format images. H. Zhu et al.: LLISP: Low-Light Image Signal Processing Net via Two-Stage Network enhanced images for the following reasons: First, most lowlight enhancement methods cannot handle images taken in extremely dark conditions that contain severe noise and color degradation. Zhu et al.: LLISP: Low-Light Image Signal Processing Net via Two-Stage Network enhanced images for the following reasons: First, most lowlight enhancement methods cannot handle images taken in extremely dark conditions that contain severe noise and color degradation Under these conditions, JPEG-format images cannot provide enough information due to the information loss during the traditional ISP. Under extremely low-light conditions, they may either enhance both the noise and scene details, or fail to recover the low visibility of low-light images Compared with these methods, our LLISP brightens up the image while preserving the inherent color and details via a proper image processing pipeline and efficient utilization of the raw data. We adopt a simple but effective pre-denoising module so that we can avoid the disruption of severe noise on the subsequent enhancement

IMAGE SIGNAL PROCESSING PIPELINE

DATA PREPARING

STAGE I

STAGE II

Findings

CONCLUSION