Abstract
Lossy video stream compression is performed to reduce the bandwidth and storage requirements. Moreover also image compression is a need that arises in many circumstances.It is often the case that older archive are stored at low resolution and with a compression rate suitable for the technology available at the time the video was created. Unfortunately, lossy compression algorithms cause artifact. Such artifacts, usually damage higher frequency details also adding noise or novel image patterns. There are several issues with this phenomenon. Low-quality images can be less pleasant to persons. Object detectors algorithms may have their performance reduced. As a result, given a perturbed version of it, we aim at removing such artifacts to recover the original image. To obtain that, one should reverse the compression process through a complicated non-linear image transformation. We propose a deep neural network able to improve image quality. We show that this model can be optimized either traditionally, directly optimizing an image similarity loss (SSIM), or using a generative adversarial approach (GAN). Our restored images have more photorealistic details with respect to traditional image enhancement networks. Our training procedure based on sub-patches is novel. Moreover, we propose novel testing protocol to evaluate restored images quantitatively. Differently from previously proposed approaches we are able to remove artifacts generated at any quality by inferring the image quality directly from data. Human evaluation and quantitative experiments in object detection show that our GAN generates images with finer consistent details and these details make a difference both for machines and humans.
Highlights
A huge number of videos are produced, streamed and shared on the web, and many more are used within private systems, such as mobile phones, cameras and surveillance systems
Many image and video compression algorithms (e.g. JPEG, JPEG2000, VP9, H.264/AVC, H.265/HEVC) use color spaces that separate luminance from chrominance information, like YCrCb. This allows to better de-correlate color components leading to a more e cient compression; it permits a first step of lossy compression through chrominance sub-sampling, based on the fact that the human visual system has reduced sensitivity to its variations
Recent works have shown that this category of tasks can be conveniently solved using generative approaches, i.e. learning a fully convolutional neural network (FCN) [19] that given a certain input image is able to generate, as an output, an improved version of it
Summary
A huge number of videos are produced, streamed and shared on the web, and many more are used within private systems, such as mobile phones, cameras and surveillance systems. To store e ciently and transmit these videos compression is necessary This allows to reduce bandwidth and storage. Compressions is tipically lossy, given the need to deal with large quantities of data, such as HD and 4K resolutions which are more and more common. These algorithms application results in a more or less strong loss of content quality, to achieve a better compression ratio. MPEG-based algorithms such as H.264 and H.265/AVC or AV1, are the most common and recent algorithm used nowadays In such case artifacts are due to subsampling of chroma (i.e. dropping of color information) and the DCT coe cient quantization; Due to how the original frame is partitioned blocking artifacts arise.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
