Optimizing UHD Production with Super Resolution Techniques

Abstract

Super resolution (SR) is a technique that is getting increased attention. It was first deployed in high-end TVs to upconvert content, and now content providers are looking at how it can be used on the production side to guarantee the same high quality on all devices. Different techniques can be used to apply SR to standard definition (SD) and high definition (HD) content for transmitting ultra high definition (UHD) resolution. During a recent trial with France TV at the French Open, SD content with a 16:9 aspect ratio was processed offline to create UHD content, using a hybrid pipeline of deep restoration and SR methods. A pretrained deep neural network was used to produce an intermediate upscaled and deblurred HD version, which was then passed on to a faster, less memory-intensive, and machine-learning (ML)-based SR filter to produce the final UHD 4K resolution output. The result is a significantly better subjective viewing experience in terms of details and sharpness compared with classical filters used in live systems. Adequately addressing analog tape recordings, which have been digitized, is a significantly bigger technical challenge, which will need to be resolved in the future. While a one-model-fits-all approach for video enhancement sounds attractive due to its simplicity, the best visual quality is produced by applying purpose-built models trained for addressing the general types of degradation found in the source video.