Registration for Super-Resolution: Theory, Algorithms, and Applications in Image and Mobile Video Enhancement

Abstract

The relation between sensor resolution and the optics of a digital camera is determined by the Nyquist sampling theorem: the sampling frequency should be larger than twice the maximum frequency of the image content coming out of the optical system. If a lower resolution is used, the output is aliased. Aliasing in digital images is often considered as a nuisance and (both optical and digital) filters are designed to avoid aliasing in digital cameras. However, aliasing also contains extra high-frequency information with additional details about the scene. Super-resolution algorithms extract the information present in the aliasing to reconstruct a higher resolution image. Super-resolution algorithms typically combine multiple aliased images with small relative motion, and create a single high resolution image. The input can be a set of pictures taken with a digital camera from approximately the same point of view. An application could be to use a low resolution camera (with a good optical system), capture a set of images while holding the camera manually in approximately the same position, and use the small camera shake to reconstruct a high resolution image. This would allow to take multiple images with a cheap camera, and combine them to a higher resolution image as if it had been taken with a more expensive camera. Other applications can be found for example in situations where a camera sensor can not be easily replaced, such as in satellites. It is (almost) impossible to install a new camera sensor, while a modification of the software allows to take a series of images of approximately the same subject.