Multiple Sub-images Research Articles

Superimposed video disambiguation for increased field of view

Many infrared optical systems in wide-ranging applications such as surveillance and security frequently require large fields of view (FOVs). Often this necessitates a focal plane array (FPA) with a large number of pixels, which, in general, is very expensive. In a previous paper, we proposed a method for increasing the FOV without increasing the pixel resolution of the FPA by superimposing multiple sub-images within a static scene and disambiguating the observed data to reconstruct the original scene. This technique, in effect, allows each sub-image of the scene to share a single FPA, thereby increasing the FOV without compromising resolution. In this paper, we demonstrate the increase of FOVs in a realistic setting by physically generating a superimposed video from a single scene using an optical system employing a beamsplitter and a movable mirror. Without prior knowledge of the contents of the scene, we are able to disambiguate the two sub-images, successfully capturing both large-scale features and fine details in each sub-image. We improve upon our previous reconstruction approach by allowing each sub-image to have slowly changing components, carefully exploiting correlations between sequential video frames to achieve small mean errors and to reduce run times. We show the effectiveness of this improved approach by reconstructing the constituent images of a surveillance camera video.

Imaging interferometric microscopy

Imaging interferometric microscopy (IIM) is a synthetic aperture imaging approach providing resolution to the transmission medium (refractive index n) linear systems limit extending to greater, similarlambda/4n using only low-numerical-aperture (low-NA) optics. IIM uses off-axis illumination to access high spatial frequencies along with interferometric reintroduction of a zero-order reference beam on the low-NA side of the optical system. For a thin object normal to the optical axis, the frequency space limit is [(1+NA)n/lambda], while tilting the object plane allows collection of diffraction information up to the material transmission bandpass-limited spatial frequency of 2n/lambda. Tilting transforms the spatial frequencies; the algorithm to reset to the correct image frequencies is described. IIM involves combining multiple subimages; the image reconstruction procedures are discussed. A mean-square-error metric is introduced. For binary objects, sigmoidal filtering of the image provides significant resolution improvement.

TOMBO sensor with scene-independent superresolution processing

One approach to flat sensor design is to use a lenslet array to form multiple subimages of a scene and then combine the subimages to recover a fully sampled image by using a superresolution algorithm. Previously, superresolution image assembly has been based on information derived from the observed scene. For lenslet arrays, we propose a new scene-independent approach based only on known imager properties in which relative subimage shifts are accurately estimated with a calibration procedure using point source imaging. Thus, the relative resolution enhancement provided by the scene-independent superresolution algorithm is impervious to changes in subimage content, contrast, sharpness, and noise.

Analysis and application of multiframe superresolution processing for conventional imaging systems and lenslet arrays

Low-cost compact sensors for ultrasmall systems are a pressing need in many new applications. One potential solution is a shallow aspect ratio system using a lenslet array to form multiple undersampled subimages of a scene on a single focal plane array, where processing techniques then produce an upsampled restored image. We have investigated the optimization and theoretical limits of the performance of such arrays. We have built a hardware simulator and developed algorithms to process imagery similar to that of a full lenslet imaging sensor, which allowed us to quickly test optical components, algorithms, and complete system designs for future lenslet imaging systems.

Single sub-image matching based low complexity motion estimation for digital image stabilization using constrained one-bit transform

A fast motion estimation approach based on single sub-image bit-plane matching for low complexity image stabilization is presented in this paper. The recently proposed one-bit transform (IBT) based motion estimation method for digital image stabilization which employs multiple sub-images has lower computational complexity, however the motion estimation accuracy is relatively low. Single sub-image matching proposed in this paper enhances motion estimation performance without causing additional computational load. Furthermore, the multiplication operation required for IBT is avoided using a new kernel. The recently proposed constrained IBT (C-IBT) that restricts pixels with values adjacent to the transform threshold is employed to increase performance at the cost of a slightly higher computational load. Experimental results show that the proposed approach outperforms other sub-image based approaches

Simultaneous out-of-focus blur estimation and restoration for digital auto-focusing system

We propose a simultaneous out-of-focus blur estimation and restoration technique for a fully digital auto-focusing system. The proposed auto-focusing system can make in-focused images by using only digital image processing techniques, and it requires neither infrared light or ultrasound nor a focusing lens assembly operated by a motor. As a result, it can provide savings in the volume and cost of the imaging system, which contains the proposed auto-focusing system. The proposed system first divides the input image into multiple sub-images or blocks, determines the set of blocks which contain linear edges, estimates the step response across the edge for each block in the selected set, computes the one-dimensional step response by averaging the step responses, estimates the two-dimensional point spread function (PSF) from the averaged step response, and finally provides in-focused image by an image restoration filter based on the estimated PSF. There is a limit on the amount of out-of-focus, which can be recovered by the proposed auto-focusing system. The proposed system operates under the assumption that an input image contains at least one piece-wise linear boundary between an object and the background. In spite of the above mentioned limitations, the proposed system can make an acceptable quality of focused image by using only digital signal processing.