Sub-pixel Shifts Research Articles

Digital superresolution and the generalized sampling theorem

The technique of reconstructing a higher-resolution (HR) image of size MLxML by digitally processing LxL subpixel-shifted lower-resolution (LR) copies of it, each of size MxM, has now become well established. This particular digital superresolution problem is analyzed from the standpoint of the generalized sampling theorem. It is shown both theoretically and by computer simulation that the choice of regularly spaced subpixel shifts for the LR images tends to maximize the robustness and minimize the error of reconstruction of the HR image. In practice, since subpixel-level control of LR image shifts may be nearly impossible to achieve, however, a more likely scenario, which is also discussed, is one involving random subpixel shifts. It is shown that without reasonably tight bounds on the range of random shifts, the reconstruction is likely to fail in the presence of even small amounts of noise unless either reliable prior information or additional data are available.

Optical projection and image processing approach for mine wall monitoring

Wall movement in an underground mine must be continually monitored to provide real-time assessment of dynamic stress changes around mining excavation and to warn against rock failure. An in situ approach using optical spot projection and image processing can measure submillimeter movements. The system can be easily automated and poses no obstruction to mine traffic. The position of an off-normal laser spot will shift as the target wall moves. The centroid of this projected spot can be measured through image processing of a digital image. Multiple laser spots give the system simultaneous information over a larger sampled area. Software processing can measure subpixel shifts of the spot centroid. The resolution depends on the pixel count, the magnification of the camera lens (and hence the field of view), the optical-beam angle, and the spot characteristics. A laboratory table-top experiment demonstrated the system for distances up to 3 m and for a resolution of 0.1 mm. The results were linear in target movement, with standard errors between 0.83% and 5.38%. An automated system is described for a mine environment.

Analysis of Artifacts in Subpixel Remote Sensing Image Registration

Subpixel accuracy image registration is needed for applications such as digital elevation model extraction, change detection, pan-sharpening, and data fusion. In order to achieve this accuracy, the deformation between the two images to be registered is usually modeled by a displacement vector field which can be estimated by measuring rigid local shifts for each pixel in the image. In order to measure subpixel shifts, one uses image resampling. Sampling theory says that, if a continuous signal has been sampled according to the Nyquist criterion, a perfect continuous reconstruction can be obtained from the sampled version. Therefore, a shifted version of a sampled signal can be obtained by interpolation and resampling with a shifted origin. Since only a sampled version of the shifted signal is needed, the reconstruction needs only to be performed for the new positions of the samples, so the whole procedure comes to computing the value of the signal for the new sample positions. In the case of image registration, the similarity between the reference image and the shifted versions of the image to be registered is measured, assuming that the maximum of similarity determines the most likely shift. The image interpolation step is thus performed a high number of times during the similarity optimization procedure. In order to reduce the computation cost, approximate interpolations are performed. Approximate interpolators will introduce errors in the resampled image which may induce errors in the similarity measure and therefore produce errors in the estimated shifts. In this paper, it is shown that the interpolation has a smoothing effect which depends of the applied shift. This means that, in the case of noisy images, the interpolation has a denoising effect, and therefore, it increases the quality of the similarity estimation. Since this blurring is not the same for every shift, the similarity may be low for a null shift (no blurring) and higher for shifts close to half a pixel (strong blurring). This paper presents an analysis of the behavior of the different interpolators and their effects on the similarity measures. This analysis will be done for the two similarity measures: the correlation coefficient and the mutual information. Finally, a strategy to attenuate the interpolation artifacts is proposed

Superresolution Digital Image Enhancement by Subpixel Image Translation With a Scanning Micromirror

This paper reports on a superresolution digital image enhancement technique using subpixel image translation with a scanning micromirror. We demonstrate that the spatial resolution of digital photomicrography, employing typical black-and-white pixelated image sensors, can be improved by acquiring multiple images with predetermined subpixel shifts and combining them with a superresolution image reconstruction algorithm. A two-axis gimbaled micromirror fabricated with a bulk-micromachining process is used to control the translational image displacements with subpixel accuracy

An Improved Observation Model for Super-Resolution Under Affine Motion

Super-resolution (SR) techniques make use of subpixel shifts between frames in an image sequence to yield higher resolution images. We propose an original observation model devoted to the case of nonisometric inter-frame motion as required, for instance, in the context of airborne imaging sensors. First, we describe how the main observation models used in the SR literature deal with motion, and we explain why they are not suited for nonisometric motion. Then, we propose an extension of the observation model by Elad and Feuer adapted to affine motion. This model is based on a decomposition of affine transforms into successive shear transforms, each one efficiently implemented by row-by-row or column-by-column one-dimensional affine transforms. We demonstrate on synthetic and real sequences that our observation model incorporated in a SR reconstruction technique leads to better results in the case of variable scale motions and it provides equivalent results in the case of isometric motions.

A New Scene-Based Non-Uniformity Correction Algorithm for Infrared Focal Plane Array

Non-uniformity exists in all IRFPA detectors. A new scene-based algorithm is developed to compensate for bias non-uniformity in focal-plane arrays. The technique is based on use of estimates of interframe sub-pixel shifts in an image sequence, in conjunction with a linear interpolation model for the motion, to exact information on the bias non-uniformity algebraically. Through a serial of experiment on real IRFPA images, we find the algorithm is effective and is more promising in comparison with the statistical approaches.

Superresolution image reconstruction from a sequence of aliased imagery

We present a superresolution image reconstruction from a sequence of aliased imagery. The subpixel shifts (displacement) among the images are unknown due to the uncontrolled natural jitter of the imager. A correlation method is utilized to estimate subpixel shifts between each low-resolution aliased image with respect to a reference image. An error-energy reduction algorithm is derived to reconstruct the high-resolution alias-free output image. The main feature of this proposed error-energy reduction algorithm is that we treat the spatial samples from low-resolution images that possess unknown and irregular (uncontrolled) subpixel shifts as a set of constraints to populate an oversampled (sampled above the desired output bandwidth) processing array. The estimated subpixel locations of these samples and their values constitute a spatial domain constraint. Furthermore, the bandwidth of the alias-free image (or the sensor imposed bandwidth) is the criterion used as a spatial frequency domain constraint on the oversampled processing array. The results of testing the proposed algorithm on the simulated low- resolution forward-looking infrared (FLIR) images, real-world FLIR images, and visible images are provided. A comparison of the proposed algorithm with a standard interpolation algorithm for processing the simulated low-resolution FLIR images is also provided.

Sub-pixel image registration with a maximum likelihood estimator

We present a new method based on a maximum likelihood (ML) estimation of the sub-pixel shift between images of a given object observed with a single instrument. We first study the case of two noisy images and give the ML approach of the registration problem. By means of simulations, we show the gain obtained with this ML solution compared to a classical registration method with an academic noise model (stationary white Gaussian), and then demonstrate the relevance of this ML estimation with a more realistic noise model. We then address the problem of a sequence of low signal frames of the same object. We develop a joint ML approach in which we simultaneously estimate the reference (i.e. the noiseless) image and the shift parameters. The registration accuracy is increased at low photon levels as the number of frames grows, reaching the sub-pixel domain at very low SNR (about 1), when considering 100 frames. When applied to experimental data (thermal IR images of a faint galaxy), both ML methods show their efficiency to recover the resolution in averaged frames and totally outperform the classical cross-correlation.

Superresolution performance for undersampled imagers

Enhancement of undersampled imager performance has been demonstrated using superresolution techniques. In these techniques, the optical flow of the scene or the relative subpixel shifts among various snapshots of the scene are calculated, and a high-resolution grid is populated with spatial data using various algorithms. Performance enhancement has been demonstrated for the case of a static image with the undersampled imager compared with a static image that has been acquired through a frame series in a dynamic scene. In this research, the performance is compared for four cases: static image with undersampled imager, static image with superresolution frame sequence, dynamic image with undersampled imager, and dynamic image with superresolution frame sequence.

Extended object wavefront sensing based on the correlation spectrum phase

In this paper we investigate the performance of a Fourier based algorithm for fast subpixel shift determination of two mutually shifted images subjected to noise. The algorithm will be used for Shack-Hartmann based adaptive optics correction of images of an extended object subjected to dynamical atmospheric fluctuations. The performance of the algorithm is investigated both analytically and by Monte Carlo simulations. Good agreement is achieved in relation to how the precision of the shift estimate depends on image parameters such as contrast, photon counts and readout noise, as well as the dependence on sampling format, zero-padding and field of view. Compared to the conventional method for extended object wavefront sensing, a reduction of the computational cost is gained at a marginal expense of precision.

PIV optimization for the study of turbulent flow using spectral analysis

Particle image velocimetry (PIV) is a measurement technique which is well adapted to the study of the structure of turbulent flows, as it allows us to obtain quantitative information on the spatial structure of the velocity field. This contribution presents an experimental approach to characterize the measurement noise of a PIV system and the spatial response of such a method. This approach is based on a specific spectral analysis of the velocity vector field deduced from several PIV experiments. This study was done in two steps. The first step was to measure the noise level of PIV and to determine a model for the PIV transfer function from a series of displacement fields measured in a quiet liquid. This model shows the effect of the interrogation window size and introduces a spectral noise density which is constant for a given recording set-up. The second step was to compute spectra from velocity fields obtained in a turbulent boundary layer in a plane parallel to the wall. These spectra show that PIV behaves as a band pass filter. This series of experiments allows us to build a model for the prediction of the PIV spectrum knowing the real one. This model confirms that the PIV noise is white. It allows us to optimize the interrogation window size in order to obtain the best compromise between the spectral response and the spatial resolution. The rms value of the noise can be estimated from the noise density, allowing us to quantify the measurement accuracy. The improvement of sub-pixel window shift is also discussed, leading to a small decrease in the noise level. An analysis is proposed to identify the main sources of noise: particles cut by the border of the interrogation window, isolated particles, etc.

An algebraic algorithm for nonuniformity correction in focal-plane arrays.

A scene-based algorithm is developed to compensate for bias nonuniformity in focal-plane arrays. Nonuniformity can be extremely problematic, especially for mid- to far-infrared imaging systems. The technique is based on use of estimates of interframe subpixel shifts in an image sequence, in conjunction with a linear-interpolation model for the motion, to extract information on the bias nonuniformity algebraically. The performance of the proposed algorithm is analyzed by using real infrared and simulated data. One advantage of this technique is its simplicity; it requires relatively few frames to generate an effective correction matrix, thereby permitting the execution of frequent on-the-fly nonuniformity correction as drift occurs. Additionally, the performance is shown to exhibit considerable robustness with respect to lack of the common types of temporal and spatial irradiance diversity that are typically required by statistical scene-based nonuniformity correction techniques.

Effect of sub-pixel misregistration on the determination of the point spread function of a CT imaging system

The point spread function (PSF) of an imaging system provides a complete, quantitative description of its resolution. In computed tomography (CT) it can be obtained conveniently and directly from the image of a thin metal wire. However, the shape of the observed PSF is affected by arbitrary sub-pixel shifts in alignment between the wire object and the imaging raster, causing partial volume averaging. We investigated the loss of symmetry of the PSF due to this misregistration, using a thin wire at a small angle to the axis of the CT scanner. We were able to identify the optimal registration, and hence the best estimate of the PSF, from the minimum skewness of a series of images. Determination of an exact PSF is important in the imaging of structures of sub-pixel size, such as small blood vessels or trabeculae in spongy bone. The method is particularly useful for modalities with coarse rasters such as magnetic resonance imaging and nuclear medicine.

Fast registration and reconstruction of aliased low-resolution frames by use of a modified maximum-likelihood approach

During the process of microscanning a controlled vibrating mirror typically is used to produce subpixel shifts in a sequence of forward-looking infrared (FLIR) images. If the FLIR is mounted on a moving platform, such as an aircraft, uncontrolled random vibrations associated with the platform can be used to generate the shifts. Iterative techniques such as the expectation-maximization (EM) approach by means of the maximum-likelihood algorithm can be used to generate high-resolution images from multiple randomly shifted aliased frames. In the maximum-likelihood approach the data are considered to be Poisson random variables and an EM algorithm is developed that iteratively estimates an unaliased image that is compensated for known imager-system blur while it simultaneously estimates the translational shifts. Although this algorithm yields high-resolution images from a sequence of randomly shifted frames, it requires significant computation time and cannot be implemented for real-time applications that use the currently available high-performance processors. The new image shifts are iteratively calculated by evaluation of a cost function that compares the shifted and interlaced data frames with the corresponding values in the algorithm's latest estimate of the high-resolution image. We present a registration algorithm that estimates the shifts in one step. The shift parameters provided by the new algorithm are accurate enough to eliminate the need for iterative recalculation of translational shifts. Using this shift information, we apply a simplified version of the EM algorithm to estimate a high-resolution image from a given sequence of video frames. The proposed modified EM algorithm has been found to reduce significantly the computational burden when compared with the original EM algorithm, thus making it more attractive for practical implementation. Both simulation and experimental results are presented to verify the effectiveness of the proposed technique.

Aliasing reduction in staring infrared imagers utilizing subpixel techniques

We introduce and analyze techniques for the reduction of aliased signal energy in a staring infrared imaging system. A standard staring system uses a fixed two-dimensional detector array that corresponds to a fixed spatial sampling frequency determined by the detector pitch or spacing. Aliasing will occur when sampling a scene containing spatial frequencies exceeding half the sampling frequency. This aliasing can significantly degrade the image quality. The aliasing reduction schemes presented here, referred to as microscanning, exploit subpixel shifts between time frames of an image sequence. These multiple images are used to reconstruct a single frame with reduced aliasing. If the shifts are controlled, using a mirror or beam steerer for example, one can obtain a uniformly sampled microscanned image. The reconstruction in this case can be accomplished by a straightforward interlacing of the time frames. If the shifts are uncontrolled, the effective sampling may be nonuniform and reconstruction becomes more complex. A sampling model is developed and the aliased signal energy is analyzed for the microscanning techniques. Finally, a number of experimental results are presented that illustrate the perlormance of the microscanning methods.

Optical interpolation and differencing of 2-D image data

We present a new 1-D acoustooptic architecture to subtract two signals that are subpixel shifted. The architecture allows for optical interpolation and electrical control of the shift between frames. We demonstrate the use of 1-D projections of 2-D image frames to achieve subpixel shifts and interpolation between successive image frames for the purpose of detecting and tracking subpixel sized targets.