Sequence Of Low-resolution Images Research Articles

A Total Variation Regularization Based Super-Resolution Reconstruction Algorithm for Digital Video

Super-resolution (SR) reconstruction technique is capable of producing a high-resolution image from a sequence of low-resolution images. In this paper, we study an efficient SR algorithm for digital video. To effectively deal with the intractable problems in SR video reconstruction, such as inevitable motion estimation errors, noise, blurring, missing regions, and compression artifacts, the total variation (TV) regularization is employed in the reconstruction model. We use the fixed-point iteration method and preconditioning techniques to efficiently solve the associated nonlinear Euler-Lagrange equations of the corresponding variational problem in SR. The proposed algorithm has been tested in several cases of motion and degradation. It is also compared with the Laplacian regularization-based SR algorithm and other TV-based SR algorithms. Experimental results are presented to illustrate the effectiveness of the proposed algorithm.

Blind superresolution imaging

Imaging plays a key role in many diverse areas of application, such as astronomy, remote sensing, microscopy, and tomography. Owing to imperfections of measuring devices (e.g., optical degradations, limited size of sensors) and instability of the observed scene (e.g., object motion, media turbulence), acquired images can be indistinct, noisy, and may exhibit insufficient spatial and temporal resolution. In particular, several external effects blur images. We will call these effects volatile blurs to emphasize their unpredictable and transitory behavior. Techniques for recovering the original image include blind deconvolution (to remove blur) and superresolution.1, 2 The stability of these methods depends on having more than one image of the same frame. Differences between images are necessary to provide new information, but they can be almost imperceivable, for example, subtle spatial shifts or slight modification of the acquisition parameters, such as focus length and aperture size. For a single observation g(i, j) the problem of recovering the original undegraded image is underdetermined (i.e., there are more unknowns than knowns) and lacks a stable solution. To partially overcome this equivocation, we need to have K (K > 1) images of the original scene and then face the so-called multichannel (multiframe) dilemma. Superresolution (SR) is the process of combining a sequence of low-resolution (LR) images to produce a better result. It is unrealistic to assume that the superresolved image can recover the original scene o(x, y) exactly. Rather, a reasonable goal of SR is a discrete version of o(x, y), which has spatial resolution higher than that of the LR images and which is free of blurs (deconvolved). The acquisition model then becomes

Superresolved imaging of remote moving targets

We present a superresolving approach that allows one to exceed the diffraction limit and recover highly resolved contours of moving targets from a sequence of low-resolution images. The presented approach is suitable for remote sensing applications. The resolution decoding algorithm that is used to recover the high-resolution features of the target can be run partially via optical means and that way can be used to reduce the required computational complexity.

An image super-resolution algorithm for different error levels per frame

In this paper, we propose an image super-resolution (resolution enhancement) algorithm that takes into account inaccurate estimates of the registration parameters and the point spread function. These inaccurate estimates, along with the additive Gaussian noise in the low-resolution (LR) image sequence, result in different noise level for each frame. In the proposed algorithm, the LR frames are adaptively weighted according to their reliability and the regularization parameter is simultaneously estimated. A translational motion model is assumed. The convergence property of the proposed algorithm is analyzed in detail. Our experimental results using both real and synthetic data show the effectiveness of the proposed algorithm.

Super-Resolution for Synthetic Zooming

Optical zooming is an important feature of imaging systems. In this paper, we investigate a low-cost signal processing alternative to optical zooming-synthetic zooming by super-resolution (SR) techniques. Synthetic zooming is achieved by registering a sequence of low-resolution (LR) images acquired at varying focal lengths and reconstructing the SR image at a larger focal length or increased spatial resolution. Under the assumptions of constant scene depth and zooming speed, we argue that the motion trajectories of all physical points are related to each other by a unique vanishing point and present a robust technique for estimating its 3D coordinate. Such a line-geometry-based registration is the foundation of SR for synthetic zooming. We address the issue of data inconsistency arising from the varying focal length of optical lens during the zooming process. To overcome the difficulty of data inconsistency, we propose a two-stage Delaunay-triangulation-based interpolation for fusing the LR image data. We also present a PDE-based nonlinear deblurring to accommodate the blindness and variation of sensor point spread functions. Simulation results with real-world images have verified the effectiveness of the proposed SR techniques for synthetic zooming.

Super-resolution reconstruction of image sequence using multiple motion estimation fusion

Super-resolution reconstruction algorithm produces a high-resolution image from a low-resolution image sequence. The accuracy and the stability of the motion estimation (ME) are essential for the whole restoration. In this paper, a new super-resolution reconstruction algorithm is developed using a robust ME method, which fuses multiple estimated motion vectors within the sequence. The new algorithm has two major improvements compared with the previous research. First, instead of only two frames, the whole sequence is used to obtain a more accurate and stable estimation of the motion vector of each frame; second, the reliability of the ME is quantitatively measured and introduced into the cost function of the reconstruction algorithm. The algorithm is applied to both synthetic and real sequences, and the results are presented in the paper.

Simultaneous estimation of super-resolved scene and depth map from low resolution defocused observations

This paper presents a novel technique to simultaneously estimate the depth map and the focused image of a scene, both at a super-resolution, from its defocused observations. Super-resolution refers to the generation of high spatial resolution images from a sequence of low resolution images. Hitherto, the super-resolution technique has been restricted mostly to the intensity domain. In this paper, we extend the scope of super-resolution imaging to acquire depth estimates at high spatial resolution simultaneously. Given a sequence of low resolution, blurred, and noisy observations of a static scene, the problem is to generate a dense depth map at a resolution higher than one that can be generated from the observations as well as to estimate the true high resolution focused image. Both the depth and the image are modeled as separate Markov random fields (MRF) and a maximum a posteriori estimation method is used to recover the high resolution fields. Since there is no relative motion between the scene and the camera, as is the case with most of the super-resolution and structure recovery techniques, we do away with the correspondence problem.

A genetic algorithm approach to image sequence interpolation

Image sequence interpolation, or to obtain an up-sampled image sequence equivalently from a corresponding low-resolution image sequence, is an ill-posed inverse problem. In this study, three processing steps, namely, regularization, discretization and optimization, are used to convert the image sequence interpolation problem into a solvable optimization problem. In regularization, a fitness function combining a set of spatial and temporal performance measures for rating the quality of the interpolated (up-sampled) images is defined, which is used to convert the original ill-posed interpolation problem into a well-posed optimization problem. Discretization transforms the well-posed problem into a discrete one so that it can be solved numerically. Genetic algorithms (GAs) are used to optimize the solution in the discrete solution space using three basic operations, namely, reproduction, crossover and mutation. In the proposed approach, instead of only the spatial information within the current image frame employed in most existing methods, both the spatial and temporal information within the image sequence can be employed. Based on the experimental results obtained in this study, the interpolation results by the proposed approach are always better than those from the three existing approaches used for comparison. This shows the feasibility of the proposed approach.

Spatial reconstruction of marker trajectories from high-speed video image sequences

A novel method is presented for the spatial reconstruction of marker trajectories, which describe the motion of articulating segments of the human or animal body or of a technical construction. In human motion analysis, such markers may be attached to landmark points defining the configuration of the subject. A high-speed video-recording system comprising two cameras is used for motion data acquisition. The particular images are composed of the synchronous recordings of both cameras; the resolution of one image is 239×192 pixels, which is comparatively low. To obtain high precision, a new centre estimation method has been developed to calculate the image coordinates of the centres of the recorded spherical markers to subpixel precision. Image coordinates of the centres in subsequent frames are obtained by first applying a tracing algorithm, which calculates the position of a marker to pixel precision, and then using the new centre estimation method. The reconstruction of the spatial coordinates from the image coordinates is based on a three-dimensional photogrammetric calibration and results in deviations of about 0.1%. The instantaneous acceleration of a falling ball has been calculated. The average deviation from the gravitational constant was comparatively small. From the accuracy of the results, it can be concluded that the novel method is applicable in connection with the high-speed video-recording system presently used. Moreover, the techniques are suitable for reconstructing three-dimensional marker trajectories from any low-resolution video image sequences recorded simultaneously with at least two cameras.

Extraction of high-resolution frames from video sequences

The human visual system appears to be capable of temporally integrating information in a video sequence in such a way that the perceived spatial resolution of a sequence appears much higher than the spatial resolution of an individual frame. While the mechanisms in the human visual system that do this are unknown, the effect is not too surprising given that temporally adjacent frames in a video sequence contain slightly different, but unique, information. This paper addresses the use of both the spatial and temporal information present in a short image sequence to create a single high-resolution video frame. A novel observation model based on motion compensated subsampling is proposed for a video sequence. Since the reconstruction problem is ill-posed, Bayesian restoration with a discontinuity-preserving prior image model is used to extract a high-resolution video still given a short low-resolution sequence. Estimates computed from a low-resolution image sequence containing a subpixel camera pan show dramatic visual and quantitative improvements over bilinear, cubic B-spline, and Bayesian single frame interpolations. Visual and quantitative improvements are also shown for an image sequence containing objects moving with independent trajectories. Finally, the video frame extraction algorithm is used for the motion-compensated scan conversion of interlaced video data, with a visual comparison to the resolution enhancement obtained from progressively scanned frames.