Input Image Pairs Research Articles

Single-Image Super-Resolution Using Sparse Regression and Natural Image Prior

This paper proposes a framework for single-image super-resolution. The underlying idea is to learn a map from input low-resolution images to target high-resolution images based on example pairs of input and output images. Kernel ridge regression (KRR) is adopted for this purpose. To reduce the time complexity of training and testing for KRR, a sparse solution is found by combining the ideas of kernel matching pursuit and gradient descent. As a regularized solution, KRR leads to a better generalization than simply storing the examples as has been done in existing example-based algorithms and results in much less noisy images. However, this may introduce blurring and ringing artifacts around major edges as sharp changes are penalized severely. A prior model of a generic image class which takes into account the discontinuity property of images is adopted to resolve this problem. Comparison with existing algorithms shows the effectiveness of the proposed method.

Training an Active Random Field for Real-Time Image Denoising

Many computer vision problems can be formulated in a Bayesian framework based on Markov random fields (MRF) or conditional random fields (CRF). Generally, the MRF/CRF model is learned independently of the inference algorithm that is used to obtain the final result. In this paper, we observe considerable gains in speed and accuracy by training the MRF/CRF model together with a fast and suboptimal inference algorithm. An active random field (ARF) is defined as a combination of a MRF/CRF based model and a fast inference algorithm for the MRF/CRF model. This combination is trained through an optimization of a loss function and a training set consisting of pairs of input images and desired outputs. We apply the ARF concept to image denoising, using the Fields of Experts MRF together with a 1-4 iteration gradient descent algorithm for inference. Experimental validation on unseen data shows that the ARF approach obtains an improved benchmark performance as well as a 1000-3000 times speedup compared to the Fields of Experts MRF. Using the ARF approach, image denoising can be performed in real-time, at 8fps on a single CPU for a 256 x 256 image sequence, with close to state-of-the-art accuracy.

Fractional Stereo Matching Using Expectation-Maximization

In our fractional stereo matching problem, a foreground object with a fractional boundary is blended with a background scene using unknown transparencies. Due to the spatially varying disparities in different layers, one foreground pixel may be blended with different background pixels in stereo images, making the color constancy commonly assumed in traditional stereo matching not hold any more. To tackle this problem, in this paper, we introduce a probabilistic framework constraining the matching of pixel colors, disparities, and alpha values in different layers, and propose an automatic optimization method to solve a Maximizing a Posterior (MAP) problem using Expectation-Maximization (EM), given only a short-baseline stereo input image pair. Our method encodes the effect of background occlusion by layer blending without requiring a special detection process. The alpha computation process in our unified framework can be regarded as a new approach by natural image matting, which handles appropriately the situation when the background color is similar to that of the foreground object. We demonstrate the efficacy of our method by experimenting with challenging stereo images and making comparisons with state-of-the-art methods.

Optimizing the High-Pass Filter Addition Technique for Image Fusion

Pixel-level image fusion combines complementary image data, most commonly low spectral-high spatial resolution data with high spectral-low spatial resolution optical data. The presented study aims at refining and improving the High-Pass Filter Additive (HPFA) fusion method towards a tunable and versatile, yet standardized image fusion tool. HPFA is an image fusion method in the spatial domain, which inserts structural and textural details of the higher resolution image into the lower resolution image, whose spectral properties are thereby largely retained. Using various input image pairs, workable sets of HPFA parameters have been derived with regard to high-pass filter properties and injection weights. Improvements are the standardization of the HPFA parameters over a wide range of image resolution ratios and the controlled trade-off between resulting image sharpness and spectral properties. The results are evaluated visually and by spectral and spatial metrics in comparison with wavelet-based image fusion results as a benchmark.

A new image registration scheme based on curvature scale space curve matching

We propose a new image registration scheme for remote sensing images. This scheme includes three steps in sequence. First, a segmentation process is performed on the input image pair. Then the boundaries of the segmented regions in two images are extracted and matched. These matched regions are called confidence regions. Finally, a non-linear optimization is performed in the matched regions only to obtain a global set of transform parameters. Experiments show that this scheme is more robust and converges faster than registration of the original image pair. We also develop a new curve-matching algorithm based on curvature scale space to facilitate the second step.

An Efficient Picture-Rate Up-Converter

The importance of low-power design is not just critical to portable devices but also to line powered equipment like TV products. Power dissipation strongly influences the price of the chip, since the packaging and cooling costs increase dramatically with increasing power dissipation. In this work, we analyze and optimize algorithm and architecture of a picture rate up-conversion module. We perform algorithm/architecture co-design in which we meet high quality specification while keeping the power dissipation low. In the algorithm front, we focus on the motion estimation which is computationally the most intensive part of the picture-rate up-conversion application. We analyze the following parameters of the motion estimation algorithm: The number of motion estimation iterations per input image pair and the image scanning order of individual iterations. Further, we apply novel pre-processing technique to address the issue of reducing the already extremely low number of motion vector candidate evaluations. However, optimal selection of motion vector candidates is a necessity to achieve high picture quality. In the architectural front, to cope with the large memory bandwidth requirements of the application, we use multi-level caching to exploit locality of reference. Further, we apply data compression to the image data stored in memory, to reduce the memory capacity and bandwidth requirements. Both the above techniques significantly reduce the overall power dissipation.

Piecewise affine registration of biological images for volume reconstruction

This manuscript tackles the reconstruction of 3-D volumes via mono-modal registration of series of 2-D biological images (histological sections, autoradiographs, cryosections, etc.). The process of acquiring these images typically induces composite transformations that we model as a number of rigid or affine local transformations embedded in an elastic one. We propose a registration approach closely derived from this model. Given a pair of input images, we first compute a dense similarity field between them with a block matching algorithm. We use as a similarity measure an extension of the classical correlation coefficient that improves the consistency of the field. A hierarchical clustering algorithm then automatically partitions the field into a number of classes from which we extract independent pairs of sub-images. Our clustering algorithm relies on the Earth mover’s distribution metric and is additionally guided by robust least-square estimation of the transformations associated with each cluster. Finally, the pairs of sub-images are, independently, affinely registered and a hybrid affine/non-linear interpolation scheme is used to compose the output registered image. We investigate the behavior of our approach on several batches of histological data and discuss its sensitivity to parameters and noise.

Implementation of disparity information‐based 3D object tracking

In this paper, a new 3D object tracking system using the disparity motion vector (DMV) is presented. In the proposed method, the time‐sequential disparity maps are extracted from the sequence of the stereo input image pairs and these disparity maps are used to sequentially estimate the DMV defined as a disparity difference between two consecutive disparity maps Similarly to motion vectors in the conventional video signals, the DMV provides us with motion information of a moving target by showing a relatively large change in the disparity values in the target areas. Accordingly, this DMV helps detect the target area and its location coordinates. Based on these location data of a moving target, the pan/tilt embedded in the stereo camera system can be controlled and consequently achieve real‐time stereo tracking of a moving target. From the results of experiments with 9 frames of the stereo image pairs having 256×256 pixels, it is shown that the proposed DMV‐based stereo object tracking system can track the moving target with a relatively low error ratio of about 3.05 % on average.

A Systolic Multi-Layer and Multi-Line Stereo Vision System for 3D Visual Inspection

A Systolic Multi-Layer and Multi-Line Stereo Vision System for 3D Visual Inspection

Geometric–photometric approach to monocular shape estimation

Monocular, reflectance map-based shape estimation has traditionally relied exclusively on photometric data, but the recently introduced disparity-based photometric stereo (DBPS) and Green's function shape from shading (GSFS) have brought more geometry into it by incorporating a matching equation, and can appropriately be termed a geometric–photometric approach. In DBPS, the matching equation is used for obtaining disparities from the input image pair, while in GSFS it is used for generating the matching image, a uniform disparity field being considered. In both cases, depth can be recovered if one assumes the disparities to result from the displacement of the irradiance pattern over the imaged surface. Starting from the analysis of such displacement under perspective projection, we show that the original DBPS/GSFS formulation must be amended: reconstructions with no free parameters are no longer feasible, but we are able to propose an approximate procedure which yields higher quality estimates, up to a multiplicative factor.

A new stereo object tracking system using disparity motion vector

In this paper, a new stereo object tracking system using the disparity motion vector (DMV) is presented. In the proposed method, the time-sequential disparity maps are extracted from the sequence of the stereo input image pairs and then, the DMV defined as a disparity difference between two consecutive disparity maps is sequentially estimated from these disparity maps. Just like the motion vectors in the conventional video signals, the DMV provides us with motion information of a moving target by showing a relatively large change of the disparity values in the target areas. Accordingly, using this DMV the target area and its location coordinates can be detected. Basing on these locational data of a moving target, the pan/tilt embedded in the stereo camera system can be controlled and as a result, real-time stereo tracking of a moving target can be achieved. From some experiments with 9 frames of the stereo image pairs having 256 × 256 pixels, it is shown that the proposed DMV-based stereo object tracking system can track the moving target with a relatively low error ratio of about 3.05% on average.

A level-set approach to image blending

This paper presents a novel method for blending images. Image blending refers to the process of creating a set of discrete samples of a continuous, one-parameter family of images that connects a pair of input images. Image blending has uses in a variety of computer graphics and image processing applications. In particular, it ran be used for image morphing, which is a method for creating video streams that depict transformations of objects in scenes based solely on pairs of images and sets of user-defined fiducial points. Image blending also has applications for video compression and image-based rendering. The proposed method for image blending relies on the progressive minimization of a difference metric which compares the level sets between two images. This strategy results in an image blend which is the solution of a pair of coupled, nonlinear, first-order partial differential equations that model multidimensional level-set propagations. When compared to interpolation this method produces more natural appearances of motion because it manipulates the shapes of image contours rather than simply interpolating intensity values. This strategy results in a process that has the qualitative property of deforming greyscale objects in images rather than producing a simple fade from one object to another. This paper presents the mathematics that underlie this new method, a numerical implementation, and results on real images that demonstrate its effectiveness.

Two-dimensional, frequency domain, adaptive system modeling using three-dimensional spatiotemporal inputs

In this paper, an adaptive, frequency domain, steepest descent algorithm for two-dimensional (2-D) system modeling is presented. The algorithm is derived here for the equation error model, and models the 2-D spatially linear and invariant unknown system by a 2-D auto-regressive, moving-average (ARMA) process. The proposed technique is implemented in the 3-D spatiotemporal domain. At each iteration, corresponding to a given pair of input and output images, the algorithm is formulated to minimize the energy of an error-function in the frequency-domain by adjusting the coefficients of the 2-D ARMA model. Signal dependent, optimal convergence factors, referred to as the homogenous convergence factors, are developed. Computer simulations demonstrate the algorithm's excellent adaptation accuracy and convergence speed. For illustration, the proposed algorithm is successfully applied to modeling a time varying 2-D system. >

A 40*40 CCD/CMOS absolute-value-of-difference processor for use in a stereo vision system

An analog VLSI processor chip with application in a high-speed binocular stereo vision system used for the recovery of scene depth is presented. The authors have attempted to exploit the principal advantages of analog VLSI, namely, its small area, high speed, and low power, while minimizing the effects of its limited accuracy, inflexibility, and lack of storage capacity. A CCD/CMOS stereo system implementation, capable of processing several thousand image frame pairs per second for 40*40-pixel binocular images, is proposed. A 40*40-pixel absolute-value-of-difference (AVD) array, a core processor of the stereo system, was fabricated in a 2- mu m CCD/CMOS process. Individual unit cells in the array were characterized and tested. The array functionality was tested by imbedding it in a computerized stereo system and using both real-scene and computer-generated input image pairs. The system output is compared with full computer simulations for the same image pairs, showing good correlation.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>