Motion-compensated Deinterlacing Research Articles

An HVS-inspired video deinterlacer based on visual saliency

Video deinterlacing is a technique wherein the interlaced video format is converted into progressive scan format for nowadays display devices. In this paper, a spatial saliency-guided motion-compensated deinterlacing method is proposed which accounts for the properties of the Human Visual System (HVS): our algorithm classifies the field according to its texture and viewer’s region of interest and adapts the motion estimation and compensation, as well as the saliency-guided interpolation to ensure high-quality frame reconstruction. Two different saliency models, namely the graph-based visual saliency (GBVS) model and the spectral residual visual saliency (SRVS) model, have been studied and compared in terms of visual quality performances as well as computational complexity. The experimental results on a great variety of video test sequences show significant improvement of reconstructed video quality with the GBVS-based proposed method compared to classical motion-compensated and adaptive deinterlacing techniques, with up to 4.5 dB gains in terms of PSNR. Simulations also show that the SRVS-based deinterlacing process can result to significant reductions of complexity (up to 25 times a decrease of the computation time compared with the GBVS-based method) at the expense of a PSNR decrease.

Corrections to “Motion Compensated Deinterlacing With Efficient Artifact Detection for Digital Television Displays” [Jun 08 218-228

In the above paper (ibid., vol. 4, no. 2, pp. 218-228, Jun. 08), the authors have discovered two errors: 1) in the byline and 2) in the References section. Those errors are corrected here.

True Motion-Compensated De-Interlacing Algorithm

This paper presents a true motion-compensated de-interlacing (TMCD) algorithm based on a fast true motion estimation scheme. The fast true motion estimation scheme, designated as a variable block size true motion estimation for a translated motion model (VBTME), finds block-based true motion vectors within interlaced fields and then uses these true motion vectors to construct corresponding de-interlaced frames. Although the real motions of a majority of the objects within an interlaced field can be accurately represented by using block-based true motion vectors, the motions of certain objects (e.g., those which disappear suddenly or are occluded by other objects) cannot be adequately compensated by using such vectors, and therefore, the corresponding pixels must be further refined by an alternative method. Accordingly, a TMCD applies a set of three decisional rules and then selects either a new spatial edge direction indicated interpolation method or a traditional temporal mean filter to interpolate those pixels which cannot be adequately compensated by using true motion vectors. The experimental results show that by comparing the PSNR values for CIF format video sequences obtained by a 4-field adaptive motion-compensated (4F-AMC) de-interlacing scheme and a selective motion-compensated (SMC) de-interlacing scheme to a TMCD scheme, an average PSNR improvement of 1.37 and 6.36 dB is achieved, respectively. Furthermore, compared to the motion estimation schemes used in a 4F-AMC and a SMC, the VBTME reduces motion estimation time by 96 and 69%, respectively. Finally, it is shown that the visual quality of videos with a CCIR601 format de-interlaced using a TMCD is better than that obtained by using a 4F-AMC de-interlacing scheme.

Deinterlacing Using Variational Methods

We present a variational framework for deinterlacing that was originally used for inpainting and subsequently redeveloped for deinterlacing. From the framework, we derive a motion adaptive (MA) deinterlacer and a motion compensated (MC) deinterlacer and test them together with a selection of known deinterlacers. To illustrate the need for MC deinterlacing, the problem of details in motion (DIM) is introduced. It cannot be solved by MA deinterlacers or any simpler deinterlacers but only by MC deinterlacers. The major problem in MC deinterlacing is computing reliable optical flow [motion estimation (ME)] in interlaced video. We discuss a number of strategies for computing optical flows on interlaced video hoping to shed some light on this problem. We produce results on challenging real world video data with our variational MC deinterlacer that in most cases are indistinguishable from the ground truth.

Performance analysis of motion-compensated de-interlacing systems

A lot of research has been conducted on motion-compensated (MC) de-interlacing, but there are very few publications that discuss the performances of de-interlacing quantatively. The various methods are compared through their performance on known video sequences. Linear system analysis of interlaced video and de-interlacer are proposed in. It is well established that the performance of the MC methods outperform the fixed or motion-adaptive methods when the motion vectors used are reliable and true to the scene content. Being an open-loop process the performance of the MC de-interlacers degrade drastically when there are motion vector errors. In this paper, a linear system analysis of MC video upconversion systems is presented and the effects of motion vector accuracy on system performance are analyzed. We investigate the various factors that contribute to the motion vector inaccuracy, such as incorrect motion modelling, acceleration between the frames, and insufficient interpolation kernel.

Deinterlacing with selective motion compensation

We propose a new deinterlacing algorithm with selective motion compensation. It has been reported that deinterlacing methods using motion compensation produce significantly improved results, although they tend to yield undesired results in fast moving areas. This is due to weak correlations between the previous and current frames. The proposed algorithm solves this problem by selectively applying motion-compensated deinterlacing. We first apply intrafield interpolation in the spatial domain, and then selectively apply motion compensations according to the type of motion vectors. Experimental results show that the proposed method produces noticeably improved performance compared to existing motion-compensated deinterlacing methods.

On the performance bounds of motion-compensated deinterlacing

Abstract The terms ‘proscan up-conversion’ or ‘deinterlacing’ stand for algorithms for the conversion of interlaced image sequences into progressive format by interpolating the lines which are missing due to interlaced scanning. Although deinterlacing is already known to improve TV image quality, such techniques will become even more important in future. Firstly, standard interlaced image sequences will be increasingly processed and displayed on computer platforms and thus usually require progressive scanning and an increased temporal resolution. Secondly, the emerging market of light valve projection equipment demands high-quality proscan up-conversion since the image quality of these display techniques relies heavily on the light efficiency of non-interlaced scanning. By describing deinterlacing as a problem of linear prediction theory, this paper derives upper MSE performance bounds for these algorithms. Thus, a theoretical framework is provided which allows to analyse the effects of various parameters like motion vector inaccuracy, temporal integration and memory constraints on the efficiency of motion-compensated deinterlacing. It is, for example, shown that these algorithms can be very sensitive to motion vector inaccuracies especially if the image sequence is recorded with very short exposure time. Finally, it is briefly demonstrated that the presented analysis can easily be extended to the case of field- and frame-rate up-conversion of interlaced (‘100 Hz’) and non-interlaced image sequences.