Frame Rate Up-conversion Algorithm Research Articles

New Frame Rate Up-Conversion Algorithms With Low Computational Complexity

This paper proposes a new frame rate up-conversion (FRUC) algorithm to reduce the computational complexity and to improve the peak signal-to-noise ratio (PSNR) performance. The proposed FRUC algorithm includes prediction-based motion vector smoothing (PMOS), partial average-based motion compensation (PAMC), and intrapredicted hole interpolation (IPHI). PMVS can efficiently remove outliers using motion vectors of neighboring blocks and PAMC performs motion compensation with the region-based partial average to reduce blocking artifacts of the interpolated frames. For hole interpolation, IPHI uses intraprediction of H.264/AVC to eliminate blurring and also uses the fixed weights implemented using only shift operations, which result in low computational complexity. Compared to the existing algorithms, which use bilateral motion estimation, the proposed algorithm improves the average PSNR of the interpolated frames by 3.44 dB and lowers PSNR performance only by 0.13 dB than the existing algorithm that employs unilateral ME; however, it can significantly reduce the computational complexity of FRUC about 89.3% based on the absolute difference.

Confidence-based adaptive frame rate up-conversion

In this article, we propose a new frame rate up-conversion (FRU) method for temporal quality enhancement. The proposed FRU algorithm employs gradual and adaptive motion estimation based on confidence priority for selecting more accurate motion vectors (MVs). In order to estimate accurate MVs, we adaptively alternate a search range and the order of blocks to be searched depending on the confidence level in hierarchical motion estimation. The precedence of the proposed algorithm is conducted based on the confidence level that is decided by the complexity of pixel values in a block. In addition, we perform bi-directional motion compensation and spatial linear interpolation to fill occlusion regions. In our experiments, we found that the proposed algorithm is about 2 dB better than several conventional methods. Furthermore, block artifacts and blur artifacts are significantly diminished by the proposed algorithm.

Frame rate up-conversion for high-definition video applications

In this paper, a novel frame rate up-conversion (FRUC) scheme to achieve better visual quality for high-definition videos is proposed. The proposed algorithm has three main advantages over conventional FRUC algorithms. First, to increase the accuracy of motion estimation, this paper considers the diversity of motions, the retiming motion vectors of the previous motion field, and an adaptive update strategy. Second, the motion vector that is selected by using the sum of the absolute differences criterion is outputted after a geometric-based motion vector restriction, thereby improving the consistency of the motion vectors. Third, post-processing is performed on the estimated motion vectors based on localized global motions which are calculated by using motion histograms. Experimental results show that the performance of the proposed algorithm significantly outperforms that of traditional methods1.

움직임 보상 보간 프레임에 대한 프레임 적응적 왜곡 예측 기법

비디오 프레임 율 증가 변환은 가전 분야에서 매우 다양한 응용으로 인해 매우 많은 관심을 받아 오고 있다. 대 부분의 진보된 FRUC 알고리즘은 보간된 프레임들의 움직임 벡터장을 결정하는 움직임 보간 기술을 사용하고 있다. 그러나 몇 개의 응용 분야에서는 움직임 보상 보간 프레임이 얼마나 잘 복원되었는지에 대한 정보를 필요로 한다. 이와 같은 목적을 위해 본 논문에서는 프레임 기반의 적응적 예측에 기초한 움직임 보상 보간 프레임의 왜곡 예측 기법을 제안한다. 제안된 기법은 대칭형 움직임 탐색 및 보상 보간 기법에 적용되며, 세 가지 다른 예측 기법 즉, 순방향, 역방향 그리고 적응적 양방향 예측 기법으로 분석된다. 모의 실험을 통하여 제안된 적응적 양방향 왜곡 예측 방식이 다른 두 방식에 비해 성능이 우수함을 보인다. Video FRUC (Frame Rate Up Conversion) has been a technique of great interest due to its diversified applications in consumer electronics. Most advanced FRUC algorithms adopt a motion interpolation technique to determine the motion vector field of interpolated frames. But, in some applications, it is necessary to evaluate how well the MCI (Motion Compensated Interpolation) frame is reconstructed. For this aim, this paper proposes a distortion estimation for motion compensated interpolation frame using frame-adaptive distortion estimation. The proposed method is applied for the symmetric motion estimation and compensated scheme and then analyzed by three different approaches, that is, forward estimation, backward estimation and adaptive bi-directional estimation schemes. Through computer simulations, it is shown that the proposed bi-directional estimation method outperforms others and can be effectively applied for FRUC.

움직임 보상 보간 프레임에 대한 블록 적응적 정합 특성을 이용한 왜곡 예측 기법

비디오 프레임 율 증가 변환은 최근에 가전 분야에서 비디오 정보원과 더불어 디스플레이 포맷의 폭발적인 증가에 힘입어 매우 많은 관심의 대상이 되었다. 대부분의 진보된 FRUC 알고리즘은 보간된 프레임들의 움직임 벡터장을 결정하는 효과적인 움직임 보간 기술을 사용하고 있다. 그러나 수신기의 후처리 분야와 같은 응용 분야에서는 움직임 보상 보간 프레임이 얼마나 잘 복원되었는지에 대한 정보를 필요로 한다. 이와 같은 목적을 달성하기 위해, 먼저, 본 논문에서는 움직임 보상 보간 프레임의 블록 단위로 신뢰도를 측정할 수 있는 비용 함수들을 도입한다. 그런 후에 이러한 함수들을 사용하여 움직임 보상 보간 프레임에서 얼마나 많은 잡음이 포함되어 있는지를 평가할 수 있는 두 가지의 왜곡 예측 모델을 제안한다. 모의실험을 통하여 제안된 왜곡 예측 방식은 움직임 보상 보간 프레임의 잡음을 효과적으로 예측할 수 있음을 보인다. Video FRUC (Frame Rate Up Conversion) is one of the main issues that have arisen in recent years with the explosive growth of video sources and display formats in consumer electronics. Most advanced FRUC algorithms adopt an efficient motion interpolation technique to determine the motion vector field of interpolated frames. But, in some application areas such as post processing in receiver side, it is necessary to evaluate how well the MCI (Motion Compensated Interpolation) frame was reconstructed. In order to achieve this aim, first, this paper introduces some cost functions to estimate the reliability of a block in the MCI frame. Then, by using these functions, this paper proposes two distortion estimation models for evaluating how much noise was produced in the MCI frame. Through computer simulations, it is shown that the proposed estimation methods perform effectively in estimating the noises of the MCI frame.

Frame Rate Up-Conversion Using Trilateral Filtering

Frame rate up-conversion (FRUC) can enhance the visual quality of low frame rate video presented on liquid crystal display. To minimize the difference between a reference block and an interpolated block, an effective FRUC algorithm partitions a large block into several sub-blocks of smaller size and estimates their motions. Motion estimation searches for the block which has the minimum difference (cost) with the processed block in terms of some block matching distortion and motion discontinuity. As convexity and convergence of the cost function are not guaranteed, the computational cost for such motion estimation is usually extensive or unpredictable. In our proposed FRUC method, the two predictions of a frame to be interpolated are generated through shifting its nearest neighbor frames in the previous and following directions with the motion vectors estimated between them. The initial interpolated frame and its pixel's reliability are subsequently estimated from these two predictions. We then apply a trilateral filter on the initial prediction to correct the unreliable pixels and to restore the missing pixels. Our proposed method not only reduces the computation for refining motion vectors, but also suppresses the interpolation noises and misregistration errors. We have conducted extensive experiments and the results show that the proposed algorithm outperforms the existing methods with better objective and subjective visual quality, and achieves about 3 dB on-average peak signal-to-noise ratio improvement.

Low complexity algorithm for robust video frame rate up-conversion (FRUC) technique

Two challenging situations for video frame rate up-conversion (FRUC) are first identified and analyzed; namely, when the input video has abrupt illumination change and/or a low frame rate. Then, a low-complexity processing technique and robust FRUC algorithm are proposed to address these two issues. The proposed algorithm utilizes a translational motion vector model of the first- and the second-order and detects the continuity of these motion vectors. Additionally, in order to improve perceptual quality of interpolated frame, spatial smoothness criterion is employed. The superior performance of the proposed algorithm has been tested extensively and representative examples are given in this work.

Multiframe-based bilateral motion estimation with emphasis on stationary caption processing for frame rate up-conversion

In this paper, we present a new motion compensated frame rate up-conversion algorithm that uses multiframes to enhance the accuracy of motion estimation. We also develop adaptive motion vector smoothing to correct outliers in a motion vector field. In addition, stationary caption processing is carried out for removing block artifacts from stationary captions. Finally, we propose adaptive-weighted bidirectional motion compensated interpolation to reduce object blurring. In experiments using benchmark test sequences, the proposed algorithm improves the average PSNR of interpolated frames by up to 5.34 dB compared to the conventional algorithm.

Adaptive frame rate up-conversion based on motion classification

In this paper, a new technique on video frame rate up-conversion (FRUC) is presented by combining the adaptive motion classification (AMC) for image sequences and the mixed motion estimation (ME). In the proposed FRUC scheme, the AMC classifies sequences into global and local motion sequences (G/LMS) whose advantages are fully utilized. The mixture of bidirectional ME (BDME) and unidirectional ME (UDME) is proposed to avoid hole and “background replace object” phenomenon in interpolated frames. Moreover, the motion vectors (MVs) refinement technique is employed to smooth MVs field. For LMS, the motion detection is used to deal with static background and occlusions are located by using forward prediction of motion regions. Correctional sum of absolute difference (CSAD) which gives more precise physical MVs field than conventional SAD is applied during all of the ME process. Simulation results demonstrate the effectiveness of the proposed FRUC algorithm.

Motion Compensated Frame Rate Up-Conversion Using Extended Bilateral Motion Estimation

This paper presents a motion compensated frame rate up-conversion algorithm that uses the expanded range of the motion trajectory to enhance the accuracy of motion estimation. In the proposed algorithm, an adaptive motion vector refinement is also proposed to correct false motion vectors. In addition, a recursive motion vector smoothing is proposed to smooth motion vectors considering true neighboring motion vectors only. Finally, we propose the weighted index-based bidirectional motion compensated interpolation to reduce artifacts. In the experiments using benchmark test sequences, the proposed algorithm improves the average PSNR of interpolated frames by up to 4.84 dB, when compared to the conventional algorithm.

Frame rate up-conversion using perspective transform

In this paper, we propose a frame rate up-conversion (FRUC) algorithm to increase the temporal resolution of video sequences at the decoder side. First, the proposed algorithm segments a frame into several objects. Then, perspective transforms are used to motion-compensate each object, while the translational block matching algorithm (BMA) is applied to the background. Furthermore, the overlapped block motion compensation (OBMC) technique is used to reduce blocking artifacts in boundary blocks. Experimental results show that the proposed algorithm provides better performance than the conventional approach.

Weighted-adaptive motion-compensated frame rate up-conversion

In this paper, we propose a frame rate upconversion algorithm using the weighted-adaptive motion-compensated interpolation (WAMCI) that reduces the block artifacts due to the failure of motion estimation and block-based processing. The proposed method is based on the interpolation scheme by weighted sum of multiple motion-compensated interpolation (MCI) images. Also, in the proposed method, the block artifacts on the block boundaries are reduced by applying a technique similar to the overlapped block motion compensation (OBMC). To reduce the blurring of overlapping processing, the proposed method uses the motion analysis to determine the type of motion and applies the OBMC adaptively. Experimental results indicate good performance of the proposed scheme with significantly reduced block artifacts.

Adaptive motion-compensated interpolation for frame rate up-conversion

We propose a new frame rate up-conversion (FRC) algorithm using the adaptive motion compensation (MC) that reduces the blocking artifacts due to block-based processing. In the proposed scheme, after conventional motion estimation (ME) between two adjacent frames is performed to construct the motion vectors for the frame to be interpolated, the motion analysis is used to determine the type of motion and the motion-compensated interpolation (MCI) is applied adaptively. Unlike conventional MCI algorithms, the proposed technique utilizes similar neighboring motion vectors to produce multiple motion trajectories. When the proposed MCI is applied, multiple motion trajectories are considered in order to increase the accuracy of the MCI. The proposed method provides high quality format conversion with significantly reduced blocking artifacts.

Adaptive motion-compensated frame rate up-conversion

A frame rate up-conversion algorithm using the adaptive motion compensation that reduces the blocking artifacts due to block-based processing is proposed. The proposed method uses the motion analysis to determine the type of motion and applies the motion-compensated interpolation (MCI) adaptively. When the MCI is applied, multiple motion trajectories are considered in order to increase the accuracy of the MCI. Experiment results indicate good performance of the proposed MCI scheme with significantly reduced blocking artifacts.

New frame rate up-conversion using bi-directional motion estimation

We propose a new frame rate up-conversion algorithm for high quality video. In the proposed scheme, bi-directional motion estimation (ME) is performed to construct the motion vector (MV) field for the frame to be interpolated. Unlike conventional motion-compensated interpolation (MCI) algorithms, the proposed technique does not produce any overlapped pixel and hole region in the interpolated frame, and thus can utilize the overlapped block motion compensation technique to reduce the blocking artifacts. The proposed algorithm is very simple to implement on consumer products when compared to conventional MCI methods. Computer simulation shows a high visual performance of the proposed frame rate up-conversion algorithm.