Deinterlacing Methods Research Articles

Video expert assessment of high quality video for Video Assistant Referee (VAR): A comparative study

The International Football Association Board decided to introduce Video Assistant Referee (VAR) in 2018. This led to the need to develop methods for quality control of the VAR-systems. This article focuses on the important aspect to evaluate the video quality. Video Quality assessment has matured in the sense that there are standardized, commercial products and established open-source solutions to measure it with objective methods. Previous research has primarily focused on the end-user quality assessment. How to assess the video in the contribution phase of the chain is less studied. The novelties of this study are two-fold: 1) The user study is specifically targeting video experts i.e., to assess the perceived quality of video professionals working with video production. 2) Six video quality models have been independently benchmarked against the user data and evaluated to show which of the models could provide the best predictions of perceived quality. The independent evaluation is important to get unbiased results as shown by the Video Quality Experts Group. An experiment was performed involving 25 video experts in which they rated the perceived quality. The video formats tested were High-Definition TV both progressive and interlaced as well as a quarters size format that was scaled down half the size in both width and height. The videos were encoded with both H.264 and Motion JPEG for the full size but only H.264 for the quarter size. Bitrates ranged from 80 Mbit/s down to 10 Mbit/s. We could see that for H.264 that the quality was overall very good but dropped somewhat for 10 Mbit/s. For Motion JPEG the quality dropped over the whole range. For the interlaced format the degradation that was based on a simple deinterlacing method did receive overall low ratings. For the quarter size three different scaling algorithms were evaluated. Lanczos performed the best and Bilinear the worst. The performance of six different video quality models were evaluated for 1080p and 1080i. The Video Quality Metric for Variable Frame Delay had the best performance for both formats, followed by Video Multimethod Assessment Fusion method and the Video Quality Metric General model.

Rethinking Deinterlacing for Early Interlaced Videos

In recent years, high-definition restoration of early videos have received much attention. Real-world interlaced videos usually contain various degradations mixed with interlacing artifacts, such as noises and compression artifacts. Unfortunately, traditional deinterlacing methods only focus on the inverse process of interlacing scanning, and cannot remove these complex and complicated artifacts. Hence, this paper proposes an image deinterlacing network (DIN), which is specifically designed for joint removal of interlacing mixed with other artifacts. The DIN is composed of two stages, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , a cooperative vertical interpolation stage for splitting and fully using the information of adjacent fields, and a field-merging stage to perceive movements and suppress ghost artifacts. Experimental results demonstrate the effectiveness of the proposed DIN on both synthetic and real-world test sets.

Triangular fuzzy interpolation deinterlacing algorithm method for image edge detection

Edge detection is the first step in image as it can be applied in the process of feature extraction, segmentation and classification. This study proposes a new fuzzy logic based edge detection method and computationally efficient algorithm based on triangular fuzzy interpolation technique for intra field deinterlacing of images that reduces the loss in image quality at the edges. The proposed algorithm improves image deinterlacing quality with respect to missing pixels in the edge region. When interpolating the missing pixels, characteristics or features of the region around the pixel are interpolated first. The proposed algorithm is tested with five test images which have both uniform surface and edges. The subjective and objective performance of the proposed algorithm shows greater improvement in the edge quality when correlated with the traditional deinterlacing methods. The designed fuzzy rules become an attractive solution for the betterment of the quality of edges as much as possible. The result reveals that the proposed algorithm provides greater robustness to contrast and lighting variations.

Intelligent color image restoration system using cognitive fuzzy concept

Deinterlacing is the procedure of reworking interlaced video signal into a non-interlaced format. The interlaced video signal is generally used in both of analog TV and HDTV with 1080i. The deinterlacing method was designed to solve a legacy issue, therefore the interlaced signal which was requested by analog TV must be trans-formed to be displayed on HDTV. This paper introduces fuzzy concept-based deinterlacing method and shows how it can be executed on various test set. The proposed method includes weight calculation stage and deinterlacing implementation stage. In weight calculation stage, I compute correct weight values on each edge direction, which are calculated using fuzzy concepts. In deinterlacing implementation stage, the computed weight values are used to obtain final deinterlacing pixels by multiplying them with candidate existent pixels. The presented method provides outstanding subjective and objective results in various data set.

Weighted aggregation and fuzzy-concept-guided signal resemblance and expansion for video format conversion

In this paper, we propose an efficient deinterlacing method for HDTV that preserves image structures, edges, and details. In the human visual system, the eyes are more sensitive to high-frequency information such as edge details than low-frequency information such as image background. Therefore, averaging low-pass filter results is not effective for image enhancement. The proposed method is a weighted filtering approach that generates a half-pixel 9-by-9 edge-based line average window. We also propose pixel-resemblance- and pixel-expansion-based fuzzy weights, which are assigned using a triangular membership function. Compared to conventional format conversion methods, the proposed method outperforms all benchmarks in terms of both objective and subjective qualities.

디인터레이싱을 위한 C4.5 분류화 기법의 적용 및 구현

디인터레이싱이란 두 개의 필드(짝수 필드 및 홀수 필드)로 구성된 인터레이스 영상을 프로그레시브 영상으로 변환하는 기술이다. 이는 크게 공간영역에서의 디인터레이싱과 시간영역에서의 디인터레이싱 기술로 나뉠 수 있다. 공간영역에서의 기법은 하나의 독립적인 필드만을 사용하여 디인터레이싱을 수행하는 것으로 하드웨어의 구성은 용이하지만 디인터레이싱 대상 화소의 정보가 해당 필드에 존재하지 않는 경우 화질 열화가 발생할 수 있다. 반면 시간영역에서의 기법은 메모리 사용량이 높고 하드웨어의 구성이 어렵지만 보다 높은 객관적 화질을 얻을 수 있다. 하지만 움직임 추정이 잘못된 경우 현저한 화질열화가 발생한다. 제안하는 방법은 공간영역에서의 디인터레이싱 기법으로 대상화소 주변의 통계적 특성에 따라 서로 다른 기법을 사용하여 디인터레이싱을 수행한다. 이 과정에서 최적의 디인터레이싱 방법의 선택을 위해 엔트로피 기반의 대표적인 분류 알고리즘인 C4.5 알고리즘을 적용한다. 실험결과 제안하는 알고리즘은 이전의 여러 방법들과 비교하여 높은 객관적 화질을 가지는 것을 볼 수 있었으며, 주관적 화질 또한 상대적으로 유사하거나 높은 것을 볼 수 있었다. Deinterlacing is a method to convert interlaced video, including two fields (even and odd), to progressive video. It can be divided into spatial and temporal methods. The deinterlacing method in the spatial domain can easily be hardware-implemented, but yields image degradation if information about the deinterlaced pixel does not exist in the same field. On the other hand, the method in the temporal domain yields a deinterlaced image with higher quality but uses more memory, and hardware implementation is more difficult. Furthermore, the deinterlacing method in the temporal domain degrades image quality when motion is not estimated properly. The proposed method is for deinterlacing in the spatial domain. It uses several deinterlacing methods according to statistical characteristics in neighboring pixel locations. In this procedure, the proposed method uses the C4.5 algorithm, a typical classification algorithm based on entropy for choosing optimal methods from among the candidates. The simulation results show that the proposed algorithm outperforms previous deinterlacing methods in terms of objective and subjective image quality.

수평방향의 3차 스플라인 보간과 ELA을 이용한 개선된 디인터레이싱 연구

본 논문에서는 하나의 필드 만을 사용하여 비월 주사 영상을 순차 주사 영상으로 변환하는 개선된 디인터레이싱 방법을 제안한다. 우선, 구하고자 하는 화소의 위와 아래 각각 4개 화소를 이용하여 화소 사이의 3개 점을 3차 스플라인 보간을 적용하여 구한다. 알려진 위와 아래 화소값과 3차 스플라인 보간으로 구한 위와 아래 화소 사이의 보간값으로 부터 구하고자 하는 화소를 중심으로 9방향으로 세분화하여 각각의 상관 관계를 구한다. 구하고자 하는 화소에서 위 아래 상관 관계가 가장 최소가 되는 방향으로 에지의 방향성을 예측하고 화소 사이의 상대적인 값이 최소가 되는 방향으로 화소의 방향성을 결정한다. 결정된 방향의 위 아래 화소값의 평균으로 화소값을 계산한다. 모의실험 결과 기존의 디인터레이싱 방법에 비해 약간의 소요 시간이 걸리는 문제점은 있으나 CPU의 개선으로 시간적 제약은 없다고 판단된다. 기존의 연구된 여러 디인터레이싱 방법들과의 비교를 통하여 본 논문에서 제시된 방법이 주관적 화질에 있어서 개선이 있었으며 객관적 화질에 있어서도 정량적으로 PSNR이 0.1~0.5dB 향상됨을 보이고 있다. This paper proposes an improved de-interlacing method that converts interlaced images into progressive images from one field. First, it calculates inter-pixel values applying third-order spline interpolation for the horizontal direction from four upper lower pixel values of missing pixels. From inter-pixel values obtained from spline interpolation and upper lower pixels with value, the proposed method makes an accurate estimate of the direction by applying the correlation between upper and lower pixels. The correlation between upper and lower pixels is calculated in nine directions of a missing pixel by using values obtained from spline interpolation and pixels with value. The direction of an edge is determined as the direction in which the correlation between upper and lower pixels is at its minimum. Thus, a missing pixel is calculated by taking the average of upper lower pixels obtained from the predicted direction of an edge. From the simulation results, there are problems in that it takes a bit more time for processing, but it is expected that the time problem will be improved by increasing CPU processing speed. As for image quality, it is shown that the proposed method improves both subjective and objective image quality and quantitatively improves picture signal-to-noise ratio (PSNR) in the range between 0.1 dB to 0.5 dB, as compared with previously presented de-interlacing methods.

An HVS-inspired video deinterlacer based on visual saliency

Video deinterlacing is a technique wherein the interlaced video format is converted into progressive scan format for nowadays display devices. In this paper, a spatial saliency-guided motion-compensated deinterlacing method is proposed which accounts for the properties of the Human Visual System (HVS): our algorithm classifies the field according to its texture and viewer’s region of interest and adapts the motion estimation and compensation, as well as the saliency-guided interpolation to ensure high-quality frame reconstruction. Two different saliency models, namely the graph-based visual saliency (GBVS) model and the spectral residual visual saliency (SRVS) model, have been studied and compared in terms of visual quality performances as well as computational complexity. The experimental results on a great variety of video test sequences show significant improvement of reconstructed video quality with the GBVS-based proposed method compared to classical motion-compensated and adaptive deinterlacing techniques, with up to 4.5 dB gains in terms of PSNR. Simulations also show that the SRVS-based deinterlacing process can result to significant reductions of complexity (up to 25 times a decrease of the computation time compared with the GBVS-based method) at the expense of a PSNR decrease.

Efficient Adaptive Deinterlacing Algorithm Using Bilateral Filter

This paper introduces an efficient intra-field deinterlacing algorithm using bilateral filter while considering both closeness and similarity between the interpolated pixel and the neighbor pixels. Instead of estimating the edge orientations via limited candidate directions as previous intra-field deinterlacing methods, we propose an adaptive spatial locality- and similarity-based deinterlacing method. Many deinterlacing methods described in the literature only consider the similarity between neighbor pixels and interpolated pixels; however awareness of distance correlation and spatial locality has not yet been studied. Experimental results indicate that the proposed scheme outperforms a number of existing approaches, in terms of both the objective and subjective performance.

Deinterlacing algorithm using gradient‐guided interpolation and weighted average of directional estimation

This study presents a novel intra-field deinterlacing algorithm using gradient-guided interpolation (GGI) and weighted average of directional estimation (WADE). In this method, the authors classify each missing pixel into two categories according to different local region gradient features. After the classification of the local region, one of the two different interpolation methods is chosen for the corresponding region. The two interpolation methods are GGI method and the interpolation method using WADE. The average weighted method is used for smooth or texture regions because of its high interpolation performance by adopting the directional detail information among the neighbouring pixels. On the other hand, the GGI method is used for complex regions with strong or weak edges since it has high edge-preserving ability because of considering not only the similarity of pixel intensities, but also the pixel gradient features. The WADE and GGI methods are the main contributions of this study. Compared with the traditional deinterlacing methods, the proposed method improves both the objective and subjective qualities of the interpolated images. In addition, the proposed method possesses a very low computational burden compared with most intra-field deinterlacing methods.

Spatiotemporal video deinterlacing using control grid interpolation

With the advent of progressive format display and broadcast technologies, video deinterlacing has become an important video-processing technique. Numerous approaches exist in the literature to accomplish deinterlacing. While most earlier methods were simple linear filtering-based approaches, the emergence of faster computing technologies and even dedicated video-processing hardware in display units has allowed higher quality but also more computationally intense deinterlacing algorithms to become practical. Most modern approaches analyze motion and content in video to select different deinterlacing methods for various spatiotemporal regions. We introduce a family of deinterlacers that employs spectral residue to choose between and weight control grid interpolation based spatial and temporal deinterlacing methods. The proposed approaches perform better than the prior state-of-the-art based on peak signal-to-noise ratio, other visual quality metrics, and simple perception-based subjective evaluations conducted by human viewers. We further study the advantages of using soft and hard decision thresholds on the visual performance.

디인터레이싱을 위한 방향지향 보간법의 개선

ABSTRACT This paper presents, a deinterlacing method by improving the Direction-Oriented Interpolation (DOI) technique. The technique is considered to be a very strong tool for intrafield-based deinterlacing. However, DOI has some problems such as long processing time, wrong edge detection in periodic pattern. To remedy this problem, we replace the full search in DOI by a two-step search to reduce processing time and introduces two additional processes to improve image quality. In the proposed method, the spatial direction vectors (SDVs) misread data are reconsidered to prevent them utilizing in the next interpolation step, resulting in an accurate deinterlacing method. We conduct experiments with ISO experimental images to compare the proposed method with the existing methods including line evarage (LA), edge-based line averaging (ELA), DOI, selective deinterlacing algorithm (SDA). Expeimental results show the proposed method gives better performance in objective and subjective quality than existing deinterlacing methods.

GPU-Parallel Implementation of the Edge-Directed Adaptive Intra-Field Deinterlacing Method

This paper proposes an efficient GPU-based massively parallel implementation of the edge-directed adaptive intra-field deinterlacing method which interpolates the missing pixels based on the deinterlaced covariance estimated from the interlaced covariance according to the geometric duality between the interlaced and the deinterlaced covariance. Although the edge-directed adaptive intra-field deinterlacing method can obtain better visual quality than conventional intra-field deinterlacing methods, the time-consuming computation is usually the bottleneck of this deinterlacing method. In order to tackle the problem, Graphics Processing Units (GPUs), as opposed to traditional CPU architectures, are better candidates to speed up the computation process. The proposed method interpolates more than one missing pixel at a time in order to gain a significant speedup compared to the case of interpolating just one missing pixel at a time. Experimental results show that we obtained a speedup of 94.6 × when the I/O transfer time was taken into account, compared to the original single-threaded C CPU code with the -O2 compiling optimization.

Edge-adaptive spatial video de-interlacing algorithms based on fuzzy logic

Since the human visual system is especially sensitive to image edges, edge-dependent spatial interpolators have been proposed in literature as a means of successfully restoring edges while avoiding the staircase effect of linear spatial algorithms. This paper addresses the application of video de-interlacing, which constitutes an indispensable stage in video format conversion. Classic edge-adaptive de-interlacing algorithms introduce annoying artifacts when the edge directions are evaluated incorrectly. This paper presents two ways of exploiting fuzzy reasoning to reinforce edges without an excessive increase in computational complexity. The performance of the proposed algorithms is analyzed by deinterlacing a wide set of test sequences. The study compares the two proposals both with each other and with other edge-adaptive de-interlacing methods reported in the recent literature.

Quality-efficient syntax element-based de-interlacing method for H.264-coded video sequences with various resolutions

In this paper, we propose a novel quality-efficient de-interlacing method for H.264-coded video sequences with various resolutions. In the proposed method, using the syntax elements provided in H.264 bitstreams, four new and efficient strategies are delivered for inter-coded macroblocks to improve the quality of de-interlaced video sequences as well as alleviate the error propagation side effect. Based on the real and generated interlaced video sequences with various common resolutions, experimental results demonstrate the proposed de-interlacing method achieves better quality in terms of both objective and subjective measures when compared with the recently published method by Dong and Ngan.

A De-Interlacing Method for Video Surveillance in Intelligent Substation

A de-interlacing method for video surveillance in intelligent substation is presented in this paper. Firstly, a framework for performance evaluation is proposed so that the subjective performance of de-interlacing will not matter in applications such as video compression. Then pre-filtering is performed to remove potential noise, and a hierarchical structure is introduced for same-parity field pictures or opposite-parity field pictures to get more reliable optical flow results for compression. Finally the optical flow results are used to improve the accuracy of block-based methods for each partition of the to-be-interpolated image and the final motion vectors are determined. Experimental results show that the proposed solution can achieve both better peak signal-to-noise ratio (PSNR) performance and visual effects.

Deinterlacing Using Taylor Series Expansion and Polynomial Regression

This paper introduces an efficient intra-field deinterlacing algorithm that is based on Taylor series expansion and polynomial regression. In order to estimate the value of an interpolated point using the given data, we rely on a generic local approximation function around this point for estimating the missing data. The well known $N$ -term Taylor series expansion is regarded as a local representation of the approximation function, and we use polynomial regression to find the optimal local approximation of the function. Instead of estimating the edge orientations as in previous intra-field deinterlacing methods, such as an edge-based line average, we propose an efficient deinterlacing method, which does not consider directional difference measurements that use limited candidate directions. When compared with existing deinterlacing algorithms, the proposed algorithm improves the peak signal-to-noise-ratio while maintaining a high efficiency.

Interpolation scheme based on the Bayes classifier

Our purpose is to present an intrafield deinterlacing method using the Bayes classifier. The conventional intrafield deinterlacing methods interpolate the pixel along the local edge direction, but they yield interpolation errors when the local edge direction is determined to be wrong. On the basis of the Bayes classifier, the proposed algorithm performs region-based deinterlacing. The proposed algorithm utilizes an input feature vector that includes five directional correlations, which are used to extract the characteristics of the local region, to classify the local region. After the classification of the local region, one of the three simple interpolation methods, which possesses the highest probability to be used among the three, is chosen for the corresponding local region. In addition, we categorized the range of the feature vector to reduce the computational complexity. Simulation results show that the proposed Bayes classifier-based deinterlacing method minimizes interpolation errors. Compared to the traditional deinterlacing methods and Wiener filter-based interpolation method, the proposed method improves the subjective quality of the reconstructed image, and maintains a higher peak signal-to-noise ratio level.

Iterative second-order derivative-based deinterlacing algorithm

In this paper, we introduce an iterative second-order derivative-based intra-field deinterlacing algorithm (ISDD). The proposed ISDD is composed of two steps: (1) preprocessing by a fast second-order derivative-based deinterlacing algorithm (FSDD), (2) iteratively refining the interpolated pixel by minimizing a mass second-order function. The function includes three components: the derivative difference, directional derivative, and isophote smoothing. The derivative difference is used to detect any intensity discontinuity in an image; the directional derivative is a supplement to the derivative difference to avoid oversmoothing; and isophote smoothing can reduce artifacts. The proposed ISDD can preserve image features and reduce the artifacts caused by conventional deinterlacing methods. Compared with existing deinterlacing algorithms, the proposed algorithm improves the peak signal-to-noise ratio while maintaining high efficiency.

Deinterlacing with reduced flickering

Deinterlacing has been used to convert interlaced video signals into progressive video signals. Conventional deinterlacing methods with low levels of complexity can lead to deteriorating video quality, particularly in edge areas. Another problematic artifact is flickering, which occurs more frequently as the video size increases. For example, this flickering artifact is more frequently observed in high-definition television signals. In this paper, we propose a flickering artifact reduction algorithm for deinterlacing, which noticeably increases the overall visual quality.