Interlaced Video Research Articles

Rethinking Deinterlacing for Early Interlaced Videos

In recent years, high-definition restoration of early videos have received much attention. Real-world interlaced videos usually contain various degradations mixed with interlacing artifacts, such as noises and compression artifacts. Unfortunately, traditional deinterlacing methods only focus on the inverse process of interlacing scanning, and cannot remove these complex and complicated artifacts. Hence, this paper proposes an image deinterlacing network (DIN), which is specifically designed for joint removal of interlacing mixed with other artifacts. The DIN is composed of two stages, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , a cooperative vertical interpolation stage for splitting and fully using the information of adjacent fields, and a field-merging stage to perceive movements and suppress ghost artifacts. Experimental results demonstrate the effectiveness of the proposed DIN on both synthetic and real-world test sets.

Multiframe Joint Enhancement for Early Interlaced Videos.

Early interlaced videos usually contain multiple and interlacing and complex compression artifacts, which significantly reduce the visual quality. Although the high-definition reconstruction technology for early videos has made great progress in recent years, related research on deinterlacing is still lacking. Traditional methods mainly focus on simple interlacing mechanism, and cannot deal with the complex artifacts in real-world early videos. Recent interlaced video reconstruction deep deinterlacing models only focus on single frame, while neglecting important temporal information. Therefore, this paper proposes a multiframe deinterlacing network joint enhancement network for early interlaced videos that consists of three modules, i.e., spatial vertical interpolation module, temporal alignment and fusion module, and final refinement module. The proposed method can effectively remove the complex artifacts in early videos by using temporal redundancy of multi-fields. Experimental results demonstrate that the proposed method can recover high quality results for both synthetic dataset and real-world early interlaced videos. At the same time, the method also won the first place in the MSU Deinterlacer Benchmark. The code is available at: https://github.com/anymyb/MFDIN.

Review on Deinterlacing Algorithms

Interlacing is a commonly used technique for doubling the perceived frame rate without adding bandwidth in television broadcasting and video recording. During playback, however, it exhibits disturbing visual artifacts such as flickering and combing. As a result in modern display devices, video deinterlacing is used where the interlaced video format is converted to progressive scan format to overcome the limitations of interlaced video. This conversion is achieved through interpolating interlaced video. Current deinterlacing approaches either neglect temporal information for real-time performance but poor visual quality, or estimate motion for better deinterlacing but higher computational cost. This paper focuses on surveying the deinterlacing algorithms which apply both spatial and temporal-based methods and focus on different aspects of both motion-adaptive, non-motion adaptive, and the time complexity through these implementations.

Intelligent color image restoration system using cognitive fuzzy concept

Deinterlacing is the procedure of reworking interlaced video signal into a non-interlaced format. The interlaced video signal is generally used in both of analog TV and HDTV with 1080i. The deinterlacing method was designed to solve a legacy issue, therefore the interlaced signal which was requested by analog TV must be trans-formed to be displayed on HDTV. This paper introduces fuzzy concept-based deinterlacing method and shows how it can be executed on various test set. The proposed method includes weight calculation stage and deinterlacing implementation stage. In weight calculation stage, I compute correct weight values on each edge direction, which are calculated using fuzzy concepts. In deinterlacing implementation stage, the computed weight values are used to obtain final deinterlacing pixels by multiplying them with candidate existent pixels. The presented method provides outstanding subjective and objective results in various data set.

A rapid local backlight dimming method for interlaced scanning video

AbstractThis paper presents a rapid local backlight dimming method for interlaced video to improve the efficiency of displaying interlaced video. In the proposed method, we combine the line average deinterlacing algorithm with the local backlight dimming algorithm. In order to reduce the computational complexity and the backlight power consumption, three novel methods are proposed. The first is to add the sparse method to process the deinterlaced image. Second, the improved dynamic threshold method is used to calculate the dimming factor. And third, the odd field and even field of the same frame use the same backlight brightness to do backlight dimming. The simulation results show that the proposed method can reduce the computational complexity and power consumption, and the experiment results verify that the method can provide high display effect.

拡大されたインタレース映像の通常インタレース状態への復元方法

拡大されたインタレース映像の通常インタレース状態への復元方法

Reconstructing Interlaced High-Dynamic-Range Video Using Joint Learning

For extending the dynamic range of video, it is a common practice to capture multiple frames sequentially with different exposures and combine them to extend the dynamic range of each video frame. However, this approach results in typical ghosting artifacts due to fast and complex motion in nature. As an alternative, video imaging with interlaced exposures has been introduced to extend the dynamic range. However, the interlaced approach has been hindered by jaggy artifacts and sensor noise, leading to concerns over image quality. In this paper, we propose a data-driven approach for jointly solving two specific problems of deinterlacing and denoising that arise in interlaced video imaging with different exposures. First, we solve the deinterlacing problem using joint dictionary learning via sparse coding. Since partial information of detail in differently exposed rows is often available via interlacing, we make use of the information to reconstruct details of the extended dynamic range from the interlaced video input. Second, we jointly solve the denoising problem by tailoring sparse coding to better handle additive noise in low-/high-exposure rows, and also adopt multiscale homography flow to temporal sequences for denoising. We anticipate that the proposed method will allow for concurrent capture of higher dynamic range video frames without suffering from ghosting artifacts. We demonstrate the advantages of our interlaced video imaging compared with the state-of-the-art high-dynamic-range video methods.

디인터레이싱을 위한 C4.5 분류화 기법의 적용 및 구현

디인터레이싱이란 두 개의 필드(짝수 필드 및 홀수 필드)로 구성된 인터레이스 영상을 프로그레시브 영상으로 변환하는 기술이다. 이는 크게 공간영역에서의 디인터레이싱과 시간영역에서의 디인터레이싱 기술로 나뉠 수 있다. 공간영역에서의 기법은 하나의 독립적인 필드만을 사용하여 디인터레이싱을 수행하는 것으로 하드웨어의 구성은 용이하지만 디인터레이싱 대상 화소의 정보가 해당 필드에 존재하지 않는 경우 화질 열화가 발생할 수 있다. 반면 시간영역에서의 기법은 메모리 사용량이 높고 하드웨어의 구성이 어렵지만 보다 높은 객관적 화질을 얻을 수 있다. 하지만 움직임 추정이 잘못된 경우 현저한 화질열화가 발생한다. 제안하는 방법은 공간영역에서의 디인터레이싱 기법으로 대상화소 주변의 통계적 특성에 따라 서로 다른 기법을 사용하여 디인터레이싱을 수행한다. 이 과정에서 최적의 디인터레이싱 방법의 선택을 위해 엔트로피 기반의 대표적인 분류 알고리즘인 C4.5 알고리즘을 적용한다. 실험결과 제안하는 알고리즘은 이전의 여러 방법들과 비교하여 높은 객관적 화질을 가지는 것을 볼 수 있었으며, 주관적 화질 또한 상대적으로 유사하거나 높은 것을 볼 수 있었다. Deinterlacing is a method to convert interlaced video, including two fields (even and odd), to progressive video. It can be divided into spatial and temporal methods. The deinterlacing method in the spatial domain can easily be hardware-implemented, but yields image degradation if information about the deinterlaced pixel does not exist in the same field. On the other hand, the method in the temporal domain yields a deinterlaced image with higher quality but uses more memory, and hardware implementation is more difficult. Furthermore, the deinterlacing method in the temporal domain degrades image quality when motion is not estimated properly. The proposed method is for deinterlacing in the spatial domain. It uses several deinterlacing methods according to statistical characteristics in neighboring pixel locations. In this procedure, the proposed method uses the C4.5 algorithm, a typical classification algorithm based on entropy for choosing optimal methods from among the candidates. The simulation results show that the proposed algorithm outperforms previous deinterlacing methods in terms of objective and subjective image quality.

An HVS-inspired video deinterlacer based on visual saliency

Video deinterlacing is a technique wherein the interlaced video format is converted into progressive scan format for nowadays display devices. In this paper, a spatial saliency-guided motion-compensated deinterlacing method is proposed which accounts for the properties of the Human Visual System (HVS): our algorithm classifies the field according to its texture and viewer’s region of interest and adapts the motion estimation and compensation, as well as the saliency-guided interpolation to ensure high-quality frame reconstruction. Two different saliency models, namely the graph-based visual saliency (GBVS) model and the spectral residual visual saliency (SRVS) model, have been studied and compared in terms of visual quality performances as well as computational complexity. The experimental results on a great variety of video test sequences show significant improvement of reconstructed video quality with the GBVS-based proposed method compared to classical motion-compensated and adaptive deinterlacing techniques, with up to 4.5 dB gains in terms of PSNR. Simulations also show that the SRVS-based deinterlacing process can result to significant reductions of complexity (up to 25 times a decrease of the computation time compared with the GBVS-based method) at the expense of a PSNR decrease.

Non-iterative phase hologram computation for low speckle holographic image projection.

Phase-only spatial light modulators (SLMs) are widely used in holographic display applications, including holographic image projection (HIP). Most phase computer generated hologram (CGH) calculation algorithms have an iterative structure with a high computational load, and also are prone to speckle noise, as a result of the random phase terms applied on the desired images to mitigate the encoding noise. In this paper, we present a non-iterative algorithm, where simple Discrete Fourier Transform (DFT) relations are exploited to compute phase CGHs that exactly control half of the desired image samples (those on even - or odd - indexed rows - or columns) via a single Fast Fourier Transform (FFT) and trivial arithmetic operations. The encoding noise appearing on the uncontrolled half of the image samples is reduced by the application of structured, non-random initial phase terms so that speckle noise is also kept low. High quality reconstructions are obtained under temporal averaging of several SLM frames. Interlaced video within half of the addressable image area is readily deliverable without frame rate division. Our algorithm provides about 6X and 20X reduction in computational cost compared to IFTA and FIDOC algorithms, respectively. Simulations and experiments verify that the algorithm constitutes a promising option for real-time computation of phase CGHs.

Efficient deinterlacing method using simple edge slope tracing

This paper presents a low-complexity interpolation method that minimizes image quality losses at edges, which are easily perceivable by the human eye. Deinterlacing, which converts an interlaced video into a progressive video, is a problem in image interpolation that doubles the number of vertical lines. Applying averaging, or any linear algorithm, achieves time-efficient deinterlacing but produces artifacts. However, applying other complex methods tends to reduce unwanted artifacts but at the cost of high compu- tation time. The proposed deinterlacing scheme is based on an algorithm called edge slope tracing which simply predicts the slope on the basis of information on adjacent slopes. Predicted slopes are used to perform deinterlacing in slope-based line averaging. The simulation results show that this scheme provides better results and reduces complexity compared to conventional state-of-the-art algorithms. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original

Fast Text Processing for Interlaced Sports or News Videos

Interlaced scanning has been widely used as a trade-off solution between picture quality and transmission bandwidth since the invention of television. During the past decades, various interlaced-to-progressive conversion algorithms have been proposed to improve subjective quality or coding efficiency. However, almost all the researchers concentrate on general cases, without making full use of specific application scenarios. Based on extensive investigations, eliminating visual artifacts in areas of subtitles and station captions for interlaced sports and news videos is still an unsolved problem, which will be addressed in this paper. Firstly, motion estimation is performed between field pictures. Secondly, text edge detection is proposed for sports and news videos. Finally, different processing strategies are applied to text regions and non-text regions. Experimental results show that the proposed method can generate much better text content than existing algorithms. In addition, it is quite stable for non-text parts.

Motion-compensated TV-reconstruction-based video deinterlacing

Interlaced video acquisition and transmission, used as a simple way of increasing motion quality in video sequences, all by respecting the bandwidth constraints of progressive format representation, has been extensively used by analogue transmission systems. However, considering that nowadays the television sets and the flat screens can display only progressive format input, ensuring a smooth and high quality interlaced to progressive format conversion represents a challenging task. If till present, interpolation-based like solutions have been preferred for performing this conversion, in this paper we use an inverse problem formulation for video deinterlacing and propose a motion-compensated total-variation-minimisation-based reconstruction algorithm for solving it. A first recovery of the progressive frame is obtained using a contour-preserving interpolation method and further used for triggering a bidirectional motion-compensated prediction for the current field. A motion-compensated residual is calculated as difference between the current field and the projection of its temporal prediction using the same parity sampling matrix. This field residual is further reconstructed using a total-variation regularisation method and added back to the motion-compensated prediction to form the final progressive frame. As shown by the experimental results, the proposed deinterlacing method presents high quality progressive frame reconstructions compared to classical deinterlacing approaches.

Quality-efficient syntax element-based de-interlacing method for H.264-coded video sequences with various resolutions

In this paper, we propose a novel quality-efficient de-interlacing method for H.264-coded video sequences with various resolutions. In the proposed method, using the syntax elements provided in H.264 bitstreams, four new and efficient strategies are delivered for inter-coded macroblocks to improve the quality of de-interlaced video sequences as well as alleviate the error propagation side effect. Based on the real and generated interlaced video sequences with various common resolutions, experimental results demonstrate the proposed de-interlacing method achieves better quality in terms of both objective and subjective measures when compared with the recently published method by Dong and Ngan.

Deinterlacing of depth-image-based three-dimensional video for a depth-image-based rendering system

Wide bandwidth is usually required to transmit depth-image-based three-dimensional (3-D) video. An effective way to reduce the bandwidth requirement is converting the progressive 3-D video to an interlaced one. Since the interlaced video cannot be displayed on an end directly, the video must be deinterlaced. A deinterlacing algorithm for depth-image-based 3-D video is proposed. The algorithm first performs deinterlacing on an interlaced reference image and its interlaced depth map by an intrafield interpolation algorithm based on direction. The interpolation algorithm works well in eliminating jagged effect by adaptively selecting the direction of interpolation; it also provides higher peak SNR in comparison with other commonly used algorithms on interpolating interlaced depth map. Then, 3-D image warping is applied to the progressive reference image and its depth map to generate a progressive destination image. Finally, hole-filling is performed on the destination image to get the virtual view. A method to modify the algorithm to facilitate hardware implementation is also discussed. Experimental results show that the proposed deinterlacing algorithm can provide progressive virtual views with quite good image quality and the performance is robust.

Optimal Analysis Window Size Investigation in Accord with Number of Edge Direction

The display monitors have been developed rapidly. The traditional broadcasting systems employed interlaced scanning format to hide motion artifacts. The procedure of converting interlaced video into a progressive formatted video called frame rate conversion. However, due to the variety of progressive scanning formats, all interlaced scanned contents must be transformed progressive format. In this paper, we study optimal number of window size for frame format conversion method. The objective and subjective performance are shown in performance analysis section.

Restoring Video Bitstream Integrity by Resolving Field Reversal and Mixed Pulldown Errors

The quality of the decoded video is affected by errors occurring at various stages of the production chain. In this paper, the errors occurring in the editing layer, due to the coexistence of different video standards in the broadcast market are addressed. The problems investigated are field reversal and mixed pulldown. Field reversal is caused when the interlaced video fields are not shown in the same order as they were captured. This results in a shaky video display, as the fields are not displayed in a chronological order. Whereas, mixed pulldown artefacts occur when the video frame-rate is up-sampled and down-sampled. These errors destroy the integrity of the bitstream and continue to corrupt the videos with which it is edited. The current solution is to visually inspect the video stream. This paper investigates these errors and describes a new technique that eliminates the visual inspection process and automatically restores the bitstream integrity. The methods are based on the principles of motion flow estimation and image correlation. The new algorithms were designed for real-time implementation and outperform existing techniques with superior performance in quality and speed.

Special issue on “Challenges, trends and solutions for 3-D imaging techniques”

We are pleased to present Challenges, Trends and Solutions for 3-D Imaging Techniques Special Issue. The interest in 3-D imaging systems and applications has been growing at a fast pace. Applications ranging from entertainment to computer vision, including augmented reality and teleoperation systems are tending to increasingly rely on 3-D imaging technologies. The papers in this special issue span awide range of problems arising in the development of algorithms for 3-D applications, technologies and services. Among the topics addressed are 3-D video transmission, quality assessment of 3-D content, virtual reality systems, production audit systems and 3-D reconstruction techniques. For this Special Issue we have received 13 submissions. Each one has undergone rigorous peer-review, and based on these reviews we have selected 7 papers for publication. The first two papers are on frame-compatible interpolation techniques for 3-D video transmission. The first one of them, Mode selective interpolation for stereoscopic 3D video in frame-compatible top-bottom packing, by C. S. Won, proposes a frame-compatible topand-bottom packing with a horizontal line offset. It exploits the horizontal parallax between the left and right viewpoints improving the quality of the reconstructed video when compared to conventional interpolation. The second paper,Analysis of frame-compatible subsampling structures for efficient 3DTV broadcast, byR. Mohedano et al., deals with alternative subsampling schemes for 3-D framecompatible broadcasting. It analyzes and compares different spatial multiplexing arrangementswith thewell known side-by-side (SbS) and top-and-bottom (TaB) formats. The authors propose, for progressive and interlaced video, schemes for frame-compatible packing that present higher objective quality for the reconstructed video than the SbS and TaB structures.

Hardware Implementation of Back-Propagation Neural Networks for Real-Time Video Image Learning and Processing

This paper presents a digital hardware Back- Propagation (BP) model for real-time learning in the field of video image processing. The model is a layer parallel architecture with a 16-bit fixed point specialized for video image processing. We have compared our model with a standard BP model that used a double-precision floating point. Simulation results show that our model has equal capabilities to those of the standard BP model. We have implemented the model on an FPGA board that we originally designed and developed for experimental use as a platform for real-time video image processing. Experimental results show that our model performed 100,000 epochs/frame learning that corresponds to 90 MCUPS and was able to test all pixels on interlace video images.

Deinterlacing with reduced flickering

Deinterlacing has been used to convert interlaced video signals into progressive video signals. Conventional deinterlacing methods with low levels of complexity can lead to deteriorating video quality, particularly in edge areas. Another problematic artifact is flickering, which occurs more frequently as the video size increases. For example, this flickering artifact is more frequently observed in high-definition television signals. In this paper, we propose a flickering artifact reduction algorithm for deinterlacing, which noticeably increases the overall visual quality.