Error Concealment Method Research Articles

Error concealment Pada Transmisi Video H.264/AVC dengan Frequency Selective Extrapolation

Video transmission is encoded by block-based techniques such as MPEG in sensitive environment. It’s very susceptible to noise which can cause block losses and even missing frames during transmission. A technique to overcome this problem is applied Error concealment (EC) on the decoder. There are two approaches used in this EC, spacial EC which utilizes information around the image and temporal EC by utilizing motion information. The method of EC that used in this research is Frequency Selective Extrapolation (FSE). The video is encoded by the H.264 / AVC standard (MPEG-Part10). The isolated block losses was added to the video as an error simulation, then applied the EC-FSE method on the decoder. There are two method of EC-FSE used, FSE-2D (two dimensions) and FSE-3D (three dimensions). The measurement results were observed by the PSNR and MOS values. The simulation results show that H.264 / AVC video concealed with FSE-3D has a better performance than FSE-2D.

A review of temporal video error concealment techniques and their suitability for HEVC and VVC

Despite of the recent progresses in reliable and high bandwidth communication, packet loss is still probable and needs special attention in real-time video streaming applications. Congestion and bit error rate, which sometimes are more than the protection capability of the channel codes, are the sources of packet loss in video communication. One common approach to deal with video packet loss is to use error concealment techniques, which estimate the non-received data as close as possible to the actual data. This article reviews the temporal video error concealment methods that have been developed over the past 30 years. The techniques are categorized into 8 groups, and the methods are covered with enough details. The strengths and weaknesses of the 8 groups are also tabulated, and some suggestions for future work and open areas for research are provided.

An Error Resilience and Concealment Method for Line-based Wavelet Image Compressions

Recently, some wavelet-based line compression methods for low-cost high-resolution image transmission have been studied. With these line compression methods, which are a combination of the multilevel discrete wavelet transform, predictive coding, and entropy coding, communications errors on the transmission channel can cause an image quality–degradation phenomenon of various line shapes. In this paper, we analyze image degradation patterns of wavelet-based hybrid line compression techniques caused by communications errors. Then, we propose an image quality–improvement method including a low-cost resynchronization technique by using a periodic reset to prevent the error propagation phenomenon of predictive coding, as well as an error concealment technique that uses only one additional line memory in the variable reference line–based decoder to improve degraded image quality. Experimental results showed that the proposed method achieved better image quality than the conventional techniques at the small expense of compression ratio while satisfying low-cost design conditions pursued by existing line compression systems.

Modified Boundary Matching algorithm for Error detection and Error Concealment for Video Communication

As the demand of video transmission over communication network has grown rapidly, the data compression and error correction in video processing have shown significant improvement day by day. When the error occurs in a single frame, the visual quality of the subsequent frames gets degraded due to error propagation. Thus, the error control techniques are required for the recovery. Concealment of error at the receiver (decoder) side feats the spatial and temporal characteristics of the frame. Without the requirement of the extra bandwidth and retransmission delay, it enhances the quality of the reconstructed video. However, the output of the error concealment may get affected if the error located before is misleading. Thus error detection also plays an important role while reconstructing the video. However, the output of the error concealment may get affected if the error located before is misleading. This paper proposes error detection and concealment approach for the recovery of lost Macro Block (MB) in video. The spatio-temporal techniques has been used for the error detection followed by the MB type decision applied for classifying the damaged macro block .For the concealment method a new method i.e. Modified Spatio-Temporal Boundary Matching Algorithm (MSTBMA) has been proposed. The proposed work is compared with various existing method for spatial and temporal error concealment. The comparison has been done for various types of error such as block error (single, multiple), burst error and random error generated by the software. Performance is improves in terms of PSNR and visual quality by considering the type of lost MB.

Error Concealment-Aware Encoding for Robust Video Transmission

In this paper an error concealment (EC)-aware encoding scheme is proposed to improve the quality of decoded video in broadcast environments subject to transmission errors and data loss. The proposed scheme is based on a scalable coding approach where the best EC methods to be used at the decoder are optimally determined at the encoder and signalled to the decoder through supplemental enhancement information messages. Such optimal EC modes are found by simulating transmission losses, followed by a Lagrangian optimization of the signalling rate—EC distortion cost. A generalized saliency-weighted distortion is used and the residue between coded frames and their EC substitutes is encoded using a rate-controlled enhancement layer. When data loss occurs the signalling information is used by the decoder to improve the reconstruction quality. The simulation results show that the proposed method achieves consistent quality gains in comparison with other reference methods and previous works. Using only the EC mode signalling, i.e., without any residue transmitted in the enhancement layer, an average PSNR gain up to 2.95 dB is achieved, while using the full EC-aware scheme, i.e., including residue encoded in the enhancement layer, the proposed scheme outperforms other comparable methods, with a PSNR gain up to 3.79 dB.

Research on anti-conflict extraction method of multimedia video information based on machine learning

In order to solve the problems of long time, poor denoising performance, low quality of multimedia video information and low efficiency of information extraction in current methods of extracting multimedia video information. An anti-collision extraction method of multimedia video information based on machine learning is proposed, which combines K-SVD algorithm and Batch-OMP algorithm to remove noise in multimedia video information. Considering the error concealment method used by decoder, channel condition, importance and lifetime of multimedia video information package, C uses improved proportional fair scheduling algorithm to complete the scheduling of multimedia video information. It calculates the time curve of multimedia video information, gets the local extremum points in the time curve, determines the length and location of the extracted video clips according to the local extremum points, calculates the importance of the video clips, and completes the extraction of multimedia video information according to the importance. In order to verify the effectiveness of the proposed method, a simulation experiment is carried out. The experimental results show that compared with the traditional methods, the proposed method has better denoising performance and higher information extraction efficiency. The above results show that the proposed method has good application prospects.

Bi-Sequential Video Error Concealment Method Using Adaptive Homography-Based Registration

This paper proposes a novel video error concealment method for the reconstruction of all unknown regions in frames. The proposed method first performs temporal error concealment (TEC) sequentially in forward direction with the propagation of source information for the unknown regions of frames in group-of-frames (GOF). Then, it performs the TEC sequentially in backward direction for the remaining unknown regions in the GOF. For the reconstruction of unknown regions in each target frame in forward and backward TEC stages, we propose an adaptive homography-based registration of the reference frame. The proposed method selects adaptively global or local homography-based registration for the reconstruction of each unknown region. After the backward direction, the TEC is performed in each frame and the spatial error concealment is applied to reconstruct completely the remaining unknown regions with relatively small sizes. The experimental results show that the proposed method gives significantly improved objective and subjective video quality compared with the traditional reconstruction methods. The proposed method also gives comparably good reconstruction results for large size unknown regions and is feasible for practical real-time applications.

Image Error Concealment Based on Deep Neural Network

In this paper, we propose a novel spatial image error concealment (EC) method based on deep neural network. Considering that the natural images have local correlation and non-local self-similarity, we use the local information to predict the missing pixels and the non-local information to correct the predictions. The deep neural network we utilize can be divided into two parts: the prediction part and the auto-encoder (AE) part. The first part utilizes the local correlation among pixels to predict the missing ones. The second part extracts image features, which are used to collect similar samples from the whole image. In addition, a novel adaptive scan order based on the joint credibility of the support area and reconstruction is also proposed to alleviate the error propagation problem. The experimental results show that the proposed method can reconstruct corrupted images effectively and outperform the compared state-of-the-art methods in terms of objective and perceptual metrics.

Optimal voice packet classification for enhanced VoIP over priority-enabled networks

This paper proposes a method for optimal classification of voice packets to enhance the quality of voice communications over priority-enabled networks when poor transmission conditions occur. Either high or low priority is assigned to each packet according to the relevance of its payload (voice segment) for the voice intelligibility. Then, in case of constrained networking conditions, by discarding first the voice packets of lower importance, the network always delivers those segments that most contribute to the perceptual quality. The proposed method is based on a dynamic programming optimisation algorithm that finds the optimal subset of m high priority voice segments in each utterance of size n > m. Such optimal subset minimizes the reconstruction distortion over all possible subsets with the same size m (i.e., the distortion incurred by a utterance reconstructed from only m segments). The simulation results show that the proposed method consistently achieves higher mean opinion scores (MOS) in comparison with non-selective packet drop under the same random network loss conditions, yielding better quality of experience (QoE) for the same packet loss rates (PLR). The priority classification algorithm is independent from error concealment methods and distortion metrics used in the optimisation process, which allows generalisation for diverse communication networks and applications.

No-reference real-time video transmission artifact detection for video signals

Video signals are a very important part of multimedia applications. Due to limited network bandwidth, video signals are subjected to the compression process, which introduces different compression artifacts. During network transmission, additional artifacts are introduced in video signals due to random bit errors and packet loss (PL). Both mentioned artifact types degrade visual quality of the video signal and thus, it has to be continuously monitored to ensure the required quality of service (QoS) provided to end users. An important component of the video quality monitoring system deals with video transmission artifact detection. In this paper, a no-reference (NR) pixel-based video transmission artifact detection algorithm is proposed, called the packet loss area measure (PLAM) algorithm. When detecting video transmission artifacts, the PLAM algorithm takes into account spatial and temporal information of a video signal. The performance of the proposed PLAM algorithm has been compared to those of the three existing different PL detection algorithms on a broad set of significantly different video signals from two publicly available video databases. One of these databases, called the Referent Packet Loss (RPL) database, has been created within this research and is presented in this paper. The algorithm performance testing results show that PLAM achieves high performance and overcomes other tested algorithms. Furthermore, the results show that the PLAM algorithm is very robust when detecting video transmission artifacts in video signals of different contents, with distinct degradation levels and PL error-concealment methods used in decoder post-processing. Due to its low computational complexity, the PLAM algorithm is capable of processing Full HD and Ultra HD video signals with the frame rate up to 100 and 25 frames per second (fps), respectively, in real time, in the case when high-end CPU is used.

Temporal error concealment using quad-tree prediction and coherency sensitive hashing for HEVC

This paper presents a temporal video error concealment method specially designed for H.265/HEVC. We propose the quad-tree partitioning prediction and the coherency sensitive hashing in order for better error concealment performance in the corrupted frames via HEVC codec. First, we try to deduce the most probable partitioning of the missing coding tree unit (CTU) using the proposed quad-tree partitioning prediction, which generates several CUs that constitute the CTU. Then, a coding unit (CU) priority choosing method is applied to select the best one from these CUs for prior concealment. Last, the coherency sensitive hashing is adopted for concealing the chosen best CU for better searching quality. The experiments shows that the recovery performance of the purposed method surpasses the compared state-of-the-art methods since the quad-tree partitioning prediction, the priority choosing process, and the coherency sensitive hashing help to improve the overall performance.

Novel Pixel Recovery Method Based on Motion Vector Disparity and Compensation Difference

As compressed videos are transmitted in the communication networks, video packet loss inevitably occurs. This problem can be solved by error concealment method. We used the motion vector of the available neighboring blocks to estimate the lost motion vector for the lost block. These estimates propagate to predict all other missing motion vectors. We further improved the work by using the idea of the motion vector disparities between neighboring available blocks to modify the motion vector weightings. Furthermore, the differences between the compensated pixels and the decoded pixels in the neighboring blocks are computed for another weighting for improvement. These two novelties are combined as a final indicator to prediction weightings. By comparison against the state-of-the-art method, the four proposed algorithms increase the average peak signal-to-noise ratio (PSNR) by up to 1.86, 1.93, 1.94, and 2.04 dB on average, showing the gradual improvement of our design systems. For other video quality measurements, the average gains of the proposed work against the state-of-the-art work can be up to 0.0575 in structural similarity index metric (SSIM), −0.0278 in video quality metric (VQM) (the lower the better), −0.0008 in motion-based video integrity evaluation (MOVIE) (the lower the better), and 2.77 in subjective evaluation. The proposed work performs slightly worse than a pixel-based state-of-the-art method in PSNR and SSIM but performs better in VQM and MOVIE (both correlate better with human perception) and subjective experiments, with much lower computational complexity.

Performance Evaluation of Frame-loss Error-concealment Solutions for the SHVC Standard

Scalable High Efficiency Video Coding (SHVC) has been emerging as an efficient video coding solution for adaptive video streaming and conferencing. SHVC is designed with a layered coding structure in which High Efficiency Video Coding (HEVC) tools are employed as core elements. However, compression efficiency and error sensitivity associated with efficient HEVC tools make this scalable codec less attractive for practical video transmission, especially when the loss of packets or video frames occurs. This problem severely impacts the display quality of received video. To address this problem, we propose efficient error concealment (EC) methods that compensate for a whole frame that is lost and that mitigate the error propagation problem occurring in practical video transmissions using SHVC. The presented EC methods mainly rely on the decoded information; thus, it is easily integrated into the SHVC decoder as a post-processing component. The experimental results obtained for both subjective and objective quality assessments show that the inter-layer correlation-based EC approach typically provides the highest concealed frame quality; thus, it is highly recommended for practical video transmission.

A video error concealment method using data hiding based on compressed sensing over lossy channel

The video error concealment with data hiding (VECDH) method aims to conceal video errors due to transmission according to the auxiliary data directly extracted from the received video file. It has the property that can well reduce the error propagated between spatially/temporally correlated macro-blocks. It is required that, the embedded information at the sender side should well capture/reflect the video characteristics. Moreover, the retrieved data should be capable of correcting video errors. The existing VECDH algorithms often embed the required information into the corresponding video frames to gain the transparency. However, at the receiver side, the reconstruction process may loss important information, which could result in a seriously distorted video. To improve the concealment performance, we propose an efficient VECDH algorithm based on compressed sensing (CS) in this paper. For the proposed method, the frame features to be embedded in every video frame are generated from the frame residuals CS measurements and scrambled with other frame features as marked data. The marked data is embedded into the corresponding frames by modulating color-triples for its least impacts on the carriers. For the receiver, the extracted data is used to reconstruct residuals to conceal errors. Error positions are located using the set theory. Since the CS has the ability to sample a signal within a lower sampling rate than the Shannon–Nyquist rate, the original signal could be reconstructed very well in theory. This indicates that the proposed method could benefit from the CS, and therefore keep better error concealment behavior. The experimental results show that the PSNR values gain about 10 dB averagely and the proposed scheme in this paper improves the video quality significantly comparing with the exiting VECDH schemes.

Recovery of Lost Color and Depth Frames in Multiview Videos.

In this paper, we consider an integrated error concealment system for lost color frames and lost depth frames in multiview videos with depths. We first proposed a pixel-based color error-concealment method with the use of depth information. Instead of assuming that the same moving object in consecutive frames has minimal depth difference, as is done in a state-of-the-art method, a more realistic situation in which the same moving object in consecutive frames can be in different depths is considered. In the derived motion vector candidate set, we consider all the candidate motion vectors in the set, and weight the reference pixels by the depth differences to obtain the final recovered pixel. Compared with the two state-of-the-art methods, the proposed method has average peak signal-to-noise ratio gains of up to 8.73 and 3.98 dB, respectively. Second, we proposed an iterative depth frame error-concealment method. The initial recovered depth frame is obtained by depth-image-based rendering from another available view. The holes in the recovered depth frame are then filled in the proposed priority order. Preprocessing methods (depth difference compensation and inconsistent pixel removal) are performed to improve the performance. Compared with a method that uses the available motion vector in a color frame to recover the lost depth pixels, the hybrid motion vector extrapolation method, the inpainting method and the proposed method have gains of up to 4.31, 10.29, and 6.04 dB, respectively. Finally, for the situation in which the color and the depth frames are lost at the same time, our two methods jointly perform better with a gain of up to 7.79 dB.

Fast‐efficient shape error concealment technique based on block classification

This study presents a fast-efficient error concealment method for recovering information related to shape. The proposed technique comprises block classification, edge direction interpolation and filtering interpolation. Missing blocks are classified into four categories: transparent, opaque, edge and isolated blocks. Most of the computation is spent on edge blocks and isolated blocks to maximise the cost and performance tradeoff. For the recovery of edge blocks, the edge slope is computed by referring to the nearest available block, from which the missing shape is interpolated parallel to the edge. Isolated blocks are dealt with using a cascade filter to approximate the actual shape. Experimental results show that the proposed method provides better cost performance in the restoration of shapes than that afforded by comparable algorithms, both in numerical parameters and the resulting shapes. The processing speed is approximately two to three times faster than previous methods and low computational load makes the proposed technique applicable to real-time MPEG-4 systems.

Video error concealment scheme based on tensor model

Since compressed video sequences may be corrupted or lost when transmitted over error-prone networks, error concealment techniques are very important for video communication. As a video sequence is a group of high-dimensional data, it can be considered as a big 3 r d -order tensor. The methodologies that matricize high-dimensional data and then apply matrix-based method for further analysis often cause a loss of the internal structure information. Therefore, we built a tensor model to process such data, in order to preserve the natural multilinear structure. The key idea of our tensor model includes two parts. The first part is to construct a small tensor consist of the corrupted block and its several reference blocks. This part could be accomplished by block matching, but the traditional block matching might give a wrong match when the corrupted area is large and continuous. In order to overcome such situation, we proposed a flexible block matching scheme (FBM). The second part is to figure out the data in the corrupted part by tensor low rank approximation. Unlike the traditional low rank approximation, we did not fix the rank of core tensor as a constant number. Instead, the rank is flexible in our method, which is adaptive to the situation. Compared with some other error concealment method in the experiments, our method is able to achieve significantly higher PSNR (Peak Signal to Noise Ratio) as well as better visual quality.

Spatial error concealment for intra-coded depth maps in multiview video-plus-depth

Transmission errors or packet loss in depth maps have great impact on the decoding quality and view synthesis of 3D and multiview video. Thus efficient methods to recover corrupted depth data are critical functions for accurate view rendering. This paper proposes an error concealment method for intra-coded depth maps, based on spatial intra and inter-view methods, which exploit neighbouring data of depth and colour images received error-free. A novel three-stage processing algorithm is devised to reconstruct sharp depth transitions (i.e. lost depth contours), using a disparity map and geometric interpolation based on parametric Bezier curves. The simulation results obtained from different views of various MVD sequences, for different packetisation modes and a wide range of packet loss rates (PLR), show that the proposed method consistently leads to quality improvement of synthesised images in comparison with reference methods.

Adaptive error concealment for temporal–spatial multiple description video coding

With the development of 3G/4G and WiFi networks, there has been a growing demand for video delivery over wireless channels. This increase of video traffic creates a significant challenge for efficient, reliable and adaptable video coding techniques. Multiple description coding (MDC) is an error resilient video coding method that has been proposed for streaming applications when retransmission is unacceptable. In this paper, we present error concealment approaches for two different four-description MDC architectures. In our first architecture, four descriptions are generated by temporal–spatial subsampling and encoded independently. To improve the error robustness, several adaptive error concealment methods are described. One of them is a frame-level method based on error tracking. It takes into account the distortion due to error concealment and error propagation. The other methods are macroblock-level methods based on foreground–background mapping and distortion mapping respectively. Our second MDC partition architecture aims at better coding efficiency and improved error robustness. Here spatial partitioning is done after prediction so that each of the two descriptions shares the same prediction loop. Experimental results are presented to demonstrate the efficacy of our proposed methods.

Frame Interpolation for Cloud-Based Mobile Video Streaming

Cloud-based High Definition (HD) video streaming is becoming popular day by day. On one hand, it is important for both end users and large storage servers to store their huge amount of data at different locations and servers. On the other hand, it is becoming a big challenge for network service providers to provide reliable connectivity to the network users. There have been many studies over cloud-based video streaming for Quality of Experience (QoE) for services like YouTube. Packet losses and bit errors are very common in transmission networks, which affect the user feedback over cloud-based media services. To cover up packet losses and bit errors, Error Concealment (EC) techniques are usually applied at the decoder/receiver side to estimate the lost information. This paper proposes a time-efficient and quality-oriented EC method. The proposed method considers H.265/HEVC based intra-encoded videos for the estimation of whole intra-frame loss. The main emphasis in the proposed approach is the recovery of Motion Vectors (MVs) of a lost frame in real-time. To boost-up the search process for the lost MVs, a bigger block size and searching in parallel are both considered. The simulation results clearly show that our proposed method outperforms the traditional Block Matching Algorithm (BMA) by approximately 2.5 dB and Frame Copy (FC) by up to 12 dB at a packet loss rate of 1%, 3%, and 5% with different Quantization Parameters (QPs). The computational time of the proposed approach outperforms the BMA by approximately 1788 seconds.