Error Concealment Research Articles

An improved multi-directional interpolation for spatial error concealment

Error concealment can improve the video quality at receiver side when the video frames are corrupted during transmission. A spatial error concealment algorithm based on the improved multi-directional interpolation is presented in this paper. The significant edges of corrupted MB are estimated using adaptive thresholding, an approximation for each missing pixel along each significant edge is computed. For two boundary pixels along each significant edge direction, proposed method computes the sum of magnitudes of gradients, which has the same quantized direction level as the edge direction. Finally a weighted average of multiple approximations is computed using the sum of gradient magnitudes. Proposed approach improves correctness of multi-directional interpolation by considering the edge directional tendendy of two boundary pixels to be the weight for directional interpolation. Experimental results show that our proposed method achieves better quality in terms of objective and subjective evaluations compared with the previous algorithms using multi-directional interpolation.

Temporal error concealment using quad-tree prediction and coherency sensitive hashing for HEVC

This paper presents a temporal video error concealment method specially designed for H.265/HEVC. We propose the quad-tree partitioning prediction and the coherency sensitive hashing in order for better error concealment performance in the corrupted frames via HEVC codec. First, we try to deduce the most probable partitioning of the missing coding tree unit (CTU) using the proposed quad-tree partitioning prediction, which generates several CUs that constitute the CTU. Then, a coding unit (CU) priority choosing method is applied to select the best one from these CUs for prior concealment. Last, the coherency sensitive hashing is adopted for concealing the chosen best CU for better searching quality. The experiments shows that the recovery performance of the purposed method surpasses the compared state-of-the-art methods since the quad-tree partitioning prediction, the priority choosing process, and the coherency sensitive hashing help to improve the overall performance.

An effective temporal error concealment in H.264 video sequences based on scene change detection-PCA model

This paper proposes a new temporal error concealment algorithm in H.264 video sequences based on scene change detection and PCA model. In order to detect scene change, dynamic threshold and image similarity metric are presented using coding prediction mode and DCT AC energy in H.264 baseline. UPCA (Updated PCA) model is presented by combining the scene change feature with Index transformation-Buffer updating approach. The lost images are concealed by Projection onto Convex Sets algorithm with UPCA model. Experimental results show that the proposed algorithm can achieve better error concealment performance for the higher motion and the frequent scene change, compared with the related method.

Proposed adaptive joint error‐resilience concealment algorithms for efficient colour‐plus‐depth 3D video transmission

The authors propose efficient hybrid error resilience and error concealment (ER-EC) algorithms for H.264 3D video-plus-depth (3DV + D) transmission over error-prone channels. At the encoder, content-adaptive pre-processing ER mechanisms are implemented by applying the context adaptive variable length coding (CAVLC), the slice structured coding, and the explicit flexible macro-block ordering. At the decoder, a post-processing EC algorithm with multi-proposition schemes is implemented to recover the lost 3DV colour frames. The convenient EC hypothesis is adopted based on the lost macro-blocks size mode, the faulty view, and the frame types. For the recovery of the lost 3DV depth frames, an encoder-independent decoder-dependent depth-assisted EC algorithm is suggested. It exploits the previously estimated colour disparity vectors (DVs) and motion vectors (MVs) to estimate more additional depth-assisted MVs and DVs. After that, the optimum colour-plus-depth DVs and MVs are accurately selected by employing the directional interpolation EC algorithm and the decoder MV estimation algorithm. Finally, a weighted overlapping block motion and disparity compensation scheme is utilised to reinforce the performance of the proposed hybrid ER-EC algorithms. Experimental results on standard 3DV + D sequences show that the proposed hybrid algorithms have superior objective and subjective performance indices.

Proposed dynamic error control techniques for QoS improvement of wireless 3D video transmission

SummaryError control techniques like error resilience (ER) and error concealment (EC) are efficient techniques to ameliorate the lost macroblocks (MBs) in the 3D video (3DV) communication system. In this paper, we propose efficient and adaptive hybrid ER‐EC algorithms for 3DV transmission over error‐prone wireless channels. At the encoder, adaptive preprocessing ER mechanisms are proposed through using the context adaptive variable length coding entropy, slice structured coding modes, and explicit flexible macroblock ordering mapping. They are used to assist the suggested EC techniques at the decoder to accurately reconstruct the erroneous MBs and frames. At the decoder, an efficient postprocessing EC technique with multiproposition methods is proposed to dynamically select the convenient EC hypothesis method based on the size of the lost MBs, the faulty view, and the frame type. It conceals the received erroneous MBs of intra‐encoded and inter‐encoded frames of the transmitted 3DV by exploiting the temporal, spatial, and inter‐view correlations among frames and views. To further improve the decoded 3DV quality, a weighted overlapping block motion and disparity compensation technique is used to reinforce the performance of the suggested ER‐EC techniques. Experimental results on various 3DV streams prove that the suggested techniques have considerably acceptable subjective and objective 3DV performance. They achieve an improved average peak signal‐to‐noise ratio gain by almost 2.85 dB compared to the conventional error control algorithms at a packet loss rate = 40%.

A Joint Framework for QoS and QoE for Video Transmission over Wireless Multimedia Sensor Networks

With the emergence of Wireless Multimedia Sensor Networks (WMSNs), the distribution of multimedia contents have now become a reality. Without proper management, the transmission of multimedia data over WMSNs affects the performance of networks due to excessive packet-drop. The existing studies on Quality of Service (QoS) mostly deal with simple Wireless Sensor Networks (WSNs) and as such do not account for an increasing number of sensor nodes and an increasing volume of data. In this paper, we propose a novel framework to support QoS in WMSNs along with a light-weight Error Concealment (EC) scheme. The EC schemes play a vital role to enhance Quality of Experience (QoE) by maintaining an acceptable quality at the receiving ends. The main objectives of the proposed framework are to maximize the network throughput and to cover-up the effects produced by dropped video packets. To control the data-rate, Scalable High efficiency Video Coding (SHVC) is applied at multimedia sensor nodes with variable Quantization Parameters (QPs). Multi-path routing is exploited to support real-time video transmission. Experimental results show that the proposed framework can efficiently adjust large volumes of video data under certain network distortions and can effectively conceal lost video frames by producing better objective measurements.

Performance evaluation of High Definition video streaming over Mobile Ad Hoc Networks

Video Service Providers (VSPs) can collect and analyze an enormous amount of multimedia data from various cloud storage centers using real-time big data systems for supporting various online customers. The infrastructure-less nature of Mobile Ad Hoc Networks (MANETs) makes the video streaming a challenging task for VSPs. High packet-loss probability in MANETs can create a notable distortion in the received video quality. In this paper, High Definition (HD) videos are streamed over MANETs. First, a transmission model is designed followed by a distortion model to estimate network distortions, such as packet-loss rate and end-to-end delay. Based on the proposed models, a video streaming framework is designed to efficiently utilize the available bandwidth in MANETs, minimize the network distortions, and improve Quality of Service (QoS). Later, an Error Concealment (EC) technique is used to conceal the lost/dropped video frames to improve the Quality of Experience (QoE). Experimental results show that our proposed video streaming framework outperforms the state-of-the-art routing protocols designed for MANETs, such as Destination-Sequenced Distance Vector (DSDV) and Optimized Link Sate Routing (OLSR) protocols. In the end, both subjective and objective evaluations are performed to evaluate the perceptual quality of the concealed video data.

Proposed optimized hybrid error recovery techniques for performance improvement of wireless 3D-MVC communication

The transmission of 3D Video (3DV) over wireless channels has become a hot issue because of the limited resources and the existence of severe channel errors. In the 3DV system, the compressed bit streams may be dropped down due to the propagation of transmission errors in the time, space, and view scopes. So, it is important to recover the Disparity Vectors (DVs) and Motion Vectors (MVs) of the corrupted Macro-Blocks (MBs) at the decoder utilizing efficient Error Concealment (EC) post-processing techniques. To enhance the received 3DV quality, we suggest optimized hybrid techniques to reconstruct the erroneous MBs of color-plus-depth inter-encoded and intra-encoded frames. A hybrid approach of Circular Scan Order Interpolation (CSOI) technique and Partitioning Motion Compensation (PMC) technique is proposed for the color intra-frames loss concealment. For the corrupted color inter-frames, a joint approach of Directional Textural Motion Coherence (DTMC) technique and Directional Interpolation Error Concealment (DIEC) technique is suggested. To estimate more additional depth-aided DVs and MVs for recovering the erroneous depth frames, a depth-assisted error recovery technique is suggested. These depth-estimated motion and disparity vectors are then added together with the estimated candidate texture DVs and MVs for reconstructing the corrupted color-plus-depth frames. Finally, the best color-plus-depth MVs and DVs are chosen by the Decoder Motion Vector Estimation (DMVE) and DIEC techniques. Simulation outcomes on various 3D video frames elucidate that the suggested hybrid color plus depth EC techniques achieve high robustness at high Packet Loss Rates (PLRs).

Recursive Bayesian Filtering-Based Error Concealment Scheme for 3D Video Communication Over Severely Lossy Wireless Channels

The three-dimensional video (3DV) is composed of variable-length stream sequences captured via diversified cameras surrounding an object. Thus, it is an urgent task to accomplish sufficient encoding to be compatible with incoming bandwidth demands, while achieving a recommended 3DV reception performance. In the 3DV compression framework, the lost macro-blocks (MBs) might propagate into the following frames and the adjoining views. Therefore, it is obligatory to avoid error propagation by concealing the corrupted MBs at the decoder through the utilization of appropriate post-processing error concealment (EC) techniques. The existing EC algorithms fundamentally exploit the temporal, inter-view, and spatial matching within the 3DV frames and views to reconstruct the disparity vectors (DVs) and motion vectors (MVs) of the corrupted MBs. Unluckily, in the state of high severe corruptions and heavily erroneous MBs, these concealment algorithms are predominantly unreliable and might give unreliable 3DV quality. Thence, in this work, we suggest the utilization of the outer block boundary matching algorithm to estimate the MVs and the directional interpolation EC algorithm to estimate the DVs of the erroneous MBs. After that, the Bayesian Kalman filter (BKF) is employed because of its efficiency to filter out the inherent errors in the previously predicted DVs and MVs to accomplish better 3D video performance. Experimental results on standard 3DV sequences demonstrate that the suggested BKF-based EC scheme is more powerful with heavy losses. It subjectively and objectively outperforms the traditional concealment techniques at severely random and bursty packet loss rates (PLRs).

Interleaving and Error Concealment to Mitigate the Impact of Packet Loss in Resource-Constrained TDLAS/WMS Data Acquisition

Tomographic imaging of pollutant gas emissions from aeroengines is attractive for the development of engines and fuels. A 126-beam tomographic setup has previously been proposed utilizing tunable diode laser absorption spectroscopy aiming for fast spatially resolved measurement of CO2 concentration. The custom data acquisition system uses a distributed architecture with at-site digital lock-in amplification, but remains resource constrained. A calibrated model is fitted to quadrature the first and second harmonic data, however, packet loss in ethernet and/or wireless networks can cause nondeterministic errors in the curve fitting and increased errors in recovered gas concentrations. Packet loss in this case, is a product of the available protocol, the high-vibration and high-noise industrial testing environment, the high network utilization expected, and the interrupt behavior of the embedded microprocessors. In this paper, the structure of the data acquisition system and the curve fitting approach are briefly discussed. Packet loss is then performed numerically to demonstrate the introduction of errors, as this cannot be swept experimentally without introducing other factors and increasing additive noise. An interleaving and error concealment mitigation approach is reported, that reduces this error, and can be applied to other resource-constrained remote acquisition systems such as Internet of Things applications. This approach is evaluated over parameters including extent of packet loss, interleaving ratio, and number of wavelength samples per packet. Viewing packet loss as a measurement SNR modifier, interleaving is shown to recover some SNR, but is ultimately limited. Processing of the received data using error concealment prior to spectrographic fitting is shown to increase tolerance.

Missing Surface Estimation Based on Modified Tikhonov Regularization: Application for Destructed Dental Tissue.

Estimation of missing digital information is mostly addressed by one or two-dimensional signal processing methods; however, this problem can emerge in multi-dimensional data including 3D images. Examples of 3D images dealing with missing edge information are often found using dental micro-CT, where the natural contours of dental enamel and dentine are partially dissolved or lost by caries. In this paper, we present a novel sequential approach to estimate the missing surface of an object. First, an initial correct contour is determined interactively or automatically, for the starting slice. This contour information defines the local search area and provides the overall estimation pattern for the edge candidates in the next slice. The search for edge candidates in the next slice is performed in the perpendicular direction to the obtained initial edge in order to find and label the corrupted edge candidates. Subsequently, the location information of both initial and nominated edge candidates are transformed and segregated into two independent signals (X-coordinates and Y-coordinates) and the problem is changed into error concealment. In the next step, the missing samples of these signals are estimated using a modified Tikhonov regularization model with two new terms. One term contributes in the denoising of the corrupted signal by defining an estimation model for a group of mildly destructed samples, and the other term contributes in the estimation of the missing samples with the highest similarity to the samples of the obtained signals from the previous slice. Finally, the reconstructed signals are transformed inversely to edge pixel representation. The estimated edges in each slice are considered as initial edge information for the next slice and this procedure is repeated slice by slice until the entire contour of the destructed surface is estimated. The visual results as well as quantitative results (using both contour-based and area-based metrics) for seven image datasets of tooth samples with considerable destruction of the dentin-enamel junction (DEJ) demonstrates that the proposed method can accurately interpolate the shape and the position of the missing surfaces in computed tomography images in both two and three dimensions (e.g. 14.87 ±3.87 μ m of mean distance (MD) error for the proposed method versus 7.33 ±0.27 μm of MD error between human experts and 1.25 ±~0 % error rate (ER) of the proposed method versus 0.64 ±~0 % of ER between human experts (~1% difference)).

On Detection Improvement in MC-CDMA Image Watermarking on Fading Channel

Code division multiple access (CDMA) at its multicarrier version of image watermarking scheme that is robust against random gain operation as well as enables high embedding rate is developed in this paper. Watermark embedding is done on mutually orthogonal set of host image samples wherein data bits are embedded using an orthogonal set of code patterns. Watermark decoding is also proposed by exploiting the principle of minimum mean square error combining. The decision variable for the binary watermark is the resultant weighted decision statistics. A generalized model of attack that consists of multiplicative and additive degradation on the watermarked image is considered. The first one follows Rayleigh distribution and the second one Gaussian distribution. We mathematically show that the proposed decoder structure incorporates the effect of random gain (fading attack) unlike the widely used correlator decoder. The concept of multiuser detection principle which is widely used in CDMA is then explored to cancel the effect of multiple bit interference in watermarking. Two variants of combined interference cancellation (CIC), namely threshold combined interference cancellation (TCIC) and group combined interference cancellation (GCIC) are then developed. Mathematical analysis and the results through simulations show that TCIC and CIC offer similar decoding error rate performance. On the other hand, a significant improvement in decoding error performance and payload are achieved through GCIC. Finally, the proposed method is studied for image error concealment and performance comparison is reported over the existing methods.

Intra-Frame Error Concealment Scheme Using 3D Reversible Data Hiding in Mobile Cloud Environment

In mobile cloud environment, data are mainly transmitted via wireless noisy channels, which may result in random transmission errors (RTE) with a high probability. For video transmission, noisy channels may cause significant degradation of the content. Improving or keeping video quality over lossy channel is, therefore, a very important research topic. Error concealment with data hiding (ECDH) is an effective way to conceal the errors introduced by channels, and it can reduce error propagation between neighbor blocks/frames. The existing video ECDH methods often embed the motion vectors (MVs) into the specific locations. Nevertheless, specific embedding locations cannot resist against RTE. To compensate the RTE in mobile cloud environment, we propose a video ECDH scheme using 3D reversible data hiding (RDH), in which we create multiple duplicate copies of each MV to improve the robustness of the ECDH scheme. Furthermore, all the duplicate copies and original MVs are embedded into macroblocks randomly. However, embedding more data would make quality of marked video decreased. In addition, satisfactory trade-off between the introduced distortion and the reconstructed video quality can be achieved by tuning the number of duplicate copies. For random embedding, the lost probability of the MVs decreases rapidly which can result in better error concealment performance. Experimental results show that the peak signal to noise ratio values gain about 5 dB at least comparing with the existing ECDH methods. Meanwhile, the proposed method improves the video quality significantly.

Spiral-Like Pixel Reconstruction Algorithm for Spatiotemporal Video Error Concealment

This paper proposes efficient error concealment (EC) algorithms that use a spiral-like pixel reconstruction (SPR) scheme on the H.264/AVC joint model simulation platform. The algorithms provide low complexity and high accuracy, and their subjective and objective evaluation results are superior to those of extant EC algorithms. First, edge matching is applied to the boundary of lost macroblocks (MBs). Second, the directional edge group with the highest magnitude is selected and symmetric pixel referencing is performed along its orthogonal symmetry axis. Finally, the lost MBs are estimated using the novel SPR scheme with reference mode selection according to the edge ratio. Experimental results revealed that, compared with the extant edge-oriented PR approaches of EC, the proposed total- and partial-spiral ordering modes in pixel referencing can be used to reconstruct lost MBs, with a desirable decoder peak signal-to-noise ratio and high visual quality.

An Effective System for Video Transmission and Error Recovery Mechanisms in Multimedia Networks

In this paper, an effective system has been proposed for video transmission. This system solves the problems arised due to occurrence of errors in transmitted video and delivers video with required quality of services. The reconstructed video maintains the quality of services at the decoder side. Video dynamics-based error concealment algorithm is applied for recovering errors occurred during transmission. The performance of the proposed system is measured by means of simulations using JM reference software.

Novel Pixel Recovery Method Based on Motion Vector Disparity and Compensation Difference

As compressed videos are transmitted in the communication networks, video packet loss inevitably occurs. This problem can be solved by error concealment method. We used the motion vector of the available neighboring blocks to estimate the lost motion vector for the lost block. These estimates propagate to predict all other missing motion vectors. We further improved the work by using the idea of the motion vector disparities between neighboring available blocks to modify the motion vector weightings. Furthermore, the differences between the compensated pixels and the decoded pixels in the neighboring blocks are computed for another weighting for improvement. These two novelties are combined as a final indicator to prediction weightings. By comparison against the state-of-the-art method, the four proposed algorithms increase the average peak signal-to-noise ratio (PSNR) by up to 1.86, 1.93, 1.94, and 2.04 dB on average, showing the gradual improvement of our design systems. For other video quality measurements, the average gains of the proposed work against the state-of-the-art work can be up to 0.0575 in structural similarity index metric (SSIM), −0.0278 in video quality metric (VQM) (the lower the better), −0.0008 in motion-based video integrity evaluation (MOVIE) (the lower the better), and 2.77 in subjective evaluation. The proposed work performs slightly worse than a pixel-based state-of-the-art method in PSNR and SSIM but performs better in VQM and MOVIE (both correlate better with human perception) and subjective experiments, with much lower computational complexity.

Performance Evaluation of Frame-loss Error-concealment Solutions for the SHVC Standard

Scalable High Efficiency Video Coding (SHVC) has been emerging as an efficient video coding solution for adaptive video streaming and conferencing. SHVC is designed with a layered coding structure in which High Efficiency Video Coding (HEVC) tools are employed as core elements. However, compression efficiency and error sensitivity associated with efficient HEVC tools make this scalable codec less attractive for practical video transmission, especially when the loss of packets or video frames occurs. This problem severely impacts the display quality of received video. To address this problem, we propose efficient error concealment (EC) methods that compensate for a whole frame that is lost and that mitigate the error propagation problem occurring in practical video transmissions using SHVC. The presented EC methods mainly rely on the decoded information; thus, it is easily integrated into the SHVC decoder as a post-processing component. The experimental results obtained for both subjective and objective quality assessments show that the inter-layer correlation-based EC approach typically provides the highest concealed frame quality; thus, it is highly recommended for practical video transmission.

Effective multi-stage error control algorithms for robust 3D video transmission over wireless networks

The Three-Dimensional Video (3DV) contains diverse video streams taken by different cameras around an object. Thence, it is an imperative assignment to fulfill efficient compression to match the future resource limitations, whilst preserving a decisive reception 3DV quality. The efficient 3DV communication over wireless networks has become a recent considerable hot issue due to the restricted resources and the presence of severe channel errors. The high-rate 3DV encoding and transmission over mobile or Internet are vulnerable to packet losses due to the existence of heavy channel losses and limited bandwidth. Therefore, this paper presents efficient multi-stage error control algorithms for reliable 3DV transmission over error-prone wireless channels. At the encoder, the error resilience schemes of context adaptive variable length coding, slice structured coding, and explicit flexible macro-block ordering are utilized. At the decoder, a joint approach of a directional interpolation error concealment algorithm and a directional textural motion coherence algorithm is proposed to conceal the corrupted color frames. For the concealment of the lost depth frames, an encoder independent decoder dependent depth-assisted error concealment algorithm is suggested. Moreover, the weighted overlapping block motion and disparity compensation algorithm is exploited to choose the candidate concealment Motion Vectors (MVs) and Disparity Vectors (DVs). Furthermore, an improved recursive Bayesian filtering algorithm is utilized as a refinement stage to smooth the remaining errors in the previously selected candidate MVs and DVs for achieving better 3DV quality. Simulation results on several 3DV sequences show that the proposed algorithms achieve adequate objective and subjective 3DV quality performance at severe packet loss rates compared to the state-of-the-art algorithms.

Improved Depth-Assisted Error Concealment Algorithm for 3D Video Transmission

In this paper, a whole frame loss error concealment algorithm for three-dimensional video coding is proposed. The main concept of the proposed algorithm is to extrapolate the motion vectors for concealing a current error block by jointly considering the available motion vector and the depth information. In addition, the depth information is adopted to help the derivation of reference pixels for concealing errors in the case that suitable motion vectors cannot be obtained by the motion vector extrapolation process alone. Simulation results demonstrate that our proposed algorithm can achieve up to 0.52 dB PSNR, as well as subjective quality improvement, compared to previous work.

A video error concealment method using data hiding based on compressed sensing over lossy channel

The video error concealment with data hiding (VECDH) method aims to conceal video errors due to transmission according to the auxiliary data directly extracted from the received video file. It has the property that can well reduce the error propagated between spatially/temporally correlated macro-blocks. It is required that, the embedded information at the sender side should well capture/reflect the video characteristics. Moreover, the retrieved data should be capable of correcting video errors. The existing VECDH algorithms often embed the required information into the corresponding video frames to gain the transparency. However, at the receiver side, the reconstruction process may loss important information, which could result in a seriously distorted video. To improve the concealment performance, we propose an efficient VECDH algorithm based on compressed sensing (CS) in this paper. For the proposed method, the frame features to be embedded in every video frame are generated from the frame residuals CS measurements and scrambled with other frame features as marked data. The marked data is embedded into the corresponding frames by modulating color-triples for its least impacts on the carriers. For the receiver, the extracted data is used to reconstruct residuals to conceal errors. Error positions are located using the set theory. Since the CS has the ability to sample a signal within a lower sampling rate than the Shannon–Nyquist rate, the original signal could be reconstructed very well in theory. This indicates that the proposed method could benefit from the CS, and therefore keep better error concealment behavior. The experimental results show that the PSNR values gain about 10 dB averagely and the proposed scheme in this paper improves the video quality significantly comparing with the exiting VECDH schemes.