Error-resilient Scheme Research Articles

Concealment Support and Error Resilience for HEVC to Improve Consumer Quality of Experience

Consumer electronics have started to support 8K/4K/UHD/SUHD videos in order to improve the perceived QoE. It is beneficial to assess the consumer’s perceived quality degradation at the output prior to transmitting the video in an error prone channel, to include concealment support and error resilient features to minimize these artifacts. While existing error resilient schemes have attempted to include source coding error and the channel’s error propagation, they have ignored the error resilience and concealment accuracy at the decoder when encoding the video which allows for end-to-end error capture and mitigation to improve consumer QoE. This paper proposes a two-pass codec encoding scheme that determines the motion vectors and the coding modes which not only mitigate the source coding error and the channel error propagation, but also improve the accuracy of the error concealment operation. The proposed method incorporates an end-to-end probabilistic distortion model and utilizes the concealment error in the subsequently encoding frames while estimating the motion vectors for the current frame. The experiment achieves up to 1.47dB PSNR gain for diverse video contents in comparison to the state-of-the-art techniques at a 10% packet error rate. This objective gain was also supported by the user perceived QoE.

RRAM-DNN: An RRAM and Model-Compression Empowered All-Weights-On-Chip DNN Accelerator

This article presents an energy-efficient deep neural network (DNN) accelerator with non-volatile embedded resistive random access memory (RRAM) for mobile machine learning (ML) applications. This DNN accelerator implements weight pruning, non-linear quantization, and Huffman encoding to store all weights on RRAM, enabling single-chip processing for large neural network models without external memory. A four-core parallel and programmable architecture adapts to various neural network configurations with high utilization. We introduce a customized RRAM macro with a dynamic clamping offset-canceling sense amplifier (DCOCSA) that achieves sub-microampere input offset. The on-chip decompression and memory error-resilient scheme enables 16 million (M) 8-bit (decompressed) weights on a single-chip using 24 Mb RRAM. The proposed RRAM-DNN is the first digital DNN accelerator featuring 24 Mb RRAM as all-on-chip weight storage to eliminate energy-consuming off-chip memory accesses. The fabricated design performs the complete inference process of the ResNet-18 model while consuming 127.9 mW power in TSMC-22 nm ULL CMOS. The RRAM-DNN accelerator achieves peak performance of 123 GOPs with 8-bit precision, exhibiting measured energy efficiency of 0.96 TOPs/W.

Effective multi-stage error control algorithms for robust 3D video transmission over wireless networks

The Three-Dimensional Video (3DV) contains diverse video streams taken by different cameras around an object. Thence, it is an imperative assignment to fulfill efficient compression to match the future resource limitations, whilst preserving a decisive reception 3DV quality. The efficient 3DV communication over wireless networks has become a recent considerable hot issue due to the restricted resources and the presence of severe channel errors. The high-rate 3DV encoding and transmission over mobile or Internet are vulnerable to packet losses due to the existence of heavy channel losses and limited bandwidth. Therefore, this paper presents efficient multi-stage error control algorithms for reliable 3DV transmission over error-prone wireless channels. At the encoder, the error resilience schemes of context adaptive variable length coding, slice structured coding, and explicit flexible macro-block ordering are utilized. At the decoder, a joint approach of a directional interpolation error concealment algorithm and a directional textural motion coherence algorithm is proposed to conceal the corrupted color frames. For the concealment of the lost depth frames, an encoder independent decoder dependent depth-assisted error concealment algorithm is suggested. Moreover, the weighted overlapping block motion and disparity compensation algorithm is exploited to choose the candidate concealment Motion Vectors (MVs) and Disparity Vectors (DVs). Furthermore, an improved recursive Bayesian filtering algorithm is utilized as a refinement stage to smooth the remaining errors in the previously selected candidate MVs and DVs for achieving better 3DV quality. Simulation results on several 3DV sequences show that the proposed algorithms achieve adequate objective and subjective 3DV quality performance at severe packet loss rates compared to the state-of-the-art algorithms.

Robust Multiview Three-Dimensional Video Communications Based on Distributed Video Coding

With the rapid development of electronic and communications technology, multiview video services are becoming more and more feasible in practice. However, robustly delivering many views over error-prone channels is a rather challenging task. Due to the recent development of distributed video coding, this paper proposes two alternative error-resilient schemes for multiview video transmission based on the Wyner–Ziv (WZ) coding technique. A lightweight-encoder error-resilient scheme requires no communications between the cameras at the encoder, whereas the temporal and interview redundancy can be explored to generate the side information at the decoder. In this case, it is not only robust to channel losses but also has independent encoders with low encoding complexity. On the other hand, a complex-encoder error-resilient scheme refers to the scheme that employs an auxiliary redundant stream encoded according to a WZ video coding approach to protect a primary stream encoded with a standard multiview video coding codec. At the receiver, the error-concealed reconstructed frames are used as the side information by the WZ decoder. Therefore, this method maintains the original multiview bitstream unaltered by simply adding WZ bits for protection. Experimental results conducted over a simulated error-prone channel reveal that the two proposed schemes have better performance than the forward error correction by means of the Reed–Solomon and intrarefresh approaches.

Delivering High-Frame-Rate Video to Mobile Devices in Heterogeneous Wireless Networks

High-frame rate (HFR) video streaming is emerging as a new paradigm in popular multimedia applications (e.g., mobile cloud gaming) to achieve smooth viewing experience perceived by end users. However, it is severely challenging to guarantee the delivery quality of HFR video over wireless platforms with regard to the high transmission rate and limited network resources. Multihoming capability enables mobile devices to concurrently receive video data with different radio interfaces (e.g., cellular and Wi-Fi). To effectively deliver mobile HFR video over multiple wireless access networks, this paper develops an application-layer transmission scheme dubbed joint FRAme Scheduling and Error Resilience (FRASER). First, we propose an unequal frame scheduling approach by taking advantage of interlaced forward error correction coding and reliability-aware data allocation to minimize the total distortion. Second, we introduce an error resilience scheme at the receiver side to proactively leverage the out-of-order and overdue video packets to mitigate the error propagations. The proposed FRASER is able to substantially reduce the probability of I (Intra) frame loss and consecutive P (Predicted) frame drops. We conduct the performance evaluation through extensive emulations in Exata involving HFR video streaming encoded with H.264 codec. Evaluation results show that FRASER outperforms the reference transmission schemes in terms of video peak signal-to-noise ratio, end-to-end delay, and received frame rate. Thus, FRASER is recommended for delivering HFR video streaming to multihomed mobile devices in heterogeneous wireless networks.

A Performance Degradation Tolerable Cache Design by Exploiting Memory Hierarchies

Performance degradation tolerance (PDT) has been shown to be able to effectively improve the yield, reliability, and lifetime of an electronic product. The focus of PDT is on the particular performance degrading faults (pdef) that only incur some performance degradation of a system without inducing any computation errors. The basic idea is that as long as the defective chips containing only the pdef can provide acceptable performance for some applications, they may still be marketable. Critical issues of PDT to be addressed include the portion of the pdef in a faulty chip and their induced performance degradation. For a typical cache design, most of the possible faults are not pdef. In this brief, we propose a cache redesign method, called PDT cache, where all functional faults in the data-storage cells of a cache (major part of the cache) can be transformed into pdef. By transforming this large number of faults into pdef, a faulty cache becomes much more likely to be still marketable. The proposed design exploits the existing hardware resources and the inherent error resilience scheme to reduce the incurred hardware overhead. The logic synthesis results show that the incurred hardware overhead is only 6.29% for a 32-kB cache. We also evaluate the induced performance degradation under various fault densities using the CPU2000 and CPU2006 benchmark programs. The results show that for a 32-kB cache design, when the fault density is <1%, only 0.31% performance degradation is incurred. In addition, the scalability of the PDT cache is also evaluated. The results show that a smaller hardware overhead is required for a larger cache, and the performance degradation is independent of the cache associativity and can even be smaller for a larger cache under a given fault density.

Leveraging the Delay-Friendliness of TCP With FEC Coding in Real-Time Video Communication

Transmission control protocol (TCP) is pervasively employed as the transport-layer solution in popular video applications (e.g., Skype, Google+, HTTP-based adaptive video streaming, etc.) for firewall traversal and network-friendliness. To remedy the shortcomings of the data retransmission mechanism in TCP, forward error correction (FEC) coding is commonly used as the application-layer error-resilient scheme in live media streaming systems. An important measurement study reveals that TCP exhibits delay-friendliness (i.e., delay-performance bias) towards traffic flows composed of small-size packets. Motivated by leveraging the delay-friendliness of TCP to optimize the real-time streaming video quality, we propose a novel FEC coding scheme dubbed C oded L ive vide O S treaming ov E r T CP (CLOSET). To achieve the optimal video quality over the lossy communication networks, we analytically formulate the constrained optimization problem of joint FEC coding and packet interleaving to minimize the effective packet loss rate. Then, we provide an approximate analysis to derive the solution for online adaption of FEC redundancy, packet size, and interleaving level. The performance of CLOSET is evaluated through extensive semi-physical emulations in Exata involving real-time encoded H.264 video streaming. Experimental results show that CLOSET outperforms the reference FEC coding schemes in terms of video peak signal-to-noise ratio (PSNR), end-to-end delay, ratio of overdue video frames, and goodput. Therefore, we recommend CLOSET for TCP-based real-time video communication systems.

A Rate–Distortion Optimized Error-Resilient Algorithm for Multi-view Video Coding

In this paper, a new rate---distortion (R---D) optimized error-resilient scheme is proposed to improve error resilience for multi-view video transmission over lossy networks. Based on the study on the characteristics of multi-view video coding and the propagating behavior of channel errors, a recursive model to estimate the end-to-end distortion is first developed, in which the channel-induced distortion takes into consideration both motion and disparity compensation prediction. Then, based on the estimated distortion and the derived total bit rate, R---D optimized motion and disparity estimation are employed to find the reliable motion vector and disparity vector to achieve the optimum trade-off between prediction efficiency and error robustness. Finally, R---D optimization is applied again to perform coding mode decision, which explicitly considers intra, inter and inter-view prediction modes. Extensive experimental results demonstrate significant performance gains can be achieved for multi-view video communications against transmission errors.

Inter-embedding error-resilient mechanism in scalable video coding

Scalable video coding (SVC), an extended version of H.264/AVC, is designed to transmit high-quality video bitstreams over heterogeneous networks. However, such video bitstreams are sensitive to transmission error, thereby severely degrading its quality. Inter-layer prediction in SVC causes errors to be propagated not only to subsequent frames but also to frames in the upper layers, when the errors have occurred in the lower layers. This paper presents an inter-embedding error-resilient scheme to reduce the distortion caused by the loss of inter-layer prediction information in SVC. The proposed algorithm exploits the reversible data embedding scheme to hide essential information of the lower layer without damaging the original data. Experimental results demonstrate that the proposed method provides a better PSNR performance than the frame copy by an average of 4.89dB in a 2-layer SVC decoder, and an average of 4.54dB in a 3-layer SVC decoder in the case of whole frame loss.

Error-resilient multi-view video coding using Wyner-Ziv techniques

In this paper, a Wyner-Ziv (WZ) coding based error-resilient scheme is proposed for multi-view video transmission over error-prone channels. At the encoder, the key frames of the odd views are protected by WZ encoding to generate the auxiliary bit-stream alongside the multi-view video coded bit-stream. At the decoder, error-concealed multi-view decoded frames are used as the side information (SI) for WZ decoding. Based on the study on the characteristics of multi-view video coding (MVC) and the propagating behavior of channel errors, a recursive model to estimate the transmission distortion is developed in the transform domain, in which the channel-induced distortion takes into consideration both motion and disparity compensation. With the proposed model, we propose a rate control strategy for WZ encoding to infer the minimum bit rate so as to correct the SI errors. The WZ bit rate estimation method exploits the correlation between the original bit-planes and the SI bit-planes as well as the bit-plane interdependency. Extensive experimental results show that the proposed error-resilient scheme outperforms Reed Solomon based forward error correction method by about 1.1 dB and outperforms the adaptive intra refresh algorithm by approximately 1.6 dB at the packet loss rate 10 %.

English

English

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.

Analysis of Video Coding and Error Resilience Tools of H. 264/AVC in Wireless Environment

Recently more and more telecommunication systems are supporting different kinds of real-time transmission, video transmission being one of the most important application.In wireless environments, channel bandwidth and high packet loss rate are to main limitations in the way of delivering of a good quality video to the end user. Therefore, in applications such as video over wireless networks, a video codec should have ability to handle the erroneous situations of the channels well as the bandwidth limitations.H.264/AVC is the newest international video coding standard, jointly developed by groups from ISO/IEC and ITUT.It has several error resilience techniques to make a video bit stream robust in the erroneous channels conditions and also achieves a significant improvement in the compression efficiency. We analyze various error resilienceschemes and innovative featuresof H.264/AVC for real time video streaming .The focus of the work is to test video coding and error resilience tools of H.264/AVC in real timeenvironment over wireless networks.

A Novel Error Resilient Scheme for Wavelet-based Image Coding Over Packet Networks

this paper presents a robust transmission strategy for wavelet based scalable bit stream over packet erasure channel. By taking the advantage of the bit plane coding and the multiple description coding, the proposed strategy adopts layered multiple description coding (LMDC) for the embedded wavelet coders to improve the error resistant capability of the important bit planes in the meaning of D(R) function. Then, the post-compression rate-distortion (PCRD) optimization process is used to improve the overall performance. At decoding, a simple error concealment technique is used to compensate the data loss. The proposed method is applied on Set Partitioning in Hierarchical Trees (SPIHT) coded image and the experimental results show graceful degradation for variable packet erasure rate.

An efficient FMO-based error-resilient scheme for video surveillance coding

The encoded video bitstreams have to suffer from quality degradation when transmitted over unreliable channels. In this paper, a novel flexible macroblock ordering (FMO)-based error-resilient scheme is proposed to cope with this problem, which is aimed at the video surveillance scenarios. This scheme could not only meet the need of prioritized transmission environment, but also take advantage of FMO's inherent error-resilient function. A motion region extraction algorithm is first advised and utilized, and each motion region's error sensitivity is then estimated based on both pictures' features and transmission condition. According to the error sensitivity value, the optimal FMO encoding mode for each motion region is selected, which enables the error-resilient intensity to be more adaptive. The intra refresh enhancement is introduced at the same time to further improve the error-resilient performance of the proposed FMO structure. To reduce the complexity, most information for calculations is derived from the encoding process. Experiments demonstrate that the proposed scheme performs better than the conventional schemes, and improves the video quality efficiently.

Error Protection of Wavelet Scalable Video Streaming Using Wyner-Ziv Technique over a Lossy Network

Abstract This paper presents a novel error resilience scheme for wavelet scalable video coding. We use Wyner-Ziv codec to produce extra bits protecting the important parts of the embedded video streaming. At the same time these bits also as the second description of important parts are transmitted over auxiliary channel to the receiver for error resilience. The errors in the embedded video streaming can be corrected by Wyner-Ziv description which regards the decoded frame as side information. Moreover, Wyner-Ziv decoder utilizes a coarse estimated version of the corrupted parts exploiting frame correlation in wavelet video decoder to generate a refine version. Simulation results show that our proposed method can achieve much better performance compared with Forward Error Correction code. Meanwhile, this error resilient algorithm can achieve 2-3 dB PSNR gains over the motion compensation error concealment.

A Novel Layered Error Resilient Method for SPIHT-Coded Images over Packet Networks

This paper presents a layered error resilient scheme for Set Partitioning in Hierarchical Trees (SPIHT) coding which incorporates the Multiple Description Coding (MDC) into the data partition. The major feature of this scheme is that the MDC is adopted in the process of packetizing the spatial orientation trees, so as to protect the important information bits according to different importance and channel state. The proposed scheme can allocate redundancy accurately and flexibly, by differentiating importance of bits through different bit planes. Experimental results show that the proposed method achieves good performance over packet erasure channels.

New Error Resilience Technique Using Adaptive FMO and Intra Refresh for H.264 Video Transmission

In this paper, we propose an error resilience scheme for wireless video coding based on adaptive flexible macroblock ordering (FMO) and intra refresh. An FMO explicit map is generated frame-by-frame by using prior information. This information involves estimated locations of guard and burst sections in the channel and estimated effect of error propagation (EEP) from the previous frame to the current frame. In addition, the role of the current frame in propagating an error to the next frame is also considered. A suitable intra refresh rate which is adaptive to the channel state is used to reduce the dependence between frames and thus can stop the EEP. The results in experiments show that the proposed method gains some improvements in terms of peak signal-to-noise rate (PSNR) as compared with some other methods that have not considered the channel condition and the error propagation in generating an FMO map.

Error-Resilient Scheme for Wavelet Video Codec Using Automatic ROI Detection and Wyner-Ziv Coding Over Packet Erasure Channel

The error-resilient for video transmission over the Internet in which regarded as the packet erasure channel is always a tough task and has gained lots of attentions. The main contradictory problem lies between error-resilient and bandwidth usage. Additional redundant data has to be added to achieve robust transmission which leads to huge bandwidth usage. In this paper, an error-resilient scheme called Wyner-Ziv Error-Resilient (WZER) based on a receiver driven layered Wyner-Ziv (WZ) coding framework is proposed. The WZER purposely emphasizes on the protection of the Region of Interest (ROI) area in the frame thus to achieve the better tradeoff between the bandwidth usage and error-resilience. WZER is designed to work for the scenario of wavelet based video coding over packet erasure channel, where several techniques including automatic ROI detection, ROI mask generation, Rate distortion optimization (RDO) quantization, WZ coding with layer design, and packet level Low Density Parity Check (LDPC) code are used. The performances of the proposed WZER are simulated based on average PSNR of luminance, perceptual reconstruction and bandwidth usage and compared with normal Forward Error Correction (FEC) full protection scheme and no protection scheme. The results show the advantages of the proposed WZER over traditional FEC protection, especially in the aspects of the recovery of the subject area and bandwidth efficiency.

Hybrid Joint Source Channel Decoding for Progressive JPEG Image Transmission

In this paper, we combine two Joint Source Channel Decoding (JSCD) approaches and apply them to progressive JPEG image transmission. The first JSCD approach consists of optimizing the allocation of the channel code-rates to the different layers of the JPEG image, and is commonly referred as Unequal Error Protection (UEP). The second JSCD strategy exploits a priori probabilities obtained from source statistics as well as the residual redundancy of the source to improve channel decoding. Additionally, we propose an error resilient sub-categorization scheme for the DC layer so as to reduce the effect of the channel-induced distortion. The hybrid JSCD scheme is implemented with Rate Compatible Punctured Turbo Codes and also with the bandwidth efficient Rate Compatible Punctured Turbo Trellis Coded Modulation. Gains of over 10 dB in PSNR are obtained with the hybrid JSCD scheme as compared to a conventional JPEG image transmission scheme.