Multiple Description Video Coding Research Articles

Spatial-frequency HEVC multiple description video coding with adaptive perceptual redundancy allocation

Multiple description coding (MDC) approaches improve the error-resilient performance of video transmission by introducing redundancy. The existing multiple description video coding (MDVC) schemes are rarely designed for particular coding structure of HEVC, and the characteristics of the human visual system (HVS) are seldom considered. In this paper, a spatial-frequency multiple description video coding with adaptive perceptual redundancy allocation framework, named SF-PMDVC, is proposed for HEVC. For descriptions generation, after polyphase down-sampling in spatial domain, a transformation based on integer discrete cosine transform (DCT) is expended to adapt to the flexible coding unit partitioning process in HEVC, and the frequency coefficients are segmented and mapped to reduce the bitrate of each description. To further improve the performance of MDVC, an adaptive perceptual redundancy allocation strategy based on visual saliency is proposed, which improves the coding efficiency adapting to the visual perception. Experimental results show that the proposed scheme improves the error resiliency of HEVC by achieving superior objective and subjective reconstructed video quality as compared to the state-of-the-art MDVC methods for HEVC.

Spatial-Frequency Hevc Multiple Description Video Coding with Adaptive Perceptual Redundancy Allocation

Spatial-Frequency Hevc Multiple Description Video Coding with Adaptive Perceptual Redundancy Allocation

HEVC compatible perceptual multiple description video coding for reliable video transmission over packet networks

Multiple description coding (MDC) has been an effective scheme for reliable transmission of videos over error prone networks but requires higher data rates. Recently, perceptual video coding schemes are able to encode videos at a lower data rate, but with the same perceived decoding quality. Such schemes exploit human visual system (HVS) properties to encode videos. Although HVS characteristics have been used in MDC, but only for residual information and are not fully standard compatible. To this end, we propose a high efficiency video coding (HEVC) standard compatible perceptual multiple description video coding (PMDVC) framework. Our proposed framework temporally sub-samples the input video into even and odd frame sub-sequences, which are individually encoded (using HEVC) in a perceptual manner. We encode each description using a context based visual saliency model to obtain visual saliency mask, which is optimally thresholded into salient and non-salient regions. Coding tree unit (CTU) level perceptual relevance mask of each frame is generated by dividing binary saliency mask into five different groups. The quantization parameter at CTU level of each perceptual relevant group is adjusted in such a manner that data rate is minimized while maintaining the perceived quality. Our proposed HEVC based PMDVC scheme is evaluated under lossless and lossy channel conditions. Our results show better performance in data rate reduction, perceptual peak signal-to-noise-ratio, multi- scale structural similarity index, and difference mean opinion score, when compared with HEVC based temporally sub-sampled multiple description video coding scheme.

Visual saliency based redundancy allocation in HEVC compatible multiple description video coding

Video delivery over error prone channels suffer from packet loss, which significantly deteriorates the reconstructed video quality. Multiple description coding (MDC) is an effective error resilient coding scheme for transmission of video over unreliable networks, where video is encoded into multiple descriptions having redundancy between them. The amount of redundancy between these descriptions has an important role in MDC, because it provides error resilience. In this paper, a new redundancy allocation scheme using visual saliency model is presented with an aim of achieving high decoding quality, when one or more than one descriptions are received at the decoder. The proposed MDC method splits the input video sequence into even and odd subsequences, which are independently encoded by using high efficiency video coding (HEVC) encoder. The missing frames in each description are predicted by using pixel interpolation and motion interpolation. The residual information is generated by using the interpolated frame and its adjacent frame from the other description, which represents the redundancy in each description. Residual information is encoded based on a visual saliency mask, which is generated using a global contrast based visual saliency model. In this work, two different modes are proposed for visual saliency based redundancy allocation to provide better perceptual quality, while decoding the descriptions. The proposed scheme is implemented on HEVC reference software HM 16.2 and compared with two state-of-the-art temporal subsampling based multiple description video coding methods. Experiments show that the proposed method performs better than the reference methods, both in lossless and packet erasure channel conditions.

HEVC and 3D dual-tree discrete wavelet transform based multiple description video coding

HEVC and 3D dual-tree discrete wavelet transform based multiple description video coding

Redundancy Allocation Based on the Weighted Mismatch-Rate Slope for Multiple Description Video Coding

Multiple description coding (MDC) is a robust coding technique for video transmission over error prone networks, whereby the video is encoded into multiple descriptions with some redundancy between the descriptions. This redundancy leads to error resiliency in the case of packet loss during the network transport. However, the amount of this redundancy has a critical role in MDC performance. Therefore, a crucial problem in MDC is to find what the optimum amount of redundancy budget is, and then how this redundancy budget can be optimally allocated to the frames. To solve this problem, we propose a scheme in which the redundancy budget is allocated to the frames based on the weighted mismatch-rate slopes so that this additional bitrate can attain maximum distortion reduction. The redundancy is added gradually so that fine tuning of the utilized bitrate is achievable. We have verified our proposed scheme by implementing it in H.264/AVC reference software JM16.0, and running experiments against two representative reference methods. Our experiments show that our scheme not only minimizes the end-to-end distortion with a rate-distortion performance that is better than the reference methods, especially for high PLRs, but also entirely uses the available bandwidth, unlike the reference methods.

Scalable Video Multicast for MU-MIMO Systems With Antenna Heterogeneity

In contemporary multiuser multiple-input multiple-output systems, it is common for the reception devices to have a varying number of antennas. When multicast is performed, the number of concurrent spatial streams is limited by the device with the least number of antennas, which prevents more capable devices from getting higher rates. In this paper, we address the antenna heterogeneity in wireless video multicast by the innovative design of multiple similar description (MSD) video coding and multiplexed space–time block coding (M-STBC). MSD coding generates multiple descriptions of a video and features that any linear combinations of the descriptions are decodable. The descriptions comprising of real numbers are further processed by transform and power allocation steps for efficient transmission in a power-constrained system. M-STBC puts symbols in similar descriptions to corresponding space–time positions and ensures decodability under any antenna settings and channel conditions. As a result, we build up a scalable video multicast system, named AirScale, which allows receivers with a various number of antennas to decode from a single transmission, and the reconstructed video quality improves with the number of equipped antennas. Evaluations on Sora shows that, in a $\{1,2,3,4\} \times 4$ system, AirScale provides baseline quality for one-antenna receiver and a much higher quality for multiantenna receivers. The gain over SoftCast is up to 3.5, 3.9, and 4.1 dB for two-, three-, and four-antenna receivers, respectively.

Layered multiple description video coding using dual-tree discrete wavelet transform and H.264/AVC

A novel hybrid layered multiple description video coding (HLMDVC) algorithm is proposed to provide a robust and flexible video transmission via unreliable networks, like Internet or the wireless ne...

Multiple Description Video Coding Based on Human Visual System Characteristics

In this paper, a novel multiple description video coding scheme is proposed based on the characteristics of the human visual system (HVS). Due to the underlying spatial-temporal masking properties, human eyes cannot sense any changes below the just noticeable difference (JND) threshold. Therefore, at an encoder, only the visual information that cannot be predicted well within the JND tolerance needs to be encoded as redundant information, which leads to more effective redundancy allocation according to the HVS characteristics. Compared with the relevant existing schemes, the experimental results exhibit better performance of the proposed scheme at same bit rates, in terms of perceptual evaluation and subjective viewing.

Multiple Description Video Coding Using Inter- and Intra-Description Correlation at Macro Block Level

Multiple Description Video Coding Using Inter- and Intra-Description Correlation at Macro Block Level

End-to-End Rate-Distortion Optimized Description Generation for H.264 Multiple Description Video Coding

In this paper, H.264/AVC primary and redundant slices interlaced multiple description video coding (PRSI-MDVC) is studied due to its high coding efficiency and ease of constructing multiple descriptions by interleaving primary and redundant slices, where the problem of optimal description generation in the rate-distortion sense is addressed. Optimal description generation requires minimization of end-to-end distortion consisting of both source coding distortion determined by quality of primary slices and channel distortion associated with redundant slice coding, subject to a rate constraint. The relevant existing works on the description generation mainly focus on the estimation of channel distortion to determine the amount of inserted redundancy (quality of redundant slices) but ignore the source distortion estimation for the quality of primary slices, thus a comprehensive end-to-end rate-distortion optimization is still unavailable. In this paper, we attack the minimization of the end-to-end distortion by fully exploring temporal coding dependency. Specifically, on one hand, a most recently developed source distortion temporal propagation model is employed to determine coding options of primary slices in the PRSI-MDVC. On the other hand, channel distortion estimation is mainly concerned with the mismatch error estimation when primary slices are lost. Unlike the existing channel distortion estimation approach under the asymptotic fine quantization assumption which is not valid in most practical cases (e.g., at low or medium coding rates), we develop a novel and more feasible estimation scheme, based on which coding parameters of the redundant slices can be better determined. Simulation results show the effectiveness of the proposed frame-level rate-distortion optimized description generation scheme compared with the relevant approaches.

Efficiency of Dynamic GOP Length in Video Stream

Multiple description codes are designed for multiple path video streaming with channel diversities and an efficient MDC technique based on the applied spatial domain processing features. In this paper, we investigate the performance of multi-path video streaming using a multiple description coding (MDC) technique, especially the impacts of several codec variables, such as different lengths of the Group of Pictures (GOP), and the different startup times of I-frame in each independent MDC sub-streams. The overall performance of the system is evaluated by the frame dropout rate, which indicates the probability of playing back frozen frames of the reconstructed video at the receiver devices. To address dynamic channel conditions for video transmissions, we propose an adaptive coding technique by adjusting the GOP lengths according to the channel condition and combinatorial mathematics optimization in length of GOP adjustment. Different approaches, such as sub-dividing GOPs for a single stream and multiple sub-streams, and an adaptive MDC video streaming system, are examined in this paper.

Video streaming over an Ad Hoc network using data partitioning and path diversity

Multiple description video coding can exploit path diversity to improve error resiliency by generating several independently decodable video descriptions and sending them over different paths. Such paths are readily available in ad hoc wireless networks. To approach the original quality, all descriptions need to be successfully received. However, in a wireless channel, error bursts or link disruption may still affect one or more descriptions, resulting in significant drops in video quality. This article describes a lower complexity yet effective scheme for video streaming over such wireless networks. In order to improve error resiliency, the scheme combines unequal packet importance with the advantages of path diversity. The scheme can be implemented with a minimal increase in computational complexity, as spatial decomposition of video frames is employed to generate the descriptions. Experimental results for an ad hoc wireless network show that a significant objective quality gain can be achieved (up to 3 dB) in the presence of signal shadow fading.

Rate-distortion optimized mode selection method for multiple description video coding

Multiple description coding (MDC) is a potential solution for video transmission over error-prone networks because it shows promising enhancement of error resilient capability. The MDC systems encode a single video stream into two or more equally important independent sub-streams, called descriptions. Therefore, if some of the descriptors get lost, remaining descriptors can be used to recover the video. Much research has proposed different information distribution methods. Since each method has different characteristic, we proposed a general rate-distortion optimization framework for MDC systems in this paper. By sophisticated rate-distortion analysis and optimization, the framework enables MDC systems to adaptively encode video considering contents and channel variation. Experimental results showed that, by comparing with the work in Tsai and You (IEEE Trans Circ Syst Video Technol 22(2):309---320, 2012), the proposed technique improves the R-D performance significantly. The improvement can be up to 2.4 dB for the channels with 0 %?~?20 % packet loss rates, and it can be even more if the loss rate increases. The proposed framework is not restricted to specific MDC tools. Ones can easily integrate their proposed coding tools into the framework and achieve better performance as long as the macroblock's bitrate and distortion information can be measured.

Adaptive multiple description video coding and transmission for scene change

In view of perfect compatibility with the standard source and channel codec, temporal sampling-based multiple description coding (MDC) has become a better choice for practical applications. However, for the frames change from one scene to another temporal correlation may be destroyed by temporal sampling extremely, which results in the false estimation when the related frames are lost at the side decoder. Therefore, in this article the frames containing scene change are detected and duplicated before temporal sampling, which maintains better temporal correlation in each description. Furthermore, for better rate distortion performance temporal sampling is employed adaptively, that is, frame skipping or up-sampling according to the motion characteristics in original video. The experimental results exhibit better performance of the proposed scheme than other schemes whether in the on–off MDC environment or packet lossy network, especially about 15 dB improvements for the frames with scene change. Therefore, it may be a promising choice for video transmission over error-prone channels, especially over wireless networks.

Multiple Description Video Coding Based on Hierarchical B Pictures Using Unequal Redundancy

Multiple description video coding (MDC) is one of the approaches for reducing the detrimental effects caused by transmission over error-prone networks. In this paper, a MDC model based on hierarchical B pictures is proposed to optimize the tradeoff between coding efficiency and error resilience. The model produces two descriptors by applying different MDC techniques such as duplication, spatial splitting and temporal splitting on the different frames of video sequences, taking into account unequal importance of frames at different hierarchical levels. Duplication (high redundancy) is for key frames: spatial splitting (medium redundancy) for reference B frames, and temporal splitting (low redundancy) for nonreference B frames. For one descriptor loss, the model applies different estimation methods, but for the two descriptor loss case, the same temporal estimation is employed. As a consequence, better error resilience can be achieved at high coding efficiency. The advantages of the proposed model are demonstrated in error-free and packet loss networks.

Inter-frame dependency in multiview multi-description video streaming

Multiview video coding has been attracting increased popularity for facilitating stereo-vision and free-viewpoint video applications. Compares to conventional single view video coding, multiview video coding utilizes inter and intra view motion compensations to yield an improved coding efficiency, and multiple description video coding helps improving video streaming over multiple lossy channels; on the other hand, multiview and multiple description coding techniques increase video frame dependencies, with trades off video streaming quality with an increased drifting error probability. This paper examines the frame dependencies of multi-view video coding against configurable parameters such as the length of group-of-pictures, and the number of viewpoints.

Transmission of Successful Route Error Message(RERR) in Routing Aware Multiple Description Video Coding Over Mobile Adhoc Network

Video transmission over mobile ad-hoc networks is becoming important as these networks become more widely used in the wireless networks. We propose a routing-aware multiple description video coding approach to support video transmission over mobile ad-hoc networks with single and multiple path transport. We build a model to estimate the packet loss probability of each packet transmitted over the network based on the standard ad-hoc routing messages and network parameters without losing the RERR message. We then calculate the frame loss probability in order to eliminate error without any loss of data.

P2P video streaming combining SVC and MDC

P2P video streaming combining SVC and MDC In this paper we propose and evaluate a combined SVC-MDC (Scalable Video Coding & Multiple Description Video Coding) video coding scheme for Peer-to-Peer (P2P) video multicast. The proposed scheme is based on a full cooperation established between the peer sites, which contribute their upload capacity during video distribution. The source site splits the video content into many small blocks and assigns each block to a single peer for redistribution. Our solution is implemented in a fully meshed P2P network in which peers are connected to each other via UDP (User Datagram Protocol) links. The video content is encoded by using the Scalable Video Coding (SVC) method. We present a flow control mechanism that allows us to optimize dynamically the overall throughput and to automatically adjust video quality for each peer. Thus, peers with different upload capacity receive different video quality. We also combine the SVC method with Multiple Description Coding (MDC) to alleviate the packet loss problem. We implemented and tested this approach in the PlanetLab infrastructure. The obtained results show that our solution achieves good performance and remarkable video quality in the presence of packet loss.

Optimal Bandwidth Assignment for Multiple-Description-Coded Video

In video streaming over multicast network, user bandwidth requirement is often heterogeneous possibly with orders of magnitude difference (say, from hundreds of kb/s for mobile devices to tens of Mb/s for high-definition TV). Multiple description coding (MDC) can be used to address this bandwidth heterogeneity issue. In MDC, the video source is encoded into multiple independent descriptions. A receiver, depending on its available bandwidth, joins different descriptions to meet their bandwidth requirements. An important but challenging problem for MDC video multicast is how to assign bandwidth to each description in order to maximize overall user satisfaction. In this paper, we investigate this issue by formulating it as an optimization problem, with the objective to maximize user bandwidth experience by taking into account the encoding inefficiency due to MDC. We prove that the optimization problem is NP-hard. However, if the description number is larger than or equal to a certain threshold (e.g., if the minimum and maximum bandwidth requirements are 100 kb/s and 10 Mb/s, respectively, such threshold is seven descriptions), there is an exact and simple solution to achieve maximum user satisfaction, i.e., meeting all the bandwidth requirements. For the case when the description number is smaller, we present an efficient heuristic called simulated annealing for MDC bandwidth assignment (SAMBA) to assign bandwidth to each description given the distribution of user bandwidth requirement. We evaluate our algorithm using simulations. SAMBA achieves virtually the same optimal performance based on exhaustive search. By comparing with other assignment algorithms, SAMBA significantly improves user satisfaction. We also show that, if the coding efficiency decreases with the number of descriptions, there is an optimal description number to achieve maximal user satisfaction.