3D Video Transmission Research Articles

Binocular Rivalry Impact on Macroblock-Loss Error Concealment for Stereoscopic 3D Video Transmission

Three-dimensional video services delivered through wireless communication channels have to deal with numerous challenges due to the limitations of both the transmission channel's bandwidth and receiving devices. Adverse channel conditions, delays, or jitters can result in bit errors and packet losses, which can alter the appearance of stereoscopic 3D (S3D) video. Due to the perception of dissimilar patterns by the two human eyes, they can not be fused into a stable composite pattern in the brain and hence try to dominate by suppressing each other. Thus, a psychovisual sensation that is called binocular rivalry occurs. As a result, undetectable changes causing irritating flickering effects are seen, leading to visual discomforts such as eye strain, headache, nausea, and weariness. This study addresses the observer's quality of experience (QoE) by analyzing the binocular rivalry impact on the macroblock (MB) losses in a frame and its error propagation due to predictive frame encoding in stereoscopic video transmission systems. To simulate the processing of experimental videos, the Joint Test Model (JM) reference software has been used as it is recommended by the International Telecommunication Union (ITU). Existing error concealing techniques were then applied to the contiguous lost MBs for a variety of transmission impairments. In order to validate the authenticity of the simulated packet loss environment, several objective evaluations were carried out. Standard numbers of subjects were then engaged in the subjective testing of common 3D video sequences. The results were then statistically examined using a standard Student's t-test, allowing the impact of binocular rivalry to be compared to that of a non-rivalry error condition. The major goal is to assure error-free video communication by minimizing the negative impacts of binocular rivalry and boosting the ability to efficiently integrate 3D video material to improve viewers' overall QoE.

Efficient secure optical DWT-based watermarked 3D video transmission over MC-CDMA wireless channel

Efficient secure optical DWT-based watermarked 3D video transmission over MC-CDMA wireless channel

No-Reference Quality Assessment of Transmitted Stereoscopic Videos Based on Human Visual System

Provisioning the stereoscopic 3D (S3D) video transmission services of admissible quality in a wireless environment is an immense challenge for video service providers. Unlike for 2D videos, a widely accepted No-reference objective model for assessing transmitted 3D videos that explores the Human Visual System (HVS) appropriately has not been developed yet. Distortions perceived in 2D and 3D videos are significantly different due to the sophisticated manner in which the HVS handles the dissimilarities between the two different views. In real-time video transmission, viewers only have the distorted or receiver end content of the original video acquired through the communication medium. In this paper, we propose a No-reference quality assessment method that can estimate the quality of a stereoscopic 3D video based on HVS. By evaluating perceptual aspects and correlations of visual binocular impacts in a stereoscopic movie, the approach creates a way for the objective quality measure to assess impairments similarly to a human observer who would experience the similar material. Firstly, the disparity is measured and quantified by the region-based similarity matching algorithm, and then, the magnitude of the edge difference is calculated to delimit the visually perceptible areas of an image. Finally, an objective metric is approximated by extracting these significant perceptual image features. Experimental analysis with standard S3D video datasets demonstrates the lower computational complexity for the video decoder and comparison with the state-of-the-art algorithms shows the efficiency of the proposed approach for 3D video transmission at different quantization (QP 26 and QP 32) and loss rate (1% and 3% packet loss) parameters along with the perceptual distortion features.

Soft transmission of 3D video for low power and low complexity scenario

Thanks to the rapid development of naked-eye 3D and wireless communication technology, 3D video related applications on mobile devices have attracted a lot of attention. Nevertheless, the time-varying characteristics of the wireless channel is very challenging for conventional source-channel coding based transmission strategy. Also, the high complexity of source-channel coding based transmission scheme is undesired for low power mobile terminals. An advanced transmission scheme named Softcast was proposed to achieve efficient transmission performance for 2D image/video. Unfortunately, it cannot be directly applied to wireless 3D video transmission with high efficiency. This paper proposes a more efficient soft transmission scheme for 3D video with a graceful quality adaptation within a wide range of channel Signal-to-Noise Ratio (SNR). The proposed method first extends the linear transform to 4 dimensions with additional view dimension to eliminate the view redundancy, and then metadata optimization and chunk interleaving are designed to further improve the transmission performance. Meanwhile, a synthesis distortion based chunk discard strategy is developed to improve the overall 3D video quality under the condition of limited bandwidth. The experimental results demonstrate that the proposed method significantly improves the 3D video transmission performance over the wireless channel for low power and low complexity scenarios.

Efficient security framework for reliable wireless 3D video transmission

Efficient security framework for reliable wireless 3D video transmission

Synthesis-Distortion-Aware Hybrid Digital Analog Transmission for 3D Videos

The hybrid digital-analog (HDA) video transmission scheme can be used to avoid the cliff effect and saturation effect. However, directly using HDA in 3D video transmission requires too much bandwidth. This paper addresses synthesis-distortion-aware HDA transmission for 3D videos to improve transmission performance. First, a 3D HDA framework that transmits both texture and depth videos in HDA mode is designed. Second, a recursive synthesis distortion estimation model, called RSDE-3D-HDA, is derived, where the transmission errors of both the texture and depth sequences are considered. Third, we optimize the power allocation between digital and analog signals based on the RSDE-3D-HDA. Finally, simulation results show that our model is accurate and the proposed 3D-HDA achieves better performance in terms of synthesis quality than state-of-the-art methods.

Dynamic adaptive streaming over HTTP‐based 3D resource allocation algorithm

With the improvement of people's quality of life, the demand for three-dimensional (3D) video has gradually grown in recent years. However, there are many challenges to providing high-definition and high-fidelity 3D video. One of the most difficult challenges for users to provide 3D video is a large amount of energy lost during 3D video transmission. In this study, the authors present a 3D resource allocation system architecture, which can effectively alleviate the problem of large transmission energy loss. They propose that multiple users in a small cell can share video resources in the downlink from the cloud to the client. In the cloud processing, multiple users can share part of the cloud computing work cycle. In the downlink, multiple users can share part of video resources. They presume that some share output bits can be multicast to overall users, and the other bits can be sent to every user in unicast. They model this and derive the simulation results. The simulation results show that the resource allocation scheme in this study achieves greater energy savings under certain conditions than the traditional case without sharing the resource.

Combined Inter-layer FEC and Hierarchical QAM for Stereoscopic 3D Video Transmission

We consider an inter-layer FEC (IL-FEC) technique combined with hierarchical QAM for an efficient and reliable transmission of stereoscopic 3D video. The transmit sources are left and right video streams, and one of them is considered a more important stream for guarantee of conventional 2D service. The source node transmits both left and right videos to a destination node by using the IL-FEC and hierarchical 16QAM schemes, in which video peak signal-to-noise ratio (PSNR) performance varies with the hierarchical value $$(\alpha )$$. In this paper, we present the suitable hierarchical values to achieve the target PSNR performances even in low SNR environment. Numerical simulation results show that the proposed system using IL-FEC and hierarchial 16QAM schemes with the suitable $$\alpha$$ value can gain about 7.3–9.3 dB in the average PSNR performance, compared with the system using IL-FEC and conventional 16QAM schemes.

Efficient Adaptive Intra Refresh Error Resilience for 3D Video Communication

3D video wireless communication is being repeatedly called upon to play a greater role in the quest for economic and political stability, peace and understanding in the world. However, excessive error propagation in compressed 3D video can dramatically decrease perceived quality of experience (QoE). Recent error control strategies suggested that using adaptive intra refresh (AIR) tool is a good candidate for fulfilling quality of service (QoS) requirements for 3D video transmission over wireless networks. We propose multi-view video AIR (MVV-AIR) based on H.264/advance video coding (AVC) to mitigate transmission error propagation in a two-way communication system. With the MVV-AIR, both high active macroblocks and channel bit error rate (BER)are computed to generate MVV-AIR refresh map. The generated map is used to periodically insert a cyclic intra-refresh line macroblock to suppress spatiotemporal error propagation. Experimental results show that MVV-AIR outperforms traditional refresh scheme in error prone transmission environment with improved objective and subjective visual quality.

Proposed dynamic error control techniques for QoS improvement of wireless 3D video transmission

SummaryError control techniques like error resilience (ER) and error concealment (EC) are efficient techniques to ameliorate the lost macroblocks (MBs) in the 3D video (3DV) communication system. In this paper, we propose efficient and adaptive hybrid ER‐EC algorithms for 3DV transmission over error‐prone wireless channels. At the encoder, adaptive preprocessing ER mechanisms are proposed through using the context adaptive variable length coding entropy, slice structured coding modes, and explicit flexible macroblock ordering mapping. They are used to assist the suggested EC techniques at the decoder to accurately reconstruct the erroneous MBs and frames. At the decoder, an efficient postprocessing EC technique with multiproposition methods is proposed to dynamically select the convenient EC hypothesis method based on the size of the lost MBs, the faulty view, and the frame type. It conceals the received erroneous MBs of intra‐encoded and inter‐encoded frames of the transmitted 3DV by exploiting the temporal, spatial, and inter‐view correlations among frames and views. To further improve the decoded 3DV quality, a weighted overlapping block motion and disparity compensation technique is used to reinforce the performance of the suggested ER‐EC techniques. Experimental results on various 3DV streams prove that the suggested techniques have considerably acceptable subjective and objective 3DV performance. They achieve an improved average peak signal‐to‐noise ratio gain by almost 2.85 dB compared to the conventional error control algorithms at a packet loss rate = 40%.

Proposed optimized hybrid error recovery techniques for performance improvement of wireless 3D-MVC communication

The transmission of 3D Video (3DV) over wireless channels has become a hot issue because of the limited resources and the existence of severe channel errors. In the 3DV system, the compressed bit streams may be dropped down due to the propagation of transmission errors in the time, space, and view scopes. So, it is important to recover the Disparity Vectors (DVs) and Motion Vectors (MVs) of the corrupted Macro-Blocks (MBs) at the decoder utilizing efficient Error Concealment (EC) post-processing techniques. To enhance the received 3DV quality, we suggest optimized hybrid techniques to reconstruct the erroneous MBs of color-plus-depth inter-encoded and intra-encoded frames. A hybrid approach of Circular Scan Order Interpolation (CSOI) technique and Partitioning Motion Compensation (PMC) technique is proposed for the color intra-frames loss concealment. For the corrupted color inter-frames, a joint approach of Directional Textural Motion Coherence (DTMC) technique and Directional Interpolation Error Concealment (DIEC) technique is suggested. To estimate more additional depth-aided DVs and MVs for recovering the erroneous depth frames, a depth-assisted error recovery technique is suggested. These depth-estimated motion and disparity vectors are then added together with the estimated candidate texture DVs and MVs for reconstructing the corrupted color-plus-depth frames. Finally, the best color-plus-depth MVs and DVs are chosen by the Decoder Motion Vector Estimation (DMVE) and DIEC techniques. Simulation outcomes on various 3D video frames elucidate that the suggested hybrid color plus depth EC techniques achieve high robustness at high Packet Loss Rates (PLRs).

Efficient Adaptive Intra Refresh Error Resilience for 3D Video Communication

3D video wireless communication is being repeatedly called upon to play a greater role in the quest for economic and political stability, peace and understanding in the world. However, excessive error propagation in compressed 3D video can dramatically decrease perceived quality of experience (QoE). Recent error control strategies suggested that using adaptive intra refresh (AIR) tool is a good candidate for fulfilling quality of service (QoS) requirements for 3D video transmission over wireless networks. We propose multi-view video AIR (MVV-AIR) based on H.264/advance video coding (AVC) to mitigate transmission error propagation in a two-way communication system. With the MVV-AIR, both high active macroblocks and channel bit error rate (BER)are computed to generate MVV-AIR refresh map. The generated map is used to periodically insert a cyclic intra-refresh line macroblock to suppress spatiotemporal error propagation. Experimental results show that MVV-AIR outperforms traditional refresh scheme in error prone transmission environment with improved objective and subjective visual quality.

Perceptual rate distortion optimization of 3D–HEVC using PSNR-HVS

Improved compression efficiency is highly desirable in the transmission of 3D video and its storage. 3D–HEVC achieves higher compression efficiency compared to the simulcast HEVC or disparity-compensated multi-view video coding (MVC). In 3D–HEVC, the mean square error (MSE) is used to measure distortion in the rate distortion optimization process. However, MSE is not a good measure to use for measuring visual quality, as it poorly correlates with human perception. We propose to integrate a perceptual video quality metric inside the rate distortion optimization process of the 3D–HEVC. Specifically, in the coding unit (CU) mode selection process, PSNR-HVS is used as a measure for distortion. PSNR-HVS is based on the characteristics of the human visual system (HVS). Performance evaluations show that the proposed approach improves the compression efficiency of 3D–HEVC for multi-view videos by 2.78%.

Effective multi-stage error control algorithms for robust 3D video transmission over wireless networks

The Three-Dimensional Video (3DV) contains diverse video streams taken by different cameras around an object. Thence, it is an imperative assignment to fulfill efficient compression to match the future resource limitations, whilst preserving a decisive reception 3DV quality. The efficient 3DV communication over wireless networks has become a recent considerable hot issue due to the restricted resources and the presence of severe channel errors. The high-rate 3DV encoding and transmission over mobile or Internet are vulnerable to packet losses due to the existence of heavy channel losses and limited bandwidth. Therefore, this paper presents efficient multi-stage error control algorithms for reliable 3DV transmission over error-prone wireless channels. At the encoder, the error resilience schemes of context adaptive variable length coding, slice structured coding, and explicit flexible macro-block ordering are utilized. At the decoder, a joint approach of a directional interpolation error concealment algorithm and a directional textural motion coherence algorithm is proposed to conceal the corrupted color frames. For the concealment of the lost depth frames, an encoder independent decoder dependent depth-assisted error concealment algorithm is suggested. Moreover, the weighted overlapping block motion and disparity compensation algorithm is exploited to choose the candidate concealment Motion Vectors (MVs) and Disparity Vectors (DVs). Furthermore, an improved recursive Bayesian filtering algorithm is utilized as a refinement stage to smooth the remaining errors in the previously selected candidate MVs and DVs for achieving better 3DV quality. Simulation results on several 3DV sequences show that the proposed algorithms achieve adequate objective and subjective 3DV quality performance at severe packet loss rates compared to the state-of-the-art algorithms.

Improved Depth-Assisted Error Concealment Algorithm for 3D Video Transmission

In this paper, a whole frame loss error concealment algorithm for three-dimensional video coding is proposed. The main concept of the proposed algorithm is to extrapolate the motion vectors for concealing a current error block by jointly considering the available motion vector and the depth information. In addition, the depth information is adopted to help the derivation of reference pixels for concealing errors in the case that suitable motion vectors cannot be obtained by the motion vector extrapolation process alone. Simulation results demonstrate that our proposed algorithm can achieve up to 0.52 dB PSNR, as well as subjective quality improvement, compared to previous work.

Analysis of Packet-Loss-Induced Distortion in View Synthesis Prediction-Based 3D Video Coding.

View synthesis prediction (VSP) is a crucial coding tool for improving compression efficiency in the next generation 3D video systems. However, VSP is susceptible to catastrophic error propagation when multi-view video plus depth (MVD) data are transmitted over lossy networks. This paper aims at accurately modeling the transmission errors propagated in the inter-view direction caused by VSP. Toward this end, we first study how channel errors gradually propagate along the VSP-based inter-view prediction path. Then, a new recursive model is formulated to estimate the expected end-to-end distortion caused by those channel losses. For the proposed model, the compound impact of the transmission distortions of both the texture video and depth map on the quality of the synthetic reference view is mathematically analyzed. Especially, the expected view synthesis distortion due to depth errors is characterized in the frequency domain using a new approach, which combines the energy densities of the reconstructed texture image and the channel errors. The proposed model also explicitly considers the disparity rounding operation invoked for the sub-pixel precision rendering of the synthesized reference view. Experimental results are presented to demonstrate that the proposed analytic model is capable of effectively modeling the channel-induced distortion for MVD-based 3D video transmission.

Resource Allocation for 3D Video Transmission in LTE System Based on Fuzzy Logic

Resource Allocation for 3D Video Transmission in LTE System Based on Fuzzy Logic

Error concealment for the transmission of H.264/AVC-compressed 3D video in color plus depth format

An error concealment method for the transmission of 3D video in color/depth dual stream format (i.e., separately encoded) is proposed. We consider frame losses of color or depth along error-prone channels. Based on motion vector extrapolation from previous decoded frames, lost frames can be reconstructed. Specifically, the received information of one kind (color or depth) is intensively used when recovering a lost frame of the other kind. Our contributions come from classification of blocks into different kinds for different processing, joint color/depth matching, and shape-adaptive motion compensation. According to experiments, our proposed method, against traditional MV extrapolation (for color) and MV sharing (for depth) schemes, has shown distinctive gains in both color and depth. Our algorithm is effective in retaining integrity and smoothness of object boundaries, which is important to view synthesis for 3D display.

Scalable 3D video streaming over P2P networks with playback length changeable chunk segmentation

3D video distribution over P2P networks has been thought as a promising way for 3D video entering home. The convergence of scalable 3D video coding and P2P streaming can provide diverse 3D experiences for heterogeneous clients with high distribution efficiencies. However, the conventional chunk segmentation and scheduling algorithms originally aiming at the non-scalable 2D video streaming are not very efficient for scalable 3D video streaming over P2P networks due to the particular data characteristics of scalable 3D video. Based on this motivation, this paper first presents a playback length changeable 3D video chunk segmentation (PLC3DCS) algorithm to provide different error resilience strengths to video and depth as well as layers with different importance levels in the 3D video transmission. Then, a hybrid-priority based chunk scheduling (HPS) algorithm is proposed to be tied in with the proposed chunk segmentation algorithm to further promote the overall 3D video P2P streaming performance. The simulation results show that the proposed PLC3DCS algorithm with the corresponding HPS can increase the success delivery rates of chunks with more important levels, and further improve the user’s quality of 3D experience.

Joint Adaptive Pre-processing Resilience and Post-processing Concealment Schemes for 3D Video Transmission

3D video transmission over erroneous networks is still a considerable issue due to restricted resources and the presence of severe channel errors. Efficiently compressing 3D video with low transmission rate, while maintaining a high quality of received 3D video, is very challenging. Since it is not plausible to re-transmit all the corrupted macro-blocks (MBs) due to real time applications and limited resources. Thus it is mandatory to retrieve the lost MBs at the decoder side using sufficient post-processing schemes, such as error concealment (EC). In this paper, we propose an adaptive multi-mode EC (AMMEC) algorithm at the decoder based on utilizing pre-processing flexible macro-block ordering error resilience (FMO-ER) technique at the encoder; to efficiently conceal the erroneous MBs of intra and inter coded frames of 3D video. Experimental simulation results show that the proposed FMO-ER/AMMEC schemes can significantly improve the objective and subjective 3D video quality.