Video Transmission Scheme Research Articles

Multiple Description Coding for Best-Effort Delivery of Light Field Video Using GNN-Based Compression

In recent years, Light Field (LF) video has grabbed much attention as an emerging form of immersive media. LF collects, through a lens matrix, light information emanating in every direction, and obtains rich information about the scene, providing users with an immersive 6 Degrees of Freedom (DoF) experience. The visual content between different viewpoints is highly homogenized, suggesting the possibility of good compression and encoding. However, most fixed-structure LF coding schemes are difficult to adapt to the real-time requirements of different LF applications and best-effort network conditions causing packet loss. In this paper, we propose a dynamic adaptive LF video transmission scheme that can achieve high compression and yet provide near-distortion-free LF video when the network condition is stable. Additionally, for unstable network conditions a description scheduling algorithm is proposed, which can decode the LF video with the highest possible quality even if partial data cannot be received completely and/or timely. We achieve this by designing a Multiple Description Coding (MDC) based solution to transport the LF video compressed by a Graph Neural Network (GNN) model. Experimental results show that the scheduling algorithm can improve the quality of the decoding results by 3% to 15%. Compared with other similar schemes, our system greatly improves the reliability of the video streaming system against packet loss/error and supports heterogeneous receivers.

Research on the Design and Implementation of a Multi-scene Web-based Video Conferencing and Live Streaming System for Colleges and Universities

With the increasing popularity of distance education and online learning in colleges and universities, the network video conference and live streaming system with multiple scenes has become an important part of the information construction of college education. In this paper, for the problems of the current network video conference and live streaming system in colleges and universities, we design and implement a network video conference and live streaming system supporting multiple scenarios. The system adopts the audio and video transmission scheme based on WebRTC technology, supports multiple platforms such as Web, Android and iOS, and integrates a variety of functional modules, including screen sharing, interactive Q&A, voting, recording and playback, etc. The experimental results show that the system is stable and efficient, and it can meet the demand for web video conferencing and live streaming in multiple scenarios in universities.

A Novel Video Transmission Latency Measurement Method for Intelligent Cloud Computing

Low latency video transmission is gaining importance in time-critical applications using real-time cloud-based systems. Cloud-based Virtual Reality (VR), remote control, and AI response systems are emerging use cases that demand low latency and good reliability. Although there are many video transmission schemes that claim low latency, they vary over different network conditions. Therefore, it is necessary to develop methods that can accurately measure end-to-end latency online, continuously, without any content modification. This research brings these applications one step closer to addressing these next generation use cases. This paper analyzes the cause of end-to-end latency within a video transmission system, and then proposes three methods to measure the latency: timecode, remote online, and lossless remote video online. The corresponding equipment was designed and implemented. The actual measurement of the three methods using related equipment proved that our proposed method can accurately and effectively measure the end-to-end latency of the video transmission system.

Video Error-Resilience Encoding and Decoding Based on Wyner-Ziv Framework for Underwater Transmission

Limited by the characteristics of underwater acoustic channels, the video transmission applications targeting deep-sea detection and operation tasks are facing severe challenges such as network failure and high delay, resulting in loss of video details, color distortion, blurring, and even bit errors, which seriously affect decoding quality of the video transmission and reception. In order to solve the problems of deep-sea long-distance wireless communication, this paper proposes an improved Wyner-Ziv coding scheme (UnderWater-WZ) for video transmission through acoustic channels. The implementation process includes controlling error range by using MJPEG coding, combining motion compensation time interpolation with calibration information to generate high-quality side information. And intraframe quantization matrix is designed to weaken the change of video scene. The experimental results show that under the highest packet loss rate of 20%, this scheme can achieve 2.6~3.5 dB improvements in terms of video reconstruction compared to the previous methods, which is close to the error-free level.

DeepWiVe: Deep-Learning-Aided Wireless Video Transmission

We present <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DeepWiVe</i> , the first-ever end-to-end joint source-channel coding (JSCC) video transmission scheme that leverages the power of deep neural networks (DNNs) to directly map video signals to channel symbols, combining video compression, channel coding, and modulation steps into a single neural transform. Our DNN decoder predicts residuals without distortion feedback, which improves the video quality by accounting for occlusion/disocclusion and camera movements. We simultaneously train different bandwidth allocation networks for the frames to allow variable bandwidth transmission. Then, we train a bandwidth allocation network using reinforcement learning (RL) that optimizes the allocation of limited available channel bandwidth among video frames to maximize the overall visual quality. Our results show that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DeepWiVe</i> can overcome the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">cliff-effect</i> , which is prevalent in conventional separation-based digital communication schemes, and achieve graceful degradation with the mismatch between the estimated and actual channel qualities. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DeepWiVe</i> outperforms H.264 video compression followed by low-density parity check (LDPC) codes in all channel conditions by up to 0.0485 in terms of the multi-scale structural similarity index measure (MS-SSIM), and H.265+ LDPC by up to 0.0069 on average. We also illustrate the importance of optimizing bandwidth allocation in JSCC video transmission by showing that our optimal bandwidth allocation policy is superior to uniform allocation as well as a heuristic policy benchmark.

UAV Anti-Jamming Video Transmissions With QoE Guarantee: A Reinforcement Learning-Based Approach

Unmanned aerial vehicles (UAVs) that are widely utilized for video capturing, processing and transmission have to address jamming attacks with dynamic topology and limited energy. In this paper, we propose a reinforcement learning (RL)-based UAV anti-jamming video transmission scheme to choose the video compression quantization parameter, the channel coding rate, the modulation and power control strategies against jamming attacks. More specifically, this scheme applies RL to choose the UAV video compression and transmission policy based on the observed video task priority, the UAV-controller channel state and the received jamming power. This scheme enables the UAV to guarantee the video quality-of-experience (QoE) and reduce the energy consumption without relying on the jamming model or the video service model. A safe RL-based approach is further proposed, which uses deep learning to accelerate the UAV learning process and reduce the video transmission outage probability. The computational complexity is provided and the optimal utility of the UAV is derived and verified via simulations. Simulation results show that the proposed schemes significantly improve the video quality and reduce the transmission latency and energy consumption of the UAV compared with existing schemes.

A comprehensive theoretical evaluation of the end-to-end performance of SoftCast-based linear video delivery schemes

SoftCast-based linear video delivery (LVD) schemes have been proposed as an alternative to traditional video transmission schemes in wireless error-prone environments. The end-to-end performance of SoftCast-based schemes have been evaluated in Xiong et al. (2016), where a theoretical model based on the Peak Signal-to-Noise Ratio (PSNR) metric has been proposed. The latter is limited to the use of a Zero-Forcing (ZF) estimator at the receiver side, and does not consider bandwidth restrictions. Nevertheless, bandwidth restrictions are common and necessary in practice, especially when considering the transmission of video content. It is mandatory to take this aspect into consideration as it may drastically influence the received video quality. In this paper, we provide valid and significant extensions of the initial model. In total, three models are introduced taking into account both (1) bandwidth constraints (i.e., data compression applied), (2) the use of a Linear Least Square Error (LLSE) estimator instead of the ZF one as well as (3) the use of the optimal power allocation. We show that regardless of the bandwidth reduction applied, the type of estimator as well as the power allocation used, the end-to-end video quality can be accurately modeled and predicted at the transmitter according to the video content characteristics, the type of estimator used at the receiver and the channel conditions. The validity of these three models is assessed through extensive end-to-end simulations. These new models give solid theoretical guidelines for optimizing and studying the performance of linear video delivery schemes.

Reinforcement learning based energy-efficient internet-of-things video transmission

The video transmission in the Internet-of-Things (IoT) system must guarantee the video quality and reduce the packet loss rate and the delay with limited resources to satisfy the requirement of multimedia services. In this paper, we propose a reinforcement learning based energy-efficient IoT video transmission scheme that protects against interference, in which the base station controls the transmission action of the IoT device including the encoding rate, the modulation and coding scheme, and the transmit power. A reinforcement learning algorithm state-action-reward-state-action is applied to choose the transmission action based on the observed state (the queue length of the buffer, the channel gain, the previous bit error rate, and the previous packet loss rate) without knowledge of the transmission channel model at the transmitter and the receiver. We also propose a deep reinforcement learning based energy-efficient IoT video transmission scheme that uses a deep neural network to approximate Q value to further accelerate the learning process involved in choosing the optimal transmission action and improve the video transmission performance. Moreover, both the performance bounds of the proposed schemes and the computational complexity are theoretically derived. Simulation results show that the proposed schemes can increase the peak signal-to-noise ratio and decrease the packet loss rate, the delay, and the energy consumption relative to the benchmark scheme.

A smart transmission optimization mechanism for sports events based on edge computing

With the rise of smart cities in today's society, more and more people pay attention to sports events with high requirements on the transmission quality. However, due to the fact that the audiences of sports video are relatively concentrated, it presents a stronger locality of time and space. Hence, the traditional video transmission schemes are no longer suitable for the transmission of sports video. In this way, this paper studies the smart sports video transmission optimization scheme based on edge computing. First of all, this paper designs a cloud incremental scalable sports video view popularity prediction model, based on which an on‐demand and customized video coding mechanism is introduced to maximize the efficiency of computing resource allocation. Secondly, this paper designs a smart sports video caching mechanism based on edge computing to eliminate a large number of video transmission redundancy. Finally, a large‐scale simulation experiment is built based on the real sports video data, and the experimental results fully show that the proposed scheme can improve the user experience on video quality and relative smoothness.

A priority and path diversity-based approach for media streaming over multi-hop and unreliable networks

Despite the recent advances, media streaming over multi-hop unreliable networks, such as the internet and wireless packet networks, is still challenging because of the dynamic and unpredictable nature of these networks. This paper proposes an approach that exploits paths' redundancies offered by multi-hop networks along with the differential sensitivity inherent in the encoded video information to enhance the quality of experience of video viewers in terms of the continuity of video playback, without compromising the quality of the reconstructed video. The performance of the proposed priority-and-path-diversity-based (PPD) video transmission scheme is compared through simulations with a best effort streaming model that mimics today's internet as well as a priority-based queuing streaming model that can be used in multi-hop wireless networks. The simulation results show that the proposed PPD streaming model outperforms the other two streaming models not only in the achieved video quality, as represented by the average peak signal to noise ratio, but also in terms of the average end-to-end delay and losses which are reflected by the instantaneous occupancy of the playback buffer at the viewer side.

A priority and path diversity-based approach for media streaming over multi-hop and unreliable networks

Despite the recent advances, media streaming over multi-hop unreliable networks, such as the internet and wireless packet networks, is still challenging because of the dynamic and unpredictable nature of these networks. This paper proposes an approach that exploits paths' redundancies offered by multi-hop networks along with the differential sensitivity inherent in the encoded video information to enhance the quality of experience of video viewers in terms of the continuity of video playback, without compromising the quality of the reconstructed video. The performance of the proposed priority-and-path-diversity-based (PPD) video transmission scheme is compared through simulations with a best effort streaming model that mimics today's internet as well as a priority-based queuing streaming model that can be used in multi-hop wireless networks. The simulation results show that the proposed PPD streaming model outperforms the other two streaming models not only in the achieved video quality, as represented by the average peak signal to noise ratio, but also in terms of the average end-to-end delay and losses which are reflected by the instantaneous occupancy of the playback buffer at the viewer side.

Synthesis-Distortion-Aware Hybrid Digital Analog Transmission for 3D Videos

The hybrid digital-analog (HDA) video transmission scheme can be used to avoid the cliff effect and saturation effect. However, directly using HDA in 3D video transmission requires too much bandwidth. This paper addresses synthesis-distortion-aware HDA transmission for 3D videos to improve transmission performance. First, a 3D HDA framework that transmits both texture and depth videos in HDA mode is designed. Second, a recursive synthesis distortion estimation model, called RSDE-3D-HDA, is derived, where the transmission errors of both the texture and depth sequences are considered. Third, we optimize the power allocation between digital and analog signals based on the RSDE-3D-HDA. Finally, simulation results show that our model is accurate and the proposed 3D-HDA achieves better performance in terms of synthesis quality than state-of-the-art methods.

HDA Video Transmission Scheme for DASH

In recent years, with the development of Internet communication technology, more and more people tend to get information through the network. And people prefer multimedia video streaming services. Traditional streaming media technology cannot meet people's needs because of its limitations. Recently, HTTP-based Dynamic Adaptive Streaming Over HTTP (DASH) has emerged as a new approach. In this paper, we consider two hybrid digital and analog video transmission schemes for DASH and propose three methods of enhancing the video quality of experience (QoE) of DASH users. First of all, considering the importance of data to video quality, we propose an energy-aware recombination method of reorganizing and packaging data and define the priority of the generated DASH layer in transmission. Second, we design an adaptive bitrate allocation algorithm to improve bandwidth utilization and video quality. Finally, we propose a retransmission mechanism to address the situation in which the digital stream at the receiving end cannot be decoded due to channel distance errors. Experiments show that our two schemes are superior to traditional DASH schemes.

Temporal information based GoP adaptation for linear video delivery schemes

An original wireless video transmission scheme called SoftCast has been recently proposed to deal with the issues encountered in conventional wireless video broadcasting systems (e.g. cliff effect). In this paper, we evaluate and optimize the performance of the SoftCast scheme according to the transmitted video content. Precisely, an adaptive coding mechanism based on GoP-size adaptation, which takes into account the temporal information fluctuations of the video, is proposed. This extension denoted Adaptive GoP-size mechanism based on Content and Cut detection for SoftCast (AGCC-SoftCast) significantly improves the performance of the SoftCast scheme. It consists in modifying the GoP-size according to the shot changes and the spatio-temporal characteristics of the transmitted video. When hardware capacities, such as buffer or processor performance are limited, an alternative method based only on the shot changes detection (AGCut-SoftCast) is also proposed. Improvements up to 16 dB for the PSNR and up to 0.55 for the SSIM are observed with the proposed solutions at the cut boundaries. In addition, temporal visual quality fluctuations are reduced under 1dB in average, showing the effectiveness of the proposed methods.

A metadata-free pure soft broadcast scheme for image and video transmission

The SoftCast, as an analog-like transmission scheme, has been proposed to provide graceful video quality degradation for wireless communication scenarios where channel condition usually varies dynamically and unpredictably. It attempts to eliminate the cliff effect that conventional communication systems usually suffer from. The unequal power allocation minimizes the total distortion by optimally scaling the transform coefficients according to their statistics. However, the sender has to transmit considerable amount of metadata over a digital channel with strong channel codes to assist in decoding at the receiver. And the transmission will still break down completely if the metadata are corrupted. Hence, SoftCast can just alleviate the cliff effect. To eliminate the cliff effect while maintaining the image and video quality, a feasible metadata-free pure soft broadcast scheme for image and video transmission is proposed in this paper. Specifically, we apply an effective chunk-based blind estimation method to reconstruct the original signal from the received noisy signal without using power scaling factors. Thus, our scheme can eliminate the cliff effect. In addition, since smaller chunks can be used, better performance than SoftCast has been achieved. Experimental results show that our scheme outperforms SoftCast by about 4dB and provides graceful quality degradation for a wide channel signal to noise ratio range.

Distortion Estimation-Based Adaptive Power Allocation for Hybrid Digital–Analog Video Transmission

Hybrid digital–analog (HDA) video transmission schemes have shown advantages in avoiding the cliff effect . However, most current HDA schemes assume perfect transmission of the digital signal, which is hardly the case in practice. In this paper, we propose an adaptive recursive distortion estimate for the HDA system (ARDE-HDA), which recursively estimates the decoder-side distortion from the encoder and adaptively allocates the transmission power between the digital and analog signals in HDA. First, we derive the closed-form expression of a recursive distortion estimation (RDE) method, which does not require the digital part of the HDA output to be decoded perfectly, and both transmission error and superposition process of digital and analog parts are taken into consideration. Then, based on the deduced RDE model, an adaptive power allocation is proposed for the digital and analog parts to minimize the decoder-side distortion. Finally, simulation results are presented, which show the accuracy of the proposed RDE model and the ARDE-HDA method. Our method can achieve an average of 4.10 dB gain over existing HDA methods and 15.14-dB gain over the Softcast method in terms of the peak signal-to-noise ratio.

Design and Analysis of CQI Feedback Reduction Mechanism for Adaptive Multicast IPTV in Wireless Cellular Networks

To guarantee that all mobile users in a cell can watch video with acceptable quality, conventional multicast IPTV services use the most robust modulation and coding scheme (MCS) for video transmission, resulting in worse spectral efficiency. To enhance the spectral efficiency, the adaptive multicast IPTV using the adaptive MCS based on signal-to-noise ratios (SNRs) or channel quality indicators (CQIs) was proposed in the literature. However, the CQI feedback messages induced by the adaptive multicast IPTV incur additional signaling overhead. Thus, a novel CQI feedback scheme was proposed in this paper to effectively decrease the amount of CQI feedback messages while preserving the system performance. Additionally, the CQI feedback rates of various CQI feedback schemes were theoretically analyzed based on the multi-dimensional Markovian model. Numerical results show that the proposed CQI feedback mechanism significantly reduces the CQI feedback rate without sacrificing the system performance and outperforms other existing CQI feedback schemes. Moreover, the analytical results almost coincide with simulation results, demonstrating that the theoretic analyses are accurate.

Joint Power and Channel Resource Optimization in Soft Multi-View Video Delivery

Existing wireless multi-view video (MVV) transmission schemes use digital compression to achieve a better coding efficiency. However, the digital schemes suffer from the cliff effect, which refers to the phenomenon that the video quality is a step function of wireless channel quality. In this paper, we first consider a soft MVV transmission scheme where the correlations between the inter-view data and texture-depth data are exploited by a 5-dimensional discrete cosine transform (5D-DCT). The linearly transformed 5D-DCT signals are modulated in an analog manner so that the video quality gracefully improves when the channel quality becomes better. The cumbersome bit and rate controls in digital solutions are replaced by simple power controls. Second, as with the increase of the number of cameras and data depths, the data size of MVV increases linearly. To reduce the heavy data traffic in soft MVV transmission, we proposed efficient resource (bandwidth and power) allocation algorithms. Simulations results demonstrate that the proposed distortion-resource (DR) optimization algorithm can ensure a best viewing quality under a resource constraint and the proposed resource-distortion (RD) optimization algorithm can minimize the resource usage for a target video quality requirement. Third, the impact of power control across texture and depth frame and the impact of view positions on synthesized virtual view quality are investigated. The efficacy of the proposed algorithm on both the reference viewpoint as well as the virtual viewpoint is verified via simulations.

Buffer-Driven Rate Control and Packet Distribution for Real-Time Videos in Heterogeneous Wireless Networks

Video transmission bandwidth can be substantially increased by concurrent transmission through multiple radio interfaces introduced by heterogeneous wireless networks. However, delivering high-quality real-time videos with stringent end-to-end delay constraints is still challenging due to the time-varying multipath channel status. Thus, to integrate available multipath bandwidth and guarantee low end-to-end delay, this paper investigates a real-time concurrent multipath video transmission scheme in heterogeneous access networks. First, we present a multipath buffer analysis model (MBAM) to derive the lower and upper bounds of the target video frame bits. Second, given the low-latency and received video quality smoothness, we propose a buffer-driven rate control and packet distribution (BRCPD) algorithm aided by frame-level rate control and multipath transmission buffer awareness based on MBAM. Finally, we implement the proposed application-layer solution BRCPD algorithm in a real-world testbed. The experiment results verify that different from existing multipath video transmission schemes, the proposed algorithm is able to leverage the MBAM-based frame-level rate control to guarantee the target low end-to-end delay. In particular, the BRCPD algorithm is able to guarantee an overdue loss rate of less than 5% under strictly limited delay constraints (250 ms) in the real-world testbed.

A multiple description scalable coding scheme for adaptive video streaming over next generation cellular networks

Nowadays, billions of users are connected to the mobile infrastructure offering a plethora of emerging services, and video streams are among the main contributors to this unprecedented traffic explosion. The paradigm of dynamic spectrum access (DSA) was coined and has become a widely accepted technology to boost the capacity of next generation networks. In this work, we propose an enhanced video coding implementation jointly with a DSA protocol wherein the sender actively picks up the most appropriate access strategy contingent on the instantaneous spectrum observations. The legacy frequencies pool has been extended to include the spectral ranges left unused in the television (TV) bands. A specific coding framework using a concatenation of multiple description coding and layered coding is investigated. The video is decomposed into two sub-streams through even and odd frames separation, then the resulting sub-streams are encoded into a scalable structure producing two independent and cumulative descriptions. The sender chooses to make use of the extended spectrum pool exclusively when the default frequencies are no more advantageous in terms of network capacity. Through extensive simulations, we compare the performance of the cellular network with and without the proposed DSA strategy. It has been proven that the proposed video transmission scheme using hybrid spectrum access provides a significant enhancement in network capacity up to 6.3Mbps and the capacity gain outweighs the disadvantage of interference incidents due to licensed-cognitive, co-channel and adjacent channel coexistence, besides retrieving an improved perceived video quality at the customer side illustrated by an increment of up to 6.8dB (0.176, respectively) in terms of the achieved PSNR-HVS-M (MS-SSIM, respectively).