Adaptive Bit Research Articles

Fleet: Improving Quality of Experience for Low-Latency Live Video Streaming

Low-latency live video streaming proposes more challenges on designing Adaptive Bit Rate (ABR) algorithms than video-on-demand streaming. In this paper, we present Fleet, a solution that improves user quality of experience (QoE) in terms of video quality, video switches, video freezes, and latency. Fleet is designed to meet two operational challenges at the same time: (1) optimizing QoE under dynamic mobile network conditions and (2) ensuring a low latency experience with minimal visual quality degradation. The core of Fleet is a stochastic model predictive controller that incorporates network conditions and client states for bitrate adaptation. The idle period problem presents great challenges for bandwidth measurement and client state prediction. Fleet consists of an HTTP chunk level bandwidth measurement algorithm, and a practical live video streaming evolution model for client state prediction. Besides, a throughput probability predictor is trained to capture the mobile network’s uncertainty. And, a triple threshold playback speed controller is designed for latency management. Fleet is practically implemented in dash.js and evaluated over both synthetic and real-world mobile network conditions. Our study shows that compared with the state-of-the-art solution, Fleet achieves an average QoE improvement of 39.7% and 37.9%, respectively. Moreover, Fleet has a good generalization, outperforming existing algorithms even on new video traces and network conditions.

Prioritizing protection communication in a 5G slice: Evaluating HTB traffic shaping and UL bitrate adaptation for enhanced reliability

Abstract5G network slicing is promising in prioritizing time‐critical protection communication in wireless networks of smart grids. However, network slicing offered by telecommunication providers encompasses all smart grid applications, lacking granularity. Smart grid automation standards provide recommendations on prioritization of protection communication to improve reliability but only for wired connections. Therefore, this paper investigates hierarchical token bucket (HTB) traffic shaping and uplink (UL) bitrate adaptation of a live video stream for prioritizing protection communication within a 5G slice. An experimental setup combines controller‐hardware‐in‐the‐loop (CHIL) with a quality of service (QoS) measurement system for validation. The system under test consists of commercial 5G networks, intelligent electronic devices (IEDs), and merging units for validation with three smart grid applications: fault location, line differential, and intertrip protection. HTB traffic shaping improves protected faults by 47.57% in congested and 1.16% in non‐congested scenarios. UL bitrate of the video stream adaptation by 2 Mbps increases protected faults by 3.69%. HTB traffic shaping even improves prioritization with a wired connection without introducing additional delays. HTB traffic shaping must be deployed in routers to maintain good QoS for critical applications. Collaboration between utilities and telecommunication providers is essential to effectively deploy 5G network slicing on smart grids.

PSTile: Perception-Sensitivity-Based 360$^\circ$ Tiled Video Streaming for Industrial Surveillance

360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> video becomes increasingly attractive in smart factories due to its immersive experience for industrial surveillance. However, transmitting this kind of video requires a fairly large demand on the bandwidth due to its high-resolution and panoramic view. Moreover, low-latency responses of the video to users' head movements are required. This leads to the tradeoff between high video quality and low-latency response under limited bandwidth in factories. This paper proposes a perception-sensitivity (PS) based 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> tiled video streaming method called PSTile for industrial surveillance. Specifically, a PS tiling strategy is constructed for valid tile grouping based on the designed PS index. Then a tile bitrate adaptation problem is formulated to allocate bitrates to valid tiles. It jointly optimizes surveillance accuracy, end-to-end latency, and quality-of-experience (QoE) of users under accuracy, latency, and bandwidth constraints. Simulations demonstrate that PSTile achieves at least 32.6% lower end-to-end latency and 19.3% higher average video bitrate than grid-tiling and clus-tiling methods.

Advanced Predictive Tile Selection Using Dynamic Tiling for Prioritized 360° Video VR Streaming

The widespread availability of smart computing and display devices such as mobile phones, gaming consoles, laptops, and tethered/untethered head-mounted displays has fueled an increase in demand for omnidirectional (360°) videos. 360° video applications enable users to change their viewing angles while interacting with the video during playback. This allows users to have a more personalized and interactive viewing experience. Unfortunately, these applications require substantial network and computational resources that the conventional infrastructure and end devices cannot support. Recently proposed viewport adaptive fixed tiling solutions stream only relevant video tiles based on user interaction with the virtual reality (VR) space to use existing transmission resources more efficiently. However, achieving real-time accurate viewport extraction and transmission in response to both head movements and bandwidth dynamics can be challenging, which can impact the user’s Quality of Experience (QoE). This article proposes innovative dynamic tiling-based adaptive 360° video streaming solutions in order to achieve high viewer QoE. First, novel and easy-to-scale tiling layout selection methods are introduced, and the best tiling layouts are employed in each adaptation interval based on the prediction-assisted visual quality metric and the observed viewport divergence. Second, a novel proactive tile selection approach is presented, which adaptively extracts tiles for each selected tiling layout based on two low-complex viewport prediction mechanisms. Finally, a practical dynamic tile priority-oriented bitrate adaptation scheme is introduced, which uniformly distributes the bitrate budget among different tiles during 360° video streaming. Extensive trace-driven experiments are conducted to evaluate the proposed solutions using head motion traces from 48 VR users for five 360° videos with tiling layouts of 4 × 3, 6 × 4, and 8 × 6 and segment durations of 1s, 1.5s, and 2s. The experimental evaluations show that the dynamic video tiling solutions achieve up to 11.2% more viewport matches and an average improvement in QoE of 9.7% to 18% compared to state-of-the-art 360° streaming approaches.

Insights From Generative Modeling for Neural Video Compression

While recent machine learning research has revealed connections between deep generative models such as VAEs and rate-distortion losses used in learned compression, most of this work has focused on images. In a similar spirit, we view recently proposed neural video coding algorithms through the lens of deep autoregressive and latent variable modeling. We present these codecs as instances of a generalized stochastic temporal autoregressive transform, and propose new avenues for further improvements inspired by normalizing flows and structured priors. We propose several architectures that yield state-of-the-art video compression performance on high-resolution video and discuss their tradeoffs and ablations. In particular, we propose (i) improved temporal autoregressive transforms, (ii) improved entropy models with structured and temporal dependencies, and (iii) variable bitrate versions of our algorithms. Since our improvements are compatible with a large class of existing models, we provide further evidence that the generative modeling viewpoint can advance the neural video coding field.

Improving quality of experience in adaptive low latency live streaming

HTTP Adaptive Streaming (HAS), the most prominent technology for streaming video over the Internet, suffers from high end-to-end latency when compared to conventional broadcast methods. This latency is caused by the content being delivered as segments rather than as a continuous stream, requiring the client to buffer significant amounts of data to provide resilience to variations in network throughput and enable continuous playout of content without stalling. The client uses an Adaptive Bitrate (ABR) algorithm to select the quality at which to request each segment to trade-off video quality with the avoidance of stalling to improve the Quality of Experience (QoE). The speed at which the ABR algorithm responds to changes in network conditions influences the amount of data that needs to be buffered, and hence to achieve low latency the ABR needs to respond quickly. Llama (Lyko et al. 28) is a new low latency ABR algorithm that we have previously proposed and assessed against four on-demand ABR algorithms. In this article, we report an evaluation of Llama that demonstrates its suitability for low latency streaming and compares its performance against three state-of-the-art low latency ABR algorithms across multiple QoE metrics and in various network scenarios. Additionally, we report an extensive subjective test to assess the impact of variations in video quality on QoE, where the variations are derived from ABR behaviour observed in the evaluation, using short segments and scenarios. We publish our subjective testing results in full and make our throughput traces available to the research community.

Performance analysis of H2BR: HTTP/2-based segment upgrading to improve the QoE in HAS

HTTP Adaptive Streaming (HAS) plays a key role in over-the-top video streaming with the ability to reduce the video stall duration by adapting the quality of transmitted video segments to the network conditions. However, HAS still suffers from two problems. First, it incurs variations in video quality because of throughput fluctuation. Adaptive bitrate (ABR) algorithms at the HAS client usually select a low-quality segment when the throughput drops to avoid stall events, which impairs the Quality of Experience (QoE) of the end-users. Second, many ABR algorithms choose the lowest-quality segments at the beginning of a video streaming session to ramp up the playout buffer early on. Although this strategy decreases the startup time, clients can be annoyed as they have to watch a low-quality video initially. To address these issues, we introduced the H2BR technique (H TTP/2-B ased R etransmission) (Nguyen et al. 33) that utilizes certain features of HTTP/2 (including server push, multiplexing, stream priority, and stream termination) for late transmissions of higher-quality versions of video segments already in the client buffer, in order to improve video quality. Although H2BR was shown to enhance the QoE, limited streaming scenarios were considered resulting in a lack of general conclusions on H2BR’s performance. Thus, this article provides a profound evaluation to answer three open questions: (i) how H2BR’s performance is impacted by parameters at the server side (i.e., various encoding specifications), at the network side (i.e., packet loss rate), and at the client side (i.e., buffer size) on the performance of H2BR; (ii) how H2BR outperforms other state-of-the-art approaches in different configurations of the parameters above; (iii) how to effectively utilize H2BR on top of ABR algorithms in various streaming scenarios. The experimental results show that H2BR’s performance increases with the buffer size and decreases with increasing packet loss rates and/or video segment duration. The number of quality levels can negatively or positively impact on H2BR’s performance, depending on the ABR algorithm deployed. In general, H2BR is able to enhance the video quality by up to 17% and 14% in scalable video streaming and in non-scalable video streaming, respectively. Compared with an existing retransmission technique (i.e., SQUAD Wang et al., ACM Trans Multimed Comput Commun Applic (TOMM) 13(3s): 45, 49), H2BR shows better results with more than 10% in QoE and 9% in the average video quality.

Fine-grained Caching and Resource Scheduling for Adaptive Bitrate Videos in Edge Networks

With the easy access to mobile networks and the proliferation of video applications, video traffic is occupying a great portion of the network traffic, which poses a new challenge of how to alleviate the heavy backhaul traffic and ensure the high quality of experience for video services. As a promising solution towards addressing this challenge, video caching in edge networks has recently received significant attention, which mostly considers the video popularity and the user preference for the video. However, few studies consider the user behavior and the user preference for different parts of the video that indeed have an essential impact on caching efficiency. Hence, this article proposes a new caching and resource scheduling scheme for adaptive bitrate videos by incorporating these fine-grained factors. We first model the video service problem as a nonlinear integer programming problem, which can be divided into a cache placement problem and an online resource scheduling problem. Then, we design efficient algorithms based on several techniques, including greedy strategy, relaxation, and rounding, to solve the two problems. Extensive experimental results based on two real-world datasets show that the proposed solution achieves superior performance compared with several state-of-the-art caching approaches.

Intelligent Video Streaming at Network Edge: An Attention-Based Multiagent Reinforcement Learning Solution

Video viewing is currently the primary form of entertainment for modern people due to the rapid development of mobile devices and 5G networks. The combination of pervasive edge devices and adaptive bitrate streaming technologies can lessen the effects of network changes, boosting user quality of experience (QoE). Even while edge servers can offer near-end services to local users, it is challenging to accommodate a high number of mobile users in a dynamic environment due to their restricted capacity to maximize user long-term QoE. We are motivated to integrate user allocation and bitrate adaptation into one optimization objective and propose a multiagent reinforcement learning method combined with an attention mechanism to solve the problem of multiedge servers cooperatively serving users. Through comparative experiments, we demonstrate the superiority of our proposed solution in various network configurations. To tackle the edge user allocation problem, we proposed a method called attention-based multiagent reinforcement learning (AMARL), which optimized the problem in two directions, i.e., maximizing the QoE of users and minimizing the number of leased edge servers. The performance of AMARL is proved by experiments.

Low-complexity frequency-domain nonlinear equalizer with absolute operation for underwater wireless optical communications.

A low-complexity 3rd-order frequency-domain nonlinear equalizer (FD-NLE) with absolute operation is proposed and experimentally demonstrated for underwater wireless optical communications (UWOC). In the proposed FD-NLE scheme, absolute operation and fast Fourier transform (FFT) with multiplication operations are utilized instead of the square and convolution operations used in conventional polynomial nonlinear equalizers (PNLEs), respectively. Therefore, complexity reductions by over 77.3% and 66.9% can be achieved compared with those of PNLE and PNLE with absolute operation, respectively, with a memory length larger than 8. A UWOC system using orthogonal frequency division multiplexing (OFDM) signals with adaptive bit and power loading is also demonstrated to evaluate the performance of the proposed scheme. Experimental results show that data rate increments by ∼ 5.6% and ∼ 5.7% at BER below 7% hard-decision forward error correction (HD-FEC) limit of 3.8 × 10-3, compared with PNLE and PNLE with absolute operation, respectively, are realized using the proposed scheme. Meanwhile, the proposed scheme has an up to 14.7% complexity reduction compared with conventional frequency-domain PNLE (FD-PNLE), while maintaining similar equalization performance.

A Taxonomy for Deep Learning in Dynamic Adaptive Video Streaming Over HTTP

Deep Learning (DL) has become a fundamental technology in the field of Dynamic Adaptive Video Streaming over HTTP (DASH), enabling significant advancements in video streaming systems. This taxonomy presents a novel framework for categorizing and organizing the diverse applications and methodologies of DL in DASH. The taxonomy encompasses various aspects of DL, including video representation, quality of experience (QoE) estimation, bitrate adaptation, buffer management, content- and context-aware adaptation, and network optimization. By providing a comprehensive overview of DL in DASH, this taxonomy serves as a valuable resource for researchers and practitioners, facilitating a better understanding of the different DL techniques and their applications in enhancing video streaming performance and user experience.

To transcode or not? A machine learning based edge video caching and transcoding strategy

The variable network conditions of end-users demand different resolutions, formats, and bitrate versions of videos to be delivered over the network. Fetching each video from the Content Delivery Network (CDN) burdens all network layers. A promising solution is to leverage Mobile Edge Computing (MEC). This paper presents a Machine Learning based caching and transcoding model, which helps release the burden on the backhaul links of the network. The purposed scheme contains a task scheduler and time estimator. The time estimator predicts the job’s transcoding time based on the Virtual Machines (VMs) load. The task scheduler maps the transcoding task to different VMs regarding the cost feasibility, Quality of Service (QoS) of the users, and the cost-to-performance ratio of VMs. For this purpose, we prepare a dataset of 500 videos and transcode each video in every lower representation using Amazon Elastic Compute Cloud (EC2). The time estimator is trained on 77% of the video dataset containing more than 80,000 transcoding time data of different videos. The simulation results show that the proposed scheme outperforms its counterparts in terms of cost, average delay perceived by the user, and backhaul burden.

Toward Efficient Multicodec Streaming

One of the biggest challenges in modern-era media delivery systems is the fragmentation of the population of receiving devices in terms of codec capabilities. For example, modern Apple devices can decode and seamlessly switch between H.264/Advanced Video Coding (AVC) and high efficiency video encoding (HEVC) streams. Most new TVs or set-top boxes can also decode HEVC, but they cannot switch between HEVC and H.264/AVC streams in the same streaming session. And there are still plenty of older devices/streaming clients that can only receive and decode H.264/AVC streams. With the arrival of next-generation codecs, such as AV1 and Versatile Video Coding (VVC), the fragmentation of codec support across devices becomes even more complex. This situation prompts the question—how can we serve such a population of devices most efficiently by using codecs delivering the best performance in all cases yet producing the minimum possible number of streams such that the overall cost of media delivery is minimal? In this article, we explain how this problem can be formalized and solved at the stage of dynamic generation of encoding profiles for adaptive bitrate (ABR) streaming. The proposed solution effectively generalizes the per-title or context-aware encoding (CAE) class of techniques, considering multiple sets of renditions generated using each codec and codec usage distributions by the population of the receiving devices. We also discuss additional system-level means (proper manifest generation, HLS and Dynamic Adaptive Streaming over HTTP (DASH) quality annotations, device detection, and edge logic) needed to make the proposed solution practically deployable.

Toward Low-Latency and High-Quality Adaptive 360$^\circ$ Streaming

Advanced 360-degree video streaming is essential for high-quality communication and industrial video applications, supporting novel and interactive visual experiences that promote consumption. However, due to the inherent fluctuations in communication networks, hardware resources, and power costs, a trade-off needs to be achieved between occupying network bandwidth and guaranteeing 360-degree visual quality. To address this issue, this paper proposes a quality-aware global optimization solution that combines different types of sensory characteristics to further enhance the 360-degree streaming efficiency in cellular networks. First, considering the characteristics of 360-degree video, each frame is divided into three different regions based on a proposed field-of-view (FoV) prediction method. Second, a new region-based rate-distortion model reflecting the bitrate-quality relationship is proposed. The divided regions are represented by different rate-distortion models. In addition, a rate-distortion model parameter update strategy with robustness to region changes is proposed to further guarantee transmission performance. Finally, we propose a globally optimized adaptive bitrate allocation algorithm to optimize 360-degree mobile streaming, which uses both the rate-distortion models and viewpoint prediction results. Evaluation results indicate that the proposed method outperforms state-of-the-art approaches in terms of several quality of experience (QoE) objectives under various network conditions.

Receive-diversity-aided power-fading compensation for C-band IM/DD OFDM systems.

A receive-diversity-aided power-fading compensation (RDA-PFC) scheme is proposed and demonstrated to eliminate the chromatic dispersion (CD)-induced power fading for C-band double-sideband (DSB) intensity modulation and direct detection (IM/DD) orthogonal frequency division multiplexing (OFDM) systems. By combining the responses before and after a dispersive element using a maximal-ratio combining (MRC) algorithm, the CD-induced power fading dips within the signal bandwidth of around 50 GHz can be effectively compensated for, which results in an up to 17.6-dB signal-to-noise ratio (SNR) improvement for the fading subcarriers after transmission over 10 km of standard single-mode fiber (SSMF). Using the 16 quadrature amplitude modulation (QAM) format, a diversity receiver with the proposed RDA-PFC scheme can support 170.6-Gbit/s OFDM signal transmission over a 10-km SSMF and reduces the bit error rate (BER) by more than an order of magnitude compared with a conventional receiver. Moreover, 208.1-Gbit/s adaptive bit and power loading OFDM signal transmission over a 10-km SSMF is realized by the proposed RDA-PFC scheme, which improves the capacity by 15.3% compared with the case without RDA-PFC at a BER of 3.8 × 10-3. The proposed RDA-PFC scheme shows great potential in CD-induced power-fading compensation for high-speed IM/DD OFDM systems.

High Speed DMT for 224 Gb/s and Faster Wireline Transmission

This paper presents a high data rate wireline Discrete Multitone (DMT) transceiver capable of operating at 252 Gbps and higher for short range chip-to-chip and chip-to-module applications. The use of DMT allows for higher spectral efficiency than PAM SERDES systems, using similar bandwidth as proposed 224 Gb/s PAM-4 systems but achieves higher data rates. Data rates have improved over other published wireline DMT designs with the use of narrow sub-channels with both adaptive bit- and power-loading along with a combined transmit and receive side equalization. Simulation and measured results show that the transmit side pre-compensation has improved the BER by a factor of 104 at 250 Gb/s. The system is measured over a channel with a loss of 18 dB at 50 GHz, showing a data rate of 252 Gb/s with a BER of 10-6, and a data rate of 400 Gb/s with a BER of 10-2.

Stable yellow light emission from lead-free copper halides single crystals for visible light communication

Yellow light-emitting diodes (LEDs) as soft light have attracted abundant attention in lithography room, museum and art gallery. However, the development of efficient yellow LEDs lags behind green and blue LEDs, and the available perovskites yellow LEDs suffer from the instability. Herein, a pressure-assisted cooling method is proposed to grow lead-free CsCu 2 I 3 single crystals, which possess uniform surface morphology and enhanced photoluminescence quantum yield (PLQY) stability, with only 10% PLQY losses after being stored in air after 5000 ​h. Then, the single crystals used for yellow LEDs without encapsulation exhibit a decent Correlated Color Temperature (CCT) of 4290 ​K, a Commission Internationale de l'Eclairage (CIE) coordinate of (0.38, 0.41), and an excellent 570-h operating stability under heating temperature of 100 ​°C. Finally, the yellow LEDs facilitate the application in wireless visible light communication (VLC), which show a −3 dB bandwidth of 21.5 ​MHz and a high achievable data rate of 219.2 Mbps by using orthogonal frequency division multiplexing (OFDM) modulation with adaptive bit loading. The present work not only promotes the development of lead-free single crystals, but also inspires the potential of CsCu 2 I 3 in the field of yellow illumination and wireless VLC. The facile pressure-assisted cooling method is proposed to prepare lead-free CsCu2I3 single crystals, which promotes potential applications in both lighting for special occasions and wireless visible light communication.

Fountain Coded Streaming for SAGIN With Learning-Based Pause-and-Listen

Efficient adaptive bitrate streaming (ABS) over long-distance and lossy space-air-ground-integrated network (SAGIN) links is an important demand, where fountain transmission is attractive as it does not require feedback/retransmissions. However, the conventional fountain scheme may waste many packets due to the propagation delay of stop messages. This letter proposes a novel fountain coded ABS design with a pause-and-listen approach which, across segments of ABS, learns proper pause time in an on-line manner based on multi-armed bandit modeling. Evaluation using a real-life implementation shows that the scheme can avoid most packet wastes while still maintaining the key benefits of the fountain transmission.

Deep Reinforcement Learning-based Bitrate Adaptations in Dynamic Adaptive Streaming over HTTP

Dynamic adaptive streaming over HTTP (DASH) has been a superior video streaming technology in recent years. Bitrate adaptation function at video player plays a vital role in guaranteeing a high quality-of-experience for the users. This work evaluates the performance of several advanced deep reinforcement learning algorithms, i.e., deep Q-learning, actor-critic, and proximal policy optimization, applied in bitrate adaptations and compares them with other rate adaptation methods with real-trace datasets.

Multipath transmission aware ABR algorithm for SVC HAS

Adaptive video streaming over Hypertext Transfer Protocol (HTTP) is one of the most popular applications that have been used on the Internet in the last decade. In such applications, clients change the quality of the received video throughout the streaming session with respect to current network conditions. In these systems, the use of scalable coded video is one of the alternatives that can obtain video files encoded at different qualities. In this paper, we propose an adaptive bitrate (ABR) algorithm that aims to improve the quality of not only future segments but also the segments in the buffer. The quality of the already buffered segments is improved by downloading additional layers of the SVC-coded video, hence, preventing network resources from being wasted while increasing Quality of Experience (QoE). We use multiple paths between the server and the clients for transferring the video packets. In such systems, the ABR algorithm should determine which segment and layer should be requested from which path and when. The proposed ABR algorithm selects a group of segments to increase the quality of the buffered segments and for future segments, and selects paths for each segment in this group. Simulation results show that clients achieve higher QoE by using our approach when compared with conventional ABR algorithms. Furthermore, we show that using multiple paths cannot guarantee that QoE is maximized unless the end-system is aware of multipath transmission.