Dynamic Adaptive Streaming Over HTTP Research Articles

Video quality adaptation using CNN and RNN models for cost-effective and scalable video streaming Services

AbstractVideo streaming services require adaptive bit rate strategies that optimize video quality based on network conditions and user preferences to provide a cost-effective and scalable solution. In this manuscript, we present a novel architecture that utilizes a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to extract features from the video stream and predict optimal bit rates for each frame. The CNN feature extractor extracts relevant features from the input video stream, which are then passed on to the RNN predictor, where the optimal bit rate is predicted to ensure a personalized viewing experience tailored to the current network conditions and specific user preferences. Our experimental results demonstrate that the proposed architecture achieves significant improvements in video quality while minimizing bandwidth usage and providing a better user experience. Specifically, our results show a 37.1% improvement in average bit rate, indicating that the proposed architecture optimizes the video quality and reduces bandwidth usage by 37.1% on average. We also observed a 16.6% improvement in Quality of Experience (QoE), meaning that users will perceive the video quality as better. Additionally, rebuffering was reduced by 87.5%, indicating that users enjoy smoother video playback without interruptions. Importantly, our architecture is optimized for Dynamic Adaptive Streaming over HTTP (DASH), addressing the need for efficient streaming over the most widely used protocol in the industry.

The Protocol Play: DASHing Ahead or HLSing Back?

This paper presents a comprehensive comparative analysis of two widely adopted adaptive streaming protocols, Dynamic Adaptive Streaming over HTTP (DASH) and HTTP Live Streaming (HLS), within the contemporary landscape of video delivery. Delving into the background of both protocols, the study explores their architectures, media format support, and adaptability to diverse network conditions. A meticulous examination of strengths and weaknesses encompasses performance, scalability, user experience, device compatibility, and implementation ease. The implications for content delivery, including considerations of quality of experience, Content Delivery Network (CDN) compatibility, and cost factors, are thoroughly discussed. The paper also sheds light on current trends, emerging technologies, and the evolving nature of DASH and HLS. In conclusion, the study provides a synthesis of key insights, offering practical recommendations for protocol selection based on specific use cases and contributing valuable perspectives to the dynamic realm of adaptive video streaming.

A Visual Sensitivity Aware ABR Algorithm for DASH via Deep Reinforcement Learning

In order to cope with the fluctuation of network bandwidth and provide smooth video services, adaptive video streaming technology is proposed. In particular, the adaptive bitrate (ABR) algorithm is widely used in dynamic adaptive streaming over HTTP (DASH) to improve quality of experience (QoE). However, existing ABR algorithms still ignore the inherent visual sensitivity of human visual system (HVS). As the final receiver of video, HVS has different sensitivity to the quality distortion of different video content, and video content with high visual sensitivity needs to allocate more bitrate resources. Therefore, existing ABR algorithms still have limitations in reasonably allocating bitrate and maximizing QoE. To solve this problem, this paper designs an adaptive bitrate strategy from the perspective of user vision, studies the modeling of visual sensitivity, and proposes a visual sensitivity aware ABR algorithm. We extract a set of content features and attribute features from the video, and consider the simulation of HVS to establish a total masking effect model that reflects the visual sensitivity more accurately. Further, the network status, buffer occupancy, and visual sensitivity are comprehensively considered under a deep reinforcement learning framework to select the appropriate bitrate for maximizing QoE. We implement the proposed algorithm over a realistic trace-driven evaluation and compare its performance with several latest algorithms. Experimental results show that our algorithm can align ABR strategy with visual sensitivity to achieve better QoE in high visual sensitivity content, and improves the average perceptual video quality and overall user QoE by 18.3% and 22.8%, respectively. Additionally, we prove the feasibility of our algorithm through subjective evaluation in the real environment.

Adaptive Scalable Video Streaming (ASViS): An Advanced ABR Transmission Protocol for Optimal Video Quality

Multimedia video streaming, identified as the dominant internet data consumption service, brings forth challenges in consistently delivering optimal video quality. Dynamic Adaptive Streaming over HTTP (DASH), while prevalent, often encounters buffering problems, causing video pauses due to empty video buffers. This study introduces the Adaptive Scalable Video Streaming (ASViS) protocol as a solution. ASViS incorporates scalable video coding, a flow-controlled User Datagram Protocol (UDP), and deadline-based criteria. A model is developed to predict the behavior of ASViS across varying network conditions. Additionally, the effects of diverse parameters on ASViS performance are evaluated. ASViS adjusts data flow similarly to the Transmission Control Protocol (TCP), based on bandwidth availability. Data are designed to be discarded by ASViS according to video frame deadlines, preventing outdated information transmission. Compliance with RFC 8085 ensures the internet is not overwhelmed. With its scalability feature, ASViS achieves the highest possible image quality per frame, aligning with Scalable Video Coding (SVC) and the available data layers. The introduction of ASViS offers a promising approach to address the challenges faced by DASH, potentially providing more consistent and higher-quality video streaming.

Influence of video content type on the usefulness of reinforcement learning algorithms in DASH systems

The article presents the result of research on DASH (Dynamic Adaptive Streaming over HTTP) systems. In the proposed solution, the adaptive algorithm is based on the RL (Reinforcement Learning) paradigm. The Pensieve algorithm was chosen as the basis for the tests. This algorithm is widely discussed in the scientific literature and therefore the study and analysis of its properties is useful in a wide range of solutions using DASH. The main contribution of the presented test results to the development of knowledge on video streaming services consists in the analysis of the impact of the characteristics of video materials on the effectiveness of the adaptation process implemented by the developed RL model. The presented results show that this influence should not be omitted in any in-depth analyses of the characteristics of DASH systems.

Cache-Enabled Adaptive Video Streaming: A QoE-Based Evaluation Study

Dynamic Adaptive Streaming over HTTP (DASH) has prevailed as the dominant way of video transmission over the Internet. This technology is based on receiving small sequential video segments from a server. However, one challenge that has not been adequately examined is the obtainment of video segments in a way that serves both the needs of the network and the improvement in the Quality of Experience (QoE) of the users. One effective way to achieve this is to implement and study caching and DASH technologies together. This paper investigates this issue by simulating a network with multiple video servers and a video client. It then implements both the peer-to-many communications in the context of adaptive video streaming and the video server caching algorithm based on proposed criteria that improve the status of the network and/or the user. Specifically, we investigate the scenario of delivering DASH-based content with the help of an intermediate server, apart from a main server, to demonstrate possible caching benefits for different sizes of intermediate storage servers. Extensive experimentation using emulation reveals the interplay and delicate balance between caching and DASH, guiding such network design decisions. A general tendency found is that, as the available buffer size increases, the video playback quality increases to some extent. However, at the same time, this improvement is linked to the random cache selection algorithm.

A Taxonomy for Generative Adversarial Networks in Dynamic Adaptive Streaming Over HTTP

Generative Adversarial Networks (GANs) have emerged as a powerful tool in the field of Dynamic Adaptive Streaming over HTTP (DASH) to enhance various aspects of video streaming. This paper presents a taxonomy that categorizes the applications and techniques of GANs in the context of DASH. The taxonomy covers several key dimensions, including video generation, compression, quality enhancement, bandwidth adaptation, dynamic bitrate streaming, and cross-modal applications. Within each dimension, specific subcategories are identified to capture the diverse applications of GANs in DASH. Additionally, evaluation metrics for assessing the quality and effectiveness of GAN-based approaches are discussed. The taxonomy serves as a comprehensive framework to understand and organize the different ways in which GANs can be utilized to improve the streaming experience in DASH. By providing an organized structure, this taxonomy facilitates better understanding, comparison, and exploration of GAN-based approaches in DASH and enables researchers and practitioners to identify areas for further research and development.

A Framework for Meta-Learning in Dynamic Adaptive Streaming over HTTP

This work presents a framework with a taxonomy on meta-learning used in Dynamic Adaptive Streaming over HTTP (DASH). With the increasing complexity of network conditions and user preferences, there is a need for efficient adaptation mechanisms in DASH to provide optimal quality of experience (QoE) for users. Meta-learning, or learning to learn, has emerged as a promising approach to enhance adaptive streaming algorithms in DASH by leveraging prior knowledge and experiences. The proposed framework provides a systematic and structured approach for applying meta-learning techniques in the context of DASH. It encompasses essential components, including data collection and preprocessing, meta-model architecture, meta-training, meta-testing, fine-tuning, and continuous improvement. The taxonomy within the framework categorizes various aspects of meta-learning in DASH, such as meta-learning approaches, components, objectives, and applications

A Framework for Multi-Task Learning in Dynamic Adaptive Streaming Over HTTP

This paper presents a framework with a taxonomy for multi-task learning in the context of Dynamic Adaptive Streaming over HTTP (DASH). DASH is a widely used technology for video streaming, and multi-task learning has emerged as a promising approach to enhance the performance and user experience of DASH systems by jointly optimizing multiple related tasks. The framework provides a structured approach to design, train, and evaluate multi-task learning models in DASH, while the taxonomy categorizes the key components and approaches within the framework. The taxonomy includes task types, multitask learning approaches, input features, and training strategies. Task types encompass video quality adaptation, buffer management, bandwidth estimation, content pre-fetching, and resource allocation, representing the specific tasks involved in DASH. Multi-task learning approaches encompass methodologies such as shared representation learning, task-specific layers, multi-head architectures, knowledge distillation, and reinforcement learning, offering flexibility in model design and optimization. Input features cover video characteristics, network conditions, device capabilities, and user preferences, providing the necessary information for informed decision-making across tasks. Training strategies include joint training, alternate training, hierarchical training, task weighting, and task balancing, determining how the multi-task learning model is trained and optimized. By following the presented framework and taxonomy, researchers and practitioners can systematically approach the design, training, and evaluation of multi-task learning models in DASH. The framework enables the development of efficient and adaptive video streaming systems by leveraging the interdependencies among tasks. The taxonomy helps organize the components and approaches within the framework, aiding in a better understanding of the various aspects of multi-task learning in the DASH context. Overall, this framework and taxonomy provide a valuable resource for advancing the field of multitask learning in the dynamic and complex domain of video streaming over HTTP.

360 8K Viewport-Independent VR

Over the past few years 360 virtual reality (VR) has been deployed using different techniques. Viewport-independent technology has been used on 4K content for delivery to 4K-capable head-mounted displays (HMDs) and smartphone devices, resulting in a disappointing experience. The alternative is using viewport-dependent technology with 8K content on 4K-capable HMDs and smartphones devices, which enables a good experience, but with complexities and limitations in terms of the integration into existing over-the-top (OTT) workflows. The 8K viewport-independent technology presented uses off-the-shelf encoding techniques to compress 8K 360 VR content as a single file and to distribute it in a common media application framework (CMAF) low-latency Dynamic Adaptive Streaming over HTTP (DASH) mode to 8K-capable devices such as the Oculus Quest 2 or Galaxy S20. This article presents an end-to-end 8K VR workflow, which is entirely cloud-based and capable of delivering high-quality 8K VR digital rights management (DRM)-protected content compared with 4K content on different devices. We present the initial results of the trial performed by the VR Study Group of the Streaming Video Alliance (SVA) in collaboration with the VR Industry Forum (VRIF), where 8K VR content was encoded and streamed on different very-high-speed (fiber and data over cable service interface specification (DOCSIS)) networks.

Toward Efficient Multicodec Streaming

One of the biggest challenges in modern-era media delivery systems is the fragmentation of the population of receiving devices in terms of codec capabilities. For example, modern Apple devices can decode and seamlessly switch between H.264/Advanced Video Coding (AVC) and high efficiency video encoding (HEVC) streams. Most new TVs or set-top boxes can also decode HEVC, but they cannot switch between HEVC and H.264/AVC streams in the same streaming session. And there are still plenty of older devices/streaming clients that can only receive and decode H.264/AVC streams. With the arrival of next-generation codecs, such as AV1 and Versatile Video Coding (VVC), the fragmentation of codec support across devices becomes even more complex. This situation prompts the question—how can we serve such a population of devices most efficiently by using codecs delivering the best performance in all cases yet producing the minimum possible number of streams such that the overall cost of media delivery is minimal? In this article, we explain how this problem can be formalized and solved at the stage of dynamic generation of encoding profiles for adaptive bitrate (ABR) streaming. The proposed solution effectively generalizes the per-title or context-aware encoding (CAE) class of techniques, considering multiple sets of renditions generated using each codec and codec usage distributions by the population of the receiving devices. We also discuss additional system-level means (proper manifest generation, HLS and Dynamic Adaptive Streaming over HTTP (DASH) quality annotations, device detection, and edge logic) needed to make the proposed solution practically deployable.

Redirection and Protocol Mechanisms in Content Delivery Network-Edge Servers for Adaptive Video Streaming

Redirection and protocol techniques are key components of the infrastructure for Content Delivery Networks (CDNs) that aid in the delivery of Multimedia Internet services to end-consumers. Redirection methods are used to route the user’s request to the nearest edge server, minimizing distance, improving delivery times, and lowering latency. Protocol mechanisms, such as HTTP Live Streaming (HLS), Dynamic Adaptive Streaming over HTTP (DASH), and Real-Time Messaging Protocol (RTMP), are used to deliver adaptive video streaming. These protocols are designed to transfer the adaptive streaming and provide high-quality video playback. They also allow the system to adjust the video quality based on the network conditions of the Quality of Service (QoS). Inadequate transmission protocols and poorly designed redirection algorithms are two major challenges that might degrade Cloud–CDN performance. These challenges lead to excessive latency, poor quality of service, and significant packet loss that have potential influences on the user experience. In this paper, firstly, three protocols are proposed by preparing a case study on selecting the optimal protocol for replicating adaptive video streaming content. Secondly, a redirection algorithm based on the Modified Cuckoo Search Algorithm (MCSA) is proposed to provide an accurate redirecting process of selected edge servers to end-users. Test results indicate that, when hybrid FASP/HTTP protocols were chosen (from original server to replicate server and to end-users), the delivery of adaptive video streaming segments was fast with lower latency. The average estimated time needed for replicating video content based on FASP is 25% better than that needed for File catalyst and Signiant protocols. Therefore, the Cuckoo search method presents more efficient results for selecting the optimal edge server for 100 servers, which is 0.296 s, compared to the conventional algorithm, which was 13 s.

ATSC 3.0, DVB-I, and TV 3.0 Services via 5G Broadcast—System Design and Reference Tools

5G broadcast brings the promise of an independent or complementary linear TV experience, which can reuse existing terrestrial broadcast infrastructure to serve high definition (HD) or 4K live content to mobile devices, possibly independent of a mobile subscription. To benefit fully from this new capability, the design of a service layer is essential, including service discovery procedures and delivery protocols over IP multicast. This article examines the set of the different standardized service layers brought by the 3GPP, Advanced Television Systems Committee (ATSC), and Digital Video Broadcasting (DVB), which could target different regions, ecosystems, or regulatory requirements. To support media service providers, a full ecosystem of tools is now emerging, including the Reference Tools from the 5G Media Action Group (5G-MAG) and the GPAC open-source solution. We will introduce the consolidated work from 3GPP and 5G-MAG to make this vision a reality, in terms of standardization and development of reference tools. We will detail how DVB-I services over 5G broadcast can now be experimented with commodity tools and commercial devices. We will also identify the potential synergies with ATSC 3.0 A/3311and the possibilities for Brazil’s TV 3.0 project,2for which Real-time Object delivery over Unidirectional Transport (ROUTE)/dynamic adaptive streaming over HTTP (DASH) is the transport layer of choice and 5G broadcast is a candidate for the physical layer.

Deep Reinforcement Learning-based Bitrate Adaptations in Dynamic Adaptive Streaming over HTTP

Dynamic adaptive streaming over HTTP (DASH) has been a superior video streaming technology in recent years. Bitrate adaptation function at video player plays a vital role in guaranteeing a high quality-of-experience for the users. This work evaluates the performance of several advanced deep reinforcement learning algorithms, i.e., deep Q-learning, actor-critic, and proximal policy optimization, applied in bitrate adaptations and compares them with other rate adaptation methods with real-trace datasets.

Reinforcement learning - based adaptation and scheduling methods for multi-source DASH

Dynamic adaptive streaming over HTTP (DASH) has been widely used in video streaming recently. In DASH, the client downloads video chunks in order from a server. The rate adaptation function at the video client enhances the user?s quality-of-experience (QoE) by choosing a suitable quality level for each video chunk to download based on the network condition. Today networks such as content delivery networks, edge caching networks, contentcentric networks, etc. usually replicate video contents on multiple cache nodes. We study video streaming from multiple sources in this work. In multi-source streaming, video chunks may arrive out of order due to different conditions of the network paths. Hence, to guarantee a high QoE, the video client needs not only rate adaptation, but also chunk scheduling. Reinforcement learning (RL) has emerged as the state-of-the-art control method in various fields in recent years. This paper proposes two algorithms for streaming from multiple sources: RL-based adaptation with greedy scheduling (RLAGS) and RL-based adaptation and scheduling (RLAS). We also build a simulation environment for training and evaluation. The efficiency of the proposed algorithms is proved via extensive simulations with real-trace data.

Adaptive Partially Reliable Delivery of Immersive Media Over QUIC-HTTP/3

The increasing popularity of head-mounted displays (HMD) and depth cameras has encouraged content providers to offer interactive immersive media content over the internet. Traditionally, dynamic adaptive streaming over HTTP (DASH) is the go-to standard for video streaming. However, HTTP is built on top of protocols such as the transmission control protocol (TCP), which prioritize reliability over latency, thereby, inducing additional delay due to acknowledgments and retransmissions, especially on lossy networks. In addition, such reliable protocols suffer from head-of-line (HOL) blocking problem at various levels, leading to playout interruptions of the video streaming application. The third generation of HTTP, i.e., HTTP/3 was recently introduced to deal with the issues posed by TCP. As a major change, HTTP/3 replaces TCP with QUIC at the transport layer, which solves the HOL problem at the transport layer. Moreover, the datagram extension of QUIC allows for unreliable data delivery, just like UDP, which could substantially reduce the latency. Combining this feature of QUIC with the quality adaptation capabilities of DASH-based streaming could bring interactive immersive media delivery to the next level. This work proposes the integration of DASH with the concept of partial reliability of QUIC to reduce playout interruptions and increase the quality of the delivered immersive content on lossy networks. Here, the DASH scheme takes quality and prioritization decisions based on the changing network conditions and the user’s viewport, respectively. Then, the part of the content with the highest priority, i.e., within the viewport, is delivered reliably and the rest unreliably. To the best of our knowledge, this is the first work to combine adaptive streaming with partial reliability. Herein, we provide an implementation of a headless player which supports HTTP/3 over partially reliable QUIC as well as state-of-the-art protocols like HTTP/3 over reliable QUIC and HTTP/2 over TCP. We performed an extensive evaluation of our proposed solution using real-world 5G throughput traces and bursty packet loss conditions, using point cloud streaming as the use case. Firstly, our evaluation shows that HTTP/2 is highly intolerant to loss and not suitable for streaming immersive media. Furthermore, even at a loss as high as 5%, the partially reliable framework achieves 46% higher throughput and delivers the content with 33% fewer playout interruptions compared to the reliable counterpart. Since current point cloud decoders are sensitive to loss, we applied the forward error correction mechanism to the data sent unreliably to ensure that the client decodes the content at a probability of 99.9%. Applying this overhead to our solution provides a significant gain of 25% in the throughput compared to the state of the art.

E-Ensemble: A Novel Ensemble Classifier for Encrypted Video Identification

In recent years, video identification within encrypted network traffic has gained popularity for many reasons. For example, a government may want to track what content is being watched by its citizens, or businesses may want to block certain content for productivity. Many such reasons advocate for the need to track users on the internet. However, with the introduction of the secure socket layer (SSL) and transport layer security (TLS), it has become difficult to analyze traffic. In addition, dynamic adaptive streaming over HTTP (DASH), which creates abnormalities due to the variable-bitrate (VBR) encoding, makes it difficult for researchers to identify videos in internet traffic. The default quality settings in browsers automatically adjust the quality of streaming videos depending on the network load. These auto-quality settings also increase the challenge in video detection. This paper presents a novel ensemble classifier, E-Ensemble, which overcomes the abnormalities in video identification in encrypted network traffic. To achieve this, three different classifiers are combined by using two different combinations of classifiers: the hard-level and soft-level combinations. To verify the performance of the proposed classifier, the classifiers were trained on a video dataset collected over one month and tested on a separate video dataset captured over 20 days at a different date and time. The soft-level combination of classifiers showed more stable results in handling abnormalities in the dataset than those of the hard-level combination. Furthermore, the soft-level classifier combination technique outperformed the hard-level combination with a high accuracy of 81.81%, even in the auto-quality mode.

Machine Learning at the Mobile Edge: The Case of Dynamic Adaptive Streaming Over HTTP (DASH)

Dynamic Adaptive Streaming over HTTP (DASH) is a standard for delivering video in segments and adapting each segment’s bitrate (quality), to adjust to changing and limited network bandwidth. We study segment prefetching, informed by machine learning predictions of bitrates of client segment requests, implemented at the network edge. We formulate this client segment request prediction problem as a supervised learning problem of predicting the bitrate of a client’s next segment request, in order to prefetch it at the mobile edge, with the objective of jointly improving the video streaming experience for the users and network bandwidth utilization for the service provider. The results of extensive evaluations showed a segment request prediction accuracy of close to 90% and reduced video segment access delay with a cache hit ratio of 58%, and reduced transport network load by lowering the backhaul link utilization by 60.91%.

KQI Assessment of VR Services: A Case Study on 360-Video Over 4G and 5G

Extended Reality (XR) arises as one of the current cutting-edge educational and entertainment emergent technologies. This service differs from traditional video streaming approaches due to its immersive experience, which allows the user to enjoy omnidirectional multimedia. However, the service experience must be guaranteed to avoid side effects such as cybersickness or disorientation. This work presents a framework to assess 360-video service streaming performance over mobile networks through Key Quality Indicators (KQIs) using VR (Virtual Reality) HMD (Head Mounted Device). The testbed is composed of a 360-video client for DASH (Dynamic Adaptive Streaming over HTTP), which playbacks multimedia content from a video server located in the cloud while KQI measuring tasks are performed in the user as on the network sides. Various metrics are collected such as resolution, frame rate, initial playback time, throughput, stall events, and round trip time (RTT), among various others. Finally, a performance comparison between LTE and 5G technologies is provided. Results from the KQI measurement highlight the potential of the new generation of mobile networks in the provision of service with high-quality levels of experience.

A Novel Dynamic Bit Rate Analysis Technique for Adaptive Video Streaming over HTTP Support

Recently, there has been an increase in research interest in the seamless streaming of video on top of Hypertext Transfer Protocol (HTTP) in cellular networks (3G/4G). The main challenges involved are the variation in available bit rates on the Internet caused by resource sharing and the dynamic nature of wireless communication channels. State-of-the-art techniques, such as Dynamic Adaptive Streaming over HTTP (DASH), support the streaming of stored video, but they suffer from the challenge of live video content due to fluctuating bit rate in the network. In this work, a novel dynamic bit rate analysis technique is proposed to model client-server architecture using attention-based long short-term memory (A-LSTM) networks for solving the problem of smooth video streaming over HTTP networks. The proposed client system analyzes the bit rate dynamically, and a status report is sent to the server to adjust the ongoing session parameter. The server assesses the dynamics of the bit rate on the fly and calculates the status for each video sequence. The bit rate and buffer length are given as sequential inputs to LSTM to produce feature vectors. These feature vectors are given different weights to produce updated feature vectors. These updated feature vectors are given to multi-layer feed forward neural networks to predict six output class labels (144p, 240p, 360p, 480p, 720p, and 1080p). Finally, the proposed A-LSTM work is evaluated in real-time using a code division multiple access evolution-data optimized network (CDMA20001xEVDO Rev-A) with the help of an Internet dongle. Furthermore, the performance is analyzed with the full reference quality metric of streaming video to validate our proposed work. Experimental results also show an average improvement of 37.53% in peak signal-to-noise ratio (PSNR) and 5.7% in structural similarity (SSIM) index over the commonly used buffer-filling technique during the live streaming of video.