Adaptive Video Streaming System Research Articles

A Framework for Multi-Task Learning in Dynamic Adaptive Streaming Over HTTP

This paper presents a framework with a taxonomy for multi-task learning in the context of Dynamic Adaptive Streaming over HTTP (DASH). DASH is a widely used technology for video streaming, and multi-task learning has emerged as a promising approach to enhance the performance and user experience of DASH systems by jointly optimizing multiple related tasks. The framework provides a structured approach to design, train, and evaluate multi-task learning models in DASH, while the taxonomy categorizes the key components and approaches within the framework. The taxonomy includes task types, multitask learning approaches, input features, and training strategies. Task types encompass video quality adaptation, buffer management, bandwidth estimation, content pre-fetching, and resource allocation, representing the specific tasks involved in DASH. Multi-task learning approaches encompass methodologies such as shared representation learning, task-specific layers, multi-head architectures, knowledge distillation, and reinforcement learning, offering flexibility in model design and optimization. Input features cover video characteristics, network conditions, device capabilities, and user preferences, providing the necessary information for informed decision-making across tasks. Training strategies include joint training, alternate training, hierarchical training, task weighting, and task balancing, determining how the multi-task learning model is trained and optimized. By following the presented framework and taxonomy, researchers and practitioners can systematically approach the design, training, and evaluation of multi-task learning models in DASH. The framework enables the development of efficient and adaptive video streaming systems by leveraging the interdependencies among tasks. The taxonomy helps organize the components and approaches within the framework, aiding in a better understanding of the various aspects of multi-task learning in the DASH context. Overall, this framework and taxonomy provide a valuable resource for advancing the field of multitask learning in the dynamic and complex domain of video streaming over HTTP.

An Optimized Bandwidth Estimation for Adaptive Video Streaming Systems Using WLBWO Algorithm

The increasing popularity of streaming video is a cause of concern for the stability of the internet because most streaming video content is currently delivered via UDP without any end-to-end congestion control. Since the internet relies on end systems implementing transmit rate regulation, there has recently been significant interest in congestion control mechanisms that are both fair to TCP and effective in delivering real-time streams. Streaming video over the internet requires dealing with bandwidth and delay that vary over time. Many video streaming applications address this problem by adapting the quality of the scalable video. But it produces poor quality service, and sending data on this channel results in buffering time. To trounce these issues, this paper proposed optimized bandwidth estimation for adaptive video streaming systems using the WLBWO algorithm. Originally, the input video is compressed by using the UHE algorithm. Next, the system proposes a KEECC to securely transfer the data. Then, the encrypted data is sent to the receiver via a multipath channel. Before sending the data to the receiver, the bandwidth is estimated by using the WLBWO. Finally, the inverse process is performed. Extensive experimental results showed the effectiveness of the proposed system than conventional methods.

Deep Video Precoding

Several groups worldwide are currently investigating how deep learning may advance the state-of-the-art in image and video coding. An open question is how to make deep neural networks work in conjunction with existing (and upcoming) video codecs, such as MPEG H.264/AVC, H.265/HEVC, VVC, Google VP9 and AOMedia AV1, AV2, as well as existing container and transport formats, without imposing any changes at the client side. Such compatibility is a crucial aspect when it comes to practical deployment, especially when considering the fact that the video content industry and hardware manufacturers are expected to remain committed to supporting these standards for the foreseeable future. We propose to use deep neural networks as precoders for current and future video codecs and adaptive video streaming systems. In our current design, the core precoding component comprises a cascaded structure of downscaling neural networks that operates during video encoding, prior to transmission. This is coupled with a precoding mode selection algorithm for each independently-decodable stream segment, which adjusts the downscaling factor according to scene characteristics, the utilized encoder, and the desired bitrate and encoding configuration. Our framework is compatible with all current and future codec and transport standards, as our deep precoding network structure is trained in conjunction with linear upscaling filters (e.g., the bilinear filter), which are supported by all web video players. Extensive evaluation on FHD (1080p) and UHD (2160p) content and with widely-used H.264/AVC, H.265/HEVC and VP9 encoders, as well as a preliminary evaluation with the current test model of VVC (v.6.2rc1), shows that coupling such standards with the proposed deep video precoding allows for 8% to 52% rate reduction under encoding configurations and bitrates suitable for video-on-demand adaptive streaming systems. The use of precoding can also lead to encoding complexity reduction, which is essential for cost-effective cloud deployment of complex encoders like H.265/HEVC, VP9 and VVC, especially when considering the prominence of high-resolution adaptive video streaming.

SASRT: Semantic-Aware Super-Resolution Transmission for Adaptive Video Streaming over Wireless Multimedia Sensor Networks

There are few network resources in wireless multimedia sensor networks (WMSNs). Compressing media data can reduce the reliance of user’s Quality of Experience (QoE) on network resources. Existing video coding software, such as H.264 and H.265, focuses only on spatial and short-term information redundancy. However, video usually contains redundancy over a long period of time. Therefore, compressing video information redundancy with a long period of time without compromising the user experience and adaptive delivery is a challenge in WMSNs. In this paper, a semantic-aware super-resolution transmission for adaptive video streaming system (SASRT) for WMSNs is presented. In the SASRT, some deep learning algorithms are used to extract video semantic information and enrich the video quality. On the multimedia sensor, different bit-rate semantic information and video data are encoded and uploaded to user. Semantic information can also be identified on the user side, further reducing the amount of data that needs to be transferred. However, identifying semantic information on the user side may increase the computational cost of the user side. On the user side, video quality is enriched with super-resolution technologies. The major challenges faced by SASRT include where the semantic information is identified, how to choose the bit rates of semantic and video information, and how network resources should be allocated to video and semantic information. The optimization problem is formulated as a complexity-constrained nonlinear NP-hard problem. Three adaptive strategies and a heuristic algorithm are proposed to solve the optimization problem. Simulation results demonstrate that SASRT can compress video information redundancy with a long period of time effectively and enrich the user experience with limited network resources while simultaneously improving the utilization of these network resources.

The simulation study on the multipath adaptive video transmission

The article presents simulation studies of multipath video transmission implemented as a live monitoring system based on adaptive streaming mechanisms. The transmission network consists of the Wi-Fi infrastructure and a single LTE cell. Both wireless networks are used during the operation of the video monitoring system. The analysis of the obtained results allows us to compare the features of two currently used multipath transmission protocols, MultiPath Quick UDP Internet Connections (MPQUIC) and MultiPath TCP (MPTCP) in the context of their use in adaptive video streaming systems. Also, the article contains a summary of the perceived advantages and disadvantages of using several transmission networks simultaneously for the implementation of video surveillance and monitoring systems.

QoE-Driven Mobile Edge Caching Placement for Adaptive Video Streaming

Caching at mobile edge servers can smooth temporal traffic variability and reduce the service load of base stations in mobile video delivery. However, the assignment of multiple video representations to distributed servers is still a challenging question in the context of adaptive streaming, since any two representations from different videos or even from the same video will compete for the limited caching storage. Therefore, it is important, yet challenging, to optimally select the cached representations for each edge server in order to effectively reduce the service load of base station while maintaining a high quality of experience (QoE) for users. To address this, we study a QoE-driven mobile edge caching placement optimization problem for dynamic adaptive video streaming that properly takes into account the different rate-distortion (R–D) characteristics of videos and the coordination among distributed edge servers. Then, by the optimal caching placement of representations for multiple videos, we maximize the aggregate average video distortion reduction of all users while minimizing the additional cost of representation downloading from the base station, subject not only to the storage capacity constraints of the edge servers, but also to the transmission and initial startup delay constraints of the users. We formulate the proposed optimization problem as an integer linear program to provide the performance upper bound, and as a submodular maximization problem with a set of knapsack constraints to develop a practically feasible cost benefit greedy algorithm. The proposed algorithm has polynomial computational complexity and a theoretical lower bound on its performance. Simulation results further show that the proposed algorithm is able to achieve a near-optimal performance with very low time complexity. Therefore, the proposed optimization framework reveals the caching performance upper bound for general adaptive video streaming systems, while the proposed algorithm provides some design guidelines for the edge servers to select the cached representations in practice based on both the video popularity and content information.

Modeling and Design of Adaptive Video Streaming Control Systems

Adaptive video streaming systems aim at providing the best user experience given the user device and the network available bandwidth. With this purpose, a controller selecting the video bitrate (or level) from a discrete set $\mathcal{L}$ has to be designed. The control goal is to maximize the video bitrate while avoiding playback interruptions and minimizing video bitrate switches. In this paper, we propose a hybrid dynamical system modeling the essential features of an important class of controllers for adaptive video streaming systems. We derive tuning rules to achieve key performance goals by sizing the control system parameters. We show how to: (i) tune the controller parameters to keep the video level switching frequency below a given target; (ii) design the video levels set $\mathcal{L}$ to obtain a performance tradeff between switching frequency and storage costs at the servers; (iii) find the minimum amount of playout buffer that should be stored to avoid rebuffering events with a given probability in case of temporary bandwidth drop. The theoretical results are validated through numerical simulation and experimental evaluation.

Application of Artificial Intelligence to Software Defined Networking: A Survey

Background/Objectives: This paper surveys the application of Artificial Intelligence (AI) to the Software Defined Networking (SDN) paradigm which is a part of previous efforts to give the computer networks the ability of being programmed based on the separation between the control and forwarding planes. In SDN approach, the controller represents the central brain of the network which leads to an advanced level of flexibility and network intelligence. Methods/Statistical Analysis: Different artificial intelligence-based techniques have been applied to achieve an enhanced load balance, network security and intelligent network applications in the SDN approach. Findings: Ant colony algorithms were successful in increasing the maximal Quality of Experience (QoE) by 24.1% compared with the shortest path routing approach. Neural network based intrusion prevention system has shown a scalable performance with low false positive rate. Applying reinforcement learning based technique in adaptive video streaming system compared with the shortest path routing and greedy-based approaches has shown decreasing of the frame loss rate by 89% and 70% respectively. Applications/Improvements: This study highlights the first attempts for applying artificial intelligence in SDN paradigm. However, hybrid intelligent techniques could be the key for achieving more advanced behaviour in SDN-based networks.

A Content-Aware Video Adaptation Service to Support Mobile Video

Adaptive video streaming systems rely on the availability of different quality versions of a video. Such a system can dynamically adjust the quality of a video stream during its playback depending on the available network throughput. Even if the necessary throughput is available, mobile users can benefit from limiting the generated data traffic as most cellular network contracts have data caps. Usually, if the cap is reached, the throughput is throttled to a speed that does not allow video streaming. Existing systems react to varying network conditions but often neglect content-specific adaptation needs. Content inspection can help to save data traffic when a higher bitrate representation would not increase the perceived quality. In this work, we present the Video Adaptation Service (VAS), a support service for a content-aware video adaptation for mobile devices. Based on the video content, the adaptation process is improved for both the available network resources and the perception of the user. By leveraging the content properties of a video stream, the system is able to maintain a stable video quality and at the same time reduce the generated data traffic. The system is evaluated with different adaptation schemes and shows that content-specific adaptation can both increase the perceived quality as well as reduce the data traffic. Additionally, we demonstrate the practical feasibility of this approach by integrating the VAS into Dynamic Adaptive Streaming over the Hypertext Transfer Protocol.

Adaptive Video Streaming With Optimized Bitstream Extraction and PID-Based Quality Control

To cope with the challenges brought about by bandwidth fluctuation and improve the experience of watching online videos, an adaptive video streaming system that can adjust video quality according to actual network conditions is proposed based on the scalable video coding (SVC) extension of H.264/AVC. First, a simple and effective linear error model is proposed and verified for quality scalability of SVC. The model exploits the linear feature of pixel value errors and can be used to accurately estimate the distortion caused by discarding any combination of enhancement data packets in an SVC bitstream. On that basis, a greedy-like algorithm is designed to assign each data packet a priority value according to its rate-distortion (R-D) impact, thus enabling R-D optimized bitstream extraction under certain bitrate constraints. Finally, the proportional-integral-derivative (PID) method is utilized to control the video quality adjustment and determine a suitable bitrate for transmission. By monitoring and predicting the past, current, and future bandwidth information, the PID-based quality control algorithm is able to reduce quality fluctuation, while still preserving a high quality level. Experimental results show that compared with the baseline software, the proposed system that integrates the above algorithms can achieve much lower video quality fluctuation, with PSNR variance reduced from 1.24 to 0.69, and at the same time deliver higher video quality, with the PSNR average increased by 0.83 dB.

Learning-based approach for layered adaptive video streaming over SDN

Software-defined networking is a recently emerging paradigm that decouples the control and data planes of computer networks. It allows for the implementation of application-specific routing algorithms, and the advantages that the SDN architecture enables can be used to enhance the performance of multimedia communication applications. In this paper, we propose an adaptive video streaming system with a learning-based approach, running over SDN. In the proposed video streaming system, we use a novel learning model to determine the optimal time to re-route the traffic flows and to change the bitrate of the video. The learning model aims to minimize the packet loss rate, quality changes and controller cost while adapting the flow routes and video quality. We have tested the performance of the learning-based approach by comparing it to traditional Internet routing and the greedy approach. The results show that the proposed system significantly outperforms the traditional Internet routing approach and the greedy approach in terms of quality of experience (QoE) and network cost under different network scenarios.

Efficiency of Dynamic GOP Length in Video Stream

Multiple description codes are designed for multiple path video streaming with channel diversities and an efficient MDC technique based on the applied spatial domain processing features. In this paper, we investigate the performance of multi-path video streaming using a multiple description coding (MDC) technique, especially the impacts of several codec variables, such as different lengths of the Group of Pictures (GOP), and the different startup times of I-frame in each independent MDC sub-streams. The overall performance of the system is evaluated by the frame dropout rate, which indicates the probability of playing back frozen frames of the reconstructed video at the receiver devices. To address dynamic channel conditions for video transmissions, we propose an adaptive coding technique by adjusting the GOP lengths according to the channel condition and combinatorial mathematics optimization in length of GOP adjustment. Different approaches, such as sub-dividing GOPs for a single stream and multiple sub-streams, and an adaptive MDC video streaming system, are examined in this paper.

Design and Implementation of a Network-Adaptive Mechanism for HTTP Video Streaming

This paper proposes a network-adaptive mechanism for HTTP-based video streaming over wireless/mobile networks. To provide adaptive video streaming over wireless/mobile networks, the proposed mechanism consists of a throughput estimation scheme in the time-variant wireless network environment and a video rate selection algorithm used to increase the streaming quality. The adaptive video streaming system with proposed modules is implemented using an open source multimedia framework and is validated over emulated wireless/mobile networks. The emulator helps to model and emulate network conditions based on data collected from actual experiments. The experiment results show that the proposed mechanism provides higher video quality than the existing system provides and a rate of video streaming almost void of freezing.

Design and Implementation of a Network-Adaptive Mechanism for HTTP Video Streaming

This paper proposes a network-adaptive mechanism for HTTP-based video streaming over wireless/mobile networks. To provide adaptive video streaming over wireless/mobile networks, the proposed mechanism consists of a throughput estimation scheme in the time-variant wireless network environment and a video rate selection algorithm used to increase the streaming quality. The adaptive video streaming system with proposed modules is implemented using an open source multimedia framework and is validated over emulated wireless/mobile networks. The emulator helps to model and emulate network conditions based on data collected from actual experiments. The experiment results show that the proposed mechanism provides higher video quality than the existing system provides and a rate of video streaming almost void of freezing.

Modeling and Evaluation of a Metadata-Based Adaptive P2P Video-Streaming System

In this paper, we present a multi-parent adaptive video-streaming system (MAVSS). MAVSS is a cooperative video-streaming system, based on the peer-to-peer (P2P) content distribution concept, to simultaneously adapt and stream video contents to heterogeneous users. In order to ensure video codec independence, we emphasize on structured, metadata-based adaptation. Therefore, MPEG-21 generic Bitstream syntax description is chosen to describe the parts of the video contents selected for adaptation operations. Additionally, without an optimal solution, it is hard to judge the efficiency of such systems. Therefore, in this paper, we first present the fundamental properties of MAVSS. We then present a mathematical model of the adaptive streaming system, where we define efficient streaming as an optimization of a cost function that can be solved as an integer linear programming problem. The model illustrates the relation among all the parameters that affect the resource contribution, resource utilization, load balancing and service fairness. We use this model to analyze the trade-offs that exist between service fairness and system efficiency.

A Cross-Layered Network-Adaptive Video Streaming Using Sender-Driven Wireless Channel Monitoring

In this letter, we propose a cross-layered network- adaptive video streaming system employing sender-driven wireless channel monitoring over a wireless LAN (WLAN). An intelligent sender with an embedded wireless access point (AP) estimates time-varying available bandwidth based on the observed transmission statistics of medium access control (MAC) and physical (PHY) layers. The estimated available bandwidth is then utilized to dynamically guide the network-adaptive control of temporally-scalable video flows.

Distributed Context Monitoring for the Adaptation of Continuous Services

This paper describes a middleware designed for distributed context acquisition and reconciliation intended to support the adaptation of continuous Internet services, like e.g., multimedia streaming. These services persist in time, and are characterized by multiple transmissions of data by the service provider, as a result of a single request from the user. Adapting these services to the current context requires the continuous monitoring of context data, and a real-time adjustment of the adaptation parameters upon the detection of a relevant context change. The proposed solution is based on asynchronous context change notifications, and specific techniques have been designed to minimize the number of unnecessary updates and the re-evaluation of policies. The paper also provides experimental results obtained by developing an adaptive video streaming system and running it on top of the proposed middleware.