Quality Of Experience Metrics Research Articles

Super-resolution with perceptual quality for improved live streaming delivery on edge computing

According to Cisco projections, mobile connection throughput is anticipated to more than triple by the year 2023, with the expected typical 5G connection throughput reaching 575 Mbps. Additionally, smartphones and TV sets equipped with computational capabilities for decoding 4K videos have significantly redefined the landscape of Internet video traffic, accounting for 82% of the total Internet traffic. This surge in connection throughput, coupled with the escalating demand for video-centric content, presents an unprecedented challenge in effectively managing traffic across the backhaul infrastructure of 5G networks. In this article, we introduce and evaluate the On-edge enhanced live streaming with super-resolution (ELiveSR) framework for live video delivery, which takes advantage of throughput improvements, in the access networks, and makes use of real-time super-resolution procedure to reduce transmission rate demands in the backhaul segment of the network. This SR procedure is carried by a deep neural network based model and is capable of coping with video content of different subjects. Experimental assessments conducted revealed that the proposed framework can reduce video-related traffic in the backhaul segment by up to 88.37%. Additionally, it simultaneously enhances the quality of experience metrics observed during live video streaming sessions.

Impact of Latency on QoE, Performance, and Collaboration in Interactive Multi-User Virtual Reality

Interactive, multi-user experiences are meant to define the present and future of Virtual Reality (VR). Such immersive experiences will typically consist of remote collaborations where content is streamed and/or synchronized over a network connection. Thus, real-time collaboration will be key. In this light, the responsiveness of the system and the network will define the overall experience. As such, understanding the effect of network distortions, especially related to time delay, on end-user’s perception (in terms of Quality-of-Experience (QoE)), performance, and collaboration becomes crucial. The existing literature, however, has mostly focused on network requirements from a system point-of-view, where the key performance parameters are only provided in the form of Quality-of-Service (QoS) parameters (such as end-to-end latency). However, the translation of these network impairments to the end-user experience is often omitted. The purpose of this paper is to fill the gap by providing a thorough investigation of the impact of latency on the perception of users while performing collaborative tasks in multi-user VR. To this end, an experimental framework was designed, developed, and tested. It is based on a multi-device synchronizing architecture, enabling two simultaneous users to work together in a gamified virtual environment. The developed test environment also allows for the identification of the most prominent network requirements and objective analysis for each traffic link. To experimentally investigate the impact of latency on user perception, a user study was conducted. Participants were paired and asked to perform the collaborative task under different latency-prone scenarios. The results show that users are able to easily distinguish between distorted and non-distorted network configurations. However, making a distinction between different manifestations of latency is much less straightforward. Moreover, factors such as the user’s role in the experience and the required task, and the level of interactivity and movement have an important influence on the subjective level of perception, the strength of the user’s preferences, and the occurrence of cybersickness. In contrast, no significant differences in objective metrics, such as system performance and user completion time were observed. These results can support the creation of collective QoE metrics that model the group as a whole rather than each individual separately. As such, this work provides an important step to dynamically counteract any drops in group dynamics and performance by means of smart interventions in the transmission system and/or virtual environment.

Using QoE Metric as a Decision Criterion in Multimedia Heterogeneous Network Optimization: Challenges and Research Perspectives

This article explores the growing importance of the QoE (Quality of Experience) metric as a fundamental criterion in the optimization of heterogeneous multimedia networks. We explore the benefits of using QoE, such as improving user experience, efficient resource management, and adaptation to network conditions. However, this paradigm presents technical challenges, including accurately measuring QoE and managing the complexity of heterogeneous networks. To address these challenges, we highlight promising research prospects, such as the development of advanced algorithms, real-time measurement of QoE, and the integration of machine learning. Ultimately, the use of QoE in the optimization of heterogeneous multimedia networks contributes to a substantial improvement in the quality of multimedia services offered to end users.

DashReStreamer: Framework for Creation of Impaired Video Clips under Realistic Network Conditions

The continuous rise of multimedia entertainment has led to an increased demand for delivering outstanding user experience of multimedia content. However, modelling user-perceived Quality of Experience (QoE) is a challenging task, resulting in efforts for better understanding and measurement of user-perceived QoE. To evaluate user QoE, subjective quality assessment, where people watch and grade videos, and objective quality assessment in which videos are graded using one or many objective metrics are conducted. While there is a plethora of video databases available for subjective and objective video quality assessment, these videos are artificially infused with various temporal and spatial impairments. Videos being assessed are artificially distorted with startup delay, bitrate changes, and stalls due to rebuffering events. To conduct a more credible quality assessment, a reproduction of original user experiences while watching different types of streams on different types and quality of networks is needed. To aid current efforts in bridging the gap between the mapping of objective video QoE metrics to user experience, we developed DashReStreamer, an open-source framework for re-creating adaptively streamed video in real networks. The framework takes inputs in the form of video logs captured by the client in a non-regulated setting, along with an.mpd file or a YouTube URL. The ultimate result is a video sequence that encompasses all the data extracted from the video log. DashReStreamer also calculates popular video quality metrics like PSNR, SSIM, MS-SSIM and VMAF. Finally, DashReStreamer allows creating impaired video sequences from the popular streaming platform, YouTube. As a demonstration of framework usage we created a database of 332 realistic video clips, based on video logs collected from real mobile and wireless networks. Every video clip is supplemented with bandwidth trace and video logs used in its creation and also with objective metrics calculation reports. In addition to dataset, we performed subjective evaluation of video content, assessing its effect on overall user QoE. We believe that this dataset and framework will allow the research community to better understand the impacts of video QoE dynamics.

Multifactor QoE scores in IoT topologies

Quality of Experience (QoE) is key to Internet of Things (IoT) applications. Indeed, depending on the nature of the application, different application message loss and latency requirements lead to different QoE metrics. In this context, and considering as inspiration the Mean Option Scores (MOS) typically associated with Real Time Communication (RTC), this paper introduces QoE scores that can be used to assess the performance of IoT applications. Taking into account physical, link, network, transport, and application layer features like the transmission rate, network layer loss and latency, the topology and the session layer protocol under consideration, linear regression mechanisms are used to estimate these QoE scores. Some of these features are regular system parameters while others are extracted from standard mechanisms and protocols used for traffic analysis as Key Performance Indicators (KPIs). This model is then trained and evaluated against a large number of Wireless Personal Area Network (WPAN) and Low Power Wide Area Network (LPWAN) architectures.

Neural adaptive IoT streaming analytics with RL-Adapt

IoT stream processing is an emerging technology that plays a crucial role in enabling time-critical IoT applications, which often demand high accuracy and low latency. For example, Augmented Reality (AR) applications require high object detection precision and object localization precision, as well as low latency. Note that the exact definition of accuracy varies depending on the specific application Existing stream processing engines are insufficient to meet these requirements, since they could not integrate and respond timely to variable network conditions in the dynamic wireless environment. Recent efforts focusing on adaptive streaming support user-specified policies to adapt to the variable network conditions. However, existing works suffer from manual policies or simplified models of the deployment environment which limit their ability to achieve optimal performance across a broad set of network conditions and quality of experience (QoE) objectives. In this paper, we present a Reinforcement Learning-based Adaptive streaming system (RL-Adapt) that is capable of generating adaption policies using RL-strategy and providing declarative APIs for efficient development. RL-Adapt trains a neural network model that can automatically select the optimal policy based on the observed network conditions. RL-Adapt does not rely on pre-defined models or assumptions about the environment. Instead, it learns to make decisions solely through observations of the resulting performance of past decisions. We implemented RL-Adapt and evaluated its performance extensively in three representative real-world IoT applications. Our results show that RL-Adapt outperforms the state-of-the-art schemes in terms of QoE metrics.

Improving quality of experience in adaptive low latency live streaming

HTTP Adaptive Streaming (HAS), the most prominent technology for streaming video over the Internet, suffers from high end-to-end latency when compared to conventional broadcast methods. This latency is caused by the content being delivered as segments rather than as a continuous stream, requiring the client to buffer significant amounts of data to provide resilience to variations in network throughput and enable continuous playout of content without stalling. The client uses an Adaptive Bitrate (ABR) algorithm to select the quality at which to request each segment to trade-off video quality with the avoidance of stalling to improve the Quality of Experience (QoE). The speed at which the ABR algorithm responds to changes in network conditions influences the amount of data that needs to be buffered, and hence to achieve low latency the ABR needs to respond quickly. Llama (Lyko et al. 28) is a new low latency ABR algorithm that we have previously proposed and assessed against four on-demand ABR algorithms. In this article, we report an evaluation of Llama that demonstrates its suitability for low latency streaming and compares its performance against three state-of-the-art low latency ABR algorithms across multiple QoE metrics and in various network scenarios. Additionally, we report an extensive subjective test to assess the impact of variations in video quality on QoE, where the variations are derived from ABR behaviour observed in the evaluation, using short segments and scenarios. We publish our subjective testing results in full and make our throughput traces available to the research community.

A Cloud Computing Based Deep Compression Framework for UHD Video Delivery

Ultra-high-definition (UHD) videos has increased popularity. However, the data size of UHD videos is 4-16 times larger of HD videos. This brings many challenges to existing video delivery systems, such as the shortage of network bandwidth resources and longer network transmission latency. In this paper, we propose a cloud computing based deep compression framework named Pearl, which utilizes the power of deep learning and cloud computing to compress UHD videos. Pearl compresses UHD videos from two respects: the frame resolution and the colorful information. In pearl, an optimal compact representation of the original UHD video is learned with two deep convolutional neural networks (DCNNs): super resolution CNN (SR-CNN) and colorization CNN (CL-CNN). SR-CNN is used to reconstruct a high-resolution video from a low-resolution video, while CL-CNN is adopted to preserve the color information of the video. Pearl focuses on video content compression in two new directions. Thus, it can be integrated with any existing video compression system. We evaluate the performance of Pearl with a wide variety of network conditions, quality of experience metrics, and video properties. In all considered scenarios, Pearl can further compress 84% of video size and reduce 73% of network transmission latency.

Live360: Viewport-Aware Transmission Optimization in Live 360-Degree Video Streaming

Live 360-degree video streaming is emerging as the next disruptive innovation in the coming era. However, compared to conventional live video streaming, it is much more challenging to provide high-quality live 360-degree video streaming due to high bandwidth demand, stringent latency requirement, and diverse user viewports. In this paper, we propose a viewport-aware live 360-degree video streaming framework called Live360 to optimize end-to-end video stream transmission. To make the problem tractable, we decompose the transmission optimization problem into two sub-problems, and optimize upstream and downstream transmission in an asynchronous way. We prioritize 360-degree cameras according to users’ real-time viewing interests and upload attractive contents with a higher bitrate. We also re-define the metric of QoE (Quality-of-Experience) for live 360-degree video viewers, and solve the optimization problems efficiently using dynamic programming. Extensive experiments show that Live360 is lightweight and significantly outperforms other state-of-the-art methods in terms of various QoE metrics. The average QoE of Live360 is twice that of other baseline methods.

Industrial User Experience Index vs. Quality of Experience Models

Network operators have increased interest in understanding and quantifying the user perceived quality of experience (QoE). Appropriate QoE models allow mapping of measurable QoS parameters to QoE metrics like mean opinion scores (MOS) or poor-or-worse (PoW) ratios. Unfortunately, there are no QoE models for all applications available yet, and development and standardization is rather time-consuming. For that reason, industry often implements simple but effective approaches based on expert knowledge to get a user experience index able to deliver high value insights. This index is often defined based on thresholds of measurable QoS parameters to classify the resulting user experience. The question arises whether such a threshold-based user experience index can properly estimate the QoE in a system. We therefore answer in this work whether the user experience index is able to quantify the expected QoE in a system and if other QoE metrics like PoW ratios can be derived. Furthermore, we discuss the interpretation and the limits of the index. As a key contribution, we present a detailed systematic analysis of the threshold-based QoE concept and discuss its usability as QoE estimation methodology for industry.

Qrator: An Interest-Aware Approach to ABR Streaming Based on User Engagement

Conventional approaches to adaptive bitrate (ABR) streaming fail to further improve the quality of experience (QoE) as they are indifferent to video semantics and user behaviors by focusing solely on the network conditions. These ABR schemes cannot enhance the perceptual video quality as they neglect the users’ varying degree of interest (DoI) over video sections and may experience QoE degradation due to frequent rebuffers caused by user behaviors such as selectively watching the interesting sections. To this end, this article proposes <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Qrator</i> , which is the very first attempt to utilize timestamps and likes as user engagement data to infer users’ genuine interests and further elevate the user-perceived QoE. Based on timestamps and likes in video comments, Qrator improves the overall QoE by considering DoI variations and user behaviors in performing interest-aware rate adaptation and prefetching. Qrator can be widely applied to conventional ABR approaches without significantly modifying their implementations. Evaluation results show that Qrator can heighten the bitrates of user interest sections without degrading the average bitrate. Furthermore, applying Qrator under user behavior patterns can reduce the rebuffering ratio and the number of rebuffers by 32% and 31%, respectively, while maintaining other QoE metrics to a similar extent.

D2D-assisted user-centric adaptive video transmission in next generation cellular networks

Next-generation cellular networks need to provide seamless connectivity with higher data rates, increased capacity, and enhanced network coverage. As multimedia service demands in various heterogeneous devices grow rapidly compared to the underlying network’s capacity and bandwidth, the adaptation in multimedia streaming services is essential for providing satisfactory Quality of Experience (QoE). This paper develops a Device-to-Device (D2D)-assisted Utility-based Adaptive Multimedia (video) Streaming scheme (UAMS) using D2D communication in a 5th Generation (5G) cellular network where low-battery users may extend their streaming duration by spending lower reception energy with the help of D2D-assisted communication. The adaptation algorithm considers four utility functions: quality, power consumption, packet error ratio, and remaining battery of the user devices to adapt the bitrate dynamically and augment viewers’ experience. We formulate an optimization problem to maximize the joint utility function to provide the best adaptive multimedia content selected for transmission to the end-users either directly or via D2D Relay Nodes (DRNs) in every scheduling interval. We use a graph theoretic approach for choosing the best DRNs. Extensive simulations show the efficacy of the proposed scheme in terms of saved battery energy, churn rate, and QoE metrics compared to a few well-known existing schemes in the literature that do not use D2D communication.

Opportunistic Transmission for Video Streaming over Wild Internet

The video streaming system employs adaptive bitrate (ABR) algorithms to optimize a user’s quality of experience. However, it is hard for ABR algorithms to choose the right bitrate consistently under highly dynamic bandwidth fluctuations in wild Internet. In this article, we propose a building block on the client side named Opportunistic Chunk Replacement Mechanism (OCRM) to help existing ABR algorithms make full use of the available bandwidth to improve the network utilization and viewing experience of users. Specifically, the servers take advantages of the spare bandwidth to opportunistically transmit high-quality chunks (called opportunistic chunks ) with low priority to the client, without incurring any extra delay. Then, the client player replaces the low-quality chunks with the opportunistic ones that have high quality. We compare OCRM with state-of-the-art ABR algorithms by using trace-driven experiments spanning a wide variety of quality of experience metrics and network conditions. The test results show that OCRM effectively achieves high network utilization and improves the user’s viewing experience by up to 35%.

A Reinforcement Learning-Based Routing for Real-Time Multimedia Traffic Transmission over Software-Defined Networking

Recently, video streaming services consumption has grown massively and is foreseen to increase even more in the future. The tremendous traffic usage has negatively impacted the network’s quality of service due to network congestion and end-to-end customers’ satisfaction represented by the quality of experience, especially during evening peak hours. This paper introduces an intelligent multimedia framework that aims to optimise the network’s quality of service and users’ quality of experience by taking into account the integration of Software-Defined Networking and Reinforcement Learning, which enables exploring, learning, and exploiting potential paths for video streaming flows. Moreover, an objective study was conducted to assess video streaming for various realistic network environments and under low and high traffic loads to obtain two quality of experience metrics; video multimethod assessment fusion and structural similarity index measure. The experimental results validate the effectiveness of the proposed solution strategy, which demonstrated better viewing quality by achieving better customers’ quality of experience, higher throughput and lower data loss compared with the currently existing solutions.

Learning Tailored Adaptive Bitrate Algorithms to Heterogeneous Network Conditions: A Domain-Specific Priors and Meta-Reinforcement Learning Approach

Internet adaptive video streaming is a typical form of video delivery that leverages adaptive bitrate (ABR) algorithms to provide video services with high quality of experience (QoE) for various users in diverse and unique network conditions. Such heterogeneous network environments, which can be viewed as exogenous input processes, often lead to the unstable performance of ABR algorithms. Unfortunately, learning-based ABR algorithm which generated by state-of-the-art reinforcement learning (RL) technologies achieves <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">good average performance</i> but fails to perform well in all kinds of network conditions. In this work, considering the video playback process as the Input-driven Markov Decision Process (IMDP), we propose <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}^{2}$ </tex-math></inline-formula> BR (Adaptation of ABR), a novel meta-RL ABR approach. <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}^{2}$ </tex-math></inline-formula> BR is mainly composed of an online stage and an offline stage. It leverages meta-RL to learn an initial meta-policy with various network conditions at the offline stage and makes decisions in personalized network conditions at the online stage. At the same time, we continually optimize the meta-policy to the tailor-made ABR policy for varying the current network environment within few shots. Moreover, in order to improve the learning efficiency, we fully utilize domain knowledge for implementing a virtual player to replay the previously experienced network. Using trace-driven experiments on various scenarios including different vehicles, users, network types, and heterogeneous user-preferences, we show that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}^{2}$ </tex-math></inline-formula> BR outperforming recent ABR approaches with rapidly adapting to the personalized QoE metrics and specific network conditions. Testbed experimental results also illustrate the superiority of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}^{2}$ </tex-math></inline-formula> BR in adapting to the unseen environments.

QoE-Driven IoT Architecture: A Comprehensive Review on System and Resource Management

Internet of Things (IoT) services have grown substantially in recent years. Consequently, IoT service providers (SPs) are emerging in the market and competing to offer their services. Many IoT applications utilize these services in an integrated manner with different Quality-of-Service (QoS) requirements. Thus, the provisioning of end-to-end QoS is getting more indispensable for IoT platforms. However, provisioning the system by using only QoS metrics without considering user experiences is not sufficient. Recently, Quality of Experience (QoE) model has become a promising approach to quantify actual user experiences of services. A holistic design approach that considers constraints of various QoS/QoE metrics together is needed to satisfy requirements of these applications and services. Besides, IoT services may operate in environments with limited resources. Therefore, effective management of services and system resources is essential for QoS/QoE support. This paper provides a comprehensive survey for the state-of-the-art studies on IoT services with QoS/QoE perspective. Our contributions are threefold: (1) QoE-driven architecture is demonstrated by classifying vital components according to QoE-related functions in prior studies, (2) QoE metrics and QoE optimization objectives are classified by corresponding system and resource control problems in the architecture, and (3) QoE-aware resource management e.g., QoE-aware offloading, placement and data caching policies with recent Machine Learning approaches are extensively reviewed.

Digital Twin of Wireless Systems: Overview, Taxonomy, Challenges, and Opportunities

Future wireless services will focus on improving the quality of life by enabling various applications, such as extended reality, brain-computer interaction, and healthcare. These applications will have diverse performance requirements (e.g., user-defined quality of experience metrics, latency, and reliability) which will be challenging to be fulfilled by existing wireless systems. To meet the diverse requirements of the emerging applications, the concept of digital twins has been recently proposed. A digital twin uses a virtual representation along with security-related technologies (e.g., blockchain), communication technologies (e.g., 6G), computing technologies (e.g., edge computing), and machine learning, so as to enable the smart applications. In this tutorial, we present a comprehensive overview on digital twins for wireless systems. First, we present the fundamental concepts (i.e., design aspects, high-level architecture, and frameworks) of digital twins for wireless systems. Second, a comprehensive taxonomy is devised for two aspects, namely, twins for wireless and wireless for twins. For the twins for wireless aspect, we consider issues related to design of twin objects, physical devices, and interface, as well as prototyping, deployment trends, incentive mechanism, isolation of twins, and decoupling. For the wireless for twins aspect, we consider issues related to accessing twin objects, security and privacy, and air interface design are considered. Finally, open research challenges and opportunities are discussed.

Video streaming over IoV using IP multicast

Video traffic over Internet of vehicles is increasing day by day to provide passengers on-board infotainment services. A video streaming service over Internet of vehicles involves design challenges of vehicular gateway selection, and vehicle to vehicle collaboration. Vehicular gateway selection is challenging due to high vehicle mobility and time-varying link conditions. Existing works on vehicular gateway selection focus on either service cost minimization, or quality of experience. To achieve a balance between these two aspects, we propose a novel vehicular gateway selection protocol that minimizes streaming service cost, keeping end-to-end delivery delay as a constraint. In our work, the vehicular gateway selection problem is formulated centrally as the minimum set-covering problem, and is solved by the greedy approximation method. In existing works, vehicle to vehicle collaboration happens in the application layer ignoring the presence of network layer nodes which do not run any proprietary application. To make the vehicle to vehicle collaboration in the network layer, we propose a novel multicast protocol SS-CAST specifically designed for streaming service over vehicular networks. Compared to existing protocols, the simulation results show VSIM as the clear winner for minimizing service cost while giving a competitive performance in the quality of experience metrics.

Trusted cloud broker for estimating the reputation of cloud providers in federated cloud environment

AbstractIn federated cloud environment, it is too hard for the consumers to manage the services as multiple services may be rendered from multiple providers. Finding the trustable cloud service provider is an important and tedious process for the consumers before acquiring the services. In this proposed work, the reputation of the service providers are calculated through two different scenarios such as reputation estimation through consumer ratings and reputation estimation based on the trusted cloud broker (TCB) monitored data. The technical parameters such as availability of service, response time, bandwidth, throughput, and so forth, and nontechnical parameters such as service compliance, ease of access, support, readability of document, and so forth, of the rendered services are considered for analysis. The proposed model follows the fuzzy analytical hierarchical process principles that are used for ranking the services based on the quality of service and quality of experience metrics. The total reputations of the providers are calculated by combining the updated consumer reputation and the TCB estimated reputation. The performance of the proposed model is compared with the existing models such as RTM and CSTEM and found that the proposed model estimates the reputation with more accuracy.

Uplink vs. Downlink: Machine Learning-Based Quality Prediction for HTTP Adaptive Video Streaming.

Streaming video is responsible for the bulk of Internet traffic these days. For this reason, Internet providers and network operators try to make predictions and assessments about the streaming quality for an end user. Current monitoring solutions are based on a variety of different machine learning approaches. The challenge for providers and operators nowadays is that existing approaches require large amounts of data. In this work, the most relevant quality of experience metrics, i.e., the initial playback delay, the video streaming quality, video quality changes, and video rebuffering events, are examined using a voluminous data set of more than 13,000 YouTube video streaming runs that were collected with the native YouTube mobile app. Three Machine Learning models are developed and compared to estimate playback behavior based on uplink request information. The main focus has been on developing a lightweight approach using as few features and as little data as possible, while maintaining state-of-the-art performance.