End-user Quality Of Experience Research Articles

A Quality-of-Experience-Aware Framework for Versatile Video Coding-Based Video Transmission

Multimedia applications have expanded and resulted in huge demands for high resolution videos in order to enhance the end user Quality of Experience (QoE). Therefore, QoE becomes a critical factor that is often addressed in design considerations of Consumer Electronics (CE) devices. Furthermore, error prone transmission conditions can influence the QoE, potentially degrading the end-user multimedia experience. Although video coding standards such as Versatile Video Coding (VVC) take potential mitigation measures to improve the bandwidth reductions, it fails to address other significant network Quality of Service (QoS) parameters that impact transmission based QoE such as transmission delay and Packet Loss Rate (PLR) in the encoding processes. In response, this paper first derives a relationship between QoE that addresses the network QoS parameters and VVC dependent parameters. Subsequently, the derived QoE model is used with the VVC encoding process to adaptively optimize Lagrange parameters as well as the quantization parameter. Extensive experiments report significant improvements to VVC Test Model (VTM) 7.0, achieving superior QoE gains over the state-of-the-art research and up to 3.41% over the anchor implementations.

Survey on user perceived system factors influencing the QoE of audiovisual calls on smartphones.

With the widespread use of applications and services supporting audiovisual calls via smartphones, both in business and leisure contexts, a key challenge for service providers is meeting end user Quality of Experience (QoE) expectations and requirements. To successfully meet this challenge, there is a need to identify and analyze the key system-related factors impacting user perceived quality. In this paper, we contribute beyond state-of-the-art by conducting a large scale web-based questionnaire survey to investigate the system-related factors that subjects identify as most influential in contributing to their overall experience and quality perception. We focus in particular on leisure audiovisual calls, established via mobile devices. Our initial survey (Phase 1) was conducted in Feb. 2020, just prior to the outbreak of the COVID-19 pandemic (272 participants). To investigate if the importance of factors has changed due to increased usage of the service caused by the pandemic among the general population, we conducted a second survey (Phase 2) in October 2021 with 249 participants. Based on obtained results, we identify key system-related QoE influence factors belonging to three categories: media quality, functional support, and usability and service design. We observe no significant differences in user opinions and expectations prior to and during the period of increased service usage, despite different participant demographics and study time frames, thus contributing to generalizability of obtained results. Study results contribute to providing insights for designing future user studies investigating QoE, in terms of key factors that should be considered.

End-to-end simulation environment for mobile edge computing

Performances analysis of resource management techniques and algorithms for edge computing is a crucial step that reveals the effectiveness of edge service placement and migration strategies in terms of delay and usage. To be able to gauge how different optimisations affect the end user quality of experience a holistic approach is needed, so that the end-to-end delay can be measured in a combined setup where the edge infrastructure is integrated in the 5G system architecture. For these purposes we put forward an open source workflow based on the integration of proven simulators as SUMO, OMNeT++ and CloudSim, that will provide the means to create large scale urban mobile simulation environments. This paper discusses the benefits and challenges of such an approach, and provides example results that showcase a few aspects of the potential end-to-end delay analysis by processing the gathered simulation output.

Deep-Reinforcement-Learning-Based Resource Allocation for Content Distribution in Fog Radio Access Networks

With the rapid development of wireless communication technologies, the emerging multimedia applications make mobile Internet traffic grow explosively while putting forward higher service requirements for the next-generation wireless networks. Therefore, how to achieve low-latency content transmission by effectively allocating heterogeneous network resources to improve the network quality of service and end-user quality of experience is a key issue to be solved urgently in the current Internet. In this article, we propose a deep reinforcement learning (DRL)-based resource allocation scheme to improve content distribution in a layered fog radio access network (FRAN). We formulate the optimal resource allocation problem as a minimal delay model, where in-network caching is deployed and the same content requests from mobile users can be aggregated in the queue of each base station. To cope with the increasing user requests and overcome capacity constraints of the FRAN, moreover, a cloud–edge cooperation offloading scheme is utilized in our model, where the integrated allocation of caching, computing, and communication resources and joint optimization between in-network caching and routing are considered to promote resource utilization and content delivery. In our solution, a new DRL policy is designed to make cross-layer cooperative caching and routing decisions for the arriving content requests according to request history information and available network resources in the system. Simulation results demonstrate that our proposed model can performs much better than the existing cloud–edge cooperation schemes in the FRAN.

Toward a mixed reality domain model for time-Sensitive applications using IoE infrastructure and edge computing (MRIoEF).

Mixed reality (MR) is one of the technologies with many challenges in the design and implementation phases, especially the problems associated with time-sensitive applications. The main objective of this paper is to introduce a conceptual model for MR application that gives MR application a new layer of interactivity by using Internet of things/Internet of everything models, which provide an improved quality of experience for end-users. The model supports the cloud and fog compute layers to give more functionalities that need more processing resources and reduce the latency problems for time-sensitive applications. Validation of the proposed model is performed via demonstrating a prototype of the model applied to a real-time case study and discussing how to enable standard technologies of the various components in the model. Moreover, it shows the applicability of the model, the ease of defining the roles, and the coherence of data or processes found in the most common applications.

QoE‐aware traffic monitoring based on user behavior in video streaming services

SummaryIn recent years, online video content has gained a lot of popularity and the exponential growth of video traffic continues in every area of the connected world. Thus, understanding the quality of experience (QoE) perceived by end‐users of video streaming services is important for both network operators and over the top providers since estimating end user's QoE has become one of the main points to meet user expectations. However, it is not trivial to ensure an adequate QoE since user experience is affected by various influencing factors (e.g., context factors, human factors, and system factors), and it is still challenging to identify the QoE key influencing factors. To address these challenges, we focused on improving QoE estimation and management strategies by exploiting valuable human and context information because the influence of social contextual information and user behavior on the perceptual quality is often neglected. In this article, we first proposed a classification of influence factors into four categories which are: system factors, human factors, context factors, and social‐behavioral factors. We developed a monitoring web application where video content is played to end‐users so that subjective and objective video metrics are collected. We built a new machine learning (ML) based model for QoE prediction. We used well‐known supervised ML algorithms like decision tree, k‐nearest neighbors, and support vector machine. Finally, we proposed a QoE management approach in the context of software defined network/multi‐access edge computing that implements the proposed QoE prediction model to optimize the video delivery transmission chain.

Deep learning-based video quality enhancement for the new versatile video coding.

Multimedia IoT (M-IoT) is an emerging type of Internet of things (IoT) relaying multimedia data (images, videos, audio and speech, etc.). The rapid growth of M-IoT devices enables the creation of a massive volume of multimedia data with different characteristics and requirements. With the development of artificial intelligence (AI), AI-based multimedia IoT systems have been recently designed and deployed for various video-based services for contemporary daily life, like video surveillance with high definition (HD) and ultra-high definition (UHD) and mobile multimedia streaming. These new services need higher video quality in order to meet the quality of experience (QoE) required by the users. Versatile video coding (VVC) is the new video coding standard that achieves significant coding efficiency over its predecessor high-efficiency video coding (HEVC). Moreover, VVC can achieve up to 30% BD rate savings compared to HEVC. Inspired by the rapid advancements in deep learning, we propose in this paper a wide-activated squeeze-and-excitation deep convolutional neural network (WSE-DCNN) technique-based video quality enhancement for VVC. Therefore, we replace the conventional in-loop filtering in VVC by the proposed WSE-DCNN model that eliminates the compression artifacts in order to improve visual quality and hence increase the end user QoE. The obtained results prove that the proposed in-loop filtering technique achieves -2.85%, -8.89%, and -10.05% BD rate reduction for luma and both chroma components under random access configuration. Compared to the traditional CNN-based filtering approaches, the proposed WSE-DCNN-based in-loop filtering framework achieves efficient performance in terms of RD cost.

A Combined Field-of-View Prediction-Assisted Viewport Adaptive Delivery Scheme for 360° Videos

Recently, 360° or omnidirectional videos have become increasingly popular for both personal and enterprize use-cases. However, 360° video streaming has very high bandwidth and processing requirements. State-of-the-art viewport-based streaming solutions lower these requirements by performing selective streaming based on long-term Field-of-View (FoV) prediction mechanisms. However, sometimes user movement is extremely unpredictable during some parts of the video, and applying these solutions adversely affects the overall quality of experience (QoE). This paper proposes a novel Combined Field-of-View tile-based adaptive streaming solution (CFOV) that improves end-user QoE for 360° video streaming. CFOV performs interactive tile selection based on more accurate dynamical viewing area identification by combining the results of two FoV prediction mechanisms. It also employs an innovative priority-based bitrate adaptation approach that ensures improved bitrate budget distribution between different tiles. We evaluate the proposed solution with a comprehensive set of experiments involving four immersive videos, diverse tiling patterns (i.e., 4×3, 6×4, and 8×6), different segment lengths (i.e., 1s, 2s, and 3s), and 48 empirical head movement traces under different bandwidth settings. The evaluation employs a newly defined QoE metric specifically introduced to assess the streaming performance of 360° videos objectively. The experimental findings show that, compared to alternative approaches, our proposed solution can achieve a higher viewport match and can significantly improve the user QoE for different watching behaviors and content characteristics.

RT-BVE—Real-Time no-reference Blocking Visibility Estimation in video frames

During lossy video compression, a blocking artifact is introduced into a video. It can significantly reduce the quality of a compressed video, and consequently end-user Quality of Experience. Hence, it is very important to measure and monitor the quality of compressed videos delivered to the end-user, which can be performed only by using the no-reference (NR) approach, without the original video signal available. Many NR video quality assessment (VQA) metrics are based on measurements of different artifacts visibility, which are then used as an input for an objective NR VQA metric. Thus, reliable artifacts detection and their visibility estimation are the first step in a reliable VQA process. In this paper, a new algorithm for Real-Time NR Blocking Visibility Estimation (RT-BVE) in video frames is proposed. The performance of the proposed RT-BVE algorithm is compared to the performance of two freely available algorithms, i.e., BBT-BDA and MSU Blocking, using videos from the well-known databases (156 video sequences in total and more than 48,000 frames). While RT-BVE achieves performance comparable to BBT-BDA and MSU Blocking algorithms for MPEG-2 videos, it outperforms both algorithms for H.264 videos. The results show that the proposed RT-BVE algorithm can precisely estimate blocking visibility in video frames for videos compressed according to different compression standards by using a fixed set of algorithm parameter values. Additionally, RT-BVE is suitable for usage in real-time applications since it is capable of processing approximately 100 Full HD video frames per second on the current midrange × 86–64 platform.

Beyond QoE: Diversity Adaptation in Video Streaming at the Edge

Adaptive bitrate (ABR) algorithms are critical techniques for high quality-of-experience (QoE) Internet video delivery. Early ABR algorithms conducting the overall QoE function of fixed parameters are limited by the fact that the QoE of end-users are diverse such that the video bitrate is often chosen in a misleading way. State-of-the-art ABR algorithms like MPC and Pensieve utilize offline modeling techniques and result in performance degradation for online QoE diversity adaptation. To address this issue, we propose Elephanta, an online ABR algorithm for edge users, which incorporates user QoE perception interface and adaptation algorithm with flexible parameters. In order to avoid overhead from updating parameters online, we model video streaming as a renewal system and formulate the specific QoE function into flexible formats by setting constraints on corresponding QoE metrics. To validate parameter settings, we emulate Elephanta under 1500 throughput traces, including FCC broadband, $3G$ HSDPA data set from the Internet, as well as the $4G$ /LTE data set we collect. Evaluation results show that Elephanta achieves QoE improvement of 7% over MPC and 3% over Pensieve under QoE diversity in part because of its superior adaptability to QoE diversity. We implemented Elephanta in dash.js at the client side for subjective experiments. We observed the diverse QoE preferences across users and 19/21 users (strongly) agree that Elephanta is responsive to parameter changes while watching videos.

Evaluating the Factors Affecting QoE of 360-Degree Videos and Cybersickness Levels Predictions in Virtual Reality

360-degree Virtual Reality (VR) videos have already taken up viewers’ attention by storm. Despite the immense attractiveness and hype, VR conveys a loathsome side effect called “cybersickness” that often creates significant discomfort to the viewers. It is of great importance to evaluate the factors that induce cybersickness symptoms and its deterioration on the end user’s Quality-of-Experience (QoE) when visualizing 360-degree videos in VR. This manuscript’s intent is to subjectively investigate factors of high priority that affect a user’s QoE in terms of perceptual quality, presence, and cybersickness. The content type (fast, medium, and slow), the effect of camera motion (fixed, horizontal, and vertical), and the number of moving targets (none, single, and multiple) in a video can be the factors that may affect the QoE. The significant effect of such factors on end-user QoE under various stalling events (none, single, and multiple) is evaluated in a subjective experiment. The results from subjective experiments show a notable impact of these factors on end-user QoE. Finally, to label the viewing safety concern in VR, we propose a neural network-based QoE prediction method that can predict the degree of cybersickness influenced by 360-degree videos under various stalling events in VR. The performance accuracy of the proposed method is then compared against well-known Machine Learning (ML) algorithms and existing QoE prediction models. The proposed method achieved a 90% prediction accuracy rate and performed well against existing models and other ML methods.

Compute-Less Networking: Perspectives, Challenges, and Opportunities.

Delay-sensitive applications have been driving the move away from cloud computing, which cannot meet their low-latency requirements. Edge computing and programmable switches have been among the first steps toward pushing computation closer to end-users in order to reduce cost, latency, and overall resource utilization. This article presents the "compute-less" paradigm, which builds on top of the well known edge computing paradigm through a set of communication and computation optimization mechanisms (e.g.,, in-network computing, task clustering and aggregation, computation reuse). The main objective of the compute-less paradigm is to reduce the migration of computation and the usage of network and computing resources, while maintaining high Quality of Experience for end-users. We discuss the new perspectives, challenges, limitations, and opportunities of this compute-less paradigm.

GeoQoE-Vanet: QoE-Aware Geographic Routing Protocol for Video Streaming over Vehicular Ad-hoc Networks

Video streaming is one of the challenging issues in vehicular ad-hoc networks (VANETs) due to their highly dynamic topology and frequent connectivity disruptions. Recent developments in the routing protocol methods used in VANETs have contributed to improvements in the quality of experience (QoE) of the received video. One of these methods is the selection of the next-hop relay vehicle. In this paper, a QoE-aware geographic protocol for video streaming over VANETs is proposed. The selection process of the next relay vehicle is based on a correlated formula of QoE and quality of service (QoS) factors to enhance the users’ QoE. The simulation results show that the proposed GeoQoE-Vanet outperforms both GPSR and GPSR-2P protocols in providing the best end-user QoE of video streaming service.

IJNM Special Issue—International Journal of Network Management: QoE‐centric analysis and management of communication networks

IJNM Special Issue—International Journal of Network Management: QoE‐centric analysis and management of communication networks

A Multipath Transport Scheme for Real-Time Multimedia Services Based on Software-Defined Networking and Segment Routing

It is still a challenging task to provide satisfactory quality of experience (QoE) to users by traditional routing mechanism in current public Internet for real-time multimedia services with high bandwidth and strict end-to-end latency constraint. Multipath transmission is a promising way to improve the performance of data delivery, due to its advantage in bandwidth aggregation capacity over multiple paths to speed up high volume transfers between endpoints. However, existing multipath transmission has many disadvantages such as the requirements for end-host support (i.e. the necessary multi-homed network setting, and operating system or upper applications updates of user devices), and blindingly pursuing maximum of transmission throughput. Path allocation and end-to-end transmission control are two important factors affecting the quality of multipath transmission. As an emerging network architecture, software-defined networking (SDN) is flexible, manageable, and responsive to rapid changes in traffic requirements. In this paper, we propose a multipath transport scheme based on SDN with segment routing (SR) which can meet the bandwidth and low end-to-end transmission latency requirements of real-time interactive multimedia services. The centralized SDN controller is extended to allocate multiple disjoint paths meeting bandwidth requirement meanwhile balancing the network load. The proprietary multipath media server (MMS) dynamically controls the assignment between subflows and routing paths to reduce the end-to-end transmission latency and thus improve QoE of end-users. To show the feasibility of our approach, we construct an SDN platform using Mininet. Simulation results show that our approach performs better compared to the conventional approaches in terms of traffic balancing and end-to-end transmission delay.

Modeling Quality of IoT Experience in Autonomous Vehicles

Today’s research on Quality of Experience (QoE) mainly addresses multimedia services. With the introduction of the Internet of Things (IoT), there is a need for new ways of evaluating the QoE. Emerging IoT services, such as autonomous vehicles (AVs), are more complex and involve additional quality requirements, such as those related to machine-to-machine communication that enables self-driving. In fully autonomous cases, it is the intelligent machines operating the vehicles. Thus, it is not clear how intelligent machines will impact end-user QoE, but also how end users can alter and affect a self-driving vehicle. This article argues for a paradigm shift in the QoE area to cover the relationship between humans and intelligent machines. We introduce the term Quality of IoT-experience (QoIoT) within the context of AV, where the quality evaluation, besides end users, considers quantifying the perspectives of intelligent machines with objective metrics. Hence, we propose a novel architecture that considers Quality of Data (QoD), Quality of Network (QoN), and Quality of Context (QoC) to determine the overall QoIoT in the context of AVs. Finally, we present a case study to illustrate the use of QoIoT.

QoE Assessment of Mobile Multiparty Audiovisual Telemeetings

The expectations of modern mobile users are increasingly moving towards being able to access demanding services regardless of context or system influence factors, such as network conditions, service topology, and device processing capabilities. Multiparty audiovisual telemeetings are an example of a real-time, delay sensitive, and heavy load service, demanding to run on smartphones that are limited in display size, processing power, and battery capacity. In this paper, we first provide an overview of multiparty audiovisual calls established via mobile devices and key aspects influencing Quality of Experience (QoE). We then report on the results of five user studies conducted over the course of the past 4 years, focused on investigating the impact of video quality in terms of different video encoding parameter configurations (namely bitrate, frame rate, and resolution) on subjective QoE scores for WebRTC-based video calls. We identify lower and upper bounds on video configuration parameters when used in the context of three-party calls. Results have shown that in certain cases it is better to provide constant lower objective video quality than to switch between higher and lower qualities, since participants start to perceive impairments. Finally, we investigate the relationship between objectively measured video quality impairments (blurriness and blockiness) and subjective user scores. Obtained results indicate that the Birnbaum-Saunders distribution for blockiness and the Burr and Gamma distributions for blurriness provide good fits for quality ratings. Gathered results aim to provide input for deriving QoE-aware service adaptation strategies, enabling increased resource allocation efficiency while maintaining acceptable end-user QoE.

Real-time throughput prediction for cognitive Wi-Fi networks

Wi-Fi as a wireless networking technology has become a widely acceptable commonplace. Over the course of time, the applications landscape of Wi-Fi networks is growing tremendously. The proliferation of new services is driving the industry to adopt novel and agile approaches to ensure the quality of experience delivered to the end user. To enhance end user experience, transmission throughput is an important metric that has a strong impact on the end-user quality of experience. The accurate real-time prediction of throughput can bring several new possibilities to enhance user experience in future self-organizing cognitive networks. However the real-time prediction of transmission throughput is challenging due to the dependency on several parameters. Previous studies on throughput prediction are primarily focused on non real-time prediction in less-dynamic networks. The studies also do not provide high accuracy as required in cognitive networks for efficient decision making. The purpose of this study is to use data-driven machine learning (ML) techniques and evaluating their accuracy and efficiency to predict the transmission throughput in Wi-Fi networks. Four algorithms are used namely multi-layer perceptrons (MLP), support vector regressors (SVR), decision trees (DT) and random forests (RF). It is widely understood that the accuracy and efficiency of machine learning (ML) algorithms hugely depend upon the datasets being used to train the model. Hence, this study proposes two distinct data models for creating ML-ready datasets using feature engineering. The accuracy of each ML algorithm over these datasets is evaluated. The evaluation results show a maximum prediction accuracy of 96.2% using MLP algorithm, followed by DT (94.5%), RF (93.3%) and SVR (91.0%) respectively. Furthermore, the complexity analysis is also presented to support the adaptation of such schemes in real-time applications.

Quality of Experience Tests of an Object-based Radio Reproduction App on a Mobile Device

Object-based audio provides many new features for the end-user. The quality of end user experience covers several different dimensions. Here, the relevant dimensions are identified and specific questions to cover these dimensions are developed. The first test of the radio app took place at JOSEPHS, a venue designed for testing new products and services located in Nuremberg, Germany. The second test took place in b com's facilities in Rennes, France. Interestingly, although the methodologies were different, relatively similar conclusions can be drawn from these two quality of end-user experience tests. For the first test, two different user or application scenarios are provided: 'on-the go' and 'at home'. As result, the general acceptance of the new features and functions that come along with object-based audio is very high. The usability is rated high, but can still be improved. A very positive surprise is that the very good sound quality with surround sound or binaural reproduction impressed the listeners most, beside all other features. In the second test at the b com lab, the most appreciated feature was to set fore-/background balance. This feature was number two in the first test. The importance of speech intelligibility for Radio (and TV) is a long known and discussed issue. Now, with object-based audio and the Next Generation Audio (NGA) codec MPEG-H, solutions are at hand to address it.

People of Collections: Facilitators of Interoperability?

In March 2019, the Muséum national d’histoire naturelle, Paris (MNHN) launched the datapoc.mnhn.fr project, funded by the French research infrastructures CollEX-Persée and E-recolnat. This proof of concept was imagined and is supported by a group of partners coming from different communities working at the Muséum (specimen collection curators, librarians, researchers, data scientists, publishers). The initial motivation of this team for getting together was to imagine a way to link the massive data produced and preserved in the heterogeneous institutional collection databases and repositories of the Muséum in order to improve global access and visibility for the benefit of end-users as well as data curation processes. After a year of sharing and deliberating, the group concluded that focusing on people’s names and identification, could be a promising way to explore interoperability and alignment solutions in order to match data hosted in the different systems. The project has thus two main goals: first, to improve biodiversity and taxonomic data quality for the qualification of personal identities, publications and scientific names by resolving frequent ambiguities and issues in people’s names assignment ; second, to develop and assess machine-driven linking strategies between specimen and authorship metadata and resources derived from various institutional datasilos of interest to the research community. In order to test this idea and to experiment innovative data computing and visualization technologies, all parties involved in the project agreed to develop a proof of concept focused on a dataset of 500 names of major MNHN naturalists from its foundation until nowadays. This proof of concept will consist in building a structured authority file for people's names, which could be shared by all services producing and using biodiversity data at MNHN, as well as reusable as open data by external stakeholders and international partners. This structured file will strengthen data and databases production and maintenance workflows, but could also help improving the quality of end-user experience by allowing individuals or machines to match, link or otherwise compute and analyse data that is still difficult to handle because of the diversity of IT applications and limited standardisation practises. It is key to the project that this structured file should somehow comply with international interoperability and semantic web standards so to facilitate global access and data exchanges with similar institutions around the world. Linked datasets and related resources derived from this work will be displayed on a public website designed for researchers as well as for the public via diverse applications and formats (API, RDF). The project will be run from April 2019 to April 2020 by the core team of partners who initiated it, with the support of a private IT and data computing service called Logilab. Some of the challenges of this project include finding an efficient way for building the structured file and then succeed in aligning and disambiguising names already present existing databases. A way to approach this issue is to confront and consolidate MNHN biodiversity datasets with external repositories by using people identifiers systems like ISNI, VIAF, IdREF, which are already familiar to libraries, archives and other cultural institutions. How can those various people identifiers systems be profitable to parse MNHN "people of collections" and help disambiguise them? Is there a particular people identifier system which will prove to be most relevant for all types of collections? Which parsing method will give the best results, and how could it scale up and possibly be reused by other institutions or even future European taxonomic infrastructures? Those are some of the questions the MNHN team is eager to deal with and to share and discuss at the Biodiversity Next Symposium.