Quality Of Experience Indicators Research Articles

Usefulness of QoS in Multidimensional QoE Prediction for Haptic–Audiovisual Communications

This article investigates prediction of QoE (Quality of Experience) by comparing borrowing–from–neighbor situations and isolated ones. We demonstrate that joint utilization of multiple QoE measures enhances the accuracy of QoE prediction compared to that by a collection of individual QoE measures each regressed on QoS (Quality of Service) parameters, while the accuracy improvement with additional usage of QoS information in the former is limited. As an example of system which needs multidimensional QoE representation, the article gives haptic audiovisual interactive communications. We employ QoE and QoS data taken previously in an experiment, where 13 QoE measures (a five–point score each) and 12 QoS parameters (nonnegative continuous values each) are available at three average rates of load traffic. We build two kinds of Bayesian models for QoE prediction; one is a logistic regression model of a single QoE measure as the response variable and QoS parameters as predictors, which is a typical traditional method of discrete QoE prediction and isolated in a sense. The other is a structural equation model (SEM) with latent constructs (i.e., factors) of audiovisual quality, haptic quality and user experience quality; the original SEM, which contains only QoE indicators of three constructs (AVQ, HQ and UXQ), was proposed in one of the author’s previous studies. This article extends the SEM in order to accommodate QoS parameters. We develop two kinds of new SEMs with QoS parameters: One has three extended constructs referred to as eAVQ, eHQ and eUXQ, each of which has both QoE and QoS indicators, and the other has separate constructs for QoE and QoS, which lead to totally six constructs (AVqoe, Hqoe, UXqoe, AVqos, Hqos and UXqos). We performed Markov chain Monte Carlo (MCMC) simulation of the Bayesian models with the JAGS software in an R environment. For comparison of QoE prediction accuracy, we adopt the 10–fold cross validation method and in part WAIC (widely applicable information criterion). We then found that the three–construct models outperform the logistic regression models with respect to all subjective QoE measures and that the two kinds of the models are comparable as for the objective measure. The six–construct model exhibits almost the same accuracy as that of the three-construct one unless the number of QoE measures ( n qoe ) in the model is small. When the number n qoe is small, single–construct models may be a better choice. We have thus learned that multiple QoE measures should be utilized jointly (i.e., borrowing from neighbor) in QoE prediction rather than resorting to QoS information only.

Cache-Assisted Collaborative Task Offloading and Resource Allocation Strategy: A Metareinforcement Learning Approach

Multiaccess edge computing (MEC) provides users with better Quality of Experience (QoE) via offloading tasks to the nearby edge. However, the emergence of new Internet of Things applications with multiple tasks and repeated requests brings redundant computation and transmission to the edge. Meanwhile, the current offloading method based on deep reinforcement learning (DRL) has low sampling efficiency and slow convergence issues for training in a changing environment. Therefore, improving QoE of computation offloading services is still the ultimate challenge. In this article, we devise a collaboration of computing and cache resources among multiple edge nodes, which could reduce redundant computation and transmission. Specifically, we formulate a cache-assisted computation offloading process as a QoE-aware utility maximization problem based on multidimensional indicators. Then, we propose a cache-assisted collaborative task offloading and resource allocation strategy to solve it. This strategy is decomposed into two subproblems. First, to determine and obtain task cache state, we propose a collaborative task caching algorithm, which can improve the hit rate of tasks while balancing network overhead. Second, to acquire offloading and resource allocation decisions efficiently, we propose a metareinforcement learning-based cache-assisted computation offloading method (MCCOM), which can achieve rapid offloading decisions with a few gradient updates and samples. The optimization problem was transformed into multiple Markov decision processes (multiple MDPs). The improved learning process includes metapolicy learning that adapts to multiple Markov decision processes (MDPs) and policy learning for a specific MDP based on metapolicy. Simulation results show that our proposed method outperforms baselines in terms of QoE indicators while achieving rapid convergence and decisions.

QoE Modeling on Split Features with Distributed Deep Learning

The development of Quality of Experience (QoE) models using Machine Learning (ML) is challenging, since it can be difficult to share datasets between research entities to protect the intellectual property of the ML model and the confidentiality of user studies in compliance with data protection regulations such as General Data Protection Regulation (GDPR). This makes distributed machine learning techniques that do not necessitate sharing of data or attribute names appealing. One suitable use case in the scope of QoE can be the task of mapping QoE indicators for the perception of quality such as Mean Opinion Scores (MOS), in a distributed manner. In this article, we present Distributed Ensemble Learning (DEL), and Vertical Federated Learning (vFL) to address this context. Both approaches can be applied to datasets that have different feature sets, i.e., split features. The DEL approach is ML model-agnostic and achieves up to 12% accuracy improvement of ensembling various generic and specific models. The vFL approach is based on neural networks and achieves on-par accuracy with a conventional Fully Centralized machine learning model, while exhibiting statistically significant performance that is superior to that of the Isolated local models with an average accuracy improvement of 26%. Moreover, energy-efficient vFL with reduced network footprint and training time is obtained by further tuning the model hyper-parameters.

An Exploration-driven Reinforcement Learning Model for Video Streaming Scheduling in 5G-Powered Drone

In the process of wireless network video streaming, especially in more complex scenarios (such as video transmission of 5G-powered drones), analyzing the quality of experience (QoE) of the video streaming is a very crucial task. Thus attention should be paid to the dynamic interaction between QoE indicators including buffer starvation probability and traffic load. This paper proposes a video streaming scheduling model based on reinforcement learning. By learning the correlation between user behavior and traffic patterns, a series of resource allocation strategies that optimize QoE indicators are obtained. Since there is a certain degree of randomness in the network status at each moment in the transmission process, the model introduces exploration rewards to solve the noise problem of random environments. At the same time, this mechanism enables the model to fully explore the environment even when the reward is sparse, so as to obtain an effective scheduling strategy. Simulation experiments have proved that our model can improve the long-term QoE of video streaming in different network environments.

Nexo: Una herramienta para la visualización y análisis de indicadores QoS y QoE móviles

Con el creciente uso de las tecnologías móviles, resulta cada vez más importante tanto para los desarrolladores de aplicaciones como para prestadores de servicios brindar una óptima calidad a sus usuarios. En la actualidad, existen diversas herramientas que permiten visualizar los distintos indicadores de calidad que impactan en la experiencia del usuario; Sin embargo, los conceptos de Calidad de Servicio (QoS) y Calidad de Experiencia (QoE) usualmente son estudiados por separado, lo cual hace que el análisis completo de la calidad sea una tarea difícil. Es por esto que, en este proyecto, se presenta Nexo: una herramienta que permite recolectar indicadores de calidad (tanto QoS como QoE) en aplicaciones móviles, así como también clasificarlos y visualizarlos de manera gráfica con la finalidad de establecer vínculos entre estos. Para comprobar el correcto funcionamiento de Nexo, se llevó a cabo un caso de estudio en el que se recopilaron diversos indicadores QoS y QoE de una aplicación real (AntennaPod) y, posteriormente, dichos indicadores fueron analizados mediante la generación de distintos gráficos con la herramienta. A través de este caso de estudio, se pudo validar la relación entre la calidad del servicio y la calidad de la experiencia del usuario.

Performance assessment of three-dimensional video codecs in mobile terminals

Understanding the most relevant factors influencing the end user experience is key to design and manage multimedia services in mobile networks. Thus, user demand can be dealt with efficient strategies of resource management. In this paper, a comprehensive analysis is carried out to evaluate the Quality of Experience (QoE) obtained by three-dimensional (3D) video codecs in mobile terminals. The analysis covers the most widespread 3D video coding formats, namely the frame-compatible Side-by-Side (SbS) and Multiview Video Coding (MVC) schemes. The analysis considers both subjective and objective measurements over compression strategies that combine bit rate reduction and frame-rate dropping. Subjective tests are done by presenting 3D video sequences with different coding parameters to users who judge image quality, depth perception and visual comfort. For this purpose, a mobile phone with autostereoscopic screen is used. Then, mean opinion scores are compared with objective measurements obtained by a video quality measurement tool. Results show that MVC outperforms SbS in terms of image quality. However, when internal parameter settings are set to very restrictive configurations, the impairment of the original image are similar for both codecs. Likewise, results show that the reduction of the video bit rate is the key parameter controlling video quality, and that the use of frame-rate dropping is not enough to counteract the impairment introduced by bit rate reduction. Finally, it is confirmed that a simple QoE indicator for 3D video can be obtained from objective measurements.

Quantitative comparison of application–network interaction: a case study of adaptive video streaming

Managing quality of experience (QoE) is now widely accepted as a critical objective for multimedia applications and the supporting communication systems. In general, QoE management encompasses: (1) monitoring of the key influence factors and QoE indicators, and (2) deciding on the appropriate control actions as specified by the management goal. Many multimedia applications, e.g., video streaming and audio conferencing, are able to adjust their operational parameters so as to react to variations in the network performance. However, such an adaptation feature is mostly based on a local client view of the network conditions, which may lead to an unfair allocation of network resources among heterogeneous clients and, thus, an unfair QoE distribution. In order to tackle this issue, there is the call for a cooperation between the applications and the underlying network, which includes application–network interaction (App-Net) in terms of: (1) exchanging information on the monitored QoE indicators, and (2) coordinating the QoE control actions. Various App-Net mechanisms focusing on specific use cases and applications have been proposed to date. This paper gives an overview of App-Net mechanisms and proposes a generic App-Net model that provides the means to realize a coordinated QoE-centric management. Based on the App-Net model, we develop an evaluation methodology to compare three App-Net mechanisms for managing QoE of HTTP adaptive streaming (HAS) against a baseline HAS service. The aim of this quantitative comparison is to explore the trade-offs between QoE gains and the complexity of App-Net implementation, with respect to the number of monitoring and control messages, achieved video quality, and QoE fairness among heterogeneous clients. Our ultimate goal is to set up reproducible experiments that facilitate a holistic evaluation of different App-Net mechanisms.

WebRTC Testing: Challenges and Practical Solutions

WebRTC comprises a set of novel technologies and standards that provide Real-Time Communication on Web browsers. WebRTC makes simple the embedding of voice and video communications in all types of applications. However, releasing those applications to production is still very challenging due to the complexity of their testing. Validating a WebRTC service requires assessing many functional (e.g. signaling logic, media connectivity, etc.) and non-functional (e.g. quality of experience, interoperability, scalability, etc.) properties on large, complex, distributed and heterogeneous systems that spawn across client devices, networks and cloud infrastructures. In this article, we present a novel methodology and an associated tool for doing it at scale and in an automated way. Our strategy is based on a blackbox end-to-end approach through which we use an automated containerized cloud environment for instrumenting Web browser clients, which benchmark the SUT (system under test), and fake clients, that load it. Through these benchmarks, we obtain, in a reliable and statistically significant way, both network-dependent QoS (Quality of Service) metrics and media-dependent QoE (Quality of Experience) indicators. These are fed, at a second stage, to a number of testing assertions that validate the appropriateness of the functional and non-functional properties of the SUT under controlled and configurable load and fail conditions. To finish, we illustrate our experiences using such tool and methodology in the context of the Kurento open source software project and conclude that they are suitable for validating large and complex WebRTC systems at scale.

Spatial opportunistic transmission for Quality of Experience satisfaction

The capability of Multiple Input Multiple Output (MIMO) to deliver service simultaneously to more than one user is an exceptional characteristic for the cognitive radio (CR) communication. In this paper, statistical optimization techniques are applied to assess the performance of the Quality of Experience (QoE) in CR systems, where each user has different demands. The Multiuser scenario is considered where the transmitter runs the Multibeam Opportunistic Beamforming technique to service more than one user. Closed form expressions are derived for different scenarios and obtained for four QoE indicators in the system. The performance of primary and secondary users in such scenarios are mathematically formulated and the results are compared with computer simulations.

Quality of Experience for Spatial Cognitive Systems within Multiple Antenna Scenarios

Multiple Input Multiple Output (MIMO) adds a new dimension, the spatial one, to be optimized in Cognitive Radio (CR) by offering service simultaneously to more than one user. In this paper, statistical optimization techniques are applied to assess the performance of the Quality of Experience (QoE) in CR systems, where each user has different demands. A Multiuser scenario is considered where the transmitter can accomplish either a random or an opportunistic scheduling approach. Closed form expressions are derived for different scenarios and obtained for three QoE indicators in the system. The performance of primary and secondary users in such scenarios are mathematically formulated and compared their results to computer simulations.

Towards real-time anomalies monitoring for QoE indicators

Anomaly detection and characterization is a main topic for network managers. Although quality-of-service (QoS) indicators can help to infer problem occurrence, they do not provide immediate insight on the user’s perceived quality. Evolved service-level agreements (SLA) will likely be established in terms of quality of experience (QoE) indicators. QoS metrics composing the QoE indicators need to be monitored on a real-time basis by the SLA management tools in order to detect anomalies driving to contract violations. Monitoring SLA contracts may involve the surveillance of individual application sessions for several users. In this work, we address the problem of anomaly detection with impact on a relatively large number of users, either on one or on several types of applications simultaneously. We propose a method to characterize the state of the network, representing QoE indicators as time series and reducing the dimension of the data set. The singular spectrum analysis (SSA) method, using a combination of geometric and statistical methods, is proposed as an analysis tool in order to detect anomalies on QoE indicator evolution.

IPTV 서비스 영상에 대한 객관적 품질측정 방안 연구

최근 IP 기반의 멀티미디어 서비스가 증가하면서 IPTV에 대한 수요가 급증할 것으로 예상되고 있다. IPTV는 케이블을 이용하는 기존의 TV와 달리 IP 네트워크 기반의 다양한 서비스 제공이 가능하다. 그러나 IPTV 서비스 품질은 패킷 손실, 지터와 같은 네트워크 열화에 크게 영향을 받기 때문에 언제나 안정된 서비스를 보장하기가 힘들다. 본 논문에서는 다양한 영상 열화에 대하여 주관적 평가를 반영한 IPTV 서비스 품질측정 방안을 도출한다. 이를 위하여 시청자에게 인지적으로 영향을 줄 수 있는 열화 항목을 선정하였으며 선정된 각 열화 항목에 대하여 측정 메트릭을 제안하였다. 각 열화 항목에 대하여 MOS(Mean Opinion Score) 기반의 주관적 평가를 수행하였으며 제안한 메트릭의 측정 결과를 주관적 평가에 사상시켜 주관적 인지 상태를 고려하는 항목별 객관적 품질측정 기준을 도출한다. 다양한 영상에 대한 실험을 통해 개발된 품질측정 메트릭이 주관적 측면을 잘 반영하고 있음을 확인할 수 있다. With the advent of IP-based multimedia service based on IP network, there is a rapidly increasing demand for IPTV. Unlike the previous coaxial cable based TV, IPTV provides a variety of convergence services based on IP newark. However, since the IPTV service quality is a lot affected by the network degradation such as packet loss and jitter, it may not be guaranteed. In this paper, we propose an objective measure for various degradations of IPTV-based videos considering subjective assessment. To this end, we first determine QoE(Quality of Experience) indicators, which can affect human visual perception. Then we develop the video quality metric for each QoE indicator. Subjective assessment based on MOS is conducted and used to construct mapping relationship between each measure and perceived visual quality. Experiments are performed on various videos to confirm the efficiency and robustness of the proposed method and show high correlation with subjective assessment.