Quality Of Experience Research Articles

Cloud media video encoding: review and challenges

AbstractIn recent years, Internet traffic patterns have been changing. Most of the traffic demand by end users is multimedia, in particular, video streaming accounts for over 53%. This demand has led to improved network infrastructures and computing architectures to meet the challenges of delivering these multimedia services while maintaining an adequate quality of experience. Focusing on the preparation and adequacy of multimedia content for broadcasting, Cloud and Edge Computing infrastructures have been and will be crucial to offer high and ultra-high definition multimedia content in live, real-time, or video-on-demand scenarios. For these reasons, this review paper presents a detailed study of research papers related to encoding and transcoding techniques in cloud computing environments. It begins by discussing the evolution of streaming and the importance of the encoding process, with a focus on the latest streaming methods and codecs. Then, it examines the role of cloud systems in multimedia environments and provides details on the cloud infrastructure for media scenarios. After doing a systematic literature review, we have been able to find 49 valid papers that meet the requirements specified in the research questions. Each paper has been analyzed and classified according to several criteria, besides to inspect their relevance. To conclude this review, we have identified and elaborated on several challenges and open research issues associated with the development of video codecs optimized for diverse factors within both cloud and edge architectures. Additionally, we have discussed emerging challenges in designing new cloud/edge architectures aimed at more efficient delivery of media traffic. This involves investigating ways to improve the overall performance, reliability, and resource utilization of architectures that support the transmission of multimedia content over both cloud and edge computing environments ensuring a good quality of experience for the final user.

Impact of Latency on QoE, Performance, and Collaboration in Interactive Multi-User Virtual Reality

Interactive, multi-user experiences are meant to define the present and future of Virtual Reality (VR). Such immersive experiences will typically consist of remote collaborations where content is streamed and/or synchronized over a network connection. Thus, real-time collaboration will be key. In this light, the responsiveness of the system and the network will define the overall experience. As such, understanding the effect of network distortions, especially related to time delay, on end-user’s perception (in terms of Quality-of-Experience (QoE)), performance, and collaboration becomes crucial. The existing literature, however, has mostly focused on network requirements from a system point-of-view, where the key performance parameters are only provided in the form of Quality-of-Service (QoS) parameters (such as end-to-end latency). However, the translation of these network impairments to the end-user experience is often omitted. The purpose of this paper is to fill the gap by providing a thorough investigation of the impact of latency on the perception of users while performing collaborative tasks in multi-user VR. To this end, an experimental framework was designed, developed, and tested. It is based on a multi-device synchronizing architecture, enabling two simultaneous users to work together in a gamified virtual environment. The developed test environment also allows for the identification of the most prominent network requirements and objective analysis for each traffic link. To experimentally investigate the impact of latency on user perception, a user study was conducted. Participants were paired and asked to perform the collaborative task under different latency-prone scenarios. The results show that users are able to easily distinguish between distorted and non-distorted network configurations. However, making a distinction between different manifestations of latency is much less straightforward. Moreover, factors such as the user’s role in the experience and the required task, and the level of interactivity and movement have an important influence on the subjective level of perception, the strength of the user’s preferences, and the occurrence of cybersickness. In contrast, no significant differences in objective metrics, such as system performance and user completion time were observed. These results can support the creation of collective QoE metrics that model the group as a whole rather than each individual separately. As such, this work provides an important step to dynamically counteract any drops in group dynamics and performance by means of smart interventions in the transmission system and/or virtual environment.

GreenABR+: Generalized Energy-Aware Adaptive Bitrate Streaming

Adaptive bitrate (ABR) algorithms play a critical role in video streaming by making optimal bitrate decisions in dynamically changing network conditions to provide a high quality of experience (QoE) for users. However, most existing ABRs suffer from limitations such as predefined rules and incorrect assumptions about streaming parameters. They often prioritize higher bitrates and ignore the corresponding energy footprint, resulting in increased energy consumption, especially for mobile device users. Additionally, most ABR algorithms do not consider perceived quality, leading to suboptimal user experience. This paper proposes a novel ABR scheme called GreenABR+, which utilizes deep reinforcement learning to optimize energy consumption during video streaming while maintaining high user QoE. Unlike existing rule-based ABR algorithms, GreenABR+ makes no assumptions about video settings or the streaming environment. GreenABR+ model works on different video representation sets and can adapt to dynamically changing conditions in a wide range of network scenarios. Our experiments demonstrate that GreenABR+ outperforms state-of-the-art ABR algorithms by saving up to 57% in streaming energy consumption and 57% in data consumption while providing up to 25% more perceptual QoE due to up to 87% less rebuffering time and near-zero capacity violations. The generalization and dynamic adaptability make GreenABR+ a flexible solution for energy-efficient ABR optimization.

A communication link lifetime prediction-supported V2V partial computation offloading scheme for autonomous driving

Vehicle Edge Computing (VEC) is a promising technique to improve the quality of service (QoS) and quality of experience (QoE) in autonomous driving by exploiting the resources at the network edge. However, the high mobility of the vehicles leads to stochastic communication link duration, and the tasks generated by various applications in autonomous driving incur fierce competition for resources. These challenges cause excessive task completion delays. In this paper, we propose a vehicle-to-vehicle (V2V) partial computation offloading scheme that leverages the prediction results of the communication link lifetime between vehicles. A History track, Current interactions and Future planning trajectory-aware Gated Recurrent Units (HCF-GRU) network is built to capture the essential factors to improve the prediction accuracy. Then, we design a GRU-based Proximal Policy Optimization (GRU-PPO) algorithm to obtain an optimal one-to-many offloading decision to minimize the task execution cost. The HCF-GRU prediction algorithm is evaluated on a real world vehicle trajectory dataset, and the performance of the GRU-PPO algorithm is analyzed on extensive numerical simulations. Experimental results demonstrate that our prediction network and offloading decision algorithm outperform the baseline methods in terms of prediction accuracy and task execution cost.

MetaABR: A Meta-Learning Approach on Adaptative Bitrate Selection for Video Streaming

Video streaming is one of the most popular Internet applications that makes up a large amount of Internet traffic. A fundamental mechanism in video streaming is adaptive bitrate (ABR) selection which decides the proper compression level for each chunk of a video to optimize the users' quality of experience (QoE). The existing ABR algorithms require significant tuning and do not generalize to diverse network conditions and personalized QoE objectives. In this paper, we propose a novel framework for meta-learning based ABR design and discuss challenges of deploying learning based ABR mechanism in real-world video streaming systems. We utilize the proposed framework to design MetaABR, a novel adaptive bitrate selection algorithm based on meta-reinforcement learning to maximize users' QoE. By jointly training multiple learning tasks with a shared meta-critic, it can provide transferrable meta-knowledge to supervise bitrate selection across tasks, and can be applied to efficiently learn a new task in unseen environment with only a few trials. We implement MetaABR on an emulation platform which connects to the Linux network protocol stack through virtual network interfaces. Extensive experiments based on real-world traces and wireless testbed show that MetaABR achieves the best comprehensive QoE compared with the state-of-the-art ABR algorithms in a variety of network environments.

Hybrid learning strategies for multivariate time series forecasting of network quality metrics

This work addresses the challenge of forecasting temporal metrics that characterize cellular traffic behavior. The ultimate goal is to provide network operators with a valuable tool for modeling mobile network traffic and optimizing connected resources. The idea is to estimate beforehand the temporal evolution of some Quality-of-Experience (QoE) and Quality-of-Service (QoS) metrics, which is helpful for accurately tuning the allocation of network resources. Remarkably, these metrics (expressed as time series) are typically correlated, and changes in one time series can affect others in a variety of ways and to different extents. For example, high network delay (a QoS-related metric) is associated with degradation in voice quality over time (a QoE-related metric). Accordingly, we address the problem of cellular traffic forecasting with correlated time series, proposing three innovative hybrid learning strategies designed by combining the advantages of two approaches: (i) a statistical approach, implemented through the Vector Autoregressive (VAR) model, which encodes each metric as a combination of past values of the same metric along with a combination of values of other related metrics, resulting in a multivariate structure; and (ii) an approach based on deep learning techniques (specifically, CNN, LSTM, and GRU) which operate on such a multivariate structure to perform the forecasting. The resulting performance demonstrates the benefits of the proposed hybrid schemes (VAR-CNN, VAR-LSTM, VAR-GRU) over their pure counterparts, with a significant reduction in forecasting errors. The network metrics were gathered in a real urban cellular environment, where the presence of exogenous factors (e.g., interferences, weather conditions, etc.) makes the forecasting assessment particularly challenging.

A Study on Immersion and Intention to Pay in AR Broadcasting: Validating and Expanding the Hedonic Motivation System Adoption Mode

With the rapid growth of online entertainment live streaming, how to continuously innovate and achieve long-term sustainability has become a major challenge for the industry. Augmented reality (AR) technology offers users immersive interactive experiences and potentially addresses this challenge. The aim of this study is to explore how AR technology influences key components of user online experience—immersion and intention to pay—using survey data. Building upon the Hedonic Motivation System Adoption Model (HMSAM), this research incorporates aesthetic variables to theoretically expand the model in order to gain a deeper understanding of the mechanisms influencing user behavior. A questionnaire survey was conducted to collect 450 valid samples. Detailed analysis was conducted using structural equation modeling. The findings confirm that aesthetic design significantly impacts users’ judgments of content value and perceived ease of use, generating positive effects at the perceptual level. Additionally, AR applications enhance the quality of user experience, thereby stimulating intrinsic motivations such as curiosity and joy. Further analysis indicates that users’ curiosity and perceived behavioral control directly influence the level of immersion and intention to pay. Overall, the research results offer important insights into industry applications. This study successfully expands the HMSAM theoretically by incorporating aesthetic variables to enhance the explanatory power of user judgment mechanisms. The analytical framework proposed aids in understanding the potential mechanisms of new technologies on customer experience and commercial value creation. The research findings provide guidelines for technological design and marketing strategies of streaming platforms.

Online Banking User Experience: A User Experience Questionnaire (UEQ) Assessment in South Africa

Online banking has seen tremendous growth driven by the emergence of the fourth industrial (4IR) and innovative technologies in every aspect of our daily activities. Hence, there is an emergence of areas of research around aspects of online banking, such as the need for user experience. This paper evaluates online banking user experience (UX) using the user experience questionnaire (UEQ). The study collected 725 survey responses from UEQ in South Africa, and the findings show a high quality of UX based on a comparison against the UEQ benchmark data set. More specifically, the hedonic quality scale aspect is at the highest level compared to the pragmatic quality aspects of UX. The findings provide practical contributions to online banking designers and developers in retail banks to optimise areas of strength and improve on those aspects that need improvements.

Survey on Quality of Experience Evaluation for Cloud-Based Interactive Applications

A cloud-based interactive application (CIA) is an application running in the cloud with stringent interactivity requirements, such as remote desktop and cloud gaming. These services have experienced a surge in usage, primarily due to the adoption of new remote work practices during the pandemic and the emergence of entertainment schemes similar to cloud gaming platforms. Evaluating the quality of experience (QoE) in these applications requires specific metrics, including interactivity time, responsiveness, and the assessment of video- and audio-quality degradation. Despite existing studies that evaluate QoE and compare features of general cloud applications, systematic research into QoE for CIAs is lacking. Previous surveys often narrow their focus, overlooking a comprehensive assessment. They touch on QoE in broader contexts but fall short in detailed metric analysis. Some emphasise areas like mobile cloud computing, omitting CIA-specific nuances. This paper offers a comprehensive survey of QoE measurement techniques in CIAs, providing a taxonomy of input metrics, strategies, and evaluation architectures. State-of-the-art proposals are assessed, enabling a comparative analysis of their strengths and weaknesses and identifying future research directions.

Video quality adaptation using CNN and RNN models for cost-effective and scalable video streaming Services

Video streaming services require adaptive bit rate strategies that optimize video quality based on network conditions and user preferences to provide a cost-effective and scalable solution. In this manuscript, we present a novel architecture that utilizes a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to extract features from the video stream and predict optimal bit rates for each frame. The CNN feature extractor extracts relevant features from the input video stream, which are then passed on to the RNN predictor, where the optimal bit rate is predicted to ensure a personalized viewing experience tailored to the current network conditions and specific user preferences. Our experimental results demonstrate that the proposed architecture achieves significant improvements in video quality while minimizing bandwidth usage and providing a better user experience. Specifically, our results show a 37.1% improvement in average bit rate, indicating that the proposed architecture optimizes the video quality and reduces bandwidth usage by 37.1% on average. We also observed a 16.6% improvement in Quality of Experience (QoE), meaning that users will perceive the video quality as better. Additionally, rebuffering was reduced by 87.5%, indicating that users enjoy smoother video playback without interruptions. Importantly, our architecture is optimized for Dynamic Adaptive Streaming over HTTP (DASH), addressing the need for efficient streaming over the most widely used protocol in the industry.

Improving Usability in Mobile Apps for Residential Energy Management: A Hybrid Approach Using Fuzzy Logic

This paper presents a study that evaluates the usability and user experience of a mobile application interface for residential energy management, adopting a hybrid approach that integrates quantitative and qualitative methods within a user-centered design framework. For the evaluation, metrics and tools such as the User Experience Questionnaire Short (UEQ-S) and the System Usability Scale (SUS) were used, in addition to the implementation of a fuzzy logic model to interpret and contrast the data obtained through these metrics, allowing a more accurate assessment of usability and user experience, reflecting the variability and trends in the responses. Three aspects evaluated stand out: satisfaction with the interface, ease of use, and efficiency. These are fundamental to understanding how users perceive the system. The results indicate a high likelihood of user recommendation of the system and a high overall quality of user experience. This study significantly contributes to mobile application usability, especially in residential energy management, offering valuable insights for designing more intuitive and effective user interfaces on mobile devices.

Design and development of prime herder optimization based BiLSTM congestion predictor model in live video streaming

High-quality content for the user in video streaming services depends critically on the ability to predict the continuous user’s quality of experience (QoE). However, continuous QoE prediction has proven challenging due to the complexity imposed by the temporal dependencies in QoE data and the non-linear correlations among QoE impact elements. In this research congestion prediction model is developed using the prime herder optimization-based BiLSTM (PHO-based BiLSTM). The input database is first gathered from the NIMS and darpa99 week 1 database and, the data collection is analyzed and the packet information is extracted after that the extracted features are then fed into the optimized BiLSTM classifier to train the classifier. The classifier’s hyperparameters are successfully tuned by the recommended prime herder optimization, which is made by fusing the herding characteristics of a prime sheepdog and herder optimization. Based on the traffic congestion prediction achievements, at training percentage (TP) 90, the accuracy is 94.81%, specificity is 94.90%, and mean square error (MSE) is 4.91 respectively for D1, similarly based on D2 the accuracy is 95.62%, specificity is 95.96%, and MSE is 0.38 respectively.

Using QoE Metric as a Decision Criterion in Multimedia Heterogeneous Network Optimization: Challenges and Research Perspectives

This article explores the growing importance of the QoE (Quality of Experience) metric as a fundamental criterion in the optimization of heterogeneous multimedia networks. We explore the benefits of using QoE, such as improving user experience, efficient resource management, and adaptation to network conditions. However, this paradigm presents technical challenges, including accurately measuring QoE and managing the complexity of heterogeneous networks. To address these challenges, we highlight promising research prospects, such as the development of advanced algorithms, real-time measurement of QoE, and the integration of machine learning. Ultimately, the use of QoE in the optimization of heterogeneous multimedia networks contributes to a substantial improvement in the quality of multimedia services offered to end users.

Speech Inpainting Based on Multi-Layer Long Short-Term Memory Networks

Audio inpainting plays an important role in addressing incomplete, damaged, or missing audio signals, contributing to improved quality of service and overall user experience in multimedia communications over the Internet and mobile networks. This paper presents an innovative solution for speech inpainting using Long Short-Term Memory (LSTM) networks, i.e., a restoring task where the missing parts of speech signals are recovered from the previous information in the time domain. The lost or corrupted speech signals are also referred to as gaps. We regard the speech inpainting task as a time-series prediction problem in this research work. To address this problem, we designed multi-layer LSTM networks and trained them on different speech datasets. Our study aims to investigate the inpainting performance of the proposed models on different datasets and with varying LSTM layers and explore the effect of multi-layer LSTM networks on the prediction of speech samples in terms of perceived audio quality. The inpainted speech quality is evaluated through the Mean Opinion Score (MOS) and a frequency analysis of the spectrogram. Our proposed multi-layer LSTM models are able to restore up to 1 s of gaps with high perceptual audio quality using the features captured from the time domain only. Specifically, for gap lengths under 500 ms, the MOS can reach up to 3~4, and for gap lengths ranging between 500 ms and 1 s, the MOS can reach up to 2~3. In the time domain, the proposed models can proficiently restore the envelope and trend of lost speech signals. In the frequency domain, the proposed models can restore spectrogram blocks with higher similarity to the original signals at frequencies less than 2.0 kHz and comparatively lower similarity at frequencies in the range of 2.0 kHz~8.0 kHz.

The Quality of Experience of Integrated Management Tahfiz Model (IM-Tahfiz) in Malaysian Education

Nowadays, the Islamic educational system in Malaysia demands a new technology for teaching and learning in Tahfiz education management. This Tahfiz learning management relatively has issues due to insufficient resources, infrastructure, and facilities. This paper proposes the usefulness of the Integrated Management Tahfiz Model (IM-Tahfiz) towards the effectiveness of education satisfaction at the heuristic level. The focus will be on Quality of Experience (QoE) based on user satisfaction due to implementing the IM-Tahfiz Model in the random Tahfiz institution in Malaysia.

A digital twin-based energy-efficient wireless multimedia sensor network for waterbirds monitoring

Wetlands play a critical role in maintaining the global climate, regulating the hydrological cycle, and protecting human health. However, they are rapidly disappearing due to human activities. Waterbirds are valuable bio-indicators of wetland health, but it is challenging to monitor them effectively. Wireless Multimedia Sensor Networks (WMSNs) offer a promising technology for monitoring wetlands. Nonetheless, these networks are constrained in terms of energy, and also encounter challenges associated with large-scale deployments under natural environmental conditions. These conditions introduce harsh circumstances that may not have been anticipated during the pre-deployment testing phase. This paper proposes a Digital Twin (DT) based energy-efficient WMSN monitoring system specifically tailored for waterbirds in wetlands. The system utilizes a unique approach that combines local audio identification and image compression with DT technology to optimize network performance and minimize energy consumption. To reduce unnecessary image transmissions, the system employs a real-time, low-complexity local audio identification phase before triggering image capture. A denoising step is employed to achieve highly accurate bird recognition despite surrounding noises. Each image undergoes a low-complexity compression scheme prior to transmission, further enhancing energy efficiency. To enhance the system’s overall efficiency and effectiveness, DT technology is integrated to create real-time replicas of the WMSN and the monitoring application. A synergistic interaction between the two DTs enables cooperative data-making decision that ensures both QoS (Quality of Service) and QoE (Quality of Experience) requirements are met. Transmission rate control is done using a fuzzy logic decision-making technique. Real-time feedback provides rapid and accurate analysis of the current state of the WMSN, allowing for dynamic adjustments. The ”what-if scenarios” feature of the implemented DTs has been effectively leveraged to find the most suitable settings for the controller. The effectiveness and performance enhancements achieved by integrating DT into our WMSN-based surveillance system are validated through comprehensive experiments in scenarios that correspond to a real-world wetland. Comparative analyses demonstrate the undeniable benefits of the DT-integrated system compared to a conventional WMSN-based surveillance setup. In particular, the results demonstrate the system’s superior performance in terms of energy efficiency, real-time monitoring capabilities, and ability to handle multiple video sources.

Physiological Data for User Experience and Quality of Experience: A Systematic Review (2018–2022)

The evaluation of human responses in multimedia experiences using physiological data has a well-established presence in the academic literature. However, this field is currently undergoing transformative changes, driven by the accessibility of diverse and cost-effective devices, innovative software analysis methods, and the emergence of novel application domains such as Virtual and Augmented Reality and mulsemedia. To address the imperative of contextualizing these evolving trends in a contemporary context, this paper presents a systematic review with the objective of delineating the array of physiological data utilized in assessing Quality of Experience (QoE) and User Experience (UX) in multimedia studies. It also examines the devices employed for data collection and the analytical techniques applied to interpret the acquired data. While our review exposes both constraints and promising discoveries in these domains, it also emphasizes the escalating significance and practicality of leveraging physiological data in user assessments, especially as the boundaries between the physical and digital domains continue to blur.

Design and implementation for a UAV-based streaming media system

The Unmanned Aerial Vehicle (UAV)-based video streaming transmission is critical for real-time communication (RTC) applications in flying ad-hoc networks (FANET). The communication links between the nodes, however, are often unstable, especially in harsh network environments. This article first presents an architecture of a UAV-based streaming media system (USMS) for more reliable transmission, where the UAVs play multiple roles simultaneously in terms of video capturing, encoding, transmission, caching, decoding, and evaluation. To promote fluency of video transmission, we propose a distributed algorithm to explore the Tradeoff between the bandWidth, the Load, and the Video parameters (TWLV) based on the UAV flying status and Quality of Service (QoS). To enhance the Quality of Experience (QoE) significantly, we design an application-oriented multi-link transmission system with a novel multi-module stacking technology. Leveraging this design, we implement an extended splitting-merging streaming (ESMS) algorithm to deal with heavy payloads and reduce transmission latency. We also propose a multi-link retransmission (MLR) algorithm to eliminate consecutive packet losses and achieve more efficient video rebuffering. Sufficient experimental results verify that compared with the commercially available products that utilize application-layer forward error correction (AL-FEC), MultiPath TCP (MPTCP), and Deadline-aware Transport Protocol (DTP), our proposed USMS outperforms in throughput promotion, latency reduction, congestion control, packet recovery, and rebuffering efficiency.

Feature selection applied to QoS/QoE modeling on video and web-based mobile data services: An ordinal approach

Nowadays, mobile service providers perceive the user experience as a reliable indicator of the quality associated to a service. Given a set of Quality of Service (QoS) factors, the aim is to predict the Quality of Experience (QoE), measured in terms of the Mean Opinion Score (MOS). Although this problem is receiving much attention, there are still some challenges that require more research in order to find effective solutions for meeting user’s expectation in terms of service quality. A core challenge in this topic refers to the analysis of the contribution of each factor to the QoS/QoE Model. In this work, we study the mapping between QoS and QoE on video and web-based services using a machine learning approach. For such purpose, we design a lab-testing methodology to emulate different cellular transmission network scenarios. Then, we address the problem of inducing a predictive model and identifying relevant QoS factors. Results suggest that bandwidth is a key factor when analyzing user’s perception of service quality.

Stable QoE-Aware Multi-SFCs Cooperative Routing Mechanism Based on Deep Reinforcement Learning

The explosive development of the Internet of Things (IOT) has stimulated the sudden and dynamic demand pattern of potential Service Function Chain (SFC) traffic. The virtual network built based on the requirements of Multiple SFCs (Multi-SFCs), sharing the resource, should provide better Quality of Experience (QoE) for Multi-SFCs according to the real-time network status. However, simultaneous Multi-SFCs in the routing process compete for the VNF resources of the same node and preempt the same link. The instability of QoE caused by node invalidation and link faults is highlighted. Therefore, to provide a more efficient and stable service to Multi-SFCs, quantitative QoE utility and stability models and cooperative Multi-SFCs path allocation are needed. Considering the limited resources, this paper designs a stable QoE-aware Multi-SFCs Cooperative Routing Mechanism (CRM) based on Deep Reinforcement Learning (DRL). Firstly, we model the problem of Multi-SFCs cooperative routing aiming at optimizing diverse QoEs of utility, stability and delay, and formulate it as a multi-objective optimization problem. Moreover, the QoE stable queueing model based on node and link availability is formulated by Lyapunov drift, which can potentially mitigate the QoE fluctuations. Then we associate triple Dueling Deep Q-Networks (DDQNs) and propose a cooperative computing multi-task DRL to obtain the optimal path allocation policy. Experiments are conducted to verify the effectiveness and the results show that our method outperforms its peers on QoE, delay, throughput and stability.