Deep Learning Model for Multimedia Quality of Experience Prediction Based on Network Flow Packets

Abstract

Quality of experience (QoE) is the overall acceptability of an application or service, as perceived subjectively by the end user. In particular, for video quality the QoE is dependent of video transmission parameters. To monitor and control these parameters is critical in modern network management systems, but it would be better to be able to monitor the QoE itself (in terms of both interpretation and accuracy) rather than the parameters on which it depends. In this article we present the first attempt to predict video QoE based on information directly extracted from the network packets using a deep learning model. The QoE detector is based on a binary classifier (good or bad quality) for seven common classes of anomalies when watching videos (blur, ghost, etc.). Our classifier can detect anomalies at the current time instant and predict them at the next immediate instant. This classifier faces two major challenges: first, a highly unbalanced dataset with a low proportion of samples with video anomaly, and second, a small amount of training data, since it must be obtained from individual viewers under a controlled experimental setup. The proposed classifier is based on a combination of a convolutional neural network (CNN), recurrent neural network, and Gaussian process classifier. Image processing, which is the common domain for a CNN, has been expanded to QoE detection. Based on a detailed comparison, the proposed model offers better performance metrics than alternative machine learning algorithms, and can be used as a QoE monitoring function in edge computing.