Inclusion of End User Playback-Related Interactions in YouTube Video Data Collection and ML-Based Performance Model Training

Abstract

Solutions relying on machine learning (ML) models that address the challenge of in-network QoE estimation for HTTP adaptive video streaming often neglect user behavior and its impact on performance estimation. End user playback-related interactions impact network traffic characteristics, thus having a (predominantly negative) impact on the performance of models that estimate Key Performance Indicators (KPIs) from encrypted traffic. The biggest challenge in incorporating user interactions when training and testing ML models lies in the wide range of different potential interactions, multiple interaction occurrences, various combinations of different interactions, and different time points of execution spanning across a video streaming session. With the aim of training models applicable for deployment in real networks, but also in an effort to optimize the overall process of model training, we systematically investigate the relationship between classification accuracy of models trained on data with and without certain user interactions. Our results for YouTube videos, played using the native YouTube app on a mobile device under emulated broadband network conditions, show that the impact of interactions on model performance highly depends on the target KPI being classified. In certain cases, the model training process may be simplified by reducing the need to consider a wide range of interaction scenarios.