End-to-end internet speed analysis of mobile networks with mapReduce

Abstract

Packet-based mobile networks are increasingly carrying internet traffic mostly for data intensive-applications. Mobile internet usage, with 67% share worldwide, is very important for mobile users. Because of the explosive growth of mobile data usage demand, data speed analysis is a very essential way to characterize network quality and user experience for service providers. Key Performance Indicators (KPIs), measured with data traffic analysis on network segments by current sophisticated systems, may not correctly capture users' Quality of Experience (QoE) due to actual throughput and latency. This is simply because; the data traffic analysis is not end-to-end. End-to-end analysis is very insuperable issue to measure service quality for user experience scaling and foreseeing user complaints. In this study we develop a distributed end-to-end internet speed test and analysis system for mobile networks. The system runs speed tests with actual end-to-end usage scenarios, traces communication packets and calculates download, upload and RTTs in mobile networks from user's point of view. Test results collected as large data sets are processed and analyzed in distributed fashion on cluster of computers using MapReduce Programming Model on Hadoop. The system is also able to render regional internet speed characteristics map and proactively characterizes QoE. As a result, it provides decision makers with indicators of user experience and getting ready for possible upcoming user complaints due to service quality degradation. The system is tested with a leading mobile ISP provider of Turkey and an experimental evaluation has been presented.