Call Detail Records Research Articles

Efficient data access and performance improvement model for virtual data warehouse

This paper presents a model for improving query performance in Virtual Data Warehouse (VDW) by simulating VDW environment on a cellular phone billing and customer care system which involves processing millions of Call Detail Records (CDRs) generated by thousands of counters across the country. Processing aggregations on millions of CDRs requires expensive systems, especially when analysing customers’ traffic trends and encompasses several performance optimization techniques used for improvement of query performance in VDW. In this regard, VDW offers several advantages such as real-time analytic reports, reduced maintenance, low cost solution and flexible data integration, but performance is still one of its critical shortcomings. This paper enhances performance of VDW by using techniques like partitioned materialized views, index performance optimization, query rewrite in materialized views, analytic functions, sub-queries and enabling parallel execution etc. The study uses Oracle 10g as a backend database; Oracle management console and SQL query analyser are used for monitoring performance concerns during validation of VDW model; standard PL/SQL developer is used for extracting and loading test data; and finally, Hyperion Development suite is used for testing time comparisons of datasets both in normal OLTP and simulated VDW environments.

Real-Time Recognition of Calling Pattern and Behaviour of Mobile Phone Users through Anomaly Detection and Dynamically-Evolving Clustering

In the competitive telecommunications market, the information that the mobile telecom operators can obtain by regularly analysing their massive stored call logs, is of great interest. Although the data that can be extracted nowadays from mobile phones have been enriched with much information, the data solely from the call logs can give us vital information about the customers. This information is usually related with the calling behaviour of their customers and it can be used to manage them. However, the analysis of these data is normally very complex because of the vast data stream to analyse. Thus, efficient data mining techniques need to be used for this purpose. In this paper, a novel approach to analyse call detail records (CDR) is proposed, with the main goal to extract and cluster different calling patterns or behaviours, and to detect outliers. The main novelty of this approach is that it works in real-time using an evolving and recursive framework.

Big Data Analytics for User-Activity Analysis and User-Anomaly Detection in Mobile Wireless Network

The next generation wireless networks are expected to operate in fully automated fashion to meet the burgeoning capacity demand and to serve users with superior quality of experience. Mobile wireless networks can leverage spatio-temporal information about user and network condition to embed the system with end-to-end visibility and intelligence. Big data analytics has emerged as a promising approach to unearth meaningful insights and to build artificially intelligent models with assistance of machine learning tools. Utilizing aforementioned tools and techniques, this paper contributes in two ways. First, we utilize mobile network data (Big Data)—call detail record—to analyze anomalous behavior of mobile wireless network. For anomaly detection purposes, we use unsupervised clustering techniques namely k-means clustering and hierarchical clustering. We compare the detected anomalies with ground truth information to verify their correctness. From the comparative analysis, we observe that when the network experiences abruptly high (unusual) traffic demand at any location and time, it identifies that as anomaly. This helps in identifying regions of interest in the network for special action such as resource allocation, fault avoidance solution, etc. Second, we train a neural-network-based prediction model with anomalous and anomaly-free data to highlight the effect of anomalies in data while training/building intelligent models. In this phase, we transform our anomalous data to anomaly-free and we observe that the error in prediction, while training the model with anomaly-free data has largely decreased as compared to the case when the model was trained with anomalous data.

Discovering and Understanding City Events with Big Data: The Case of Rome

The increasing availability of large amounts of data and digital footprints has given rise to ambitious research challenges in many fields, which spans from medical research, financial and commercial world, to people and environmental monitoring. Whereas traditional data sources and census fail in capturing actual and up-to-date behaviors, Big Data integrate the missing knowledge providing useful and hidden information to analysts and decision makers. With this paper, we focus on the identification of city events by analyzing mobile phone data (Call Detail Record), and we study and evaluate the impact of these events over the typical city dynamics. We present an analytical process able to discover, understand and characterize city events from Call Detail Record, designing a distributed computation to implement Sociometer, that is a profiling tool to categorize phone users. The methodology provides an useful tool for city mobility manager to manage the events and taking future decisions on specific classes of users, i.e., residents, commuters and tourists.

Activity-Based Human Mobility Patterns Inferred from Mobile Phone Data: A Case Study of Singapore

In this study, with Singapore as an example, we demonstrate how we can use mobile phone call detail record (CDR) data, which contains millions of anonymous users, to extract individual mobility networks comparable to the activity-based approach. Such an approach is widely used in the transportation planning practice to develop urban micro simulations of individual daily activities and travel; yet it depends highly on detailed travel survey data to capture individual activity-based behavior. We provide an innovative data mining framework that synthesizes the state-of-the-art techniques in extracting mobility patterns from raw mobile phone CDR data, and design a pipeline that can translate the massive and passive mobile phone records to meaningful spatial human mobility patterns readily interpretable for urban and transportation planning purposes. With growing ubiquitous mobile sensing, and shrinking labor and fiscal resources in the public sector globally, the method presented in this research can be used as a low-cost alternative for transportation and planning agencies to understand the human activity patterns in cities, and provide targeted plans for future sustainable development.

How friends share urban space: An exploratory spatiotemporal analysis using mobile phone data

AbstractThis study proposes a framework to investigate the roles of urban spaces in connecting social contacts (i.e., “friends”). The framework is applied to a Call Detail Record (CDR) dataset collected in Singapore. First, a comparative analysis is performed to understand how friends share urban space differently from random people. Then, we derive two metrics to quantify the “bonding” and “bridging” capabilities of places in the city. The two metrics reflect the potential of a place in connecting friends and random people (e.g., chance encounters), respectively. Finally, we examine the temporal signature of the places’ bonding capabilities, and associate the results with various types of Points of Interest (POIs). We find that: (1) friends are more likely to share urban space than random people, and they also share more locations; (2) a place could play different roles in connecting friends vs. random people, and the relationship (between bonding and bridging) varies depending on the time and type of a day (weekdays vs. weekends); (3) the temporal signature of bonding capability is strongly related to the semantics of a place; (4) certain POI types (e.g., shopping malls) tend to have a much higher impact on bonding capability than others (e.g., sports centers).

Urban communications and social interactions through the lens of mobile phone data

The social network built on top of mobile phone data has drawn increasing attention in recent years, due to its being far more accurate than its online counterpart in mirroring people’s offline sociality. In this paper, we leverage a large dataset of mobile Call Detail Records (CDRs), provided by one of the primary Italian operators, to build the multiplex social network given by call and text message activities of the operator’s subscribers. By discussing the multiplex network characteristics – ranging from ego-networks, dyads and triads all the way to cliques – the paper offers a comprehensive and thorough overview of human sociality as carried on through mobile phone; in addition, it highlights the need to consider more than one communication media when aiming to understand people’s sociality.Finally, by investigating on-phone cliques, we show people’s inclination to gather in cohesive and restricted groups of close friends, thus providing strong ominous indicators of where many recent online social networks, namely Snapchat, WeChat and others, are leading.

Assessing the use of mobile phone data to describe recurrent mobility patterns in spatial epidemic models.

The recent availability of large-scale call detail record data has substantially improved our ability of quantifying human travel patterns with broad applications in epidemiology. Notwithstanding a number of successful case studies, previous works have shown that using different mobility data sources, such as mobile phone data or census surveys, to parametrize infectious disease models can generate divergent outcomes. Thus, it remains unclear to what extent epidemic modelling results may vary when using different proxies for human movements. Here, we systematically compare 658 000 simulated outbreaks generated with a spatially structured epidemic model based on two different human mobility networks: a commuting network of France extracted from mobile phone data and another extracted from a census survey. We compare epidemic patterns originating from all the 329 possible outbreak seed locations and identify the structural network properties of the seeding nodes that best predict spatial and temporal epidemic patterns to be alike. We find that similarity of simulated epidemics is significantly correlated to connectivity, traffic and population size of the seeding nodes, suggesting that the adequacy of mobile phone data for infectious disease models becomes higher when epidemics spread between highly connected and heavily populated locations, such as large urban areas.

Extracting regular mobility patterns from sparse CDR data without a priori assumptions

In this work we present two methods that can extract habitual movement patterns and reconstruct the underlying movement of users from their call detail records (CDR) in a way that works for users with only moderate numbers of CDRs and that does not make any prior assumptions on the behaviour of the users. The methods allow for a more comprehensive user base in large-scale studies due to the fact that users that might otherwise have to be discarded can also be analysed. The first one is computationally not overly intense and is based on association mining. The second one, which we named DAMOCLES, is based on extracting idiosyncratic daily patterns from clustered daily activities. The methods are evaluated on real data of 140 users over an average of 200 days against benchmarks using assumptions commonly found in the literature such as a work week from Monday to Friday on GPS ground truth. Both methods clearly outperform the benchmarks and for many users retrieve similar regularities. Additionally a simulation study is performed that allows to evaluate the methods in a more controlled environment.

GIS-based mobile coverage estimation to reduce uncertainty of cellular trajectories

Mobile positioning data are normally generated as scattered points in CDRs (Call Detail Records), which include the IDs of connected cell towers for call activities. To interpret mobile associations with towers, one needs to model the area where the mobile phone user is likely to be when connected. Voronoi cells, which define the area closest to each tower, are usually employed to this end. However, the accuracy of Voronoi tessellation is influenced by various environmental factors due to propagation of signals in different manners, such as diffraction, scattering and reflection. This non-specificity of cell shapes causes a major source of uncertainty in, say, trajectory reconstruction. This paper evaluates the effect of imprecision of cell shape on accuracy of CDR-based trajectory reconstruction. It further aims to reconstruct more accurate trajectories from the coarse CDR data through considering the wave propagation models in estimating the location of mobile phone users. Since many of the factors in radio wave propagation are geographical, GIS techniques will be incorporated into this procedure.

Relationship Among the Diameter of the Area of Influence & Refill Usage of Sri Lanka Using Anonymized Call Detail Records

Relationship Among the Diameter of the Area of Influence & Refill Usage of Sri Lanka Using Anonymized Call Detail Records

Multidimensional Analysis and Mining of Call Detail Records Using Pattern Cube Algorithm

In the emerging and keenly competitive telecommunications market, it is imperative for mobile telecom operators to carry out regular analysis of their massive stored call logs to maintain and manage their numerous subscribers. Such sequential stream data analysis requires an efficient data mining algorithm and techniques bearing in mind the challenges posed by its massive size. Many data mining applications have been adapted for similar purposes. However, there has not been much emphasis on an in-depth mining of call detail records (CDR) as a multidimensional sequential stream data with its attending storage overhead. This paper proposes a novel algorithm for the multidimensional analysis of call records. Pattern Cube Algorithm (PCA) was implemented with a computer program and established empirically that: a massive CDR can be meaningfully summarized into a handy record as data mart with a gain of about 90% reduction in size and the possibility of processing a huge data from any server irrespective of the size of the target data. A quantitative exploration of the various gains in IT resources is conducted through an extensive experimental study on a sample of CDR adapted from MTN Communications Nigeria Limited.

Analysis and Data Mining of Call Detail Records using Big Data Technology

Analysis and Data Mining of Call Detail Records using Big Data Technology

Phonion: Practical Protection of Metadata in Telephony Networks

Abstract The majority of people across the globe rely on telephony networks as their primary means of communication. As such, many of the most sensitive personal, corporate and government related communications pass through these systems every day. Unsurprisingly, such connections are subject to a wide range of attacks. Of increasing concern is the use of metadata contained in Call Detail Records (CDRs), which contain source, destination, start time and duration of a call. This information is potentially dangerous as the very act of two parties communicating can reveal significant details about their relationship and put them in the focus of targeted observation or surveillance, which is highly critical especially for journalists and activists. To address this problem, we develop the Phonion architecture to frustrate such attacks by separating call setup functions from call delivery. Specifically, Phonion allows users to preemptively establish call circuits across multiple providers and technologies before dialing into the circuit and does not require constant Internet connectivity. Since no single carrier can determine the ultimate destination of the call, it provides unlinkability for its users and helps them to avoid passive surveillance. We define and discuss a range of adversary classes and analyze why current obfuscation technologies fail to protect users against such metadata attacks. In our extensive evaluation we further analyze advanced anonymity technologies (e.g., VoIP over Tor), which do not preserve our functional requirements for high voice quality in the absence of constant broadband Internet connectivity and compatibility with landline and feature phones. Phonion is the first practical system to provide guarantees of unlinkable communication against a range of practical adversaries in telephony systems.

Kerberos: A real-time fraud detection system for IMS-enabled VoIP networks

In this paper we present the design, implementation and experimental evaluation of Kerberos, an architecture for the detection of frauds in current generation Voice over IP (VoIP) networks. Kerberos is fed by an On-line Charging System (OCS) generating events associated with the setup, evolution and tear-down of end-user calls in a VoIP network compliant with the IP Multimedia Subsystem (IMS) specification. Such events are properly correlated in order to identify, in real-time, patterns associated with a fraudulent utilization of the Operator's resources. The detection phase can in turn trigger the subsequent remediation actions. Communication between the OCS and Kerberos is based on an asynchronous paradigm, whereas event correlation and analysis are effectively realized through a Complex Event Processing approach. The paper will shed light on both the design and the implementation of the system, whose performance is then evaluated by relying on a real-world dataset of Call Detail Record (CDR) events provided by Tiscali, a well known Italian Operator.

High Performance CDR Processing with MapReduce

A call detail record (CDR) is a data record produced by telecommunication equipment consisting of call detail transaction logs. It contains valuable information for many purposes in several domains, such as billing, fraud detection and analytical purposes. However, in the real world these needs face a big data challenge. Billions of CDRs are generated every day and the processing systems are expected to deliver results in a timely manner. The capacity of our current production system is not enough to meet these needs. Therefore a better performing system based on MapReduce and running on Hadoop cluster was designed and implemented. This paper presents an analysis of the previous system and the design and implementation of the new system, called MS2. In this paper also empirical evidence is provided to demonstrate the efficiency and linearity of MS2. Tests have shown that MS2 reduces overhead by 44% and speeds up performance nearly twice compared to the previous system. From benchmarking with several related technologies in large-scale data processing, MS2 was also shown to perform better in the case of CDR batch processing. When it runs on a cluster consisting of eight CPU cores and two conventional disks, MS2 is able to process 67,000 CDRs/second.

Evidence and future potential of mobile phone data for disease disaster management

Global health threats such as the recent Ebola and Zika virus outbreaks require rapid and robust responses to prevent, reduce and recover from disease dispersion. As part of broader big data and digital humanitarianism discourses, there is an emerging interest in data produced through mobile phone communications for enhancing the data environment in such circumstances. This paper assembles user perspectives and critically examines existing evidence and future potential of mobile phone data derived from call detail records (CDRs) and two-way short message service (SMS) platforms, for managing and responding to humanitarian disasters caused by communicable disease outbreaks. We undertake a scoping review of relevant literature and in-depth interviews with key informants to ascertain the: (i) information that can be gathered from CDRs or SMS data; (ii) phase(s) in the disease disaster management cycle when mobile data may be useful; (iii) value added over conventional approaches to data collection and transfer; (iv) barriers and enablers to use of mobile data in disaster contexts; and (v) the social and ethical challenges. Based on this evidence we develop a typology of mobile phone data sources, types, and end-uses, and a decision-tree for mobile data use, designed to enable effective use of mobile data for disease disaster management. We show that mobile data holds great potential for improving the quality, quantity and timing of selected information required for disaster management, but that testing and evaluation of the benefits, constraints and limitations of mobile data use in a wider range of mobile-user and disaster contexts is needed to fully understand its utility, validity, and limitations.

Constructing spatiotemporal poverty indices from big data

Big data offers the potential to calculate timely estimates of the socioeconomic development of a region. Mobile telephone activity provides an enormous wealth of information that can be utilized alongside household surveys. Estimates of poverty and wealth rely on the calculation of features from call detail records (CDRs), however, mobile network operators are reluctant to provide access to CDRs due to commercial and privacy concerns. As a compromise, this study shows that a sparse CDR dataset combined with other publicly available datasets based on satellite imagery can yield competitive results. In particular, a model is built using two CDR-based features, mobile ownership per capita and call volume per phone, combined with normalized satellite nightlight data and population density, to estimate the multi-dimensional poverty index (MPI) at the sector level in Rwanda. This model accurately estimates the MPI for sectors in Rwanda that contain mobile phone cell towers (cross-validated correlation of 0.88).

Sensing Urban Patterns with Antenna Mappings: The Case of Santiago, Chile.

Mobile data has allowed us to sense urban dynamics at scales and granularities not known before, helping urban planners to cope with urban growth. A frequently used kind of dataset are Call Detail Records (CDR), used by telecommunication operators for billing purposes. Being an already extracted and processed dataset, it is inexpensive and reliable. A common assumption with respect to geography when working with CDR data is that the position of a device is the same as the Base Transceiver Station (BTS) it is connected to. Because the city is divided into a square grid, or by coverage zones approximated by Voronoi tessellations, CDR network events are assigned to corresponding areas according to BTS position. This geolocation may suffer from non negligible error in almost all cases. In this paper we propose “Antenna Virtual Placement” (AVP), a method to geolocate mobile devices according to their connections to BTS, based on decoupling antennas from its corresponding BTS according to its physical configuration (height, downtilt, and azimuth). We use AVP applied to CDR data as input for two different tasks: first, from an individual perspective, what places are meaningful for them? And second, from a global perspective, how to cluster city areas to understand land use using floating population flows? For both tasks we propose methods that complement or improve prior work in the literature. Our proposed methods are simple, yet not trivial, and work with daily CDR data from the biggest telecommunication operator in Chile. We evaluate them in Santiago, the capital of Chile, with data from working days from June 2015. We find that: (1) AVP improves city coverage of CDR data by geolocating devices to more city areas than using standard methods; (2) we find important places (home and work) for a 10% of the sample using just daily information, and recreate the population distribution as well as commuting trips; (3) the daily rhythms of floating population allow to cluster areas of the city, and explain them from a land use perspective by finding signature points of interest from crowdsourced geographical information. These results have implications for the design of applications based on CDR data like recommendation of places and routes, retail store placement, and estimation of transport effects from pollution alerts.

NASA WEBWORLDWIND: MULTIDIMENSIONAL VIRTUAL GLOBE FOR GEO BIG DATA VISUALIZATION

In this paper, we presented a web application created using the NASA WebWorldWind framework. The application is capable of visualizing n-dimensional data using a Voxel model. In this case study, we handled social media data and Call Detailed Records (CDR) of telecommunication networks. These were retrieved from the "BigData Challenge 2015" of Telecom Italia. We focused on the visualization process for a suitable way to show this geo-data in a 3D environment, incorporating more than three dimensions. This engenders an interactive way to browse the data in their real context and understand them quickly. Users will be able to handle several varieties of data, import their dataset using a particular data structure, and then mash them up in the WebWorldWind virtual globe. A broad range of public use this tool for diverse purposes is possible, without much experience in the field, thanks to the intuitive user-interface of this web app.