Trace Dataset Research Articles

GroupCast: Preference-Aware Cooperative Video Streaming With Scalable Video Coding

In this paper, we propose a preference-aware cooperative video streaming system for videos encoded using scalable video coding (SVC). In the proposed system, the collaborating users are interested in watching a video together on a shared screen. However, the willingness of each user to cooperate is subject to her own constraints such as user data plans. Using SVC, videos are segmented into chunks and each chunk is encoded using layers, where each layer can be fetched through any of the collaborating users. We formulate the problem of finding the optimal quality decisions and fetching policy of the SVC layers of video chunks subject to the available bandwidth, chunk deadlines, and cooperation willingness of the different users as an optimization problem. The objective is to optimize a QoE metric that maintains a trade-off between maximizing the playback rate of every chunk and ensuring fairness among all chunks to achieve the minimum skip (stall) duration without violating any of the imposed constraints. We propose an offline algorithm to solve the non-convex optimization problem when the bandwidth prediction is non-causally known. This algorithm has a run-time complexity that is polynomial in the video length and the number of collaborating users. Furthermore, we propose an online version of the algorithm for practical scenarios, where erroneous bandwidth prediction for a short window is used. Real implementation with android devices using SVC encoded video and a public dataset of bandwidth traces reveals the robustness and performance of the proposed algorithm and shows that the algorithm significantly outperforms round robin-based mechanisms in terms of avoiding skips/stalls and fetching video chunks at their highest quality possible.

Intelligent temporal classification and fuzzy rough set-based feature selection algorithm for intrusion detection system in WSNs

At present, Internet-based information processing systems are challenged by different kinds of threats, which lead to various types of damages that in turn result in significant loss of information in Wireless Sensor Networks (WSNs). Moreover, the stream of data communication through WSNs is continuously increasing in volume. Such huge amounts of data are targeted by attackers frequently, and thus it is crucial to provide high-level security to ensure safe and effective communication of information through the Internet. In order to secure data communication over the Internet through WSNs, Intrusion Detection Systems (IDSs) must be provided as an essential component in networks, databases and cloud systems. Over the previous decade, various approaches were utilized for developing IDSs. Unfortunately, however,none of these systems are completely flawless due to uncertainty surrounding network traffic created by normal users and attackers. Hence, the need to develop efficient IDSs has increased continuously. The present study proposes an adaptive IDS based on Fuzzy Rough sets for attribute selection and Allen’s interval algebra, which is applied on network trace datasets in order to select a huge number of attack data for effective prediction of attacks in WSNs. In addition, a fuzzy and rough set based nearest neighborhood algorithm (FRNN) is proposed in this article for effective classification of network trace dataset. This model uses a biased dataset that has 50:50 normal and attack data as opposed to the conventional datasets that have 80:20 normal and attack data. The efficiency of the proposed IDS is increased due to the use of biased data. The combination of feature selection, temporal-based dataset selection, and classification using a biased dataset reduces the false alarm rate and increases detection accuracy.

Quantifying privacy vulnerability of individual mobility traces: A case study of license plate recognition data

Emerging smart transportation applications are calling for publishing and sharing individual-based mobility trace data sets to researchers and practitioners; in the meanwhile, however, privacy issues have become a major concern given that true identities of individuals can be easily revealed from these data sets. Data synthesis In this paper, we quantitatively measure the risk of privacy disclosure in mobility trace data set caused by re-identification attacks based on the concept of k-anonymity. Using a one-month license plate recognition (LPR) data set collected in Guangzhou, China, we examine a variety of factors determining the degree of anonymity of an individual, including the temporal granularity and the size of the published data, local v.s. non-local vehicles, and continuous v.s. non-continuous observations. We find that five spatiotemporal records are enough to uniquely identify about 90% of individuals, even when the temporal granularity is set to be half a day. To publish LPR data without compromising privacy, we propose a suppression solution and a generalization solution and quantify the privacy-and-utility trade-off of them. Our results show that the suppression solution, which removes sensitive records, have a notable performance on privacy protection. The average individual anonymity identified by three spatiotemporal records increases by more than 20% at the cost of losing less than 8% of the data. We also propose a bintree-based adaptive time interval cloaking algorithm as a generalization solution. To meet a specific anonymity constraint, this algorithm adjusts the temporal resolution adaptively based on traffic counts under the principle of minimal information loss. We find that the generalization algorithm performs extremely well in satisfying different user-specified anonymity constraints and it is more flexible and reliable than the traditional uniform time interval cloaking method. We also find a strong correlation between the resulting temporal accuracy of data anonymized by the algorithm and the traffic condition. This study serves as a reminder to relevant agencies and data owners about the privacy vulnerability in individual-based mobility trace data sets and provides methodological guidance when publishing and sharing such sensitive data set.

A cost analysis of machine learning using dynamic runtime opcodes for malware detection

The ongoing battle between malware distributors and those seeking to prevent the onslaught of malicious code has, so far, favored the former. Anti-virus methods are faltering with the rapid evolution and distribution of new malware, with obfuscation and detection evasion techniques exacerbating the issue. Recent research has monitored low-level opcodes to detect malware. Such dynamic analysis reveals the code at runtime, allowing the true behaviour to be examined. While previous research uses machine learning techniques to accurately detect malware using dynamic runtime opcodes, underpinning datasets have been poorly sampled and inadequate in size. Further, the datasets are always fixed size and no attempt, to our knowledge, has been made to examine the cost of retraining malware classification models on datasets which grow continually. In the literature, researchers discuss the explosion of malware, yet opcode analyses have used fixed-size datasets, with no deference to how this model will cope with retraining on escalating datasets. The research presented here examines this problem, and makes several novel contributions to the current body of knowledge.First, the performance of 23 machine learning algorithms are investigated with respect to the largest run trace dataset in the literature. Second, following an extensive hyperparameter selection process, the performance of each classifier is compared, on both accuracy and computational costs (CPU time). Lastly, the cost of retraining and testing updatable and non-updatable classifiers, both parallelized and non-parallelized, is examined with simulated escalating datasets. This provides insight into how implemented malware classifiers would perform, given simulated dataset escalation. We find that parallelized RandomForest, using 4 cores, provides the optimal performance, with high accuracy and low training and testing times.

Seismic attribute benchmarking on instantaneous frequency

The complex seismic trace analysis is a widely applied and versatile method for computing seismic attributes. Instantaneous frequency is an important complex trace attribute, and it is generally used for identifying specific seismic events, such as abnormal attenuation and thin bed tuning. Although the definition itself is clear, in practice, the calculation varies considerably and deviates from the definition. As a result, there is little consistency in the calculation of the instantaneous frequency. We thus adopt a robust and reliable scientific process to objectively compare various implementations of the instantaneous frequency. To this end, we start by reviewing four classic algorithms for instantaneous frequency computation and efficient approximations and point out several issues and weaknesses of these algorithms. Then, the theoretical foundation for the instantaneous frequency of the sum of two sinusoids is derived from the original definition. With the above foundations, two synthetic test data sets of seismic traces, with varying frequencies and fixed amplitudes, are generated. Their synthetic ground truth instantaneous frequencies are automatically calculated by an analytic formula. In addition, regions that exhibit algorithmically interesting issues are segmented from the whole data sets and considered specifically in the proposed quality metrics. The evaluation includes four classic noncommercial algorithms and four commercial software implementations. Their quantitative and qualitative results indicate the effectiveness of the synthesized data sets and the quality metrics for instantaneous frequency evaluations. It is hoped that the inclusion of instantaneous frequency in the free academic benchmarking web service A3Mark will help to improve instantaneous frequency algorithms throughout the industry.

Social-Aware Data Dissemination via Device-to-Device Communications: Fusing Social and Mobile Networks with Incentive Constraints

Nowadays, pervasive mobile devices not only pose new challenges for existing wireless networks to accommodate the surging demands, but also offer new opportunities to support various services. For example, device-to-device (D2D) communications provide a promising paradigm for data dissemination with low resource cost and high energy efficiency. In this paper, we propose a three-phase approach for D2D data dissemination, which exploits social-awareness and opportunistic contacts with user mobility. The proposed approach includes one phase of seed selection and two subsequent phases of data forwarding. In Phase I, we build a social-physical graph model, which combines the social network and the mobile network with opportunistic transmissions. Then we partition the social-physical graph into communities using the Girvan-Newman algorithm based on edge-betweenness, and select seeds for the communities according to vertex-closeness. In Phase II, data forwarding only takes place among socially connected users. In Phase III, the base station intervenes to enable data forwarding among cooperative users. For Phases II and III, we propose the new mechanisms for message selection and cooperation pairing, which take into account both altruistic and selfish behaviors of users. The theoretical analysis for the message selection mechanism proves its truthfulness and approximation ratio in the worst case. Extensive simulation results further demonstrate the effectiveness of the proposed three-phase approach with various synthetic and real tracing datasets.

Integrating Survey Data and Digital Trace Data: Key Issues in Developing an Emerging Field

While survey research has been at the heart of social science for decades and social scientific research with digital trace data has been growing rapidly in the last few years, until now, there are relatively few studies that combine these two data types. This may be surprising given the potential of linking surveys and digital trace data, but at the same time, it is important to note that the collection and analysis of such linked data are challenging in several regards. The three key issues are: (1) data linking including informed consent for individual-level studies, (2) methodological and ethical issues impeding the scientific (re)analysis of linked survey and digital trace data sets, and (3) developing conceptual and theoretical frameworks tailored toward the multidimensionality of such data. This special issue addresses these challenges by presenting cutting-edge methodological work on how to best collect and analyze linked data as well as studies that have successfully combined survey data and digital trace data to find innovative answers to relevant social scientific questions.

An Analysis of a Real Mobility Trace Based on Standard Mobility Metrics

Better understanding mobility, being it from pedestrians or any other moving object, is practical and insightful. Practical due to its applications to the fundamentals of communication, with special attention to wireless communication. Insightful because it might pinpoint the pros and cons of how we are moving, or being moved, around. There are plenty of studies focused on mobility in mobile wireless networks, including the proposals of several synthetic mobility models. Getting real mobility traces is not an easy task, but there has been some efforts to provide traces to the public through repositories. Synthetic mobility models are usually analyzed through mobility metrics, which are designed to capture mobility subtleties. This work research on the applicability of some representative mobility metrics for real traces analysis. To achieve that goal, a case study is accomplished with a dataset of mobility traces of taxi cabs in the city of Rome/Italy. The results suggest that the mobility metrics under consideration are capable of capturing mobility properties which would otherwise require more sophisticated analytical approaches.

Modeling Spatio-Temporal App Usage for a Large User Population

With the wide adoption of mobile devices, it becomes increasingly important to understand how users use mobile apps. Knowing when and where certain apps are used is instrumental for app developers to improve app usability and for Internet service providers (ISPs) to optimize their network services. However, modeling spatio-temporal patterns of app usage has been a challenging problem due to the complicated usage behavior and the very limited personal data. In this paper, we propose a Bayesian mixture model to capture when, where and what apps are used and predict future app usage. To solve the challenge of data sparsity, we apply a hierarchical Dirichlet process to leverage the shared spatio-temporal patterns to accurately model users with insufficient data. We then evaluate our model using a large dataset of app usage traces involving 1.7 million users over 3503 apps. Our analysis shows a clear correlation between the user's location and the apps being used. Extensive evaluations show that our model can accurately predict users' future locations and app usage, outperforming the state-of-the-art algorithms by 11.7% and 11.1%, respectively. In addition, our model can be used to synthesize app usage traces that do not leak user privacy while preserving the key data statistical properties.

Destination-aware metric based social routing for mobile opportunistic networks

Although centrality is widely used to differentiate the importance of nodes for social-aware routing in mobile opportunistic networks (MONs), it is destination-agnostic since such metrics are usually measured without destination information. To this end, we propose a destination-aware social routing scheme for MONs, namely DAS, which utilizes the destination-aware betweenness centrality (DBC) to choose the right nodes as relays given the specific destination node. During the process of message dissemination, the number of replicas for a message is calculated by the source and each relay independently in respond to the network condition in a dynamic manner. Therefore, DAS disseminates only a few message copies to ensure data delivery as well as reducing routing cost. We conduct extensive simulations using real trace data sets to show improved performance with low overhead in comparison with existing social-aware routing approaches in various scenarios.

General regularization framework for DEER spectroscopy.

Tikhonov regularization is the standard processing technique for the inversion of double electron-electron resonance (DEER) data to distance distributions without assuming a parametrized model. In other fields it has been surpassed by modern regularization methods. We analyze such alternative regularization methods based on the Tikhonov, total variation (TV) and Huber penalties with and without the use of Bregman iterations. For this, we provide a general mathematical framework and its open-source software implementation. We extend an earlier approach by Edwards and Stoll for the selection of an optimal regularization parameter to all of these penalties and use their big test data set of noisy DEER traces with known ground truth for assessment. The results indicate that regularization methods based on Bregman iterations provide an improvement upon Tikhonov regularization in recognizing features and recovering distribution width at moderate signal-to-noise ratio, provided that noise variance is known. Bregman-iterative methods are robust with respect to the method used in the choice of regularization parameter.

Modeling Metropolitan-Area Ambulance Mobility Under Blue Light Conditions

Actions taken immediately following a life-threatening personal health incident are critical for the survival of the sufferer. The timely arrival of specialist ambulance crew in particular often makes the difference between life and death. As a consequence, it is critical that emergency ambulance services achieve short response times. This objective sets a considerable challenge to ambulance services worldwide, especially in metropolitan areas, where the density of incident occurrence and traffic congestion are high. Using London as a case study, in this paper, we consider the advantages and limitations of data-driven methods for ambulance routing and navigation. Our long-term aim is to enable considerable improvements to their operational efficiency through the automated generation of more effective response strategies and tactics. A key ingredient of our approach is to use a large historical dataset of incidents and ambulance location traces to model route selection and arrival times. Working on the London road network graph modified to reflect the differences between emergency and civilian vehicle traffic, we develop a methodology for the precise estimation of expected ambulance speed at the individual road segment level. We demonstrate how a model that exploits this information achieves best predictive performance by implicitly capturing route-specific persistent patterns in changing traffic conditions. We then present a predictive method that achieves a high route similarity score while minimising journey duration error. This is achieved through the combination of a technique that correctly predicts routes selected by the current LAS navigation system and our best performing speed estimation model. This hybrid approach outperforms alternative mobility models. To the best of our knowledge, this paper represents the first attempt to apply a data-driven methodology for route selection and the estimation of arrival times of ambulances travelling with blue lights and sirens on.

Crowdsourcing Incentives for Multi-Hop Urban Parcel Delivery Network

Efficient and economic parcel delivery becomes a key factor in the success of online shopping. Addressing this challenge, this paper proposes to crowdsource the parcel delivery task to urban vehicles to utilize their spare capacities, thus improving the efficiency while reducing traffic congestions. The delivery is planned as a multi-hop process, and participating vehicles will carry parcels from one shipping point to the next until they arrive at the destination, following the routes learned from the historical traffic statistics. The major contributions include an incentive framework to motivate the vehicles to participate in the delivery tasks by preserving the interests of the platform, the sender, and the crowd vehicles. Two incentive models are designed from platform-centric and user-centric perspectives, respectively. The platform-centric model first assesses an optimal reward $R$ for parcel delivery with the principle of Stackelberg game, which enables the platform to maximize its profit. The user-centric model then applies a reverse auction mechanism to select the winning bids of vehicles while minimizing the sender cost, with truthfulness guarantee. Theoretical analysis and extensive experiments on a real urban vehicle trace dataset are provided to validate the efficacy of the proposed framework.

A Data-Driven Algorithm for Indoor/Outdoor Detection Based on Connection Traces in a LTE Network

Environmental factors have a strong impact on the satisfaction of mobile users. Thus, estimating the context of a session is key to evaluating the end-user experience in mobile network management. Such a context is mainly defined by user location. In most cases, user location is derived from network measurements in the absence of handset measurements. Unfortunately, the current geolocation techniques do not have enough accuracy to detect if the user was indoor or outdoor. In this paper, a data-driven statistical model is proposed to detect if a cellular connection is originated in an indoor location based on the traffic attributes of the connection. Unlike the state-of-the-art approaches, based on application-level data, the proposed model is developed by logistic regression on data from radio connection traces stored in the network management system. The model is tested with a large trace dataset from a live Long Term Evolution (LTE) network.

Energy-Aware Incentivized Data Dissemination via Wireless D2D Communications With Weighted Social Communities

Device-to-device (D2D) communications are featured by high energy efficiency and spectrum efficiency, which offers a promising technique for data dissemination over wireless networks. In this paper, we propose a novel solution that exploits D2D communications to enable efficient data dissemination over wireless networks. To distribute some messages to a target group of users, the base station first identifies the most influential users, called initial sources or seeds, and fulfills their requests. Then, these source devices forward their received messages to remaining users via D2D communications. To incentivize the sources in data forwarding, we propose a monetary auction-based mechanism and a moneyless matching-based mechanism, which can be activated depending on the practical application scenario. The monetary mechanism obtains the global optimal solution and achieves truthfulness, but it involves transfer of monetary rewards. In contrast, the moneyless mechanism enables a lightweight implementation, and it guarantees two-sided stability so that both sides of users in data forwarding are willing to accept the pairing result. To expedite data dissemination, we take advantage of social-awareness in a manner different from many existing approaches. We exploit weighted social relationships, which can be direct or indirect links between any two users in a connected community. To evaluate the performance of the proposed approaches, we develop novel schemes to generate and preprocess synthetic and real tracing datasets. Extensive simulation results with both synthetic and real tracing datasets demonstrate that our approaches achieve high utilities.

Social‐aware content dissemination through opportunistic D2D communications

AbstractLast decade has witnessed rapid development in wireless network. The traffic demand is driven by enormous application data that have high‐bandwidth requirement. From the operators perspective, device‐to‐device (D2D) communications are expected to provide an efficient way of offloading cellular traffic. Note that, in the high‐density gathering cases, the redundant traffic in the cellular infrastructure can be drastically offloaded by disseminating the duplicated content through D2D communications. According to this observation, we propose a novel social‐aware content dissemination scheme through D2D communications to effectively offload cellular traffic. We base our proposition on the inherent social aspects of user mobility, exploit three social structural metrics, namely, centrality, contact rate, and intercontact time. The contributions of this paper are two‐fold. First, we integrate the carry‐forward mechanism and multihop forwarding into our D2D content dissemination scheme to fully utilize the potential of D2D networks. Second, based on social pattern, we design a two‐stage social‐aware delay prediction scheme (SADP) to improve the D2D offloading utility gain. The two stages are decision‐making stage and content dissemination stage. Based on real trace data set, we evaluate the proposed method through opportunistic D2D communications, which utilizes the aforementioned metrics to enhance offloading performance and D2D transmission utility gain. By comparing with other two well‐known schemes, SA‐Epidemic and SA‐Prophet, it has been shown that our method is superior to them in terms of offloading ratio and message delay and can substantially enhance the content dissemination performance. Since SADP is designed based on a distributed insight, it ensures that it can be adaptive to mobile network. To that end, we also look forward to the future that SADP is a promising approach applied in the application scenario of 5G network.

Vehicle-Based Bi-Objective Crowdsourcing

Mobile crowdsourcing is an emerging complex problem solving paradigm that makes use of pervasive mobile devices equipped with multi-functional sensors. Recently, vehicles have also been increasingly adopted for mobile crowdsourcing, as the vehicles, as well as drivers, can provide diverse sensing capability and predictable mobility. Existing mobile crowdsourcing algorithms mostly recruit workers to complete one kind of sensing tasks, i.e., location-based query tasks or automatic sensing tasks. In this paper, we investigate the possibility of recruiting a set of vehicles to simultaneously complete these two categories of tasks, so as to maximize the sensing utility of each participant. We first model the worker recruitment for vehicle-based crowdsourcing as a bi-objective optimization problem with respect to the sensing capability and predictable mobility of vehicles. The recruitment problem is proven to be NP-hard, and we design two heuristic algorithms based on the bi-objective greedy strategy and the multi-objective genetic algorithm to find the solutions. The experimental results with a real-world traffic trace data set show that the proposed algorithms outperform some existing algorithms in finding solutions that maximize both objectives.

You, the Web, and Your Device

Understanding how people interact with the web is key for a variety of applications, e.g., from the design of effective web pages to the definition of successful online marketing campaigns. Browsing behavior has been traditionally represented and studied by means of clickstreams , i.e., graphs whose vertices are web pages, and edges are the paths followed by users. Obtaining large and representative data to extract clickstreams is, however, challenging. The evolution of the web questions whether browsing behavior is changing and, by consequence, whether properties of clickstreams are changing. This article presents a longitudinal study of clickstreams from 2013 to 2016. We evaluate an anonymized dataset of HTTP traces captured in a large ISP, where thousands of households are connected. We first propose a methodology to identify actual URLs requested by users from the massive set of requests automatically fired by browsers when rendering web pages. Then, we characterize web usage patterns and clickstreams, taking into account both the temporal evolution and the impact of the device used to explore the web. Our analyses precisely quantify various aspects of clickstreams and uncover interesting patterns, such as the typical short paths followed by people while navigating the web, the fast increasing trend in browsing from mobile devices, and the different roles of search engines and social networks in promoting content. Finally, we contribute a dataset of anonymized clickstreams to the community to foster new studies.<sup;>1</sup;>

A Data Set for User Request Trace-Oriented Monitoring and its Applications

User request trace-oriented monitoring is an effective method to improve the reliability of cloud services. However, there are some difficulties in getting useful traces in practice, which hinder the development of trace-oriented monitoring research. In this paper, we release a fine-grained user request-centric open trace data set, called TraceBench, which is collected in a real-world cloud storage service deployed in a real environment. When collecting, we consider different scenarios, involving multiple scales of clusters, different kinds of user requests, various speeds of workloads, many types of injected faults, etc . To validate the usability and authenticity, we have employed TraceBench in several trace-oriented monitoring topics, such as anomaly detection, performance problem diagnosis, and temporal invariant mining. The results show that TraceBench well supports these research topics. In addition, we have also carried out an extensive data analysis based on TraceBench, which validates the high quality of the data set.

Differential privacy for publishing enterprise-scale WLAN traces

Wireless trace data play an important role in wireless network researches. However, publishing the raw WLAN traces poses potential privacy risks of network users. Therefore, it is necessary to sanitize users’ sensitive information before these traces are published, and provide high data utility for wireless network researches as well. Although some existing works based on various anonymization methods have started to address the problem of sanitizing WLAN traces, we find the anonymization techniques cannot provide strong and provable privacy guarantee by analyzing a real WLAN trace dataset. Differential Privacy is the only framework that can provide strong and provable privacy guarantee. However, our analysis shows that existing studies on differential privacy fail to provide effective data utility for query operations on multi-dimensional and large-scale datasets. Aiming at WLAN trace datasets that have unique characteristics of multi-dimensional and large-scale, this paper proposes a privacy-preserving data publishing algorithm which not only satisfies differential privacy but also realizes high data utility for query operations. We prove that the proposed sanitization algorithm satisfies $$\epsilon $$ -differential privacy. Furthermore, the theoretical analysis shows the noise variance in our sanitization algorithm is $$O(\log ^{o(1)}n/\epsilon ^2)$$ which indicates the algorithm can achieve high data utility on large-scale datasets. Moreover, from the results of extensive experiments on an enterprise-scale WLAN trace dataset, we also show that our sanitization algorithm can provide high data utility for query operations.