Real-world Traffic Traces Research Articles

A Novel Markov Model-Based Traffic Density Estimation Technique for Intelligent Transportation System

An intelligent transportation system (ITS) aims to improve traffic efficiency by integrating innovative sensing, control, and communications technologies. The industrial Internet of things (IIoT) and Industrial Revolution 4.0 recently merged to design the industrial Internet of things-intelligent transportation system (IIoT-ITS). IIoT sensing technologies play a significant role in acquiring raw data. The application continuously performs the complex task of managing traffic flows effectively based on several parameters, including the number of vehicles in the system, their location, and time. Traffic density estimation (TDE) is another important derived parameter desirable to keep track of the dynamic state of traffic volume. The expanding number of vehicles based on wireless connectivity provides new potential to predict traffic density more accurately and in real time as previously used methodologies. We explore the topic of assessing traffic density by using only a few simple metrics, such as the number of surrounding vehicles and disseminating beacons to roadside units and vice versa. This research paper investigates TDE techniques and presents a novel Markov model-based TDE technique for ITS. Finally, an OMNET++-based approach with an implementation of a significant modification of a traffic model combined with mathematical modeling of the Markov model is presented. It is intended for the study of real-world traffic traces, the identification of model parameters, and the development of simulated traffic.

An efficient dynamic service provisioning mechanism in fog computing environment: A learning automata approach

Recent advances in the Internet of Things (IoT) technology have contributed to growing the number of IoT applications in various scenarios, e.g., buildings, cities, healthcare, wearable devices, and businesses to change the way work and live. The fog computing model has appeared as an appropriate distributed platform to service at the edge of the network using the resource capacities to support and execute the real-time IoT applications. One of the most challenging issues in IoT application management is the dynamic service provisioning problem to fix changes in IoT application resource usage patterns. In this paper, we propose an efficient dynamic service provisioning mechanism using the learning automata technique to determine the service provisioning decisions to deploy or release the IoT applications over the heterogeneous and dynamic fog infrastructure. Besides, we design an autonomous dynamic service provisioning manager (DSPM) that follows a self-management control loop to provision the IoT applications on the fog infrastructure. The simulation results obtained using synthetic and real-world traffic traces show that our proposed algorithm effectively reduces service delay, cost and service delay violation compared to other algorithms.

CacheNet: Leveraging the principle of locality in reconfigurable network design

Emerging optical communication technologies support the dynamic reconfiguration of datacenter network topologies depending on the traffic they serve. However, to reap the benefits of such demand-aware networks, control logic that quickly learns and adapts to traffic patterns is required. This paper presents CacheNet, a novel approach to efficiently control demand-aware networks. CacheNet consists of two components, a demand-aware links-cache, and a demand-oblivious topology. CacheNet leverages temporal and spatial locality in the traffic by managing the reconfigurable links of the optical switches as a links-cache. Network traffic, in turn, can be served either by a link from the links-cache component or by a demand-oblivious topology component. We study several classic caching algorithms like online LFU and LRU as our caching algorithms, as well as offline optimal caching as a benchmark, and provide an analytical model which captures their performance benefits compared to an all demand-oblivious topology. Our analytical results show that based on the hit ratios and the links-cache size, when considering the average packet delay, our hybrid design outperforms a design that is based only on demand-oblivious topology. We also evaluate CacheNet empirically, using both synthetic and real-world traffic traces, confirming the potential of our approach to consider reconfigurable links as a network of links-cache.

Efficient non-segregated routing for reconfigurable demand-aware networks

More and more networks are becoming reconfigurable: not just the routing can be programmed, but the physical layer itself as well. Various technologies enable this programmability, ranging from optical circuit switches to beamformed wireless connections and free-space optical interconnects. Existing reconfigurable network topologies are typically hybrid in nature, consisting of static and a reconfigurable links. However, even though the static and reconfigurable links form a joint structure, routing policies are artificially segregated and hence do not fully exploit the network resources: the state of the art is to route large elephant flows on direct reconfigurable links, whereas the remaining traffic is left to the static network topology. Recent work showed that such artificial segregation is inefficient, but did not provide the tools to actually leverage the benefits on non-segregated routing.In this paper, we provide several algorithms which take advantage of non-segregated routing, by jointly optimizing topology and routing. We compare our algorithms to segregated routing policies and also evaluate their performance in workload-driven simulations, based on real-world traffic traces. We find that our algorithms do not only outperform segregated routing policies, in various settings, but also come close to the optimal solution, computed by a integer linear program formulation, also presented in this paper. Finally, we also provide insights into the complexity of the underlying combinatorial optimization problem, by deriving approximation hardness results.

Gaussian Process Regression-Based Traffic Load Balancing for Multimedia Multipath Systems

A Multimedia multipath (MMMP) system aims at offering higher reliability and stability utilizing a variety of communication media and paths. To satisfy quality of service (QoS) requirements, an associated router should estimate the path status and allocate traffic load to each path properly. In this paper, we propose an adaptive load balancing algorithm which does not require any information about the system and the necessary information is estimated through online learning with Gaussian Process Regression (GPR). To this end, we introduce a probing period to collect training data on delay for GPR and estimate the timeout probability of each path using the predictive distributions from GPR. We then analyze and minimize the cost function, the weighted sum of the timeout probabilities of the paths. In the analysis, we propose two approximation methods of the timeout probability without direct calculations, which allows the proposed algorithm to operate in online manner. Through extensive simulations under various scenarios with real-world traffic traces, we demonstrate that the proposed algorithm balances the traffic load properly according to dynamic system conditions. Furthermore, we analyze the impact of probing packets on the network and design the proposed algorithm to reduce the communication overhead due to probing packets.

Task-aware flow scheduling with heterogeneous utility characteristics for data center networks

With the continuous enrichment of cloud services, an increasing number of applications are being deployed in data centers. These emerging applications are often communication-intensive and data-parallel, and their performance is closely related to the underlying network. With their distributed nature, the applications consist of tasks that involve a collection of parallel flows. Traditional techniques to optimize flow-level metrics are agnostic to task-level requirements, leading to poor application-level performance. In this paper, we address the heterogeneous task-level requirements of applications and propose task-aware flow scheduling. First, we model tasks' sensitivity to their completion time by utilities. Second, on the basis of Nash bargaining theory, we establish a flow scheduling model with heterogeneous utility characteristics, and analyze it using Lagrange multiplier method and KKT condition. Third, we propose two utility-aware bandwidth allocation algorithms with different practical constraints. Finally, we present Tasch, a system that enables tasks to maintain high utilities and guarantees the fairness of utilities. To demonstrate the feasibility of our system, we conduct comprehensive evaluations with real-world traffic trace. Communication stages complete up to 1.4× faster on average, task utilities increase up to 2.26×, and the fairness of tasks improves up to 8.66× using Tasch in comparison to per-flow mechanisms.

FOGPLAN: A Lightweight QoS-Aware Dynamic Fog Service Provisioning Framework

Recent advances in the areas of Internet of Things (IoT), big data, and machine learning have contributed to the rise of a growing number of complex applications. These applications will be data-intensive, delay-sensitive, and real-time as smart devices prevail more in our daily life. Ensuring quality of service (QoS) for delay-sensitive applications is a must, and fog computing is seen as one of the primary enablers for satisfying such tight QoS requirements, as it puts compute, storage, and networking resources closer to the user. In this paper, we first introduce FOGPLAN, a framework for QoS-aware dynamic fog service provisioning (QDFSP). QDFSP concerns the dynamic deployment of application services on fog nodes, or the release of application services that have previously been deployed on fog nodes, in order to meet low latency and QoS requirements of applications while minimizing cost. FOGPLAN framework is practical and operates with no assumptions and minimal information about IoT nodes. Next, we present a possible formulation (as an optimization problem) and two efficient greedy algorithms for addressing the QDFSP at one instance of time. Finally, the FOGPLAN framework is evaluated using a simulation based on real-world traffic traces.

Vehicle-Based Bi-Objective Crowdsourcing

Mobile crowdsourcing is an emerging complex problem solving paradigm that makes use of pervasive mobile devices equipped with multi-functional sensors. Recently, vehicles have also been increasingly adopted for mobile crowdsourcing, as the vehicles, as well as drivers, can provide diverse sensing capability and predictable mobility. Existing mobile crowdsourcing algorithms mostly recruit workers to complete one kind of sensing tasks, i.e., location-based query tasks or automatic sensing tasks. In this paper, we investigate the possibility of recruiting a set of vehicles to simultaneously complete these two categories of tasks, so as to maximize the sensing utility of each participant. We first model the worker recruitment for vehicle-based crowdsourcing as a bi-objective optimization problem with respect to the sensing capability and predictable mobility of vehicles. The recruitment problem is proven to be NP-hard, and we design two heuristic algorithms based on the bi-objective greedy strategy and the multi-objective genetic algorithm to find the solutions. The experimental results with a real-world traffic trace data set show that the proposed algorithms outperform some existing algorithms in finding solutions that maximize both objectives.

It Can Drain Out Your Energy: An Energy-Saving Mechanism Against Packet Overhearing in High Traffic Wireless LANs

Energy efficiency is a critical issue of wireless devices. As the packets are broadcast to the devices in the wireless transmission media, all active neighboring devices have to spend their energy receiving the packets though the packets are not addressed to them, which is called as the packet overhearing problem . The real-world traffic trace analysis reveals that the energy cost on the packet overhearing accounts for the majority of the devices’ energy inefficiency in high traffic wireless local area networks (WLANs). In this paper, we propose a novel sample-address sample-duration (SASD) scheme to solve the energy inefficiency of the packet overhearing problem. By adding a new SASD header, which contains the critical information, in front of the data packet at the physical layer, the SASD enables the devices to discern the required information in the energy-saving downclocking mode. Consequently, the non-destination devices of the packet can switch to the sleeping mode to avoid the packet overhearing problem. We demonstrate the feasibility of the SASD through hardware experiments and evaluate its energy-saving performance through ns-2 simulations. The results show that the SASD can greatly outperform the existing approaches in the high traffic WLAN scenario.

Accurate traffic matrix completion based on multi-Gaussian models

Traffic matrix (TM) describes the volumes of traffic between a set of sources and destinations in a network. As an important parameter, TMs are used in a variety of network engineering tasks, such as traffic engineering, capacity planning and anomaly detection. However, it is a challenge to reliably measure TMs in practice. For example, due to flaws in the measurement systems and possible failure in data collection systems, missing values are unavoidable. It is important to recover the missing data from the partial direct measurements. Existing matrix completion methods do not fully consider network traffic behavior and traffic hidden characteristic. Their completion accuracy tends to be significantly worse when the data loss rate is high. In this paper, we perform a study on intrinsic characteristics of network traffic by analyzing real-world traffic trace data, which reveals that traffic has the features of temporal stability and spatial affinity. According to traffic spatial feature, we model TM as multi-Gaussian distributions, which describes the actual network traffic more accurately. Furthermore, we propose a novel matrix completion method based on multi-Gaussian models to estimate the missing traffic data. Finally, we utilize traffic temporal characteristic to further optimize traffic matrix completion for the missing data interpolation. Our proposed approach has been evaluated utilizing real-world traffic trace data. The extensive experiments demonstrate that our method achieves significantly better performance compared with the state-of-the-art interpolation methods.

Network assisted latency reduction for mobile web browsing

To load a webpage, a web browser first downloads the base HTML file of the page in order to discover the list of objects referenced in the page. This process takes roughly one round-trip time and constitutes a significant portion of the web browsing delay on mobile devices as wireless networks suffer from longer transmission and access delays compared to wired networks. In this work, we propose a solution for eliminating this initial delay, which is transparent to end systems, does not require modifying HTTP, and is well suited for web browsing on mobile devices. Our solution, called WebPro, relies on a network proxy that builds an up-to-date database of resource lists for the websites visited frequently by network users. The proxy resides in the wired part of the network, and hence can afford to pro-actively build and refresh the resource list database periodically. When a request for a webpage comes to the proxy, it simultaneously fetches the base HTML and all referenced objects required to render the webpage using the corresponding resource list stored in the local database. We also show that the benefits of WebPro become more significant by increasing the complexity of webpages as it is able to circumvent the inter-object dependencies in a webpage. We have built a working prototype of WebPro and have used real-world traffic traces along with live experiments over Wi-Fi and LTE networks to assess its performance. Our results show an average of 26% reduction in page load time for a mix of popular web sites chosen from categories such as news, sports and shopping. Moreover, in comparison to another best known proxy-based solution, WebPro provides delay reductions ranging from 5% to 51% for a variety of web sites.

Improving Accuracy and Automation of Anomaly Detectors Based on Self-Correlation

Anomaly detectors (ADs) are extremely effective in finding ＂Zero Day＂ attacks, so anomaly detection has become an important area for both academic researchers as well as commercial interests. While ADs are attractive conceptually, they suffered from low accurate and poor adaptability. In this paper, an algorithm to improve accuracy and automation of ADs based on self-correlation analysis are proposed for any given ADs. We first perform statistical and information-theoretic analyses of Ads’ anomaly scores to reveal a consistent self-correlation characteristic during benign and abnormal periods separately. We found that anomaly scores during benign periods can be modeled as a stochastic process to prove automation improvement and anomaly scores during abnormal periods can be modeled as another stochastic process. Therefore, we model anomaly scores as a random walk on one of two stochastic processes, corresponding respectively to the traffic patterns of benign and malware. The multi-windows correlation algorithm based on threshold random walk was proposed to prove accurate and automation improvement. The proposed accurate and adaptive algorithm is incorporated into five prominent ADs. These accurate and adaptive ADSs are evaluated on public and real-world traffic traces. We show that, while reducing the need for manual parameter configuration and having low-complexity, the proposed algorithm enables the ADs to achieve considerably higher accuracy.

SLIC: Self-Learning Intelligent Classifier for network traffic

Internet traffic classification plays an important role in the field of network security and management. Past research works utilize flow-level statistical features for accurate and efficient classification, such as the nearest-neighbor based supervised classifier. However, classification accuracy of supervised approaches is significantly affected if the size of the training set is small. More importantly, the model built using a static training set will not be able to adapt to the non-static nature of Internet traffic. With the drastic evolution of the Internet, network traffic cannot be assumed to be static. In this paper, we develop the concept of ‘self-learning’ to deal with these two challenges. We propose, design and develop a new classifier called Self-Learning Intelligent Classifier (SLIC). SLIC starts with a small number of training instances, self-learns and rebuilds the classification model dynamically, with the aim of achieving high accuracy in classifying non-static traffic flows. We carry out performance evaluations using two real-world traffic traces, and demonstrate the effectiveness of SLIC. The results show that SLIC achieves significant improvement in accuracy compared to the state-of-the-art approach.

A Multi-resolution Port Scan Detection Technique for High-speed Networks

In this paper, we present a novel failed flow dispersion estimation technique, called multi-window state map (MWSM), which requires a small amount of memory and a constant number of memory accesses for implementing the multi-resolution concept (e.g., MRDS). We then extended the proposed MWSM scheme into a complete port scan detector. The simulation results with real-world traffic traces indicate that the proposed estimation technique manages the expected relative error and average standard error of less than 0.8% and 9%, respectively, while limiting the memory consumption to less than 60% of MRDS. In addition, the number of false positives decreases by 61% compared to a scan detector based on MRDS when it is extended to a complete scan detector. Owing to its simple mechanism and architecture, the proposed technique is well suited to hardware implementation. Therefore, we believe that the proposed technique is practically viable in modern high-speed intrusion detection systems.

Maximizing profit in green cellular networks through collaborative games

In this paper, we deal with the problem of maximizing the profit of Network Operators (NOs) of green cellular networks in situations where Quality-of-Service (QoS) guarantees must be ensured to users, and Base Stations (BSs) can be shared among different operators.We show that if NOs cooperate among them, by mutually sharing their users and BSs, then each one of them can improve its net profit.By using a game-theoretic framework, we study the problem of forming stable coalitions among NOs. Furthermore, we propose a mathematical optimization model to allocate users to a set of BSs, in order to reduce costs and, at the same time, to meet user QoS for NOs inside the same coalition. Based on this, we propose an algorithm, based on cooperative game theory, that enables each operator to decide with whom to cooperate in order to maximize its profit.This algorithms adopts a distributed approach in which each NO autonomously makes its own decisions, and where the best solution arises without the need to synchronize them or to resort to a trusted third party.The effectiveness of the proposed algorithm is demonstrated through a thorough experimental evaluation considering real-world traffic traces, and a set of realistic scenarios. The results we obtain indicate that our algorithm allows a population of NOs to significantly improve their profits thanks to the combination of energy reduction and satisfaction of QoS requirements.

Toward Unsupervised Protocol Feature Word Extraction

Protocol feature words are byte subsequences within traffic payload that can distinguish application protocols, and they form the building blocks of many constructions of deep packet analysis rules in network management, measurement, and security systems. However, how to systematically and efficiently extract protocol feature words from network traffic remains a challenging issue. Existing approaches like those based on n-gram or Common String (CS), which simply breaks payload into equal-length pieces or attempts to find a frequent itemset, are ineffective in capturing the hidden statistical structure of the payload content. In this paper, we propose ProWord, an unsupervised approach that extracts protocol feature words from traffic traces. ProWord builds on two nontrivial algorithms. First, we propose an unsupervised segmentation algorithm based on the modified Voting Experts algorithm, such that we break payload into candidate words according to entropy information and provide more accurate segmentation than existing n-gram and CS approaches. Second, we propose a ranking algorithm that incorporates different types of well-known feature word retrieval heuristics, such that we can build an ordered structure on the candidate words and select the highest ranked ones as protocol feature words. We compare ProWord and existing prior approaches via evaluation on real-world traffic traces. We show that ProWord captures true protocol feature words more accurately and performs significantly faster.

Green Networking With Packet Processing Engines: Modeling and Optimization

With the aim of controlling power consumption in metro/transport and core networks, we consider energy-aware devices able to reduce their energy requirements by adapting their performance. In particular, we focus on state-of-the-art packet processing engines, which generally represent the most energy-consuming components of network devices, and which are often composed of a number of parallel pipelines to “divide and conquer” the incoming traffic load. Our goal is to control both the power configuration of pipelines and the way to distribute traffic flows among them. We propose an analytical model to accurately represent the impact of green network technologies (i.e., low power idle and adaptive rate) on network- and energy-aware performance indexes. The model has been validated with experimental results, performed by using energy-aware software routers loaded by real-world traffic traces. The achieved results demonstrate how the proposed model can effectively represent energy- and network-aware performance indexes. On this basis, we propose a constrained optimization policy, which seeks the best tradeoff between power consumption and packet latency times. The procedure aims at dynamically adapting the energy-aware device configuration to minimize energy consumption while coping with incoming traffic volumes and meeting network performance constraints. In order to deeply understand the impact of such policy, a number of tests have been performed by using experimental data from software router architectures and real-world traffic traces.

Flow whitelisting in SCADA networks

Supervisory control and data acquisition (SCADA) networks are commonly deployed in large industrial facilities. Modern SCADA networks are becoming more vulnerable to cyber attacks due to the common use of standard communications protocols and increased interconnections with corporate networks and the Internet. This paper describes an approach for improving the security of SCADA networks using flow whitelisting. A flow whitelist describes legitimate traffic based on four properties of network packets: client address, server address, server-side port and transport protocol.The proposed approach incorporates a learning phase in which a flow whitelist is learned by capturing network traffic over a period of time and aggregating it into flows. After the learning phase is complete, any non-whitelisted connection observed generates an alarm. The evaluation of the approach focuses on two important whitelist characteristics: size and stability. The applicability of the approach is demonstrated using real-world traffic traces captured at two water treatment plants and at an electric-gas utility.

A Distributed TCAM Coprocessor Architecture for Integrated Longest Prefix Matching, Policy Filtering, and Content Filtering

Longest Prefix Matching (LPM), Policy Filtering (PF), and Content Filtering (CF) are three important tasks for Internet nowadays. It is both technologically and economically important to develop integrated solutions to the effective execution of the three tasks. To this end, in this paper, we propose a distributed Ternary Content Addressable Memory (TCAM) coprocessor architecture. The integrated solution exploits the complementary lookup load and storage load requirements of the three tasks to balance the lookup load and storage load among the TCAMs. A prefix filtering-based CF algorithm is designed to reduce the lookup load and a novel cache system is developed to dynamically handle the lookups from overloaded TCAMs. Simulations based on real-world traffic traces show that the proposed solution can perform all three tasks given a 10 Gbps line rate using only the resources required to perform just the CF task given a 10 Gbps line rate.

On the Tradeoff Between Energy Saving and QoS Support for Video Delivery in EEE-Based FiWi Networks Using Real-World Traffic Traces

There is a tradeoff between energy saving and performance in green networks since energy saving is generally achieved by rerouting packets along a substitute path with more hops to put low-load nodes to sleep or assembling several packets before sending them to reduce the number of wake and sleep time intervals of transmitting ports. Fiber-Wireless (FiWi) access networks must be able to transport video traffic in an efficient way due to the fact that video will account for the largest part of global consumer traffic. To reduce the energy consumption of FiWi networks and guarantee the performance of video applications, issues related to the energy-efficient Quality-of-Service (QoS) support for video delivery in IEEE 803.3az Energy Efficient Ethernet (EEE)-based FiWi access networks are addressed in this paper. We extend EEE packet coalescing by considering QoS priorities and the shared nature of FiWi networks and improve the QoS support for video delivery by proposing a green QoS differentiated routing strategy. In addition, to better evaluate the performance of our scheme in a real-world setting, we carry out traffic measurements for the research on EEE-based networks and use them in our simulations. Our results show that the proposed scheme can effectively guarantee video delivery and reduce the energy consumption of EEE-based FiWi access networks.