Traffic Data Stream Research Articles

Federated continual representation learning for evolutionary distributed intrusion detection in Industrial Internet of Things

As a promising paradigm, Federated Learning (FL)-based distributed intrusion detection offers potent protection for the network security of Industrial Internet of Things (IIoT) systems. Nonetheless, the practical deployment of IIoT systems occurs in a highly-complex and dynamic distributed environment. The ever-growing and dynamically-evolving cyber attacks will render the FL-based intrusion detection model inefficient, since FL cannot gracefully learn from the dynamic traffic data streams to identify new attacks. To address this issue, we propose the evolutionary distributed intrusion detection system based on federated continual representation learning, designed to continually capture effective feature representations of emerging attacks from dynamic traffic data streams. Specifically, we develop the supervised contrastive loss and the global information-aware regularization loss to alleviate the catastrophic forgetting on the previously observed attacks and mitigate the data heterogeneity across clients. Besides, we propose the prototype variance-based memory update strategy to ensure the effective memory replay data. Extensive experimental results demonstrate our proposed method outperforms the state-of-the-art methods by 13.3%–31.5% in terms of average accuracy on a real energy intrusion detection dataset.

AE‐Integrated: Real‐time network intrusion detection with Apache Kafka and autoencoder

SummaryUnknown cyber‐attack detection in network traffic streams is challenging but crucial to ensure network security. It is observed that new security threats occur on a daily basis and make cyberspace vulnerable. In the literature, machine learning and deep learning‐based network intrusion detection systems have gained a lot of success but still face many challenges in detecting new security threats and unknown cyber‐attacks in real‐time. Additionally, high false alarm rates and real‐time detection in constantly evolving high‐dimensional network data streams are open issues for the research community. To address this issue, a DL‐based solution is developed to detect real‐time network anomalies in streaming data with high detection accuracy, precision, recall and low false negative and positive scenarios. The proposed novel algorithm, AE‐Integrated, is developed and evaluated on the latest CICIDS‐2017 dataset. The AE‐Integrated is updated with the newest network traffic data stream by the human administrator after a certain period to maintain its prediction accuracy for future inference. The simulation study is conducted with the Apache Kafka and Slack API to get real‐time anomaly alerts. Finally, we compared the result with recent state‐of‐the‐art research to evaluate the significance of the proposed algorithm. It is concluded that combining multiple lightweight autoencoders into a single large architecture provides optimal results. The accuracy, recall, and AUC of AE‐Integrated obtained are 99.54%, 99.53%, and 0.998, respectively.

An efficient architecture for processing real-time traffic data streams using apache flink

An efficient architecture for processing real-time traffic data streams using apache flink

DDoS Attack Detection Using Enhanced Neural Network Algorithm in Software-Defined Networks

With the uncommon advancement of technology and networking, the world today is totally relying on it. The most obvious drawback of this technical progress resulted in several possible dangers. A Distributed Denial of Service (DDoS) attack is one in which a large number of compromised systems work together to prevent service from being provided to the targeted system. Consequently, to protect network servers, attack detection systems must be very effective. The proposed architecture has Software Defined Networks (SDN) which comprise controllers and SFlow agents. According to the article, the anomaly detection of statistical traffic, which is performed on both normal and pathological anomaly traces in the packet header, as well as traffic volume detection, is based on the suggested work provided in this study. The k-means clustering technique is used by the statistical anomaly detection system and the attack alert aggregation system. Low-level attack detection systems are used to generate cluster dissimilar warnings, which are then investigated further. Clusters generate meta-alerts based on the information they have gathered. After that, the report of meta-alerts is sent to the security specialists. The online alert aggregation technique, which is also known as the probabilistic model, is used to identify new assaults on a system. The k-means clustering method is used to improve the quality of the traffic data streams. The Enhanced Neural Network Algorithm (ENNA) is being used to advance the intelligent attack detection system, which is currently under development. It is utilized in open-day controllers to identify attacks with a 99.7% accuracy by using Mininet and the Python simulation tool (ODL). In future work, it is possible to evaluate how the suggested detection method would be used in the event of subnet attacks and its mitigation.

A digital twin in transportation: Real-time synergy of traffic data streams and simulation for virtualizing motorway dynamics

The introduction of digital twins is expected to fundamentally change the technology in transportation systems, as they appear to be a compelling concept for monitoring the entire life cycle of the transport system. The advent of widespread information technology, particularly the availability of real-time traffic data, provides the foundation for supplementing predominated (offline) microscopic simulation approaches with actual data to create a detailed real-time digital representation of the physical traffic. However, the use of actual traffic data in real-time motorway analysis has not yet been explored. The reason is that there are no supporting models and the applicability of real-time data in the context of microscopic simulations has yet to be recognized. Thus, this article focuses on microscopic motorway simulation with real-time data integration during system run-time. As a result, we propose a novel paradigm in motorway traffic modeling and demonstrate it using the continuously synchronized digital twin model of the Geneva motorway (DT-GM). We analyze the application of the microscopic simulator SUMO in modeling and simulating on-the-fly synchronized digital replicas of real traffic by leveraging fine-grained actual traffic data streams from motorway traffic counters as input to DT-GM. Thus, the detailed methodological process of developing DT-GM is presented, highlighting the calibration features of SUMO that enable (dynamic) continuous calibration of running simulation scenarios. By doing so, the actual traffic data are directly fused into the running DT-GM every minute so that DT-GM is continuously calibrated as the physical equivalent changes. Accordingly, DT-GM raises a technology dimension in motorway traffic simulation to the next level by enabling simulation-based control optimization during system run-time that was previously unattainable. It, thus, forms the foundation for further evolution of real-time predictive analytics as support for safety–critical decisions in traffic management. Simulation results provide a solid basis for the future real-time analysis of an extended Swiss motorway network.

Interoperable Internet of Things for Smart Transportation Systems in Circular Cities

This article discusses recent developments in the area of interoperable Internet of Things, using an example of delivering data-driven transportation services. Initial results obtained by exploiting heterogeneous traffic data streams reveal interesting traffic patterns for different vehicle types.

A comparative study on online machine learning techniques for network traffic streams analysis

Modern networks generate a massive amount of traffic data streams. Analyzing this data is essential for various purposes, such as network resources management and cyber-security analysis. There is an urgent need for data analytic methods that can perform network data processing in an online manner based on the arrival of new data. Online machine learning (OL) techniques promise to support such type of data analytics. In this paper, we investigate and compare the OL techniques that facilitate data stream analytics in the networking domain. We also investigate the importance of traffic data analytics and highlight the advantages of online learning in this regard, as well as the challenges associated with OL-based network traffic stream analysis, e.g., concept drift and the imbalanced classes. We review the data stream processing tools and frameworks that can be used to process such data online or on-the-fly along with their pros and cons, and their integrability with de facto data processing frameworks. To explore the performance of OL techniques, we conduct an empirical evaluation on the performance of different ensemble- and tree-based algorithms for network traffic classification. Finally, the open issues and the future directions in analyzing traffic data streams are presented. This technical study presents valuable insights and outlook for the network research community when dealing with the requirements and purposes of online data streams analytics and learning in the networking domain.

A data-driven method of traffic emissions mapping with land use random forest models

The development of intelligent approaches to quantify and mitigate on-road emissions is essential for urban and transportation sustainability for global megacities. Here, we utilize high-density traffic monitoring data and land use data to train random forest models capable of accurately predicting dynamic, link-level vehicle emissions. A total of 272 predicting indicators, including road features, population density, and land use information, were included in model training. Our model performed well, with a spatial generalization R2 > 0.8 for both volume and speed simulations. Dynamic link-based emissions of major air pollutants and carbon dioxide (CO2) were estimated for the whole road network of Chengdu, a populous city with the second greatest vehicle population in China. We adopted a generalized additive model to identify the drivers of spatial heterogeneity of on-road emissions and energy consumption, and nonlinear relationships between emissions, demographic and land use variables were found. Fine-grained assessments of emission reductions from potential Low Emission Zone policies are explored based on the high-resolution vehicle emission mapping tool. With high computational efficiency, the method is promising for handling traffic data streams in a real-time fashion, thus offering the potential for more precise vehicle emission management and carbon footprint tracking.

Impact of COVID-19 pandemic on low-carbon shared traffic scheduling under machine learning model

The present work aims to expand the application of machine learning models in predicting and identifying traffic flow data and provide a reference for the scheduling and management of shared traffic against the Coronavirus Disease 2019 (COVID-19) pandemic. First, a time segmentation-based prediction model is proposed considering the classification superiority of Support Vector Machine (SVM) and combining the Optimal Segmentation Algorithm (OSA), denoted as OSA-SVM. Second, an algorithm for generating a shared traffic flow sequence is proposed based on the historical data of shared traffic flow. Finally, a shared traffic flow moment identification model is constructed based on the label propagation algorithm and the Random Forest (RF) model. Comparative analysis suggests that the OSA-SVM regression prediction model can accurately fit the fluctuations caused by the shared traffic flow data; however, its overall effect is not good, with deviation from the actual traffic sequence. Introducing historical data for weighting processing improves the goodness-of-fit of the regression prediction model significantly, maintaining at the level of 0.66–0.71 after one week. The stochastic gradient descent algorithm can provide a better weighted processing effect. The RF model shows the best recognition effect for the shared traffic data stream compared with other models, presenting an excellent performance in dealing with the imbalance and instability problems. The proposed model and algorithm have outstanding prediction and recognition accuracy in shared traffic scheduling, playing an active role in traffic control during COVID-19 prevention and control.

A traffic data clustering framework based on fog computing for VANETs

Vehicular Ad-hoc Networks (VANETs) are based on vehicle to infrastructure communications in which the vehicles periodically broadcast information to update a Road Side Unit (RSU). The traffic data is forwarded from all RSUs to a cloud or a central server for global analysis and detection of congestion levels on the roads. However, communication costs may considerably increase when a large amount of data is transmitted to such cloud-like service providers. In this paper, we propose a data clustering framework to perform traffic information reduction at the edge of vehicular networks by exploiting fog computing. The proposed data clustering framework defines two methods for the reduction of the traffic data stream: (i) Baseline method, which is an ordinary traffic congestion detection approach, and (ii) two adapted clustering methods for a data stream; namely, the Ordering Points to Identify the Clustering Structure (OPTICS) and the Density-Based Spatial Clustering of Applications with Noise (DBSCAN). The results have shown that the proposed traffic data framework using clustering methods is accurate even when the vehicular traffic condition is highly congested, potentially reducing the communication costs and bringing significant results for the development of VANETs.

Uncertainty quantification and reduction in aircraft trajectory prediction using Bayesian-Entropy information fusion

Eliminating accidents while maintaining the integrity of the National Airspace System is one of the central objectives of the Next Generation Air Transportation System. This paper presents a Bayesian framework for accurate trajectory and accident prediction in National Airspace System using a high-fidelity trajectory simulation platform. Various uncertainties in aircraft trajectory prediction due to pilot behavior and weather effects are included as random variables in the simulations. Bayesian-Entropy method fuses available observation data (e.g., positioning system) with existing physical constraints (e.g. runway location) to update these simulation parameters. The posterior distributions of parameters are used to predict the probability of an adverse incident and time-remaining to incident. The proposed Bayesian updating scheme offers a flexible and rigorous way for adverse incident diagnostics and prognostics in current and future Air Traffic Management. Two realistic examples are given to show that it is possible to derive advance warning using the proposed methodology. The approach integrates data from a simulation model, with real-time traffic data streams and available physical constraints, using the Bayesian-Entropy information fusion methodology. This advance warning will allow the pilots/controllers to take actions to mitigate adverse incident in the National Airspace System.

Mining Vehicles Frequently Appearing Together from Massive Passing Records

Vehicles Frequently Appearing Together, or VFATs, can be clues in solving criminal cases. Traditional sequence mining approaches help identify VFATs from passing-through records collected at monitoring sites. However, huge traffic data streams hinder fast identification of VFATs. In this paper, we present a mult i-threaded approach to fast identification of VFATs based on multi-core processors, called Frequent Sequential Mining based on Multi-Cores (FSMMC). It parallels the execution of tasks, partitions large volumes of data, and obtains VFATs by merging local candidates discovered in different threads running on different pro cessor cores. Through local parallel reduction, FSMMC eliminates the repetitive patterns and reduces computational effort. Moreover, it achieves workload balance by the dynamic distribution of tasks to a pool of threads where the thread that finishes first joins anothe r running thread. Both theoretical analysis and case studies show that FSMMC takes full advantage of multi-core computing platforms and has higher speed-up when searching VFATs among massive passing through records, compared with other approaches without multi- threading.

StationRank: Aggregate dynamics of the Swiss railway

Increasing availability and quality of actual, as opposed to scheduled, open transport data offers new possibilities for capturing the spatiotemporal dynamics of railway and other networks of social infrastructure. One way to describe such complex phenomena is in terms of stochastic processes. At its core, a stochastic model is domain-agnostic and algorithms discussed here have been successfully used in other applications, including Google's PageRank citation ranking. Our key assumption is that train routes constitute meaningful sequences analogous to sentences of literary text. A corpus of routes is thus susceptible to the same analytic tool-set as a corpus of sentences. With our experiment in Switzerland, we introduce a method for building Markov Chains from aggregated daily streams of railway traffic data. The stationary distributions under normal and perturbed conditions are used to define systemic risk measures with non-evident, valuable information about railway infrastructure.

Improved Novel Clustering Technique for Diverse and Self-motivated Traffic Data Stream for IoT Scenario

Technologies are changing day by day and IoT is worldwide data and may of great business important to various users. sTo create such reasonable data, majority adaptive and K-mediod clustering techniques are employed in data mining. In research work, it focus on comparing adaptive, K-medisod and novel clustering technique to internet-of-things data collection in ITSs (Intelligence Traffic System). In traffic DataStream is composed form online site, it challenges of 30,000 instances with 9 attributes, clusters formed after evaluation and number of clusters is identified after the evaluation. Proposed techniques are significant too easy than some other clustering techniques with respect to all computation recall and precision parameters. In traffic databases depends on the data separation and cluster enhancement that is quality of clusters. To resolve the major issues that over load the system or Centre’s in IoT which consequences the huge kind of data on internet. It evaluated a set of consequences experiments using token and manufacture data from traffic use case view where the traffic considerations from the city monitor. Comparison of clustering methods that helps in determining suitable clustering approach for the offer internet of things database which results in optimal performance metrics.

An improved algorithm for clustering uncertain traffic data streams based on Hadoop platform

During the development of intelligent transportation systems, traffic data has the characteristics of streaming, high dimension and uncertainty. In order to realize the query of uncertain traffic data streams in a distributed environment, the authors design the algorithm of Uncertain Traffic Data Stream Parallel Continuous Query algorithm (UTDSPCQ). Firstly, the sliding window mode is applied to realize the data receiving and buffering in the data stream environment, so as to adapt to the MapReduce computing framework of the Hadoop distributed structure. Then, the impact of the high dimensionality and uncertainty of the data on the feature analysis of the dataset is reduced, through the dimension reduction and data rewriting. Finally, a multi-attribute data point RePoint is newly defined, to solve the problem of data dimension increase caused by data rewriting. Experiments show that this algorithm optimizes the traditional density-based clustering algorithm, and make it more adaptable to parallel continuous queries for uncertain traffic data streams, and can fully consider the newly generated streaming traffic data.

Dynamic reliability modeling of cyber-physical edge computing network

ABSTRACT Recently, large scale cyber physical systems (LS-CPS) leverage network-cores provided by application providers (APs) to carry out analytics. These CPS-APs uses the automated cloud to gather traffic data streams, thereby reducing infrastructure and maintenance costs. However, network reliability and maintainability considering the arrival rate of user application requests presents a major challenge such as Edge-to-Fog and Fog-to-Cloud resource auto-scaling constraints. QoS dynamic reliability, as well as its flexible management, could offer an efficient network for CPS. In this paper, CloudMesh CPS architecture is presented as a promising solution to support services like Input-Output (IO) data stream, traffic engineering, service function optimization and software defined network monitoring in CPS IPv6 data-center core. Dynamic reliability modeling for QoS parameter scaling based on observable history is presented. Hence, QoS proactive auto-scaling algorithm (PASCQA) embedded with a heuristic predictor is introduced in CloudMesh CPS. IO streams are captured by the predictor which analyzes the QoS history for reliable global performance. Finally, the work realized the critical performance aspects of CPS architecture and analyzed the effects of QoS parameter configuration for real-time deployment. The implementation results show that CloudMesh offers satisfactory QoS metrics compared with Fat-tree-1 and Fat-tree-2 for CPS applications.

An Adaptive Window Size Selection Method for Differentially Private Data Publishing over Infinite Trajectory Stream

Recently, various services based on user's location are emerging since the development of wireless Internet and sensor technology. VANET (vehicular ad hoc network), in which a large number of vehicles communicate using wireless communication, is also being highlighted as one of the services. VANET collects and analyzes the traffic data periodically to provide the traffic information service. The problem is that traffic data contains user’s sensitive location information that can lead to privacy violations. Differential privacy techniques are being used as a de facto standard to prevent such privacy violation caused by data analysis. However, applying differential privacy to traffic data stream which has infinite size over time makes data useless because too much noise is inserted to protect privacy. In order to overcome this limitation, existing researches set a certain range of windows and apply differential privacy to windowed data. However, previous researches have set a fixed window size do not consider a traffic data’s property such as road structure and time-based traffic variation. It may lead to insufficient privacy protection and unnecessary data utility degradation. In this paper, we propose an adaptive window size selection method that consider the correlation between road networks and time-based traffic variation to solve a fixed window size problem. And we suggest an adjustable privacy budget allocation technique for corresponding to the adaptive window size selection. We show that the proposed method improves the data utility, while satisfying the equal level of differential privacy as compared with the existing method through experiments that is designed based on real-world road network.

Approach to discovering companion patterns based on traffic data stream

A companion of moving objects is an object group that move together in a period of time. Platoon companions are a generalised companion pattern, which describes a group of objects that move together for time segments, each with some minimum consecutive duration of time. This study proposes a method that can instantly discover platoon companions from a special kind of streaming traffic data, called automatic number plate recognition data. Compared to related approaches, the authors transform the companion discovery into a frequent sequence mining problem. The authors propose a data structure, platoon tree (PTree), to record discovered platoon companions. To reduce the cost of tree traversal during mining platoon companions, they utilise the last two together-moving objects of a group to update PTree. Finally, a lot of experiments have been carried out to show the efficiency and effectiveness of the proposed approach.

Data Driven Congestion Trends Prediction of Urban Transportation

Smart traffic prediction system provides significant benefits in solving the city traffic congestion. However, existing smart transportation system needs a lot of real-time traffic data and accurate location information to display the traffic condition. We hope that we can use the data which is easy to be obtained, and then predict a reliable congestion time. To address this problem, this paper studied a smart traffic forecasting system based on SWARIMA model. The system includes three steps: 1) use the sliding windows to calculate and process real-time data stream; 2) establish the SWARIMA model and make regression analysis; and 3) from a statistical point of view, calculate the elastic interval and predict the congestion trend. Our system is capable of accepting the real-time traffic data stream for the congestion prediction, in addition, we reduce the actual running parameters to three attributes: 1) speed; 2) time; and 3) location information. When faced with the challenges of real-time traffic congestion, the system can timely and effectively calculate the congestion trends and provide three reliable elastic intervals: 1) warning; 2) congestion; and 3) mitigation, which has significance to improve traffic condition and alleviate urban road congestion.

Detection of fake plate vehicles based on traffic data stream

Detection of fake plate vehicles based on traffic data stream