Traffic Data Processing Research Articles

Enhancing Traffic Control with AI Blockchain and Dynamic Computation Techniques

The rapid urbanization and increasing vehicular density in modern cities have led to significant challenges in traffic management and control. As urban areas continue to expand, the demand for more efficient and intelligent traffic control systems has become increasingly critical. This paper presents a novel approach to enhancing traffic management by integrating Artificial Intelligence (AI), Blockchain technology, and Dynamic Computation Techniques. AI is utilized to analyze and predict traffic patterns, enabling real-time adjustments to traffic signals and flow management. Blockchain provides a secure, transparent, and decentralized platform for data sharing and coordination among various stakeholders, ensuring data integrity and trust. The incorporation of Dynamic Computation Techniques allows for flexible and scalable processing of complex traffic data, facilitating rapid decision-making and adaptation to changing conditions. This multidisciplinary approach not only improves traffic efficiency and reduces congestion but also paves the way for more resilient and sustainable urban transportation systems. The findings highlight the transformative potential of combining AI, Blockchain, and advanced computation methods in the field of traffic control.

A computationally intelligent framework for traffic engineering and congestion management in software-defined network (SDN)

Software-defined networking (SDN) revolutionizes network administration by centralizing control and decoupling the data plane from the control plane. Despite its advantages, the escalating volume of network traffic induces congestion at nodes, adversely affecting routing quality and overall performance. Addressing congestion has become imperative due to its emergence as a fundamental challenge in network management. Previous strategies often faced drawbacks in handling congestion, with issues arising from the inability to efficiently manage heavy packet surges in specific network regions. In response, this research introduces a novel approach integrating a multiplicative gated recurrent neural network with a congestion-aware hunter prey optimization (HPO) algorithm for effective traffic management in SDN. The framework leverages machine learning and deep learning techniques, acknowledged for their proficiency in processing traffic data. Comparative simulations showcase the congestion-aware HPO algorithm's superiority, achieving a normalized throughput 3.4–7.6% higher than genetic algorithm (GA) and particle swarm optimization (PSO) alternatives. Notably, the proposed framework significantly reduces data transmission delays by 58–65% compared to the GA and PSO algorithms. This research not only contributes a state-of-the-art solution but also addresses drawbacks observed in existing methodologies, thereby advancing the field of traffic engineering and congestion management in SDN. The proposed framework demonstrates notable enhancements in both throughput and latency, providing a more robust foundation for future SDN implementations.

Data Processing and Enrichment of LiDAR-Derived Traffic Data

Data Processing and Enrichment of LiDAR-Derived Traffic Data

STBGRN: A Traffic Prediction Model Based on Spatiotemporal Bidirectional Gated Recurrent Units and Graph Convolutional Residual Networks

With the development of society and the advancement of urbanization, the development of intelligent transportation system has attracted much attention. Efficient and accurate traffic flow prediction is one of the core tasks in the research of intelligent transportation system. Most of the existing spatiotemporal traffic flow prediction models do not make full use of various periodic spatiotemporal dynamic characteristics of traffic flow, and it is difficult to effectively capture the complex spatiotemporal changes of traffic flow. To accurately predict the traffic flow of complex urban network, this paper proposes a traffic prediction model (STBGRN) based on spatiotemporal bidirectional gated cycle unit (ST-BiGRU) and graph convolution residual network (GCN-ResNet). The spatiotemporal information in traffic data is learned using the spatiotemporal bidirectional GRU coupled with the forward and backward information of the current time step, and the topology structure of the traffic network is captured by the graph convolution residual network, which is combined to complete the prediction task of the urban traffic network. In this paper, STBGRN was tested on two real data sets, SZ-taxi and Los-loop, and compared with other baseline methods, the RMSE index reached 3.857 and 5.461, and the MAE index reached 2.702 and 3.781, respectively. Accuracy index is 72.66% and 91.35%, R2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${R}^{2}$$\\end{document} index is 0.858 and 0.845, var index is 0.858 and 0.849. The experimental results show that STBGRN model can obtain spatiotemporal dependence from complex traffic data, and can be used in spatiotemporal traffic data processing.

Traffic trajectory data analysis technology based on HMM model map matching algorithm.

The rapid growth of traffic trajectory data and the development of positioning technology have driven the demand for its analysis. However, in the current application scenarios, there are some problems such as the deviation between positioning data and real roads and low accuracy of existing trajectory data traffic prediction models. Therefore, a map matching algorithm based on hidden Markov models is proposed in this study. The algorithm starts from the global route, selects K nearest candidate paths, and identifies candidate points through the candidate paths. It uses changes in speed, angle, and other information to generate a state transition matrix that match trajectory points to the actual route. When processing trajectory data in the experiment, K = 5 is selected as the optimal value, the algorithm takes 51 ms and the accuracy is 95.3%. The algorithm performed well in a variety of road conditions, especially in parallel and mixed road sections, with an accuracy rate of more than 96%. Although the time loss of this algorithm is slightly increased compared with the traditional algorithm, its accuracy is stable. Under different road conditions, the accuracy of the algorithm is 98.3%, 97.5%, 94.8% and 96%, respectively. The accuracy of the algorithm based on traditional hidden Markov models is 95.9%, 95.7%, 95.4% and 94.6%, respectively. It can be seen that the accuracy of the algorithm designed has higher precision. The experiment proves that the map matching algorithms based on hidden Markov models is superior to other algorithms in terms of matching accuracy. This study makes the processing of traffic trajectory data more accurate.

TrafficGPT: Viewing, processing and interacting with traffic foundation models

With the promotion of ChatGPT to the public, Large language models indeed showcase remarkable common sense, reasoning, and planning skills, frequently providing insightful guidance. These capabilities hold significant promise for their application in urban traffic management and control. However, large language models (LLMs) struggle with addressing traffic issues, especially processing numerical data and interacting with simulations, limiting their potential in solving traffic-related challenges. In parallel, specialized traffic foundation models exist but are typically designed for specific tasks with limited input-output interactions. Combining these models with LLMs presents an opportunity to enhance their capacity for tackling complex traffic-related problems and providing insightful suggestions. To bridge this gap, we present TrafficGPT—a fusion of multiple LLMs and traffic foundation models. This integration yields the following key enhancements: 1) empowering LLMs with the capacity to view, analyze, process traffic data, and provide insightful decision support for urban transportation system management; 2) facilitating the intelligent deconstruction of broad and complex tasks and sequential utilization of traffic foundation models for their gradual completion; 3) aiding human decision-making in traffic control through natural language dialogues; and 4) enabling interactive feedback and solicitation of revised outcomes. By seamlessly intertwining large language model and traffic expertise, TrafficGPT not only advances traffic management but also offers a novel approach to leveraging AI capabilities in this domain. The TrafficGPT demo can be found in https://github.com/lijlansg/TrafficGPT.git.

Machine Learning-Based Intrusion Detection System for Encrypted Attacks

In today's society, information and communication technology is developing rapidly. With the gradual maturity and popularization of encryption technology, more and more malicious attacks are also using encryption technology to evade the scrutiny of traditional traffic detection systems. Therefore, accurate identification of encrypted attacks has become a research hotspot in the international community. This paper proposes an encrypted traffic detection method based on convolutional neural network (CNN) technology to address the issues of tedious steps and low recognition accuracy in manually extracting traffic features. This method does not require manual or expert feature extraction, and can automatically extract advanced features through CNNs, which are then fed into XGBoost classifiers for classification processing. On the basis of the above methods, this article designs and implements an encrypted traffic intrusion detection system, which is divided into five parts: traffic collection, data processing, model detection, data visualization, and traffic blocking. Reasonable explanations and technical introductions are provided for these modules.

High-Resolution Image Processing and Spatiotemporal Data Transmission System Based on GPU Acceleration

With the development of information technology and the increasing demand for data processing, the serial mode of the central processing unit (CPU) is difficult to efficiently transmit large-scale spatiotemporal data, and the processing effect for high-resolution images is not good. This paper designed a high-resolution image processing and spatiotemporal data transmission system based on graphics processing unit (GPU) acceleration to improve the processing efficiency of large-scale spatiotemporal data. In this paper, traffic spatiotemporal data was taken as an example for analysis. Large-scale traffic image data was collected by road monitoring equipment, and image compression was performed on the collected image. Fourier transform was used to eliminate image data redundancy, and GPU-accelerated parallel processing was used to achieve fast image defogging and data transmission. This paper selected 2TB of traffic spatiotemporal data with image resolutions of 540P, 720P, 1080P, 1440P, and 2160P. GPU acceleration was performed using the Compute Unified Device Architecture (CUDA). In images with a resolution of 2160P, the processing time for CPU and GPU acceleration was 2900ms and 28ms, respectively, with an acceleration ratio of 103.6. A high-resolution image processing and spatiotemporal data transmission system based on GPU acceleration can improve the efficiency of traffic spatiotemporal data processing and have excellent concurrent processing capabilities.

Multisource Accident Datasets‐Driven Deep Learning‐Based Traffic Accident Portrait for Accident Reasoning

Traffic accident data‐based portrait plays a vital role in accident cause investigation, relationship reasoning, prevention, and control. The traffic accident data tend to be multisourced with increasingly hidden and complicated accident relationships. The existing reported research focus more on traffic drivers’ measurement of penalty, the relationship among drivers, cars, and dates, etc. How to use multisource data based on deep learning, especially based on the Chinese recent unstructured data and structured data to establish accident portrait for individual and groups of accident drivers, still lacks. Moreover, how to perform multisource accident data label extraction, identity, and relationship extraction are still challenging problems. This paper proposes a multisource accident datasets‐driven deep learning‐based traffic accident portrait method. Our multisource accident datasets‐driven deep learning model is composed of the following three submodels: (1) the structured data accident model using our accident feature‐driven bidirectional long short‐term memory (Bi‐LSTM) and accident feature‐driven bidirectional conditional random field (Bi‐CRF) model to extract labels, (2) the unstructured traffic accident data model using our accident feature‐driven piecewise convolutional neural network (PCNN) model to identify the extracted labels, and (3) the semistructured traffic accident data processing model. Moreover, to solve the problem of how to construct hidden relationship among the multisource accident data, a multisource accident data visualization method based on traffic accident knowledge graph where the accident relational inference algorithm is to complete the hidden relationship between traffic accident data labels is used and then data are visualized using the traffic accident knowledge graph. This paper uses the NER dataset of the People’s Daily and a manually labeled dataset to test the Bi‐LSTM + Bi‐CRF model, and it acquires the highest scores of 0.9562 and 0.9779 compared with several other models. This paper uses the DuIE dataset and a manually labeled dataset to test the PCNN model, and it acquires the highest scores of 0.9674 and 0.9108 compared with several other models. Experiments verified our model’s merits than other models in regards to accident label extraction, accident identity identification, and accident relationship extraction.

Distributed Intelligent Traffic Data Processing and Analysis Based on Improved Longhorn Whisker Algorithm

Distributed Intelligent Traffic Data Processing and Analysis Based on Improved Longhorn Whisker Algorithm

Research of Model Increasing Reliability Intrusion Detection Systems

The paper presents the results of the using, a recurrent neural network to detect malicious software as part of the Snort intrusion detection system.The research was conducted on datasets generated on the basis of athena, dyre, engrat, grum, mimikatz, surtr malware exploiting vulnerability CVE-2022-20685 in the Snort intrusion detection system. Processing of input traffic data was carried out before the frag-3 and modbus preprocessors. The method of k nearest neighbors was used as a mathematical apparatus. The simulation of the developed software at different iterations. All research results are presented in https://github.com/T-JN

Statistical processing of traffic flow characteristics data

In the course of statistical processing of traffic flows characteristics data, the check for the presence of anomalous measurements in the sampling should be done at the very start of processing. If anomalous measurements are detected, they should be excluded from the sampling at an early stage of the processing and not taken into considerations in further calculations. Numerous criteria have been developed to detect outliers, their effectiveness depends on the sample size. In practice, for technical and economic reasons, it is impractical to obtain a large number of measurements, as a rule the sampling should be processed on the basis of limited number of observations. In this regard, methods for detection of outliers with a small number of measurements, which include the method based on the use of the Romanovsky criterion, are of great importance. However, the analysis of literary references showed that in some recently published studies it is not recommended to use the Romanovsky criterion with the number of measurements less than 20. Therefore, the purpose of this study is to test the power of the Romanovsky criterion (test) for a small number of measurements and the possibility of its application in samplings of small size (n≤ 20). The conducted studies have shown that the power of the Romanovsky criterion is quite high and it has high reliability with a small number of measurements, which makes it possible to use it in small samples to detect anomalous measurements.

RESNETCNN: An abnormal network traffic flows detection model

Intrusion detection is an important means to protect system security by detecting intrusions or intrusion attempts on the system through operational behaviors, security logs, and data audit. However, existing intrusion detection systems suffer from incomplete data feature extraction and low classification accuracy, which affects the intrusion detection effect. To this end, this paper proposes an intrusion detection model that fuses residual network (RESNET) and parallel crossconvolutional neural network, called RESNETCCN. RESNETCNN can efficiently learn various data stream features through the fusion of deep learning and convolutional neural network (CNN), which improves the detection accuracy of abnormal data streams in unbalanced data streams, moreover, the oversampling method into the data preprocessing, to extract multiple types of unbalanced data stream features at the same time, effectively solving the problems of incomplete data feature extraction and low classification accuracy of unbalanced data streams. Finally, three improved versions of RESNETCNN networks are designed to meet the requirements of different traffic data processing, and the highest detection accuracy reaches 99.98% on the CICIDS 2017 dataset and 99.90% on the ISCXIDS 2012 dataset.

Development of vehicle driving cycles based on the real traffic dataset

The paper describes methods for generating (modeling) representative driving cycles of the vehicle used to solve engineering problems in the design of electric vehicles, such as resource calculations, determination of the required capacity of traction batteries and evaluation of power reserve, etc. The general approaches used in the processing of real traffic data and algorithms for modeling driving cycles using deterministic and probabilistic approaches are described. This paper presents driving cycles that ensure convergence with parameters corresponding to the real conditions of vehicle movement. The developed driving cycles can be used in the design of electric vehicle transmission components and allow for the analysis of operational properties.

Multitask Neural Tensor Factorization for Road Traffic Speed-Volume Correlation Pattern Learning and Joint Imputation

Missing data is a common and critical problem in the stage of traffic data collection and processing. How to impute the missing values in spatio-temporal traffic data has been a challenging topic for a long time. Recently, a variety of methods have been proposed to impute the missing values. Among them, the tensor-based methods show higher competence in multi-dimensional traffic data imputation. However, the previous studies of tensor factorization rarely considered the joint imputation of multiple correlative data such as traffic speed and traffic volume. In this paper, a novel method called Multi-Task Neural Tensor Factorization (MTNTF) is proposed to learn the non-linear correlation patterns of traffic speed-volume, and then address the joint imputation of traffic speed and traffic volume. Extensive experiments on a real dataset show our MTNTF significantly outperforms the state-of-the-art methods in element-wise missing and fiber-wise missing cases. In addition, our method can impute the slice-wise missing values of traffic volume based on incomplete traffic speed.

Security Improvements for Connected Vehicles Position-Based Routing

The constant growing on the number of vehicles is increasing the complexity of traffic in urban and highway environments. It is paramount to improve traffic management to guarantee better road usage and people’s safety. Through efficient communications, Vehicular Ad hoc Networks (VANETs) can provide enough information for traffic safety initiatives, daily traffic data processing, and entertainment information. However, VANETs are vulnerable to malicious nodes applying different types of net-work attacks, where an attacker can, for instance, forge its position to receive the data packet and drop the message. This can lead vehicles and authorities to make incorrect assumptions and decisions, which can result in dangerous situations. Therefore, any data dissemination protocol designed for VANET should consider security issues when selecting the next-hop forwarding node. In this paper, we propose a security scheme designed for position-based routing algorithms, which analyzes nodes position, transmission range, and hello packet interval. The scheme deals with malicious nodes performing network attacks, faking their positions forcing packets to be dropped. We used the Simulation of Urban MObility (SUMO) and Network Simulator-version 3 (NS-3) to compare our proposed scheme integrated with two well-known position-based algorithms. The results were collected in an urban Manhattan grid environment varying the number of nodes, the number of malicious nodes, as well as the number of source-destination pairs. The results show that the proposed security scheme can successfully improve the packet delivery ratio while maintaining low average end-to-end delay of the algorithms.

Human-Computer Interaction of Networked Vehicles Based on Big Data and Hybrid Intelligent Algorithm

Intelligent connected vehicle (ICV) refers to realizing the exchange and sharing of intelligent information between vehicles, roads, people, clouds, and so on by carrying advanced on-board sensors, controllers, actuators, and other devices and combining modern communication and network technology. In the age of big data, the information of everything can be transformed into digital resources, and transportation big data has become a basic resource. This paper constructs a big data platform for traffic data processing to realize the function of real-time collection, processing, and analysis of traffic data. Based on the proposed big data platform, the parallel programming framework of MapReduce and HDFS distributed storage system are used to process the real-time vehicle dynamic information in parallel, and the output result is used as the input of running genetic algorithm simulated annealing (GA-SA) for parallel calculation. At the same time, it studies the impact of various elements on users’ interactive behavior, constructs the demand framework and design model of automobile human-computer interaction, and then realizes fast and comprehensive search. The experimental results show that the human-computer interaction method of intelligent networked vehicle can find the optimal driving path, transmit it to each networked vehicle through the human-computer interaction system, realize human-computer interaction, reduce the impact of user unintentional operation on redundant motion, reduce the motion error accumulation of the system, and improve the performance of human-computer interaction system.

Sboing4Real: A real-time crowdsensing-based traffic management system

This work describes the architecture of the back-end engine of a real-time traffic data processing and satellite navigation system. The role of the engine is to process real-time feedback, such as speed and travel time, provided by in-vehicle devices and derive real-time reports and traffic predictions through leveraging historical data as well. We present the main building blocks and the versatile set of data sources and processing platforms that need to be combined together to form a fully-functional and scalable solution. We also present performance results focusing on meeting system requirements while keeping the need for computing resources low. The lessons and results presented are of value to additional real-time applications that rely on both recent and historical data. Finally, we discuss the application of the aforementioned solution to a successful pilot study, where the full system was deployed and processed data from 800 taxis for a period of 3 months.

Traffic Data Collection and Visualization Tool for Knowledge Discovery Using Google Maps

Traffic jam is increasingly aggravating in almost every urban area. Traffic forecast, traffic modeling, visualization can help to provide appropriate route and time for traveling and thus provides a significant impact on traffic jam reduction. For traffic forecasting, modeling and visualization, city-wide traffic data collection and analysis are needed, which is still challenging in many aspects. This paper aims to develop a tool for acquiring and processing traffic data from Google Maps that can be used for forecasting, modeling, and visualization. Dhaka city is used as a case study since there is no infrastructure available for traffic data collection. The traffic flow intensity of the road is analyzed to determine the congestion of the road. The flow intensity is used for traffic modeling, visualization, traffic prediction and many more.

MODELS AND TOOLS OF THE AUTOMATED SYSTEM FOR COMPUTER NETWORK TRAFFIC INVESTIGATION USING BERKELEY PACKET FILTER

An approach for automating the monitoring and analysis of incoming network traffic in large-scale computer networks is proposed in the paper. The authors suggest using the Linux Berkeley Packet Filter tool to automate traffic analysis in computer networks. The software structure is developed, which includes two main parts: the service machine and the user machine, it is based on the modular principle, which allows for rapid improvement and modernization of the system. The main algorithms for software functionality are built, namely: the algorithm for processing network packets using the Berkeley Packet Filter tool, and the algorithm of the user-space program for loading the Berkeley Packet Filter program to kernel space and setting up communication with it. A study model of program functioning dynamics based on the Petri net theory has been developed. As a result of the application of models based on the Petri net in the software development process, the system works correctly, all states are accessible, and there are no dead ends. A simulation model of the application of the Berkeley Packet Filter tool for the automation of computer network traffic analysis was designed, and the script was created for testing the developed software system. Implemented Python script generates a flow of network packets with random values in the sender IP address, receiver port number, and protocol. These packets, created by the IP address spoofing methodology, later are sent to the service machine's network interface. The developed computer network traffic monitoring software, that uses the Berkeley Packet Filter tool and is implemented in C, C++, and Python programming languages, provides collecting and processing of computer network traffic data. The output of the analysed results is displayed in a user-friendly form. The development and testing of the created software were carried out on the operating system Arch Linux version 5.10.3, which was previously installed on a virtual machine. The results of traffic testing in computer networks in different modes of normal operation and during DDoS attacks are given. In particular, an example of sidebar output with network traffic statistics over a long period, an example of output with network traffic spike parameters, and an example of a warning message, that the sidebar dashboard will show, are presented.