German Traffic Sign Detection Benchmark Research Articles

Traffic sign detection based on classic visual recognition models

This paper aims to assess the effectiveness of various object detection high-level architectures, including Faster R-CNN, R-FCN, SSD, and YOLO, in recognizing traffic signs. Since traffic sign recognition is a critical element of driver-assistance and autonomous driving systems, this study explores how five distinct architectures for feature extracting (ResNet V1 50, ResNet V1 101, Inception V2, Inception-ResNet-V2, and Darknet-19) can enhance the performance of these meta-architectures. To evaluate the models, the authors fine-tuned a pre-trained object recognition algorithm on the German Traffic Sign Detection Benchmark (GTSDB) dataset. While the Faster R-CNN Inception Resnet v2 proved to be the most accurate among the models, the unbalanced distribution of mandatory, prohibitory, and danger signs in the GTSDB dataset may have caused biased results in object detection. Overall, the result obtained from this study can help improve the accuracy of driver-assistance and autonomous driving systems by providing insights into the performance of various object detection models in recognizing traffic signs.

Feature-Selection-Based Attentional-Deconvolution Detector for German Traffic Sign Detection Benchmark

In this study, we propose a novel traffic sign detection algorithm based on the deeplearning approach. The proposed algorithm, which we termed the feature-selection-based attentionaldeconvolution detector (FSADD), is used along with the “you look only once” (YOLO) v5 structure for feature selection. When applying feature selection inside a detection algorithm, the network divides the extracted feature maps after the convolution layer into similar and non similar feature maps. Generally, the feature maps obtained after the convolution layers are the outputs of filters with random weights. Owing to the randomness of the filter, the network obtains various kinds of feature maps with unnecessary components, which degrades the detection performance. However, grouping feature maps with high similarities can increase the relativeness of each feature map, thereby improving the network detection of specific targets from images. Furthermore, the proposed FSADD model has modified sizes of the receptive fields for improved traffic sign detection performance. Many of the available general detection algorithms are unsuitable for the German traffic sign detection benchmark (GTSDB) because of the small sizes of these signs in the images. Experimental comparisons were performed with respect to the GTSDB to show that the proposed FSADD is comparable to the state-of-the-art while detecting 29 kinds of traffic signs with 73.9% accuracy of classification performances.

Faster Light Detection Algorithm of Traffic Signs Based on YOLOv5s-A2

Traffic sign recognition systems have been applied to advanced driving assistance and automatic driving systems to help drivers obtain important road information accurately. The current mainstream detection methods have high accuracy in this task, but the number of model parameters is large, and the detection speed is slow. Based on YOLOv5s as the basic framework, this paper proposes YOLOv5S-A2, which can improve the detection speed and reduce the model size at the cost of reducing the detection accuracy. Firstly, a data augmentation strategy is proposed by combining various operations to alleviate the problem of unbalanced class instances. Secondly, we proposed a path aggregation module for Feature Pyramid Network (FPN) to make new horizontal connections. It can enhance multi-scale feature representation capability and compensate for the loss of feature information. Thirdly, an attention detection head module is proposed to solve the aliasing effect in cross-scale fusion and enhance the representation of predictive features. Experiments on Tsinghua-Tencent 100K dataset (TT100K) show that our method can achieve more remarkable performance improvement and faster inference speed than other advanced technologies. Our method achieves 87.3% mean average precision (mAP), surpassing the original model’s 7.9%, and the frames per second (FPS) value is maintained at 87.7. To show generality, we tested it on the German Traffic Sign Detection Benchmark (GTSDB) without tuning and obtained an average precision of 94.1%, and the FPS value is maintained at about 105.3. In addition, the number of YOLOv5s-A2 parameters is about 7.9 M.

Traffic sign classification using CNN and detection using faster-RCNN and YOLOV4

Autonomous driving cars are becoming popular everywhere and the need for a robust traffic sign recognition system that ensures safety by recognizing traffic signs accurately and fast is increasing. In this paper, we build a CNN that can classify 43 different traffic signs from the German Traffic Sign Recognition benchmark dataset. The dataset is made up of 39,186 images for training and 12,630 for testing. Our CNN for classification is light and reached an accuracy of 99.20% with only 0.8 M parameters. It is tested also under severe conditions to prove its generalization ability. We also used Faster R–CNN and YOLOv4 networks to implement a recognition system for traffic signs. The German Traffic Sign Detection benchmark dataset was used. Faster R–CNN obtained a mean average precision (mAP) of 43.26% at 6 Frames Per Second (FPS), which is not suitable for real-time application. YOLOv4 achieved an mAP of 59.88% at 35 FPS, which is the preferred model for real-time traffic sign detection. These mAPs are obtained using Intersect Over Union of 50%. A comparative analysis is also presented between these models.

Traffic Sign Recognition Based on Semantic Scene Understanding and Structural Traffic Sign Location

Traffic sign recognition (TSR) plays an important role in driving assistance system and traffic safety insurance. However, existing methods focus on extracting features of traffic signs and ignore the constraints of spatial positional relationships between traffic signs and other objects in the scene. This way results in incorrectly detecting other similar objects as traffic signs and failing to detect very small traffic signs. A TSR method based on semantic scene understanding and structural traffic sign location is proposed in this study to solve the aforementioned problems. A scene structure model based on the constraints of spatial positional relationships between traffic signs and other objects is proposed to establish trusted search regions. An improved Light-weight RefineNet is used to analyze and understand a scene semantically and accurately and then segment objects in complicated environments precisely. A new network multiscale densely connected object detector (MDCOD) based on densely connected style, multiscale feature fusion, and improved K-means++ algorithms is proposed to recognize very small traffic signs. The trusted traffic signs are found by filtering false candidates outside the scene structure model. The proposed method is tested on Tsinghua-Tencent 100K and German Traffic Sign Detection Benchmark datasets and achieves accuracies of 92.8% and 99.90%, respectively, outperforming the existing methods.

A Traffic Sign Detection System Linking Hypothesis Tests and Deep Learning Networks

In this paper, we propose a computationally efficient method for traffic sign detection. Our methodology uses deep learning networks introduced for semantic image segmentation and classification of region of interests generated by image segmentation. The region of interests are detected by segmenting the road scene image dataset and hypothesis tests are applied to accept or reject the probability of being a traffic sign in the detected region of interest. For hypothesis tests, color feature, and the location of the traffic sign with respect to the segmented road information is used. The main contribution of our method is the introduction of hypothesis tests that mutually couple two deep learning networks trained by well-known datasets publicly available for benchmarking purposes. To test and evaluate our traffic sign detection system, we use the German traffic sign detection benchmark dataset with a large set of traffic signs and during its training, The cityscapes dataset offering labeled urban scenes is also leveraged by the traffic sign detection system. Our experimental results illustrate the effectiveness performance metrics are reached at 90.81%, 94.76% and 92.74% in precision, recall and F-measure, respectively. The runtime cost is around 0.4[Formula: see text]s for an image on an ordinary laptop computer.

IELMNet: Integrating Novel Improved Extreme Learning Machine and Convolutional Neural Network Model for Traffic Sign Detection.

Traffic sign detection (TSD) in real-time environment holds great importance for applications such as automated-driven vehicles. Large variety of traffic signs, different appearances, and spatial representations causes a huge intraclass variation. In this article, an extreme learning machine (ELM), convolutional neural network (CNN), and scale transformation (ST)-based model, called improved extreme learning machine network, are proposed to detect traffic signs in real-time environment. The proposed model has a custom DenseNet-based novel CNN architecture, improved version of region proposal networks called accurate anchor prediction model (A2PM), ST, and ELM module. CNN architecture makes use of handcrafted features such as scale-invariant feature transform and Gabor to improvise the edges of traffic signs. The A2PM minimizes the redundancy among extracted features to make the model efficient and ST enables the model to detect traffic signs of different sizes. ELM module enhances the efficiency by reshaping the features. The proposed model is tested on three publicly available data sets, challenging unreal and real environments for traffic sign recognition, Tsinghua-Tencent 100K, and German traffic sign detection benchmark and achieves average precisions of 93.31%, 95.22%, and 99.45%, respectively. These results prove that the proposed model is more efficient than state-of-the-art sign detection techniques.

FFSSD: Feature Fusion SSD algorithm with SE on Traffic Sign Detection

Traffic sign detection is of great importance to the development of intelligent vehicles. In this paper, we propose a modified SSD algorithm called FFSSD to the traffic environment. Firstly,we integrate low and high level features together in order to enhance the accuracy of tiny instances. Besides, we insert a feature enhancement module into it and the original SSD loss was altered by focal loss, which is designed to solve the problem of uneven sample distribution. We evaluate the algorithm on GTSDB (The German Traffic Sign Detection Benchmark dataset). The experimental result demonstrates that our algorithm has a great achievement. Compared with some up-of-date methods, it also achieves higher correctness in the traffic field.

Detection of Small Size Traffic Signs Using Regressive Anchor Box Selection and DBL Layer Tweaking in YOLOv3

Traffic sign recognition is a key module of autonomous cars and driver assistance systems. Traffic sign detection accuracy and inference time are the two most important parameters. Current methods for traffic sign recognition are very accurate; however, they do not meet the requirement for real-time detection. While some are fast enough for real-time traffic sign detection, they fall short in accuracy. This paper proposes an accuracy improvement in the YOLOv3 network, which is a very fast detection framework. The proposed method contributes to the accurate detection of a small-sized traffic sign in terms of image size and helps to reduce false positives and miss rates. In addition, we propose an anchor frame selection algorithm that helps in achieving the optimal size and scale of the anchor frame. Therefore, the proposed method supports the detection of a small traffic sign with real-time detection. This ultimately helps to achieve an optimal balance between accuracy and inference time. The proposed network is evaluated on two publicly available datasets, namely the German Traffic Sign Detection Benchmark (GTSDB) and the Swedish Traffic Sign dataset (STS), and its performance showed that the proposed approach achieves a decent balance between mAP and inference time.

Hardware Efficient Modified CNN Architecture for Traffic Sign Detection and Recognition

Traffic sign recognition is a vital part for any driver assistance system which can help in making complex driving decision based on the detected traffic signs. Traffic sign detection (TSD) is essential in adverse weather conditions or when the vehicle is being driven on the hilly roads. Traffic sign recognition is a complex computer vision problem as generally the signs occupy a very small portion of the entire image. A lot of research is going on to solve this issue accurately but still it has not been solved till the satisfactory performance. The goal of this paper is to propose a deep learning architecture which can be deployed on embedded platforms for driver assistant system with limited memory and computing resources without sacrificing on detection accuracy. The architecture uses various architectural modification to the well-known Convolutional Neural Network (CNN) architecture for object detection. It uses a trainable Color Transformer Network (CTN) with the existing CNN architecture for making the system invariant to illumination and light changes. The architecture uses feature fusion module for detecting small traffic signs accurately. In the proposed work, receptive field calculation is used for choosing the number of convolutional layer for prediction and the right scales for default bounding boxes. The architecture is deployed on Jetson Nano GPU Embedded development board for performance evaluation at the edge and it has been tested on well-known German Traffic Sign Detection Benchmark (GTSDB) and Tsinghua-Tencent 100k dataset. The architecture only requires 11 MB for storage which is almost ten times better than the previous architectures. The architecture has one sixth parameters than the best performing architecture and 50 times less floating point operations per second (FLOPs). The architecture achieves running time of 220[Formula: see text]ms on desktop GPU and 578 ms on Jetson Nano which is also better compared to other similar implementation. It also achieves comparable accuracy in terms of mean average precision (mAP) for both the datasets.

Integrated Feature Pyramid Network With Feature Aggregation for Traffic Sign Detection

Traffic sign detection is a critical task in the visual system of the Advanced Driver Assistance System (ADAS) and the Automated Driving System (ADS). Although the general object detection has achieved promising results by using Feature Pyramid Network (FPN) in recent years, we still observed that FPN cannot obtain satisfactory results in traffic sign detection because the size and class distribution of traffic signs are extremely unbalanced. To overcome this problem, a novel Plug-and-Play neck network Integrated Feature Pyramid Network with Feature Aggregation (IFA-FPN) is proposed in this paper based on the statistical characteristics of traffic signs. First, a lightweight operation is introduced to fully utilize the model and improve the inference speed of the model. Second, an Integrated Operation (IO) is introduced to solve the imbalance problem of Region-of-Interests (RoIs) in pyramid levels. Third, we introduce a Feature Aggregation (FA) structure to strengthen the feature representation capacity of feature maps, thereby enhancing the network robustness against the size discrepancy of traffic signs. The experiments are performed on three mainstream datasets, i.e., the German Traffic Sign Detection Benchmark (GTSDB), Swedish Traffic Sign Dataset (STSD), and Tsinghua-Tencent 100k dataset (TT100k). The experimental results demonstrate the superiority of the proposed IFA-FPN in the traffic sign detection tasks. Specifically, when the proposed IFA-FPN is applied to the Cascade RCNN, it achieves 80.3% mAP in GTSDB which surpasses FPN by 9.9%, 65.2% in mAP in STSD which surpasses FPN by 3.5%, and 93.6% in mAP in TT100k which surpasses FPN by 1.6%.

Fast Traffic Sign Detection Approach Based on Lightweight Network and Multilayer Proposal Network

Vision-based traffic sign detection plays a crucial role in intelligent transportation systems. Recently, many approaches based on deep learning for traffic sign detection have been proposed and showed better performance compared with traditional approaches. However, due to difficult conditions in driving environment and the size of traffic signs in traffic scene images, the performance of deep learning-based methods on small traffic sign detection is still limited. In addition, the inference speed of current state-of-the-art approaches on traffic sign detection is still slow. This paper proposes a deep learning-based approach to improve the performance of small traffic sign detection in driving environments. First, a lightweight and efficient architecture is adopted as the base network to address the issue of the inference speed. To enhance the performance on small traffic sign detection, a deconvolution module is adopted to generate an enhanced feature map by aggregating a lower-level feature map with a higher-level feature map. Then, two improved region proposal networks are used to generate proposals from the highest-level feature map and the enhanced feature map. The proposed improved region proposal network is designed for fast and accuracy proposal generation. In the experiments, the German Traffic Sign Detection Benchmark dataset is used to evaluate the effectiveness of each enhanced module, and the Tsinghua-Tencent 100K dataset is used to compare the effectiveness of the proposed approach with other state-of-the-art approaches on traffic sign detection. Experimental results on Tsinghua-Tencent 100K dataset show that the proposed approach achieves competitive performance compared with current state-of-the-art approaches on traffic sign detection while being faster and simpler.

Real time detection and classification of traffic signs based on YOLO version 3 algorithm

The issue of effective detection and classification of various traffic signs is studied. The two-stage method is proposed for creation of holistic model with end-to-end solution. The first stage includes implementation of effective localization of traffic signs by YOLO version 3 algorithm (You Only Look Once). At the first stage the traffic signs are grouped into four categories according to their shapes. At the second stage, an accurate classification of the located traffic signs is performed into one of the forty-three predefined categories. The second stage is based on another model with one convolutional neural layer. The model for detection of traffic signs was trained on German Traffic Sign Detection Benchmark (GTSDB) with 630 and 111 RGB images for training and validation, respectively. Сlassification model was trained on German Traffic Sign Recognition Benchmark (GTSRB) with 66000 RGB images on pure “numpy” library with 19 × 19 dimension of convolutional layer filters and reached 0.868 accuracy on testing dataset. The experimental results illustrated that the training of the first model deep network with only four categories for location of traffic signs produced high mAP (mean Average Precision) accuracy reaching 97.22 %. Additional convolutional layer of the second model applied for final classification creates efficient entire system. Experiments on processing video files demonstrated frames per second (FMS) between thirty-six and sixty-one that makes the system feasible for real time applications. The frames per second depended on the number of traffic signs to be detected and classified in every single frame in the range from six to one.

Performance enhancement techniques for traffic sign recognition using a deep neural network

An advanced traffic sign recognition (ATSR) system using novel pre-processing techniques and optimization techniques has been proposed. During the pre-processing of input road images, color contrasts are enhanced and edges are made clearer, for easier detection of small-sized traffic signs. YOLOv3 has been modified to build our traffic sign detector, since it is an efficient and effective deep neural network. In this YOLOv3 modifications, grid optimization and anchor box optimization were done to optimize the detection performance on small-sized traffic signs. We trained the system on our traffic sign dataset and tested the recognition performance using the Mean Average Precision (MAP) on the Korean Traffic Sign Dataset (KTSD) and German Traffic Sign Detection Benchmark (GTSDB). We used the bisection method for selecting the optimum threshold of confidence score to reduce false predictions. Our ATSR system is capable of recognizing Prohibitory, Mandatory, and Danger class traffic signs from road images. ATSR can detect small-sized traffic signs accurately along with big-sized traffic signs. It shows the best recognition performance of 98.15% on the challenging KTSD (the previously reported best performance was 90.07%) and 100% on the GTSDB. Result comparisons show that ATSR significantly outperforms ITSR, TS detector, YOLOv3, and D-patches, on KTSD.

Two‐stage traffic sign detection and recognition based on SVM and convolutional neural networks

Nowadays, traffic sign recognition is the most important task of advanced driver assistance systems since it improves the safety and comfort of drivers. However, it remains a challenging task due to the complexity of road traffic scenes. In this study, a novel two-stage approach for real-time traffic sign detection and recognition in a real traffic situation was proposed. The first stage aims to detect and classify the detected traffic signs into circular and triangular shape using HOG features and linear support vector machines (SVMs). The main objective of the second stage is to recognise the traffic signs using a convolutional neural network into their subclasses. The performance of the whole process is tested on German traffic sign detection benchmark (GTSDB) and German traffic sign recognition benchmark (GTSRB) datasets. Experimental results show that the obtained detection and recognition rate is comparable with those reported in the literature with much less complexity. Furthermore, the average processing time demonstrates its suitability for real-time processing applications.

Traffic Sign Detection Using a Multi-Scale Recurrent Attention Network

Traffic sign detection plays an important role in intelligent transportation systems. But traffic signs are still not well-detected by deep convolution neural network-based methods because the sizes of their feature maps are constrained, and the environmental context information has not been fully exploited by other researchers. What we need is a way to incorporate relevant context detail from the neighboring layers into the detection architecture. We have developed a novel traffic sign detection approach based on recurrent attention for multi-scale analysis and use of local context in the image. Experiments on the German traffic sign detection benchmark and the Tsinghua-Tencent 100K data set demonstrated that our approach obtained an accuracy comparable to the state-of-the-art approaches in traffic sign detection.

Traffic Signs Detection and Classification for European Urban Environments

Traffic signs play an important role for Advanced Driver Assistance Systems (ADAS) as well as for autonomous driving vehicles. Most of the works done focus on recognizing symbol based signs leaving apart important information provided by other type of signs like complementary panels or text based signs. In this paper, we include detection and classification of both symbol and text based signs focusing on the most common ones found in European urban environments. The system consists of three stages, traffic sign detection, refinement and classification. The detection and refinement is performed using Mask R-CNN while the classification is achieved with a proposed Convolutional Neural Network (CNN) architecture. We introduced the extended version of the German Traffic Sign Detection Benchmark (GTSDB), labeled in a pixel manner (masks) with 164 classes grouped into 8 categories. It is used for the detection and classification steps. Experimental results on German environments show that our proposed system is capable of detecting all categories of traffic signs while at the same time recognizing them with high accuracy achieving comparable performance with the state of the art.

Benchmarking Deep Learning Frameworks and Investigating FPGA Deployment for Traffic Sign Classification and Detection

We benchmark several widely-used deep learning frameworks and investigate the field programmable gate array (FPGA) deployment for performing traffic sign classification and detection. We evaluate the training speed and inference accuracy of these frameworks on the graphics processing unit (GPU) by training FPGA-deployment-suitable models with various input sizes on German Traffic Sign Recognition Benchmark (GTSRB), a traffic sign classification dataset. Then, selected trained classification models and various object detection models that we train on GTSRB's detection counterpart (i.e., German Traffic Sign Detection Benchmark) are evaluated with inference speed, accuracy, and FPGA power efficiency by varying different parameters such as floating-point precisions, batch sizes, etc. We discover that Neon and MXNet deliver the best training speed and classification accuracy on the GPU in general for all test cases, while TensorFlow is always among the frameworks with the highest inference accuracies. We observe that with the current OpenVINO release, the performance of lightweight models (e.g., MobileNet-v1-SSD, etc.) usually exceeds the requirement of real-time detection without losing much accuracy, while other models (e.g., VGG-SSD, ResNet-50-SSD) generally fail to do so. We also demonstrate that we can adjust the precision of bitstreams and the batch sizes to balance inference speed and accuracy of the applications deployed on the FPGA. Finally, we show that for all test cases, the FPGA always achieves higher power efficiency than the GPU.

Improved Faster R-CNN Traffic Sign Detection Based on a Second Region of Interest and Highly Possible Regions Proposal Network.

Traffic sign detection systems provide important road control information for unmanned driving systems or auxiliary driving. In this paper, the Faster region with a convolutional neural network (R-CNN) for traffic sign detection in real traffic situations has been systematically improved. First, a first step region proposal algorithm based on simplified Gabor wavelets (SGWs) and maximally stable extremal regions (MSERs) is proposed. In this way, the region proposal a priori information is obtained and will be used for improving the Faster R-CNN. This part of our method is named as the highly possible regions proposal network (HP-RPN). Second, in order to solve the problem that the Faster R-CNN cannot effectively detect small targets, a method that combines the features of the third, fourth, and fifth layers of VGG16 to enrich the features of small targets is proposed. Third, the secondary region of interest method to enhance the feature of detection objects and improve the classification capability of the Faster R-CNN is proposed. Finally, a method of merging the German traffic sign detection benchmark (GTSDB) and Chinese traffic sign dataset (CTSD) databases into one larger database to increase the number of database samples is proposed. Experimental results show that our method improves the detection performance, especially for small targets.

Automatic Traffic Sign Detection and Recognition Using SegU-Net and a Modified Tversky Loss Function With L1-Constraint

Traffic sign detection is a central part of autonomous vehicle technology. Recent advances in deep learning algorithms have motivated researchers to use neural networks to perform this task. In this paper, we look at traffic sign detection as an image segmentation problem and propose a deep convolutional neural network-based approach to solve it. To this end, we propose a new network, the SegU-Net, which we form by merging the state-of-the-art segmentation architectures–SegNet and U-Net to detect traffic signs from video sequences. For training the network, we use the Tversky loss function constrained by an L1 term instead of the intersection over union loss traditionally used to train segmentation networks. We use a separate network, inspired by the VGG-16 architecture, to classify the detected signs. The networks are trained on the challenge free sequences of the CURE-TSD dataset. Our proposed network outperforms the state-of-the-art object detection networks, such as the Faster R-CNN inception Resnet V2 and R-FCN Resnet 101, by a large margin and obtains a precision and recall of 94.60% and 80.21%, respectively, which is the current state of the art on this part of the dataset. In addition, the network is tested on the German Traffic Sign Detection Benchmark (GTSDB) dataset, where it achieves a precision and recall of 95.29% and 89.01%, respectively. This is on a par with the performance of the aforementioned object detection networks. These results prove the generalizability of the proposed architecture and its suitability for robust traffic sign detection in autonomous vehicles.