German Traffic Sign Recognition Benchmark Research Articles

Perturbation Augmentation for Adversarial Training with Diverse Attacks

Adversarial Training (AT) aims to alleviate the vulnerability of deep neural networks to adversarial perturbations. However, the AT techniques struggle to maintain the performance on natural samples while improving the deep model’s robustness. The absence of perturbation diversity in generated during the adversarial training degrades the generalizability of the robust models, causing overfitting to particular perturbations and a decrease in natural performance. This study proposes an adversarial training framework that augments adversarial directions from a single-step attack to address the trade-off between robustness and generalization. Inspired by feature scattering adversarial training, the proposed framework computes a principal adversarial direction with a single-step attack that finds a perturbation disrupting the inter-sample relationships in the mini-batch during adversarial training. The principal direction obtained at each iteration is augmented by sampling new adversarial directions within a region spanning 45 degrees from the principal adversarial direction. The proposed adversarial training approach does not require extra backpropagation steps in adversarial direction augmentation. Therefore, generalization of the robust model is improved without posing an additional burden on the feature scattering adversarial training. Experiments on CIFAR-10, CIFAR-100, SVHN, Tiny-ImageNet, and The German Traffic Sign Recognition Benchmark consistently improve the accuracy on adversarial with an almost pristine natural performance.

Traffic Sign Recognition using HIICNN Stack Ensemble Method

Abstract: This project introduces a novel deep convolutional neural network (CNN) model aimed at enhancing traffic sign recognition for autonomous vehicle technology and road safety. Employing a stacking ensemble approach, we combine multiple CNNs with diverse improvement techniques, including preprocessing for adverse weather conditions and shooting errors, and data augmentation to handle class imbalance. Through adjustments in learning rates during model training, we mitigate overfitting. Our ensemble model achieves an exceptional 99.4 percent test accuracy on the German Traffic Sign Recognition Benchmark (GTSRB) dataset, surpassing prior studies. We utilize gradient-weighted Class Activation Mapping (Grad-CAM) for model explain- ability and an evidential deep learning approach to quantify classification uncertainty. Our framework underscores the efficacy of combining preprocessing, ensemble learning, and transfer learning for superior perfor- mance and reliability in traffic sign recognition systems, contributing significantly to autonomous driving and road safety.

Traffic Sign Recognition and Classification using Deep Neural Networks

Deep Neural Networks such as Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs), have been applied in various applications, including machine vision and computer vision. One of the most notable emerging applications of deep neural networks is Advanced Driver Assistance System (ADAS). In ADAS model, the deep neural networks have outperformed the manual human performance in terms of traffic sign recognition and classification. to the problem of traffic sign recognition, with promising results. This study has proposed a novel Convolutional Neural Network (CNN) architecture for recognizing and classifying traffic signs. The German Traffic Sign Recognition Benchmark dataset is used for experimental analysis. The outcome shows that the proposed model performs comparatively better. The results also indicate that the implemented CNN model with Adam optimizer and RankMax activation performs well for recognizing and classifying traffic sign images.

Video frame feeding approach for validating the performance of an object detection model in real-world conditions

The challenge of evaluating deep learning-based object detection models in complex traffic scenarios, characterized by changing weather and lighting conditions, is addressed in this study. Real-world testing proves time and cost-intensive, leading to the proposal of a Video Frame Feeding (VFF) approach as a solution. The proposed Video Frame Feeding approach acts as a bridge between object detection models and simulated environments, enabling the generation of realistic scenarios. Leveraging the CarMaker (CM) tool to simulate realistic scenarios, the framework utilizes a virtual camera to capture the simulated environment and feed video frames to an object identification model. The VFF algorithm, with automated validation using simulated ground truth data, enhances detection accuracy to over 95% at 30 frames per second within 130 meters. Employing the You Only Look Once (YOLO) version 4 and the German Traffic Sign Recognition Benchmark dataset, the study assesses a traffic signboard identification model across various climatic conditions. Notably, the VFF algorithm improves accuracy by 2% to 5% in challenging scenarios like foggy days and nights. This innovative approach not only identifies object detection issues efficiently but also offers a versatile solution applicable to any object detection model, promising improved dataset quality and robustness for enhanced model performance.

Exploring Explainable Artificial Intelligence Techniques for Interpretable Neural Networks in Traffic Sign Recognition Systems

Traffic Sign Recognition (TSR) plays a vital role in intelligent transportation systems (ITS) to improve road safety and optimize traffic management. While existing TSR models perform well in challenging scenarios, their lack of transparency and interpretability hinders reliability, trustworthiness, validation, and bias identification. To address this issue, we propose a Convolutional Neural Network (CNN)-based model for TSR and evaluate its performance on three benchmark datasets: German Traffic Sign Recognition Benchmark (GTSRB), Indian Traffic Sign Dataset (ITSD), and Belgian Traffic Sign Dataset (BTSD). The proposed model achieves an accuracy of 98.85% on GTSRB, 94.73% on ITSD, and 92.69% on BTSD, outperforming several state-of-the-art frameworks, such as VGG19, VGG16, ResNet50V2, MobileNetV2, DenseNet121, DenseNet201, NASNetMobile, and EfficientNet, while also providing faster training and response times. We further enhance our model by incorporating explainable AI (XAI) techniques, specifically, Local Interpretable Model-Agnostic Explanations (LIME) and Gradient-weighted Class Activation Mapping (Grad-CAM), providing clear insights of the proposed model decision-making process. This integration allows the extension of our TSR model to various engineering domains, including autonomous vehicles, advanced driver assistance systems (ADAS), and smart traffic control systems. The practical implementation of our model ensures real-time, accurate recognition of traffic signs, thus optimizing traffic flow and minimizing accident risks.

Neural Network-based Multi-Class Traffic-Sign Classification with the German Traffic Sign Recognition Benchmark

Neural Network-based Multi-Class Traffic-Sign Classification with the German Traffic Sign Recognition Benchmark

Two-Stage Traffic Sign Classification System

Traffic Sign Recognition (TSR) has received widespread attention due to increased traffic accidents caused by a failure to recognize road traffic signs. Most Traffic Sign Recognition remains challenging due to the illumination of natural scenes, and the diversity in traffic signs [31]. To solve the problem, we proposed a two-stage classification system to overcome the above-mentioned challenges and classify the traffic sign boards efficiently. During the first phase, colour based histogram equalizations applied to eliminates the illumination factors and then traffic sign boards are classified into many shapes, which includes circular, triangular, hexagonal, and diamond traffic signs boards based on the modified HOG feature descriptors and various machine learning classifiers are compared [12]. In the second phase, circular and triangular traffic signs are further categorized into specific subclasses using circular and triangular signs trained Convolutional Neural Network (CNN) respectively to overcome intra-class variation among the similar sign board. Experimentation was conducted on German Traffic Sign Recognition Benchmark dataset (GTSRB) [30] and the results show 99.96% top-1 accuracy during the first stage of classification. In the second stage, there is accuracy of 99.40% for circular traffic signs and 97.50% for triangular traffic signs.

Zero-Shot Traffic Sign Recognition Based on Midlevel Feature Matching

Traffic sign recognition is a complex and challenging yet popular problem that can assist drivers on the road and reduce traffic accidents. Most existing methods for traffic sign recognition use convolutional neural networks (CNNs) and can achieve high recognition accuracy. However, these methods first require a large number of carefully crafted traffic sign datasets for the training process. Moreover, since traffic signs differ in each country and there is a variety of traffic signs, these methods need to be fine-tuned when recognizing new traffic sign categories. To address these issues, we propose a traffic sign matching method for zero-shot recognition. Our proposed method can perform traffic sign recognition without training data by directly matching the similarity of target and template traffic sign images. Our method uses the midlevel features of CNNs to obtain robust feature representations of traffic signs without additional training or fine-tuning. We discovered that midlevel features improve the accuracy of zero-shot traffic sign recognition. The proposed method achieves promising recognition results on the German Traffic Sign Recognition Benchmark open dataset and a real-world dataset taken from Sapporo City, Japan.

Real-time traffic monitoring system using IoT-aided robotics and deep learning techniques

The increasing number of vehicles on the road has made traffic regulations challenging to manage, particularly in large and crowded cities. Real-time traffic monitoring systems are one of the most important factors that enable efficient traffic flow and enhanced mobility. Therefore, vehicles and drivers have always needed reliable and accurate real-time traffic information. Recently, various solutions have been proposed to solve the problems and concerns in traffic situations. One alternative solution is vehicular cloud computing (VCC). Additionally, an IoT-aided robotic (IoRT) model has been developed with a modern architecture that integrates IoT sensor nodes and cameras to gather real-time traffic data. The main contributions of this research work are to implement two deep learning techniques based on modified LeNet-5 for real-time traffic sign recognition and the transfer learning-based Inception-V3 model for detecting and recognizing traffic lights. Furthermore, optimal distance was found between the ultrasonic sensors and the obstacles using ultrasonics’ waves time and speed to reduce road accidents. The data, which is collected by sensors and cameras, is processed using various image processing algorithms and it is sent to the cloud to be available for drivers and commuters through a mobile application. Test results indicate that the proposed models have significant improvements in terms of accuracy. The modified LeNet-5 achieved accuracy rates of 99.12% and 99.78% on the German Traffic Sign Recognition Benchmark (GTSRB) and extended GTSRB (EGTSRB) datasets, respectively, whereas the second model, trained on Laboratory for the Intelligent and Safe Automobiles (LISA) dataset, attained a 98.6% accuracy rate. Compared to the related traffic monitoring systems, the findings of this study outperform other works by 3.78% for traffic sign recognition and by 1.02% for traffic light detection and recognition.

Adversarial Deep Learning based Dampster–Shafer data fusion model for intelligent transportation system

Intelligent Transportation Systems (ITS) have revolutionized transportation by incorporating advanced technologies for efficient and safe mobility. However, these systems face challenges ensuring security and resilience against adversarial attacks. This research addresses these challenges and introduces a novel Dampster–Shafer data fusion-based Adversarial Deep Learning (DS-ADL) Model for ITS in fog cloud environments. Our proposed model focuses on three levels of adversarial attacks: original image level, feature level, and decision level. Adversarial examples are generated at each level to evaluate the system’s vulnerability comprehensively. To enhance the system’s capabilities, we leverage the power of several vital components. Firstly, we employ Dempster–Shafer-based Multimodal Sensor Fusion, enabling the fusion of information from multiple sensors for improved scene understanding. This fusion approach enhances the system’s perception and decision-making abilities. For feature extraction and classification, we utilize ResNet 101, a deep learning architecture known for its effectiveness in computer vision tasks. We introduced a novel Monomodal Multidimensional Gaussian Model (MMGM-DD) based Adversarial Detection approach to detect adversarial examples. This detection mechanism enhances the system’s ability to identify and mitigate adversarial attacks in real-time. Additionally, we incorporate the Defensive Distillation method for adversarial training, which trains the model to be robust against attacks by exposing it to adversarial examples during the training process. To evaluate the performance of our proposed model, we utilize two datasets: Google Speech Command version 0.01 and the German Traffic Sign Recognition Benchmark (GTSRB). Evaluation metrics include latency delay and computation time (fog–cloud), accuracy, MSE, loss, and F-score for attack detection and defense. The results and discussions demonstrate the effectiveness of our Dampster–Shafer data fusion-based Adversarial Deep Learning Model in enhancing the robustness and security of ITS in fog–cloud environments. The model’s ability to detect and defend against adversarial attacks while maintaining low-latency fog–cloud operations highlights its potential for real-world deployment in ITS.

Driver Assistant Using Deep Learning

Abstract: In the development of intelligent vehicles, accurate traffic sign detection and recognition are critical. This project proposes an improved algorithm for traffic sign detection and recognition, aiming to address limitations of traditional methods such as environmental sensitivity and poor real-time performance of deep learning-based approaches. The algorithm employs HSV-based spatial threshold segmentation for effective traffic sign detection based on shape features. Furthermore, the algorithm enhances the classical convolutional neural network model by utilizing Gabor kernel for initial convolution, incorporating batch normalization after pooling, and employing the Adam optimizer algorithm. The proposed algorithm is evaluated using the German Traffic Sign Recognition Benchmark, achieving a favorable prediction and accurate recognition rate of 99.75%, with an average processing time of 5.4ms per frame. Compared to other algorithms, the proposed approach demonstrates superior accuracy, real-time performance, generalization ability, and training efficiency. The findings of this project are expected to contribute to reducing accident rates and enhancing road traffic safety through improved traffic sign recognition in intelligent vehicles.

Switching network for mixing experts with application to traffic sign recognition

AbstractThe correct and robust recognition of traffic signs is indispensable to self-driving vehicles and driver-assistant systems. In this work, we propose and evaluate two network architectures for multi-expert decision systems that we test on a challenging Traffic Sign Recognition Benchmark dataset. The decision systems implement individual experts in the form of deep convolutional neural networks (CNNs). A gating network CNN acts as final decision unit and learns which individual expert CNNs are likely to contribute to an overall meaningful classification of a traffic sign. The gating network then selects the outputs of those individual expert CNNs to be fused to form the final decision. In this work we study the advantages and challenges of the proposed multi-expert architectures that in comparison to other network architectures allow for parallel training of individual experts with reduced datasets. Under the challenging conditions introduced by the benchmark dataset, the demonstrated multi-expert decision systems achieve a recognition performance that is superior to those of humans: with an accuracy of 99.10%, when training experts with the complete dataset and 98.94%, when individual experts are only trained with 36% of the training samples. Overall, our approach ranked fourth on the list of the applied approaches proposed for the German traffic sign Recognition Benchmark (GTSRB) dataset.

A Lightweight Convolutional Neural Network (CNN) Architecture for Traffic Sign Recognition in Urban Road Networks

Recognizing and classifying traffic signs is a challenging task that can significantly improve road safety. Deep neural networks have achieved impressive results in various applications, including object identification and automatic recognition of traffic signs. These deep neural network-based traffic sign recognition systems may have limitations in practical applications due to their computational requirements and resource consumption. To address this issue, this paper presents a lightweight neural network for traffic sign recognition that achieves high accuracy and precision with fewer trainable parameters. The proposed model is trained on the German Traffic Sign Recognition Benchmark (GTSRB) and Belgium Traffic Sign (BelgiumTS) datasets. Experimental results demonstrate that the proposed model has achieved 98.41% and 92.06% accuracy on GTSRB and BelgiumTS datasets, respectively, outperforming several state-of-the-art models such as GoogleNet, AlexNet, VGG16, VGG19, MobileNetv2, and ResNetv2. Furthermore, the proposed model outperformed these methods by margins ranging from 0.1 to 4.20 percentage point on the GTSRB dataset and by margins ranging from 9.33 to 33.18 percentage point on the BelgiumTS dataset.

Enhanced Traffic Sign Recognition with Ensemble Learning

With the growing trend in autonomous vehicles, accurate recognition of traffic signs has become crucial. This research focuses on the use of convolutional neural networks for traffic sign classification, specifically utilizing pre-trained models of ResNet50, DenseNet121, and VGG16. To enhance the accuracy and robustness of the model, the authors implement an ensemble learning technique with majority voting, to combine the predictions of multiple CNNs. The proposed approach was evaluated on three different traffic sign datasets: the German Traffic Sign Recognition Benchmark (GTSRB), the Belgium Traffic Sign Dataset (BTSD), and the Chinese Traffic Sign Database (TSRD). The results demonstrate the efficacy of the ensemble approach, with recognition rates of 98.84% on the GTSRB dataset, 98.33% on the BTSD dataset, and 94.55% on the TSRD dataset.

Accurate Image Multi-Class Classification Neural Network Model with Quantum Entanglement Approach.

Quantum machine learning (QML) has attracted significant research attention over the last decade. Multiple models have been developed to demonstrate the practical applications of the quantum properties. In this study, we first demonstrate that the previously proposed quanvolutional neural network (QuanvNN) using a randomly generated quantum circuit improves the image classification accuracy of a fully connected neural network against the Modified National Institute of Standards and Technology (MNIST) dataset and the Canadian Institute for Advanced Research 10 class (CIFAR-10) dataset from 92.0% to 93.0% and from 30.5% to 34.9%, respectively. We then propose a new model referred to as a Neural Network with Quantum Entanglement (NNQE) using a strongly entangled quantum circuit combined with Hadamard gates. The new model further improves the image classification accuracy of MNIST and CIFAR-10 to 93.8% and 36.0%, respectively. Unlike other QML methods, the proposed method does not require optimization of the parameters inside the quantum circuits; hence, it requires only limited use of the quantum circuit. Given the small number of qubits and relatively shallow depth of the proposed quantum circuit, the proposed method is well suited for implementation in noisy intermediate-scale quantum computers. While promising results were obtained by the proposed method when applied to the MNIST and CIFAR-10 datasets, a test against a more complicated German Traffic Sign Recognition Benchmark (GTSRB) dataset degraded the image classification accuracy from 82.2% to 73.4%. The exact causes of the performance improvement and degradation are currently an open question, prompting further research on the understanding and design of suitable quantum circuits for image classification neural networks for colored and complex data.

Traffic Sign Identification Using Modified LeNet-5 CNN

Abstract: In 21st century, it is obvious that automobiles have become the standard mode of transportation. With the spread of their usage, road traffic becomes more intricate; hence road safety must be promising enough avoiding accidents. Drivers may sometimes overlook the sign boards along the roads which might lead to undesirable casualties. This could be helped by a system that could assist the driver in keeping an eye on the traffic sign boards on the way. It can be achieved using CNN i.e. Convolutional Neural Networks. CNNs are preferred when the applications are concerned with computer vision or image recognition. A deep learning CNN model is built in the LeNet-5 architecture with some modifications and is trained with traffic signs images from GTSRB dataset which stands for German traffic sign recognition benchmark with the help of OpenCV, Tensorflow, Keras and other libraries. The trained model is tested with test data and is observed to be recognizing any of the traffic signs which were already learnt by the network. Vehicle cameras come handy to capture the real-time sign boards. Implementation of this supervised learning model in real-time would aid road safety through vehicle control.

Robust transformer with locality inductive bias and feature normalization

Vision transformers have been demonstrated to yield state-of-the-art results on a variety of computer vision tasks using attention-based networks. However, research works in transformers mostly do not investigate robustness/accuracy trade-off, and they still struggle to handle adversarial perturbations. In this paper, we explore the robustness of vision transformers against adversarial perturbations and try to enhance their robustness/accuracy trade-off in white box attack settings. To this end, we propose Locality iN Locality (LNL) transformer model. We prove that the locality introduction to LNL contributes to the robustness performance since it aggregates local information such as lines, edges, shapes, and even objects. In addition, to further improve the robustness performance, we encourage LNL to extract training signal from the moments (a.k.a., mean and standard deviation) and the normalized features. We validate the effectiveness and generality of LNL by achieving state-of-the-art results in terms of accuracy and robustness metrics on German Traffic Sign Recognition Benchmark (GTSRB) and Canadian Institute for Advanced Research (CIFAR-10). More specifically, for traffic sign classification, the proposed LNL yields gains of 1.1% and 35% in terms of clean and robustness accuracy compared to the state-of-the-art studies.

Wave interference network with a wave function for traffic sign recognition.

In this paper, we successfully combine convolution with a wave function to build an effective and efficient classifier for traffic signs, named the wave interference network (WiNet). In the WiNet, the feature map extracted by the convolutional filters is refined into many entities from an input image. Each entity is represented as a wave. We utilize Euler's formula to unfold the wave function. Based on the wave-like information representation, the model modulates the relationship between the entities and the fixed weights of convolution adaptively. Experiment results on the Chinese Traffic Sign Recognition Database (CTSRD) and the German Traffic Sign Recognition Benchmark (GTSRB) demonstrate that the performance of the presented model is better than some other models, such as ResMLP, ResNet50, PVT and ViT in the following aspects: 1) WiNet obtains the best accuracy rate with 99.80% on the CTSRD and recognizes all images exactly on the GTSRB; 2) WiNet gains better robustness on the dataset with different noises compared with other models; 3) WiNet has a good generalization on different datasets.

SeVuc: A study on the Security Vulnerabilities of Capsule Networks against adversarial attacks

Capsule Networks (CapsNets) preserve the hierarchical spatial relationships between objects, and thereby bear the potential to surpass the performance of traditional Convolutional Neural Networks (CNNs) in performing tasks like image classification. This makes CapsNets suitable for the smart cyber–physical systems (CPS), where a large amount of training data may not be available. A large body of work has explored adversarial examples for CNNs, but their effectiveness on CapsNets has not yet been studied systematically. In our work, we perform an analysis to study the vulnerabilities in CapsNets to adversarial attacks. These perturbations, added to the test inputs, are small and imperceptible to humans, but can fool the network to mispredict. We propose a greedy algorithm to automatically generate imperceptible adversarial examples in a black-box attack scenario. We show that this kind of attacks, when applied to the German Traffic Sign Recognition Benchmark and CIFAR10 datasets, mislead CapsNets in making a correct classification, which can be catastrophic for smart CPS, like autonomous vehicles. Moreover, we apply the same kind of adversarial attacks to a 5-layer CNN (LeNet), to a 9-layer CNN (VGGNet), and to a 20-layer CNN (ResNet), and analyze the outcome, compared to the CapsNets, to study their different behaviors under the adversarial attacks.

Traffic sign classification using CNN and detection using faster-RCNN and YOLOV4

Autonomous driving cars are becoming popular everywhere and the need for a robust traffic sign recognition system that ensures safety by recognizing traffic signs accurately and fast is increasing. In this paper, we build a CNN that can classify 43 different traffic signs from the German Traffic Sign Recognition benchmark dataset. The dataset is made up of 39,186 images for training and 12,630 for testing. Our CNN for classification is light and reached an accuracy of 99.20% with only 0.8 M parameters. It is tested also under severe conditions to prove its generalization ability. We also used Faster R–CNN and YOLOv4 networks to implement a recognition system for traffic signs. The German Traffic Sign Detection benchmark dataset was used. Faster R–CNN obtained a mean average precision (mAP) of 43.26% at 6 Frames Per Second (FPS), which is not suitable for real-time application. YOLOv4 achieved an mAP of 59.88% at 35 FPS, which is the preferred model for real-time traffic sign detection. These mAPs are obtained using Intersect Over Union of 50%. A comparative analysis is also presented between these models.