Traffic Light Recognition Research Articles

Current Status and Issues of Traffic Light Recognition Technology in Autonomous Driving System

Current Status and Issues of Traffic Light Recognition Technology in Autonomous Driving System

내장 그래픽 칩 기반 머신 러닝과 딥러닝의 알고리즘 융합을 활용한 실시간 교통 신호인지

The recognition of traffic light in unmanned autonomous driving is one of the key functions. In general, to achieve this goal, deep learning and image processing technology are mainly used in autonomous driving industry. Both technologies have high recognition rates alone, but misrecognition can occur because of interference of the environment. misrecognitions are considered important problem because it may lead to big accidents. In this paper, we suggest using machine learning based on LabVIEW and deep running grounded in Python to increase recognition rate at intersection. By using both technologies, we can minimize the impact of the environment on traffic light recognition and reduce misrecognition.

Implementation of Autonomous Driving on RC-CAR with Raspberry PI and AI Server

A lot of research is being done on autonomous driving vehicles or robots that recognize objects and drive themselves without human intervention. In order to develop autonomous driving technology, there is a fundamental difficulty in securing expensive real cars equipped with various sensors. In this paper, an autonomous driving system development platform was developed using an inexpensive RC-Car, and a test system that can test various algorithms related to autonomous driving was introduced. In the system developed in this study, the single board computer Raspberry PI was mounted on the RC-Car to control the car, and the autonomous driving-related algorithms were implemented in a separate AI server, and they communicated with the message-based ROS protocol. In addition, those who want to develop an autonomous driving system can easily attach desired sensors to the RC-Car, increasing scalability. In this paper, almost all algorithms related to autonomous driving have been implemented. A simple autonomous driving RC-Car system was actually implemented and operation was verified by designing and implementing algorithms such as lane recognition, driving along the lane, obstacle detection and stopping, traffic light recognition, driving between smooth and sharp curves, and autonomous parking. In sharp curves, the angle of the lane was tracked in a short period to prevent the vehicle from crossing the lane. In addition, we developed an Android app that can manually control the car and monitor the video from the camera in time. This study presented and solved various difficulties that could not be known by developing an autonomous driving algorithm using simulators.

IBGS: A Wearable Smart System to Assist Visually Challenged

Traditional blind guide devices are large and high cost. In this paper, a kind of intelligent blind guide system (IBGS) is introduced. GD32 is used as the main control chip, it cooperates with various functional modules to realize traffic light recognition, obstacle avoidance, payment, and navigation functions on the basis of speech recognition. At the same time, IBGS uses WIFI instead of Bluetooth to get rid of the dependence on smart phones. In addition, a cloud database is built, and the Internet of Things technology is used to realize the information interconnection between the IBGS, the database, and the guardian’s mobile terminal. In order to better realize the function of speech recognition, this paper proposes a Conv-Transformer Transducer (ConvT-T) speech recognition framework based on Weak-Attention Suppression (WAS), which improves the efficiency of multi-head attention through WAS. The proposed method achieves word error rate (WER) of 3.2% in test-clean and 7.9% in test-other on the LibriSpeech ASR corpus with only 73M parameters, which reduces the WER by 0.3% and 0.4% compared to ConvT-T respectively, indicating that WAS can effectively play the role of suppressing non-critical attention on the ConvT-T framework. At the same time, the IBGS has been tested in a comprehensive application scenario. In outdoor traffic light recognition, speech recognition, and obstacle avoidance tests, the accuracy rates of 92.33%, 90.33%, and 96.67% were achieved respectively. The results show that the IBGS can effectively deal with various challenges encountered by the blind in their daily outdoor walking.

Design of Smart Blind Crutch System Based on Arduino

The blind used traditional blind crutch to walk on the blind track in the past. With the increase of the blind and the increase of the complexity of the situation of the road, the simple blind crutch cannot give enough security to the blind. The blind crutch needs improvement in active obstacle avoidance and night safety to meet the demand. In this paper, a smart blind crutch is designed. The ultrasonic ranging sensor, laser ranging sensor, color sensor, photosensitive resistance, buzzer and vibrator are linked together with Arduino microcontrollers to realize obstacle avoidance reminder, blind lane and traffic light recognition, glowing when the light is dim, which are integrated into blind crutch to facilitate the travel of the blind. The tests show that the blind crutch is accurate in reminding obstacles in front, accurate in recognizing blind lanes and ground traffic lights, and sensitive in luminous induction at night. Compared to the traditional blind crutch, the travel safety of the blind is more guaranteed especially at night This smart blind crutch also provides referenced for smart blind crutch.

Construction of Driving image dataset under Adverse conditions for Traffic light recognition in Urban Automated Driving

Construction of Driving image dataset under Adverse conditions for Traffic light recognition in Urban Automated Driving

Traffic Light Recognition for Autonomous Driving Vehicle: Using Mono Camera and ITS

Traffic Light Recognition for Autonomous Driving Vehicle: Using Mono Camera and ITS

Traffic Lights Detection and Recognition Method Based on the Improved YOLOv4 Algorithm.

For facing of the problems caused by the YOLOv4 algorithm’s insensitivity to small objects and low detection precision in traffic light detection and recognition, the Improved YOLOv4 algorithm is investigated in the paper using the shallow feature enhancement mechanism and the bounding box uncertainty prediction mechanism. The shallow feature enhancement mechanism is used to extract features from the network and improve the network’s ability to locate small objects and color resolution by merging two shallow features at different stages with the high-level semantic features obtained after two rounds of upsampling. Uncertainty is introduced in the bounding box prediction mechanism to improve the reliability of the prediction of the bounding box by modeling the output coordinates of the prediction bounding box and adding the Gaussian model to calculate the uncertainty of the coordinate information. The LISA traffic light data set is used to perform detection and recognition experiments separately. The Improved YOLOv4 algorithm is shown to have a high effectiveness in enhancing the detection and recognition precision of traffic lights. In the detection experiment, the area under the PR curve value of the Improved YOLOv4 algorithm is found to be 97.58%, which represents an increase of 7.09% in comparison to the 90.49% score gained in the Vision for Intelligent Vehicles and Applications Challenge Competition. In the recognition experiment, the mean average precision of the Improved YOLOv4 algorithm is 82.15%, which is 2.86% higher than that of the original YOLOv4 algorithm. The Improved YOLOv4 algorithm shows remarkable advantages as a robust and practical method for use in the real-time detection and recognition of traffic signal lights.

Traffic Light and Back-light Recognition using Deep Learning and Image Processing with Raspberry Pi

Traffic light detection and back-light recognition are essential research topics in the area of intelligent vehicles because they avoid vehicle collision and provide driver safety. Improved detection and semantic clarity may aid in the prevention of traffic accidents by self-driving cars at crowded junctions, thus improving overall driving safety. Complex traffic situations, on the other hand, make it more difficult for algorithms to identify and recognize objects. The latest state-of-the-art algorithms based on Deep Learning and Computer Vision are successfully addressing the majority of real-time problems for autonomous driving, such as detecting traffic signals, traffic signs, and pedestrians. We propose a combination of deep learning and image processing methods while using the MobileNetSSD (deep neural network architecture) model with transfer learning for real-time detection and identification of traffic lights and back-light. This inference model is obtained from frameworks such as Tensor-Flow and Tensor-Flow Lite which is trained on the COCO data. This study investigates the feasibility of executing object detection on the Raspberry Pi 3B+, a widely used embedded computing board. The algorithm’s performance is measured in terms of frames per second (FPS), accuracy, and inference time.

Adversarial Attacks and Defense Technologies on Autonomous Vehicles: A Review

Abstract In recent years, various domains have been influenced by the rapid growth of machine learning. Autonomous driving is an area that has tremendously developed in parallel with the advancement of machine learning. In autonomous vehicles, various machine learning components are used such as traffic lights recognition, traffic sign recognition, limiting speed and pathfinding. For most of these components, computer vision technologies with deep learning such as object detection, semantic segmentation and image classification are used. However, these machine learning models are vulnerable to targeted tensor perturbations called adversarial attacks, which limit the performance of the applications. Therefore, implementing defense models against adversarial attacks has become an increasingly critical research area. The paper aims at summarising the latest adversarial attacks and defense models introduced in the field of autonomous driving with machine learning technologies up until mid-2021.

Investigating Visual Recognition of Color and Safe Driving in Color-Weak Drivers

This study examined the visual characteristics of drivers with color weakness to improve their safety while driving. Significantly affected by the traffic environment, drivers with color weakness are not able to recognize traffic lights rapidly and accurately, which endangers traffic safety. In the first part of the research, through a static visual recognition test of color vision using the orthogonal method, the study explored the influence of light intensity, visual recognition distance, and color weakness type on the perception of traffic light colors by participants with color weakness. In the second part, a dynamic visual recognition test of color vision was conducted through simulating the driving environment of urban roads. Eye movement indexes between participants with color weakness and those with normal vision were analyzed by means of different vehicle speeds and time periods. The results indicated that the type of color weakness was the dominant factor affecting visual recognition of traffic lights. The eye movements of participants with deuteranomaly were close to those of people with normal vision, whereas the eye movement index of those with protanomaly and dyserythrochloropsia were significantly different. Distraction, slower responses, and higher color recognition error rates for the traffic lights were major characteristics—all representing risks that increase at night. To reduce the probability of road traffic injury, the driving safety of people with color weakness should be addressed.

Traffic signal light detection and recognition based on canny operator

In this paper, five classical edge detection operators are compared, and then a traffic signal light detection and recognition scheme that can be used for intelligent connected vehicles is implemented. Firstly, the image to be processed is obtained by detecting the traffic signal light through the vision sensor. The image is preprocessed: the color space of the image is converted from RGB space to HSV space. Through the grayscale, histogram equalization, image binarization processing, using the morphological closure operation, the five operators are compared in noise sensitivity, positioning accuracy and signal-to-noise ratio, the Canny edge detection operator is selected for image edge detection, and the target recognition area is obtained. Finally, using the histogram drawn, the number of red, green and yellow pixel points in the histogram can be clearly counted, and the color with the largest number of pixel points can be identified as the color of the identified traffic signal light, and the identification of the traffic signal light can be completed. The actual pictures are simulated on the MATLAB, which verifies the feasibility of the proposed method of traffic signal light recognition method based on Canny operator in this paper, which can correctly identify the color of the traffic signal light.

Distant Traffic Light Recognition Using Semantic Segmentation

Traffic light recognition is an important task for automatic driving support systems. Conventional traffic light recognition techniques are categorized into model-based methods, which frequently suffer from environmental changes such as sunlight, and machine-learning-based methods, which have difficulty detecting distant and occluded traffic lights because they fail to represent features efficiently. In this work, we propose a method for recognizing distant traffic lights by utilizing a semantic segmentation for extracting traffic light regions from images and a convolutional neural network (CNN) for classifying the state of the extracted traffic lights. Since semantic segmentation classifies objects pixel by pixel in consideration of the surrounding information, it can successfully detect distant and occluded traffic lights. Experimental results show that the proposed semantic segmentation improves the detection accuracy for distant traffic lights and confirms the accuracy improvement of 12.8 % over the detection accuracy by object detection. In addition, our CNN-based classifier was able to identify the traffic light status more than 30 % more accurately than the color thresholding classification.

Blind guidance system based on image recognition and convolutional neural network

This paper applies straight-line driving technology and pothole detection technology in the field of unmanned driving and edge detection in image recognition to the blind guide system. Researched and produced an intelligent, popular, safe and reliable intelligent blind guide system. This blind guide system uses Python and OpenCV as the development environment under the Linux operating system to realize image recognition and integrate functions on the Jetson nano development board. Through the simulation and analysis of scenes such as the life and travel of the blind, the designed blind guidance system mainly realizes the following functions: image recognition, traffic light recognition, zebra crossing detection, road pothole detection, voice interaction, and assisting straight-line walking.

An improved traffic lights recognition algorithm for autonomous driving in complex scenarios

Image recognition is susceptible to interference from the external environment. It is challenging to accurately and reliably recognize traffic lights in all-time and all-weather conditions. This article proposed an improved vision-based traffic lights recognition algorithm for autonomous driving, integrating deep learning and multi-sensor data fusion assist (MSDA). We introduce a method to obtain the best size of the region of interest (ROI) dynamically, including four aspects. First, based on multi-sensor data (RTK BDS/GPS, IMU, camera, and LiDAR) acquired in a normal environment, we generated a prior map that contained sufficient traffic lights information. And then, by analyzing the relationship between the error of the sensors and the optimal size of ROI, the adaptively dynamic adjustment (ADA) model was built. Furthermore, according to the multi-sensor data fusion positioning and ADA model, the optimal ROI can be obtained to predict the location of traffic lights. Finally, YOLOv4 is employed to extract and identify the image features. We evaluated our algorithm using a public data set and actual city road test at night. The experimental results demonstrate that the proposed algorithm has a relatively high accuracy rate in complex scenarios and can promote the engineering application of autonomous driving technology.

딥러닝 기반 신호등 인식 시스템에 대한 특수 신호등의 영향 평가

신호등 인식 기능은 자율주행 차량 및 첨단 운전자 보조 시스템(ADAS: Advanced Driver Assistance Systems)의 필수 구성요소이다. 교통 신호등은 국제적으로 통용되는 일반적인 체계가 존재하지만, 특정 국가에서만 사용되는 특수 신호등이 존재하므로 이에 대한 고려가 필요하다. 우리나라의 경우 버스 전용차로를 위한 버스 전용 신호등을 사용하고 있다. 본 연구에서는 YOLOv3 계열의 딥러닝 모델을 이용해 신호등 인식 기능을 구현할 때 버스 전용 신호등이 신호등 인식률에 미치는 영향을 분석한다. 즉, 버스전용 신호등을 포함하는 학습 데이터셋을 만들어 이를 이용하여 학습을 수행한 신호등 인식 시스템의 인식률을 살펴보고, 버스 전용 신호등이 상대적으로 낮은 인식률을 보이는 요인을 분석한다.

Simultaneous detection and tracking using deep learning and integrated channel feature for ambint traffic light recognition

Perceiving the information about ambient traffic lights is an inevitable task for autonomous vehicles. To deal with the issue, this work develops an accurate and fast traffic light recognition strategy for autonomous vehicles by an onboard camera. In this paper, deep learning based detection and object tracking is synthesized to determine the position and color of traffic lights. First, the mechanism of simultaneous detection and tracking is founded, wherein the video reading module, convolutional neural network (CNN) module, integrated channel feature tracking (ICFT) module are run simultaneously. Then, the respective modules of detection and tracking are introduced. CNN model is designed and trained to obtain the position of traffic lights utilized as initial information for tracking. ICFT is applied to continually track the traffic light targets and determine the light color. Finally, the effectiveness of the presented method is validated via comparing with the state of art. Experiments results indicate that the proposed technique can improve the accuracy and speed of recognition. Our contributions are: (1) Establish a mechanism for simultaneous detection and tracking of traffic lights; (2) Carefully design the CNN architecture and ICFT features; (3)The precision and recall rates on traffic lights recognition reached 0.962 and 0.909, respectively, and the recognition speed reached 21.4FPS (GPU: Nvidia Titan Xp).

Machine Learning based Traffic Light Detection and IR Sensor based Proximity Sensing for Autonomous Cars

In this fast-moving digital world autonomous cars will be a norm in near future. One of the most critical features of any autonomous car is traffic light detection and recognition. The problem related to outdoor perception is a key issue faced by driver-assisted and autonomous cars, as the machine, unless taught, cannot detect traffic lights, road signs, obstacles, etc. in the direction of motion. Autonomous cars need to be capable of detecting traffic signals and recognizing their current status whereas Human beings can easily identify the relevant traffic signals. Traffic light detection and recognition needs to be integrated with the CPU of autonomous cars (which governs the car) leading to resolution of the traffic signal alignment issue. Additionally, solution is required for the detection of obstacles. Open CV2 module of machine learning works well for resolving traffic related problems. Furthermore, it has leveraged state-of-the-art detectors to recognize traffic lights from 2D camera images. Tesla is using the traffic light detection and recognition technology in their autonomous cars. Google has proposed prior map of the location of traffic signals (recognition of traffic lights) which makes it possible for autonomous cars to performance well in the night too. Another integral feature for an autonomous car is proximity sensing, it’s basically done for sensing any obstacle near the car. The car should act accordingly and well within time to avoid a collision when an obstacle is sensed in the direction it is moving. Proximity sensing is done using cameras, RADAR (Radio Detection And Ranging) and LiDAR sensors (Light Detection And Ranging); IR sensor can be used as a redundancy in case one of the above fails. Tesla, Google and BMW have come up with a software that uses cutting edge technology such as RADAR, high resolution 3-D cameras, ultrasonic sensors and LiDAR sensors. This paper talks about the traffic light detection robustly using opencv2 module, IR sensors for proximity sensing (nearby traffic, obstacles, pedestrians) interfaced with an Arduino Uno board and a computer through which the behaviour of the autonomous car can be monitored.

Predicting driver behaviour at intersections based on driver gaze and traffic light recognition

This work introduces and evaluates a model for predicting driver behaviour, namely turns or proceeding straight, at traffic light intersections from driver three-dimensional gaze data and traffic light recognition. Based on vehicular data, this work relates the traffic light position, the driver's gaze, head movement, and distance from the centre of the traffic light to build a model of driver behaviour. The model can be used to predict the expected driver manoeuvre 3 to 4 s prior to arrival at the intersection. As part of this study, a framework for driving scene understanding based on driver gaze is presented. The outcomes of this study indicate that this deep learning framework for measuring, accumulating and validating different driving actions may be useful in developing models for predicting driver intent before intersections and perhaps in other key-driving situations. Such models are an essential part of advanced driving assistance systems that help drivers in the execution of manoeuvres.

Algorithms for Railway Embedded Control Devices for Safety Manoeuvres

Abstract This study is dedicated to solve manoeuvres making task while working on the station with no marshalling hump. It is part of the project aimed at the development of intelligent safety and optimal control systems of autonomous electric vehicles and transport in general. The main manoeuvres safety depends on the lack of items and other objects on the rails as well as on the position of turnouts. In most cases rails, occupied with other wagons, as well as the wrong position of turnouts are marked with prohibiting red or blue signals of the traffic light. The authors propose an algorithm for the traffic light recognition by using a convolutional neural network (CNN) and traffic light indicator recognition. However, the situation when the locomotive needs to drive on the rails occupied with other wagons, for example, during the manoeuvres on the railway station can also appear. For this purpose, the authors have developed a CNN algorithm for the wagon recognition on the rails.