Obstacle Detection System Research Articles

Artificial intelligence-aided intelligent obstacle and trespasser detection based on locomotive-mounted forward-facing camera data

Most fatalities in the railroad industry are trespasser deaths. Additional obstacles such as fallen trees, construction materials, and other debris also pose a danger to railroad operations. Understanding where, when, and how these events occur can help railroads develop trespasser mitigation strategies and improve rail safety. However, obtaining information from forward-facing footage requires immense manual effort. We developed a forward-facing trespassing and obstacle detection system that utilizes artificial intelligence to recognize and understand unsafe events and logs them for later review and analysis. The system dynamically identifies regions of interest, trespassers, and obstacles with a customized detection methodology which uses a semantic segmentation model called DeepLab and an object detection model called YOLOv5. These models were trained by a dataset containing over 10,000 images from various sources, including open-source datasets and videos provided by industry collaborators. The novelties of this research are threefold: the development of an algorithm capable of detecting the railroad’s restricted area around the track area, optimizing the trade-off between accuracy and latency to achieve real-time performance, and proposing a universal obstacle detection algorithm. The final system was able to analyze 400-pixel x 400-pixel videos at a rate of 14.5 frames per second (FPS) in an edge-computing device which identified trespassers and obstacles with more than 92% accuracy. The system can adaptively detect both obstacles and trespassers in the train’s path, as well as the railroad property area surrounding the tracks. This research offers the railroad industry tools which can provide precise trespasser and obstacle information to improve railroad safety and operations.

Verification of Applicability of 90 GHz Band Millimeter‐Wave for Obstacle Detection to Railway

AbstractObstacles on railway tracks are a factor hindering safe and stable operation. For this reason, obstacle detection systems have been introduced at level crossings, slopes and under platforms. However, they have been introduced only in limited sections for limited applications. Therefore, the authors considered to introduce an obstacle monitoring system that combines 90 GHz band millimeter‐wave and Radio over Fiber (RoF) technology, as an obstacle monitoring system over tracks to monitor the entire line. Since the millimeter‐wave band has a wide bandwidth, high precision detection is possible when they are used in radar. In addition, since radar control units can be grouped in one place by introducing the RoF technology, we can build a system at a low cost. In order to study the feasibility of introducing a new obstacles monitoring system combining 90 GHz band millimeter‐wave and RoF technology in railway environment, a field test using prototype equipment was conducted on a railway track. As a result, it was confirmed that the prototype radar system could detect humans and non‐metallic objects entering a track at a distance of about 200 m from the location of the prototype system. These results shows that the proposed system has the potential to be introduced as a new railway obstacle monitoring system using 90 GHz band millimeter‐waves and RoF technology. © 2023 The Authors. IEEJ Transactions on Electrical and Electronic Engineering published by Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Design and Test of Obstacle Detection and Harvester Pre-Collision System Based on 2D Lidar

Aiming at the need to prevent agricultural machinery from colliding with obstacles in the operation of unmanned agricultural machinery, an obstacle detection algorithm using 2D lidar was proposed, and a pre-collision system was designed using this algorithm, which was tested on a harvester. The method uses the differences between lidar data frames to calculate the collision times between the farm machinery and the obstacles. The algorithm consists of the following steps: pre-processing to determine the region of interest, median filtering, and DBSCAN (density-based spatial clustering of applications with noise) to identify the obstacle and calculate of the collision time according to the 6σ principle. Based on this algorithm, a pre-collision system was developed and integrated with agricultural machinery navigation software. The harvester was refitted electronically, and the system was tested on a harvester. The results showed that the system had an average accuracy rate of 96.67% and an average recall rate of 97.14% for being able to stop safely for obstacles in the area of interest, with a summed average of 97% for both the accuracy and recall rates. The system can be used for an emergency stop when encountering obstacles in the automatic driving of agricultural machinery and provides a basis for the unmanned driving of agricultural machinery in more complex scenarios.

Automatic Obstacle Detection Method for the Train Based on Deep Learning

Automatic obstacle detection is of great significance for improving the safety of train operation. However, the existing autonomous operation of trains mainly depends on the signaling control system and lacks the extra equipment to perceive the environment. To further enhance the efficiency and safety of the widely deployed fully automatic operation (FAO) systems of the train, this study proposes an intelligent obstacle detection system based on deep learning. It collects perceptual information from industrial cameras and light detection and ranging (LiDAR), and mainly implements the functionality including rail region detection, obstacle detection, and visual–LiDAR fusion. Specifically, the first two parts adopt deep convolutional neural network (CNN) algorithms for semantic segmentation and object detection to pixel-wisely identify the rail track area ahead and detect the potential obstacles on the rail track, respectively. The visual–LiDAR fusion part integrates the visual data with the LiDAR data to achieve environmental perception for all weather conditions. It can also determine the geometric relationship between the rail track and obstacles to decide whether to trigger a warning alarm. Experimental results show that the system proposed in this study has strong performance and robustness. The system perception rate (precision) is 99.994% and the recall rate reaches 100%. The system, applied to the metro Hong Kong Tsuen Wan line, effectively improves the safety of urban rail train operation.

Obstacle Detection Algorithm for Railroad-Road Crossings Based on Video Stream Analysis

The paper presents a computationally efficient method developed to automatically detect obstacles that appear at railroad-road crossings. The proposed algorithm works maintenance-free and automatically detects obstacles that appear and stay or are left at the crossing. These obstacles can pose a threat for the trains causing even a trail derailment. When a detected obstacle is a person it is crucial to generate a warning in order to react and safely remove the person before the train arrives. The proposed algorithm is based on live video stream analysis. The standard image processing methods were used in order to detect obstacles in the acquired images. However, several techniques were proposed to obtain the desired results dealing with image vibrations, heavy rain and snow. Such robustness was obtained by building and maintaining a fuzzy background frame and appropriate filtering. The obstacle detection system was implemented and tested. The results confirmed its correct operation and usefulness in real-world conditions.

An ultrasonic sensor driven obstacle detection and localisation system in 3D space for visually impaired persons

An ultrasonic sensor driven obstacle detection and localisation system in 3D space for visually impaired persons

Obstacle detection technique to solve poor texture appearance of the obstacle by categorising image's region using cues from expansion of feature points for small UAV

Achieving a reliable obstacle detection system for small unmanned aerial vehicle (UAV) is very challenging due to its size and weight constraints. Prior works tend to employ the vision sensor as main detection sensor but resulting to high dependency on texture appearance while not having distance sensing capabilities. Besides, most of wide spectrum range sensors are heavy and expensive. The contribution of this work is on different based-sensor integration technique to increase reliability of detection. A method was developed to create trusted avoidance path by categorising the region in environment into two regions, which are the obstacle region and free region. Cues from expansion of the features points are used to extract the depth information of the environment and classify the region in the image frame. The results show that the proposed system able to handle multiple obstacle and create safe path regardless of the texture and size of the obstacle.

Design of Sub-Systems for GPS-Guided Autonomous Delivery Robot System

The use of autonomous robots for delivery services is a new potential goldmine. Furthermore, since the e-commerce and delivery industry are growing at a rapid rate, it is recommended that a system that could handle the high-volume traffic as well serve as a new customer attraction, be implemented. Therefore, this work aims to develop the Autonomous Delivery Robot System (ADRS) that could be utilized for delivery services. The ADRS uses the Arduino microcontroller to run a program. The developed system consists of three main components (sub-systems), namely, mobile robot, cloud server and mobile application. The mobile robot is equipped with a tracking and an obstacle detection system to maneuver it autonomously. The ultrasonic sensors are used for obstacle detection, coupled with a Global Positioning System (GPS) to enable it deliver packages to customers in a typical outdoor environment. The ADRS ensures a human-contactless and secure delivery by using a password protected container to carry the delivery packages. Only the customer can unlock the container using the provided password via mobile application. The mobile application with customized map based on Google Maps is created to provide the user with an interactive interface. A camera is also attached on the robot for real-time monitoring purpose. The performance of the ADRS has been evaluated to determine the functionality of the developed system.

Fast Obstacle Detection System for UAS Based on Complementary Use of Radar and Stereoscopic Camera

Autonomous unmanned aerial systems (UAS) are having an increasing impact in the scientific community. One of the most challenging problems in this research area is the design of robust real-time obstacle detection and avoidance systems. In the automotive field, applications of obstacle detection systems combining radar and vision sensors are common and widely documented. However, these technologies are not currently employed in the UAS field due to the major complexity of the flight scenario, especially in urban environments. In this paper, a real-time obstacle-detection system based on the use of a 77 GHz radar and a stereoscopic camera is proposed for use in small UASs. The resulting system is capable of detecting obstacles in a broad spectrum of environmental conditions. In particular, the vision system guarantees a high resolution for short distances, while the radar has a lower resolution but can cover greater distances, being insensitive to poor lighting conditions. The developed hardware and software architecture and the related obstacle-detection algorithm are illustrated within the European project AURORA. Experimental results carried out employing a small UAS show the effectiveness of the obstacle detection system and of a simple avoidance strategy during several autonomous missions on a test site.

A low cost collision avoidance system based on a ToF camera for SLAM approaches

Indoor positioning is a problem that has not yet been solved efficiently and accurately. In outdoors the most effective solution is the Global Position System (GPS), but it cannot be used indoors due to the weakening of the signal, so other solutions have been studied. These approaches could be applied to define a map for the guidance of blind people, tourism or navigation for autonomous robots. In this paper, the study, design, implementation and evaluation of a robust obstacle detection and mapping system is proposed. Thus, it can be used to alert of near objects presence and avoid possible collisions in an indoor navigation. The system is based on a Time-of-Flight (ToF) camera and a Single Board Computer (SBC) like Raspberry PI or NVIDIA Jetson Nano. In order to evaluate the system several real experiments were carried out. This kind of system can be integrated on a wheelchair and help the handicapped person to move indoors or take data from an indoor environment and recreate it in a 2D or 3D images.

Review of Obstacle Detection Systems for Collision Avoidance of Autonomous Underwater Vehicles Tested in a Real Environment

The high efficiency of obstacle detection system (ODS) is essential to obtain the high performance of autonomous underwater vehicles (AUVs) carrying out a mission in a complex underwater environment. Based on the previous literature analysis, that include path planning and collision avoidance algorithms, the solutions which operation was confirmed by tests in a real-world environment were selected for this paper consideration. These studies were subjected to a deeper analysis assessing the effectiveness of the obstacle detection algorithms. The analysis shows that over the years, ODSs being improved and provide greater detection accuracy that results in better AUV response time. Almost all analysed methods are based on the conventional approach to obstacle detection. In the future, even better ODSs parameters could be achieved by using artificial intelligence (AI) methods.

Towards Integrated Digital-Twins: An Application Framework for Autonomous Maritime Surface Vessel Development

The use of digital twins for the development of Autonomous Maritime Surface Vessels (AMSVs) has enormous potential to resolve the increasing need for water-based navigation and safety at the sea. Aiming at the problem of lack of broad and integrated digital twin implementations with live data along with the absence of a digital twin-driven framework for AMSV design and development, an application framework for the development of a fully autonomous vessel using an integrated digital twin in a 3D simulation environment has been presented. Our framework has 4 layers which ensure that simulation and real-world vessel and the environment are as close as possible. Åboat, an in-house, experimental research platform for maritime automation and autonomous surface vessel applications, equipped with two trolling electric motors, cameras, LiDARs, IMU and GPS has been used as the case study to provide a proof of concept. Åboat, its sensors, and the environment have been replicated in a commercial, 3D simulation environment, AILiveSim. Using the proposed application framework, we develop obstacle detection and path planning systems based on machine learning which leverage live data from a 3D simulation environment to mirror the complex dynamics of the real world. Exploiting the proposed application framework, the rewards across training episodes of a Deep Reinforcement Learning model are evaluated for live simulated data in AILiveSim.

Development of sensor data fusion for obstacle detection for electric mini tractor

Automation in the agriculture space has become the need of the hour. Tractors assist farmers by imparting time-efficient power, comfort, and speed to farming operations. Autonomous/Smart tractors can improve the capability and productivity of farming operations. For the same, the obstacle detection system and motion planning system are needed. So, to forge ahead, this research work aims to develop sensor fusion data for obstacle detection in electric mini tractors. Automation task has two aspects: perceiving the environment and navigation. Perception is done with help of a camera, lidar, and ultrasonic sensors. The camera sensor is used for obstacle identification and localization. The distance of the obstacle from the vehicle is captured by the lidar and ultrasonic sensors. Different features of the obstacle captured by sensor modules are combined and used to develop an obstacle detection system. The conditional algorithm is developed for the sensor data fusion to ensure the synchronous working of the system. The obstacle detection system is developed with the help of sensor data fusion for an electric mini tractor.

Obstacle Detection for Unmanned Aerial Vehicle (UAV)

This study aims to develop an obstacle detection system for unmanned aerial vehicles utilisingthe ORB feature extraction. In the past, small unmanned aerial vehicles(UAV)were typically equipped with vision-based or range-based sensors. Each sensor in the sensor-based technique possesses different advantages and disadvantages. As a result, the small unmanned aerial vehicle is unable to determine the obstacle's distance or bearing precisely. Due to physical size restrictions and payload capacity, the lightweight Pi Camera and TF Luna LiDAR sensor were selected as the most suitable sensors for integration. In algorithmdevelopmentandfiltration isused to improve the accuracy of the feature matching process, which is required for classifying the obstacle region and free region of any texture obstacle. The experimentwas under the environment of OpenCV and Spyder. In real-time experiment, the success rate for good texture (40%), poor texture (55%) and texture-less (45%) The findings indicate that the recommended method works well for detectingtextures-less obstacle even though the success rate is only 40% becauseout of 10 test only one test is fail on detecting free region. The sensor calibration and constructing convex hullfor obstacle detectionis recommended in future works to improve the efficiency of the obstacle detection system and classified the free region and obstacle region to create safe avoidance path.

Anomaly detection as vision-based obstacle detection for vehicle automation in industrial environment

Nowadays, produced cars are equipped with mechatronical actuators as well as with a wide range of sensors in order to realize driver assistance functions. These components could enable cars’ automation at low speeds on company premises, although autonomous driving in public traffic is still facing technical and legal challenges. For automating vehicles in an industrial environment a reliable obstacle detection system is required. State-of-the-art solution for protective devices in Automated Guided Vehicles is the distance measuring laser scanner. Since laser scanners are not basic equipment of today’s cars in contrast to monocameras mounted behind the windscreen, we develop a computer vision algorithm that is able to detect obstacles in camera images reliably. Therefore, we make use of our well-known operational design domain by teaching an anomaly detection how the vehicle path should look like. The result is an anomaly detection algorithm that consists of a pre-trained feature extractor and a shallow classifier, modelling the probability of occurrence. We record a data set of a real industrial environment and show a robust classifier after training the algorithm with images of only one run. The performance as an obstacle detection is on par with a semantic segmentation, but requires a fraction of the training data and no labeling.

<b>Train Frontal Obstacle Detection Method with Camera-LiDAR Fusion</b>

Recently, the importance of obstacle detection methods for railway has been increasing. In the field of automobiles, obstacle detection systems with sensors have been introduced on mass-produced vehicles. However, in railway, a practical detection system does not exist because railways require longer detection distances than do automobiles. Therefore, we have developed a train frontal obstacle detection method using a camera and LiDAR. We confirmed that our method detects a person 200 m away, which a camera alone cannot detect, with 45% accuracy at night.

Autonomous Vehicles Management in Agriculture with Bluetooth Low Energy (BLE) and Passive Radio Frequency Identification (RFID) for Obstacle Avoidance

Obstacle avoidance is a key aspect for any autonomous vehicles, and their usage in agriculture must overcome additional challenges such as handling interactions with agricultural workers and other tractors in order to avoid severe accidents. The simultaneous presence of autonomous vehicles and workers on foot definitely calls for safer designs, vehicle management systems and major developments in personal protective equipment (PPE). To cope with these present and future challenges, the “SMARTGRID” project described in this paper deploys an integrated wireless safety network infrastructure based on the integration of Bluetooth Low Energy (BLE) devices and passive radio frequency identification (RFID) tags designed to identify obstacles, workers, nearby vehicles and check if the right PPE is in use. With the aim of detecting workers at risk by scanning for passive RFID-integrated into PPE in danger areas, transmitting alerts to workers who wear them, tracking of near-misses and activating emergency stops, a deep analysis of the safety requirements of the obstacle detection system is shown in this study. Test programs have also been carried out on an experimental farm with detection ranging from 8 to 12 meters, proving that the system might represent a good solution for collision avoidance between autonomous vehicles and workers on foot.

Testing ToF Sensors for Use in Obstacle Detection Systems

ToF (Time of Flight) sensors have been gaining popularity in recent times as a cheap and accurate way to take distance measurements. They may prove to be a key component of obstacle detection systems in the near future. This paper presents a stepper motor system with two measurement modules containing ToF sensors. The results of distance measurements made with these modules relative to a flat surface for different sensor modes are presented. Standard deviations were determined for the measurement results and a polynomial fitting was performed using the linear least squares method. The results were used to select the sensor for further work in terms of its use in an obstacle detection system.

Fast and accurate obstacle detection of manipulator in complex human–machine interaction workspace

In order to solve the problems of a large amount of calculation, poor real-time performance, and insufficient detection accuracy in an existing robot obstacle detection system, this paper proposes an improved lightweight You Only Look Once Version 3 algorithm for obstacle detection and tracking, by combining the DeepSort algorithm with a three-dimensional model velocity estimation method. First, the depthwise separable convolution and convolution kernel pruning methods are used to lighten the network. Second, to meet the requirement of dynamic obstacle detection, the DeepSort method is used to track dynamic obstacles and estimate their moving speed. Third, a method of bounding-sphere modeling and discrete marking is proposed to simplify the obstacle model. The experimental results show that: the model size is reduced by 92.11%, which suggests that the proposed lightweight detection system is more suitable for industrial applications; the detection speed in the actual scene reaches 118 frames per second and the average detection accuracy is 92.6%, while the multiple objects tracking accuracy and multiple objects tracking precision are all above 0.8 in the dynamic obstacle tracking process. Static and dynamic obstacles in the workspace can be detected simultaneously with an average positioning accuracy error of 1.5 cm. The proposed model can not only effectively improve the detection efficiency of obstacles in the manipulator workspace but also enhance the safety of industrial robot systems.

Smart Automated Guided Vehicle for Flexible Manufacturing Systems

In the Flexible Manufacturing System, automation and the ability to restructure the manufacturing facility is important. The development of a discretely working Smart Automated Guided Vehicle is the need of the hour. Hence the objective is to develop a compact unit load Smart Automated Guided Vehicle to increase efficiency and productivity & to overcome the problems of conventional material handling systems and improve the efficacy of manufacturing. Smart Automated Guided Vehicle is provided with navigation, weight sensing, obstacle detection systems with other auxiliary systems instrumental in zonal setup for the Smart Automated Guided Vehicle as well as adaptable for frequent changes. This model of Smart Automated Guided Vehicle is helpful for a small operational manufacturing unit for multipurpose applications at very low cost and high customizability. The objective is to provide a safe environment to the Smart Automated Guided Vehicle & its surroundings also, to reduce human dependency.