Vision-based Obstacle Detection Research Articles

Vision-Based obstacle detection in dangerous region of coal mine driverless rail electric locomotives

In order to solve the problem of inaccurate obstacle detection as well as frequent start-stops caused by oversensitive obstacle detection in existing driverless rail electric locomotives in underground coal mines, the YOLO-Region model is proposed to realize regional obstacle detection. First, the model backbone uses InceptionNeXt block and the developed New Spatial Pyramid Pooling (NSPP) module; the model neck extends the FPN+PAN architecture; the model head uses improved task-specific context decoupling (Impro-TSCODE) head. In addition, repulsion loss is introduced to improve the detection ability of partially occluded targets. The experimental results show that the YOLO-Region achieves competitive detection performance with mAP of 98.0 % and an average detection speed of 94.5 FPS. Second, a vision-based method for defining dangerous region based on pixel coordinate points is developed and integrated into YOLO-Region, which allows the model to detect obstacles only within a specific region, thereby reducing frequent start-stops of driverless electric locomotives.

Vision-Based Obstacle Detection and Collision Prevention in Self-Driving Cars

With increasing computational power and a vast amount of data to work with, deep learning has risen to prominence since the 2010s. Numerous applications are being researched and developed using deep learning. One of the applications is computer vision in self-driving cars. Convolutional Neural Networks (CNNs) are being widely used because of their high performance compared to other alternative techniques in several perception and control tasks. The Convolutional Neural Networks (CNNs) allows the automobile to learn from different types of roads, scenarios allowing the car to forecast its route on any particular road with minimum inaccuracy. This paper proposes a working model of the autonomous car, which has a Raspberry Pi 4 Model B as the control unit and processing unit. This working model gets real-time images from the Raspberry Pi camera and these images are used by the CNN model, which predicts the direction the car must turn. The raspberry pi sends the control signals to the L298n motor driver. The trained CNN model achieved an accuracy of 96.39% with the test dataset and 84.77% with the validation dataset.

Anomaly detection as vision-based obstacle detection for vehicle automation in industrial environment

Nowadays, produced cars are equipped with mechatronical actuators as well as with a wide range of sensors in order to realize driver assistance functions. These components could enable cars’ automation at low speeds on company premises, although autonomous driving in public traffic is still facing technical and legal challenges. For automating vehicles in an industrial environment a reliable obstacle detection system is required. State-of-the-art solution for protective devices in Automated Guided Vehicles is the distance measuring laser scanner. Since laser scanners are not basic equipment of today’s cars in contrast to monocameras mounted behind the windscreen, we develop a computer vision algorithm that is able to detect obstacles in camera images reliably. Therefore, we make use of our well-known operational design domain by teaching an anomaly detection how the vehicle path should look like. The result is an anomaly detection algorithm that consists of a pre-trained feature extractor and a shallow classifier, modelling the probability of occurrence. We record a data set of a real industrial environment and show a robust classifier after training the algorithm with images of only one run. The performance as an obstacle detection is on par with a semantic segmentation, but requires a fraction of the training data and no labeling.

Computer Vision and Artificial Intelligence Based Navigation on Top-view Virtual Plane

This paper shows a vision-based obstacle detection approach in an outdoor sidewalk environment. There is a major problem of applying single camera in outdoor navigation mission, that is the differentiation of various types of barriers and clutter pavement. To settle this issue, the property that pavement margins are sub-sampled while barrier margins are oversampled when the original image is mapped to the virtual plane of the top view. Morphology filters are applied to strengthen barrier margins as margin-blobs with greater size, while sparse margins from pavement are screened. On basis of the recognized barriers, secure walking areas are predicted by tracking a polar margin-blob histogram. The test of algorithm is made in various sidewalk scenes with complicated pavements with confirmed potency.

Computer vision based obstacle detection and target tracking for autonomous vehicles

Obstacle detection and target tracking are two major issues for intelligent autonomous vehicles. This paper proposes a new scheme to achieve target tracking and real-time obstacle detection of obstacles based on computer vision. ResNet-18 deep learning neural network is utilized for obstacle detection and Yolo-v3 deep learning neural network is employed for real-time target tracking. These two trained models can be deployed on an autonomous vehicle equipped with an NVIDIA Jetson Nano motherboard. The autonomous vehicle moves to avoid obstacles and follow tracked targets by camera. Adjusting the steering and movement of the autonomous vehicle according to the PID algorithm during the movement, therefore, will help the proposed vehicle achieve stable and precise tracking.

Review on Application of Binocular Vision Technology in Field Obstacle Detection

Obstacles within the field are the main external source of safety hazard in field operation of agricultural machineries. Autonomous obstacle avoidance is a critical technology to be solved for the robotization of field operation by agricultural machineries. For this technology, a critical premise is to timely and accurately perceive static and dynamic information of field obstacles. Firstly, this paper describes main detection parameters of field obstacles. Whereafter, by compared with other detection technologies, applicability of binocular vision-based detection technology in field obstacle detection is analyzed. Afterwards, this paper summarizes the whole process of binocular vision-based obstacle detection into three steps. For step one, main detecting methods for locating obstacles in images, main principles of these methods and their applicability in field operation scene are discussed. For step two, main existing methods for detecting spatial position parameters of obstacles based on detection results of step one and their application status in field operation scene are summarized. For step three, typical target tracking methods for obtaining the motion state of obstacles are compared and summarized, and their reference value for future research is pointed out. At last, the main challenges of detecting obstacles in field scene based on binocular vision are summarized.

Vision-based monitoring of orchards with UAVs

We present an autonomous aerial system which can safely navigate inside the rows of an orchard for crop inspection, yield estimation and similar farm management tasks. Our operating environment has two practical constraints: GPS signal can be unreliable close to the ground and the trees can grow large branches that might block the robot’s predetermined path. To address these challenges, we developed components for navigation, vision-based obstacle detection and avoidance and path planning for visual coverage. Results from a field demonstration in an apple orchard are presented showcasing the ability of our system to perform successful navigation and complete yield coverage of apple tree rows.

VISION BASED OBSTACLE DETECTION USING ROVER STEREO IMAGES

Abstract. Vision based obstacle detection using stereo images is an essential way for hazard avoidance and path planning in planetary rover missions. However, due to light condition changes and topographic relief, only partial or sparse three-dimensional points may be derived by image matching and triangulation reconstruction, which is not sufficient for recognizing obstacles. In this paper, we developed a strategy to detect obstacles using rover stereo images by combining both image grayscale information and sparse 3D point information. Experiments were carried out using stereo images captured by navigation cameras mounted on the Yutu rover of Chang’e-3 mission. Moreover, how obstacle localization accuracy affected by the parameters are analysed and discussed.

Vision-Based Parking Guidance and Obstacle Detection System

Vision-Based Parking Guidance and Obstacle Detection System

Monocular Vision-Based Obstacle Detection and Avoidance for a Multicopter

This article presents a monocular vision-based algorithm for detecting obstacles and identifying obstacle-aware regions, developed to be used for collision avoidance by a multicopter. The first step of our algorithm is to predict a disparity image from a single-view image via implementing a deep encoder-decoder network. All pixels in this disparity prediction are then categorized as one of three classes, obstacle, road, or obstacle-free, by combining V-disparity analysis and a fuzzy inference system. For pixels belonging to obstacle objects, obstacle-aware regions are generated within the field of visual perception. To accommodate the safety margins of a multicopter, intermediate waypoints are then added to obtain a new flyable path that passes through an unknown environment safely. Experimental results verified the effectiveness of the detection of obstacles and the identification of obstacle-aware regions. The accuracy of disparity prediction and monocular depth estimation were quantitatively compared to support the feasibility of monocular vision in obstacle avoidance. Furthermore, the entire algorithm was successfully tested on a robotic platform, autonomously flying a hexacopter in an outdoor space with obstacles. In conclusion, the proposed monocular algorithm performs well for obstacle detection and depth estimation and is potentially an alternative to a binocular solution.

Obstacle Detection Quality as a Problem-Oriented Approach to Stereo Vision Algorithms Estimation in Road Situation Analysis

In this work we present a method for performance evaluation of stereo vision based obstacle detection techniques that takes into account the specifics of road situation analysis to minimize the effort required to prepare a test dataset. This approach has been designed to be implemented in systems such as self-driving cars or driver assistance and can also be used as problem-oriented quality criterion for evaluation of stereo vision algorithms.

UAV and obstacle sensing techniques – a perspective

PurposeThe purpose of this paper is to provide an overview of unmanned aerial vehicle (UAV) developments, types, the major functional components of UAV, challenges, and trends of UAVs, and among the various challenges, the authors are concentrating more on obstacle sensing methods. This also highlights the scope of on-board vision-based obstacle sensing for miniature UAVs.Design/methodology/approachThe paper initially discusses the basic functional elements of UAV, then considers the different challenges faced by UAV designers. The authors have narrowed down the study on obstacle detection and sensing methods for autonomous operation.FindingsAmong the various existing obstacle sensing techniques, on-board vision-based obstacle detection has better scope in the future requirements of miniature UAVs to make it completely autonomous.Originality/valueThe paper gives original review points by doing a thorough literature survey on various obstacle sensing techniques used for UAVs.

Unsupervised obstacle detection in driving environments using deep-learning-based stereovision

Abstract A vision-based obstacle detection system is a key enabler for the development of autonomous robots and vehicles and intelligent transportation systems. This paper addresses the problem of urban scene monitoring and tracking of obstacles based on unsupervised, deep-learning approaches. Here, we design an innovative hybrid encoder that integrates deep Boltzmann machines (DBM) and auto-encoders (AE). This hybrid auto-encode (HAE) model combines the greedy learning features of DBM with the dimensionality reduction capacity of AE to accurately and reliably detect the presence of obstacles. We combine the proposed hybrid model with the one-class support vector machines (OCSVM) to visually monitor an urban scene. We also propose an efficient approach to estimating obstacles location and track their positions via scene densities. Specifically, we address obstacle detection as an anomaly detection problem. If an obstacle is detected by the OCSVM algorithm, then localization and tracking algorithm is executed. We validated the effectiveness of our approach by using experimental data from two publicly available dataset, the Malaga stereovision urban dataset (MSVUD) and the Daimler urban segmentation dataset (DUSD). Results show the capacity of the proposed approach to reliably detect obstacles.

Fast Obstacle Detection for Monocular Autonomous Mobile Robots

This paper proposes a monocular vision based obstacle detection algorithm for autonomous mobile robots. Our main algorithm consists of two stages. In the first stage, we use an inverse perspective ...

Farm Workers of the Future: Vision-Based Robotics for Broad-Acre Agriculture

Farmers are under growing pressure to intensify production to feed a growing population while managing environmental impact. Robotics has the potential to address these challenges by replacing large complex farm machinery with fleets of small autonomous robots. This article presents our research toward the goal of developing teams of autonomous robots that perform typical farm coverage operations. Making a large fleet of autonomous robots economical requires the use of inexpensive sensors, such as cameras for localization and obstacle avoidance. To this end, we describe a vision-based obstacle detection system that continually adapts to environmental and illumination variations and a visionassisted localization system that can guide a robot along crop rows with a complex appearance. Large fleets of robots will become time-consuming to monitor, control, and resupply. To reduce this burden, we describe a vision-based docking system for autonomously refilling liquid supplies and an interface for controlling multiple robots.

Vision-Based Real-Time Aerial Object Localization and Tracking for UAV Sensing System

The paper focuses on the problem of vision-based obstacle detection and tracking for unmanned aerial vehicle navigation. A real-time object localization and tracking strategy from monocular image sequences is developed by effectively integrating the object detection and tracking into a dynamic Kalman model. At the detection stage, the object of interest is automatically detected and localized from a saliency map computed via the image background connectivity cue at each frame; at the tracking stage, a Kalman filter is employed to provide a coarse prediction of the object state, which is further refined via a local detector incorporating the saliency map and the temporal information between two consecutive frames. Compared to existing methods, the proposed approach does not require any manual initialization for tracking, runs much faster than the state-of-the-art trackers of its kind, and achieves competitive tracking performance on a large number of image sequences. Extensive experiments demonstrate the effectiveness and superior performance of the proposed approach.

Vision‐based Obstacle Detection and Navigation for an Agricultural Robot

This paper describes a vision-based obstacle detection and navigation system for use as part of a robotic solution for the sustainable intensification of broad-acre agriculture. To be cost-effective, the robotics solution must be competitive with current human-driven farm machinery. Significant costs are in high-end localization and obstacle detection sensors. Our system demonstrates a combination of an inexpensive global positioning system and inertial navigation system with vision for localization and a single stereo vision system for obstacle detection. The paper describes the design of the robot, including detailed descriptions of three key parts of the system: novelty-based obstacle detection, visually-aided guidance, and a navigation system that generates collision-free kinematically feasible paths. The robot has seen extensive testing over numerous weeks of field trials during the day and night. The results in this paper pertain to one particular 3 h nighttime experiment in which the robot performed a coverage task and avoided obstacles. Additional results during the day demonstrate that the robot is able to continue operating during 5 min GPS outages by visually following crop rows.

Obstacle Detection System Based on Stereo Vision and a Structured Light

Obstacle detection is a crucial issue for pilotless device guidance function and it has to be performed with high reliability to avoid any potential collision with the front object. The vision-based obstacle detection systems are regarded perfect for this purpose because they require little on all kind of condition. In this paper, an obstacle detection system using stereo vision sensors and structured light is developed. This system realizes rapid feature matching and high precision measurement distance with the help of structured light, avoiding the time-consuming of the initial corresponding pairs. After the initial detection, the system executes the tracking light strip algorithm for the obstacles. The proposed system can detect a front obstacle in vision field and obtain the size of obstacle. The proposed obstacle detection system is set up and its performance is verified experimentally

Construction of Fisheye Lens Inverse Perspective Mapping Model and Its Applications of Obstacle Detection

In this paper, we develop a vision based obstacle detection system by utilizing our proposed fisheye lens inverse perspective mapping (FLIPM) method. The new mapping equations are derived to transform the images captured by the fisheye lens camera into the undistorted remapped ones under practical circumstances. In the obstacle detection, we make use of the features of vertical edges on objects from remapped images to indicate the relative positions of obstacles. The static information of remapped images in the current frame is referred to determining the features of source images in the searching stage from either the profile or temporal IPM difference image. The profile image can be acquired by several processes such as sharpening, edge detection, morphological operation, and modified thinning algorithms on the remapped image. The temporal IPM difference image can be obtained by a spatial shift on the remapped image in the previous frame. Moreover, the polar histogram and its post-processing procedures will be used to indicate the position and length of feature vectors and to remove noises as well. Our obstacle detection can give drivers the warning signals within a limited distance from nearby vehicles while the detected obstacles are even with the quasi-vertical edges.

A Universal Autonomous Robot Navigation Method

Recently, autonomous navigation has become an important research topic. There are a lot of applications where the need for autonomous robots is obvious, either because the real or virtual human presence is impossible, dangerous, or expensive, or the tasks to be solved are against the human nature. In most of the applications where robots are to be used, the conditions/environment change along the time that results in an ever-increasing need for universal methods, which are general enough to be used at a wide range of problems. In this paper, a universal, hybrid navigation method is proposed, which is able to work in cases of known, partially known, dynamically changing, or unknown environments. The model consists of two parts which are able to co-operate or to work alone. The modules combine two techniques that deal with a priori information and sensory data separately, thus blends the intelligence and optimality of global navigation methods with the reactivity and low complexity of local ones. The first, global navigation module, based on a priori information, chooses intermediary goals for the local navigation module, for which the so called A* algorithm is used. The second part, carrying out the (local) navigation relying on sensory data, applies a fuzzy-neural representation of an improved potential field based guiding navigation tool. Vision based obstacle detection is implemented by difference detection based on a combination of RGB and HSV representations of the pixels.