Non-road Regions Research Articles

Explicit Points-of-Interest Driven Siamese Transformer for 3D LiDAR Place Recognition in Outdoor Challenging Environments

Place recognition (PR) plays a crucial role in simultaneous localization and mapping. Unfortunately, however, changes in viewpoints and conditions in large-scale environments impose tricky challenges for PR. To this end, this paper specifically proposes an explicit points-of-interest driven PR method, which consists of a road segmentation module based on grid-wise patch U-Transformer (GP_UT) and a PR module based on regions of interest Siamese Transformer NetVLAD (RI_STV). Especially for RI_STV, in the individual dimension, it is dedicated to exploring the local topological features of non-road regions of interest. In the spatial dimension, an improved Transformer is introduced to capture the global interactions between features of interest. In the cluster dimension, NetVLAD embedded with weighted pooling is created to perform weighted aggregation of feature clusters to generate discriminative and general descriptors. Evaluation on various datasets shows that our customized method is not only impressively competitive, but also strikes the best balance between accuracy and real-time performance.

A Residual Attention and Local Context-Aware Network for Road Extraction from High-Resolution Remote Sensing Imagery

Extracting road information from high-resolution remote sensing images (HRI) can provide crucial geographic information for many applications. With the improvement of remote sensing image resolution, the image data contain more abundant feature information. However, this phenomenon also enhances the spatial heterogeneity between different types of roads, making it difficult to accurately discern the road and non-road regions using only spectral characteristics. To remedy the above issues, a novel residual attention and local context-aware network (RALC-Net) is proposed for extracting a complete and continuous road network from HRI. RALC-Net utilizes a dual-encoder structure to improve the feature extraction capability of the network, whose two different branches take different feature information as input data. Specifically, we construct the residual attention module using the residual connection that can integrate spatial context information and the attention mechanism, highlighting local semantics to extract local feature information of roads. The residual attention module combines the characteristics of both the residual connection and the attention mechanism to retain complete road edge information, highlight essential semantics, and enhance the generalization capability of the network model. In addition, the multi-scale dilated convolution module is used to extract multi-scale spatial receptive fields to improve the model’s performance further. We perform experiments to verify the performance of each component of RALC-Net through the ablation study. By combining low-level features with high-level semantics, we extract road information and make comparisons with other state-of-the-art models. The experimental results show that the proposed RALC-Net has excellent feature representation ability and robust generalizability, and can extract complete road information from a complex environment.

Distance-based confidence generation and aggregation of classifier for unstructured road detection

Road region and non-road region separation in the unstructured road intends to be an important task for safe navigation and collision avoidance for autonomous driving vehicles. The road that connects rural areas and cities to the national highways are considered as unstructured roads. Absence of clear lane marking on these unstructured roads makes them more prone to accidents in comparison to highways, which have clear lane marking for indication of road and non-road regions. However, the unstructured roads have different color information from its background that paves an easy way for design and development of an efficient road detection system for recognition and classification of road and non-road regions. Hence, in this paper, we propose an efficient road detection system for the classification of unstructured roads into road and non-road regions using multiple nearest neighbors (NN) classifier and soft voting aggregation approach. The proposed system utilized the chromatic information (i.e. *a,*b, and Hue) to train the NN classifiers and aggregated their output using soft voting (SV) approach for final output response. The output results of multiple classifiers were aggregated using SV approach based on the confidence score obtained by each individual classifier.The performance of the proposed system is evaluated in terms of precision, recall, accuracy, intersection over union (IOU), true positive rate (TPR), and processing time and compared with current state of art methods reported in the literature. The proposed system achieved precision, recall, accuracy, IOU, and TPR of 96.79%, 96.92%, 97.8%, 96.08% and 96%, respectively with the processing time three times smaller than those of the existing state of art methods. The experimental results demonstrate that the proposed system can provide an effective guidance to the autonomous vehicles through recognition and classification of road and non-road regions in rural, urban, and city areas, wherein single unstructured roads connect the national highways.

Utilizing the Road Mark Training Set from Ground-Based Mapping System to Airborne Imagery in Deep Learning Framework

Abstract. Deep Learning is a kind of Machine Learning technology which utilizing the deep neural network to learn a promising model from a large training data set. Convolutional Neural Network (CNN) has been successfully applied in image segmentation and classification with high accuracy results. The CNN applies multiple kernels (also called filters) to extract image features via image convolution. It is able to determine multiscale features through the multiple layers of convolution and pooling processes. The variety of training data plays an important role to determine a reliable CNN model. The benchmarking training data for road mark extraction is mainly focused on close-range imagery because it is easier to obtain a close-range image rather than an airborne image. For example, KITTI Vision Benchmark Suite. This study aims to transfer the road mark training data from mobile lidar system to aerial orthoimage in Fully Convolutional Networks (FCN). The transformation of the training data from ground-based system to airborne system may reduce the effort of producing a large training data set.This study uses FCN technology and aerial orthoimage to localize road marks on the road regions. The road regions are first extracted from 2-D large-scale vector map. The input aerial orthoimage is 10 cm spatial resolution and the non-road regions are masked out before the road mark localization. The training data are road mark’s polygons, which are originally digitized from ground-based mobile lidar and prepared for the road mark extraction using mobile mapping system. This study reuses these training data and applies them for the road mark extraction using aerial orthoimage. The digitized training road marks are then transformed to road polygon based on mapping coordinates. As the detail of ground-based lidar is much better than the airborne system, the partially occulted parking lot in aerial orthoimage can also be obtained from the ground-based system. The labels (also called annotations) for FCN include road region, non-regions and road mark. The size of a training batch is 500 pixel by 500 pixel (50 m by 50 m on the ground), and the total number of training batches for training is 75 batches. After the FCN training stage, an independent aerial orthoimage (Figure 1a) is applied to predict the road marks. The results of FCN provide initial regions for road marks (Figure 1b). Usually, road marks show higher reflectance than road asphalts. Therefore, this study uses this characteristic to refine the road marks (Figure 1c) by a binary classification inside the initial road mark’s region.To compare the automatically extracted road marks (Figure 1c) and manually digitized road marks (Figure 1d), most road marks can be extracted using the training set from ground-based system. This study also selects an area of 600 m × 200 m in quantitative analysis. Among the 371 reference road marks, 332 can be extracted from proposed scheme, and the completeness reached 89%. The preliminary experiment demonstrated that most road marks can be successfully extracted by the proposed scheme. Therefore, the training data from the ground-based mapping system can be utilized in airborne orthoimage in similar spatial resolution.

A real-time road detection method based on reorganized lidar data.

Road Detection is a basic task in automated driving field, in which 3D lidar data is commonly used recently. In this paper, we propose to rearrange 3D lidar data into a new organized form to construct direct spatial relationship among point cloud, and put forward new features for real-time road detection tasks. Our model works based on two prerequisites: (1) Road regions are always flatter than non-road regions. (2) Light travels in straight lines in a uniform medium. Based on prerequisite 1, we put forward difference-between-lines feature, while ScanID density and obstacle radial map are generated based on prerequisite 2. According to our method, we construct an array of structures to store and reorganize 3D input firstly. Then, two novel features, difference-between-lines and ScanID density, are extracted, based on which we construct a consistency map and an obstacle map in Bird Eye View (BEV). Finally, the road region is extracted by fusing these two maps and refinement is used to polish up our outcome. We have carried out experiments on the public KITTI-Road benchmark, achieving one of the best performances among the lidar-based road detection methods. To further prove the efficiency of our method on unstructured road, the visual outcomes in rural areas are also proposed.

Extraction of road using soft computing techniques

One of the most challenging research topics is extraction of road network automatically from remotely sensed imagery. The roads are considered as backbone and important modes of transportation. For efficient automatic road network extraction, ground points and building points extracted were integrated and it was used with multistage road network extraction model. For road network extraction, a lane detection system is considered as a more important element. From the lane marking in the roads, the roads are easily detected from the remotely sensed images. In order to detect the lanes more effectively, B-snake with fuzzy C-means (FCM) clustering is proposed in this paper. The input image is divided into equal halves by using FCM clustering algorithm. Then, the lower part is binarized by analyzing RGB channel and it is analyzed by using B-snake model because it contains more information about road lane than upper part of the image. Then, the lane detection problem is formulated by finding the set of lane model control points. A good initial position to the B-snake lane model is provided by introducing Canny/Hough estimation of vanishing points. Moreover, the external force field for lane detection is constructed by using gradient vector flow. The correspondence between B-snake and the real edge image is determined by using minimum mean square error which is presented to find out the parameters of road model iteratively. The detected lane points are fused with the road and ground points by using constraint satisfaction neural network-complementary information integration. Moreover, the road and non-road regions are separated by using region part segmentation. Finally, the medial-axis-transform-based hypothesis verification is used to automatically complete the road network extraction framework. The performance of the existing road network extraction, ground point-based road extraction model, building and ground point-based road extraction model and proposed building, ground and lane point-based road extraction model is analyzed in terms of correctness and completeness.

Roads and Intersections Extraction from High-Resolution Remote Sensing Imagery Based on Tensor Voting under Big Data Environment

Currently, big data is a new and hot object of research. In particular, the development of the Internet of things (IoT) results in a sharp increase in data. Enormous amounts of networking sensors are constantly collecting and transmitting data for storage and processing in the cloud including remote sensing data, environmental data, geographical data, etc. Road information extraction from remote sensing data is mainly researched in this paper. Roads are typical man-made objects. Extracting roads from remote sensing imagery has great significance in various applications such as GIS data updating, urban planning, navigation, and military. In this paper a multistage and multifeature method to extract roads and detect road intersections from high-resolution remotely sensed imagery based on tensor voting is presented. Firstly, the input remote sensing image is segmented into two groups including road candidate regions and nonroad regions using template matching; then we can obtain preliminary road map. Secondly, nonroad regions are removed by geometric characteristics of road (large area and long strip). Thirdly, tensor voting is used to overcome the broken roads and discontinuities caused by the different disturbing factors and then delete the nonroad areas that are mixed into the road areas due to mis-segmentation, improving the completeness of extracted roads. And then, all the road intersections are extracted by using tensor voting. The experiments are conducted on different remote sensing images to test the effectiveness of our method. The experimental results show that our method can get more complete and accurate extracted results than the state-of-the-art methods.

A novel FMH model for road extraction from high-resolution remote sensing images in urban areas

With the rapid development of remote sensing satellites and sensors, the higher resolution of remote sensing images can be collected. Comparing with roads of countryside, it is more difficult to extract roads of urban areas from remote sensing images due to various interference factors including buildings and vehicles. There are regional characteristics with specific width and gray levels of the urban road in high-resolution images. In this article, a novel model named FMH is proposed to extract the main road information from the high-resolution remote sensing image in urban areas. After pre-processing of the image, fuzzy c-means algorithm is applied to binary processing, and thus the image is divided into the road part and the non-road part. Then mathematical morphology is used to eliminate more non-road regions. Next the local Hough transform is applied to extracting the road regions. Finally morphological operations are used to modify and refine shapes of roads. This model is tested on the panchromatic image with the resolution of 2m collected by Gaofen-1 satellite, which shows effective and excellent results.

A Road Extraction Method Based on Region Growing and Mathematical Morphology from Remote Sensing Images

Road traffic is the important driving factor for economic and social development. With the rapid increase of vehicle population, road traffic problems such as traffic jam and traffic accident have become the bottleneck which restricts economic development. In recent years, natural disasters frequently occur in China. Therefore, it is essential to extract road information to compute the degree of road damage for traffic emergency management. A road extraction method based on region growing and mathematical morphology from remote sensing images is proposed in this paper. According to the road features, the remote sensing image is preprocessed to separate road regions from non-road regions preliminarily. After image thresholding, region growing algorithm is used to extract connected regions. Then we sort connected regions by area to exclude the small regions which are probably non-road objects. Finally, the mathematical morphology algorithm is used to fill the holes inside the road regions. The experimental results show that the method proposed can effectively extract roads from remote sensing images. This research also has broad prospects in dealing with traffic emergency management by the government.

Combining CNN and MRF for road detection

Road detection aims at detecting the road surface ahead of the vehicle and plays a crucial role in driver assistance systems. To improve the accuracy and robustness of road detection approaches in complex environments, a new road detection method based on a convolutional neural network (CNN) and Markov random field (MRF) is proposed. The original road image is segmented into super-pixels of uniform size using the simple linear iterative clustering (SLIC) algorithm. On this basis, we train the convolutional neural network, which can automatically learn the features that are most beneficial to the classification. The trained convolutional neural network (CNN) is then applied to classify road and non-road regions. Finally, based on the relationship between the super-pixel neighborhood, we utilize Markov random field (MRF) to optimize the classification results of the convolutional neural network (CNN). The approach provides the better performance.

Road Detection Using Classification Algorithms

In this study, we present a road detection method. Proposed method consists of two phases. In the first phase, a binary image is obtained by utilizing greyscale transformation and thresholding processes. In the second phase, K-Nearest Neighbours and Naive Bayes classifiers are applied on image by utilizing colour features. Road and non-road regions are determined and these two classifiers are compared with each other. Google Maps satellite images are used since they are easily accessible on the Internet. C# based software is developed for realizing and testing the methods mentioned above.

A Multi-stage Method to Extract Road from High Resolution Satellite Image

Extracting road information from high-resolution satellite images is complex and hardly achieves by exploiting only one or two modules. This paper presents a multi-stage method, consisting of automatic information extraction and semi-automatic post-processing. The Multi-scale Enhancement algorithm enlarges the contrast of human-made structures with the background. The Statistical Region Merging segments images into regions, whose skeletons are extracted and pruned according to geometry shape information. Setting the start and the end skeleton points, the shortest skeleton path is constructed as a road centre line. The Bidirectional Adaptive Smoothing technique smoothens the road centre line and adjusts it to right position. With the smoothed line and its average width, a Buffer algorithm reconstructs the road region easily. Seen from the last results, the proposed method eliminates redundant non-road regions, repairs incomplete occlusions, jumps over complete occlusions, and reserves accurate road centre lines and neat road regions. During the whole process, only a few interactions are needed.

Visual Campus Road Detection for an UGV using Fast Scene Segmentation and Rapid Vanishing Point Estimation

Vision-based road detection plays a key role for Unmanned Ground Vehicles (UGVs) working in an unknown outdoor environment. The estimation of the vanishing point is a practical solution for general road detection using monocular vision, which however is not good enough for robust road detection in a campus environment due to the strong noises of texture orientations generated from roadside trees and buildings. In this paper, a novel system framework is proposed by combining the fast scene segmentation (FSS) and the rapid vanishing point detection. The proposed FSS algorithm can segment a single image into road and non-road regions based on the similarity analysis of color histogram, which can eliminate the inherent noises in the trees and buildings and improve the robustness of road detection effectively. Before voting for the vanishing point, we use Canny algorithm to extract the edges in the road region roughly segmented in FSS step. Since most of the strong texture orientations exist in the extracted edges, the computational complexity in the voting stage can be reduced significantly. Experimental results implemented on a real UGV platform show the validity and robustness of the proposed approach.

Drivable Road Boundary Detection for Intelligent Vehicles based on Stereovision with Plane-induced Homography

Road detection is one of the key issues for intelligent vehicles and advanced driver assistance systems (ADAS). In this paper, we present a stereovision-based approach for estimating the boundary between the drivable road region and the non-road region. It is based on a formulation of stereo with the ground-plane-induced homography in a hidden Markov model (HMM). Under this formulation, we employ the Viterbi algorithm and propose a sophisticated measure of the probability of the state sequence to find the most likely road/non-road boundary. Experimental results on a wide variety of typical but challenging real road scenes have substantiated the effectiveness as well as robustness of the proposed approach.

Road identification in monocular color images using random forest and color correlogram

This paper presents a system to identify road and non-road regions from monocular color images of paved and unpaved roads. Despite being a single object, the road in these images is subject to large changes in appearance due to environmental effects and track materials. This condition has challenged the practical application of road identification. The proposed system combines random forest with color correlogram to overcome such conditions and offers a classifier for road and non-road regions in traffic images. As a color feature, the color correlogram depicts the spatial correlation of color changes in an image. Using random forest, road identification is formulated as a learning paradigm. The combined effects of color correlograms and random forest create a robust system capable of identifying roads even in variable situations in real time. This combination is more effective than other combinations, such as a color histogram plus random forest, a color correlogram plus neural network, or a color histogram plus neural network.

Research for Improving the Process of Road Information Recognition on Grid Traffic Map

Enhancement on extracting road layer directly and automatically from raster traffic map has become key segment in manufacturing vector maps. This research goal is to improve whole process of recognizing and extracting road layer from raster map, and to make the whole process as a closed feedback system with output measurable, state controllable and recognition sub-process optimally adaptable .The scheme is building an integrated system with its output road layer measurable by selecting two (or more) methods of perfect comparatively identifications with their mechanism complementary and then parallel collecting them as the controlled sub-process objects of the integrated system. Based on approach thought of microscopic identification and macroscopic elimination, the system realized first the road layer initial recognition and last road map level completely to withdraw via some output layers re-clustering and feedback control. In view of the standard municipal transportation map, the path and the region valve value condition has determined. Through re-clustering to the noise the whole map can be completely divided into two parts of road and non-road (region). The feedback re-clustering strategies of recognition for the transportation road networks are based on the core characteristic of obtain road constitution. The convergence of the feedback re-clustering of road layer guarantees the map level optimization.

On‐road vehicle detection and tracking based on road context and the ambient lighting adaptive framework

AbstractDespite recent active research on vision‐based vehicle detection and tracking, its lack of flexibility hindered practical use in real environments. To be practical, adaptation to various illumination conditions is an essential ingredient. We propose a novel adaptation framework that can improve on this current lighting adaptation using a simple road context, feature arbiter, and a proper feature fusion scheme. In real driving environments, self‐supervised online learning can efficiently segment the road and nonroad regions in front of the host vehicle. Classification into these regions is very important to generate regions of interest (ROIs) for potential vehicle position, that is, road context. It improves on system efficiency by reducing noise and processing time. In our global and local lighting models, the feature arbiter selects an appropriate daytime or nighttime detector for each ROI. And finally, an adaptive fusion framework method can robustly track by selecting or removing the distinctive visual attributes. This system was successfully tested on real road data obtained with various ambient lighting conditions. © 2008 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 18, 283–295, 2008

Autonomous robotic vehicle road following

A description is given of the system architecture of an autonomous vehicle and its real-time adaptive vision system for road-following. The vehicle is a 10-ton armored personnel carrier modified for robotic control. A color transformation that best discriminates road and nonroad regions is derived from labeled data samples. A maximum-likelihood pixel classification technique is then used to classify pixels in the transformed color image. The vision system adapts itself to road changes in two ways; color transformation parameters are updated infrequently to accommodate significant road color changes, and classifier parameters are updated every processing cycle to deal with gradual color and intensity changes. To reduce unnecessary computation, only the most likely road region in the segmented image is selected, and a polygonal representation of the detected road region boundary is transformed from the image coordinate system to the local vehicle coordinate system based on a flat-earth assumption.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>