Line Segment Detector Research Articles

FPGA-Based Linear Detection Algorithm of an Underground Inspection Robot

Conveyor belts are key pieces of equipment for bulk material transport, and they are of great significance to ensure safe operation. With the development of belt conveyors in the direction of long distances, large volumes, high speeds, and high reliability, the use of inspection robots to perform full inspections of belt conveyors has not only improved the efficiency and scope of the inspections but has also eliminated the dependence of the traditional method on the density of sensor arrangement. In this paper, relying on the wireless-power-supply orbital inspection robot independently developed by the laboratory, aimed at the problem of the deviation of the belt conveyor, the methods for the diagnosis of the deviation of the conveyor belt and FPGA (field-programmable gate array) parallel computing technology are studied. Based on the traditional LSD (line segment detection) algorithm, a straight-line extraction IP core, suitable for an FPGA computing platform, was constructed. This new hardware linear detection algorithm improves the real-time performance and flexibility of the belt conveyor diagnosis mechanism.

Laser Scanning and Parametrization of Weld Grooves with Reflective Surfaces.

This paper presents a novel weld groove parametrization algorithm, which is developed specifically for weld grooves in typical stub and butt joints between large tubular elements. The procedure is based on random sample consensus (RANSAC) with additionally proposed correction steps, including a corner correction step for grooves with narrow root weld, and an iterative error elimination step for improving the initially obtained data fit. The problem of curved groove sides (due to the pipe geometry) is attributed and solved. In addition, the procedure detects and eliminates several types of data noise due to laser line reflections. The performance of the procedure is studied experimentally using small-scale test objects, which have been ground using typical industrial power tools to achieve a realistic level of reflections. The execution times and data fit errors of the proposed procedure are compared to a procedure based on a more conventional RANSAC approach for line segment detection.

Instance segmentation of fallen trees in aerial color infrared imagery using active multi-contour evolution with fully convolutional network-based intensity priors

Over the last several years, semantic image segmentation based on deep neural networks has been greatly advanced. On the other hand, single-instance segmentation still remains a challenging problem. In this paper, we introduce a framework for segmenting instances of a common object class by multiple active contour evolution over semantic segmentation maps of images obtained through fully convolutional networks. The contour evolution is cast as an energy minimization problem, where the aggregate energy functional incorporates a data fit term, an explicit shape model, and accounts for object overlap. Efficient solution neighborhood operators are proposed, enabling optimization through metaheuristics such as simulated annealing. We instantiate the proposed framework in the context of segmenting individual fallen stems from high-resolution aerial multispectral imagery, providing problem-specific energy potentials. We validated our approach on 3 real-world scenes of varying complexity, using 730 manually labeled polygon outlines as ground truth. The test plots were situated in regions of the Bavarian Forest National Park, Germany, which sustained a heavy bark beetle infestation. Evaluations were performed on both the polygon and line segment level, showing that the multi-contour segmentation can achieve up to 0.93 precision and 0.82 recall. An improvement of up to 7 percentage points (pp) in recall and 6 in precision compared to an iterative sample consensus line segment detection baseline was achieved. Despite the simplicity of the applied shape parametrization, an explicit shape model incorporated into the energy function improved the results by up to 4 pp of recall. Finally, we show the importance of using a high-quality semantic segmentation method (e.g. U-net) as the basis for individual stem detection, as the quality of the results degraded dramatically in our baseline experiment utilizing a simpler method. Our method is a step towards increased accessibility of automatic fallen tree mapping in forests, due to higher cost efficiency of aerial imagery acquisition compared to laser scanning. The precise fallen tree maps could be further used as a basis for plant and animal habitat modeling, studies on carbon sequestration as well as soil quality in forest ecosystems.

ULSD: Unified line segment detection across pinhole, fisheye, and spherical cameras

Image line segment detection is a fundamental problem in computer vision and remote sensing. Although numerous state-of-the-art methods have shown great performance for straight line segment detection, line segment detection for distorted images without undistortion is still a challenging problem. Besides, there is a lack of a unified line segment detection framework for both distorted and undistorted images. To address these two problems, we propose a novel learning-based Unified Line Segment Detection method (i.e., ULSD) for distorted and undistorted images in this paper. Specifically, we first propose a novel equipartition point-based Bezier curve representation to model arbitrary distorted line segments. Then the line segment detection is tackled by equipartition point regression with an end-to-end trainable neural network. Consequently, the proposed ULSD is independent of camera distortion parameters and does not need any undistortion preprocessing. In the experiments, the proposed method is firstly evaluated on the pinhole, fisheye, and spherical image datasets, respectively, as well as trained and tested on the mixed dataset with differently distorted images. The experimental results on each distortion model show that the proposed ULSD is more competitive than the state-of-the-art methods for both accuracy and efficiency, especially for the results of the unified model trained on the mixed datasets, thus demonstrating the effectiveness and generality of the proposed ULSD to real-world scenarios. The source code and datasets are available at https://github.com/lh9171338/ULSD-ISPRS.

Identifying Urban Poverty Using High-Resolution Satellite Imagery and Machine Learning Approaches: Implications for Housing Inequality

Enriching Asian perspectives on the rapid identification of urban poverty and its implications for housing inequality, this paper contributes empirical evidence about the utility of image features derived from high-resolution satellite imagery and machine learning approaches for identifying urban poverty in China at the community level. For the case of the Jiangxia District and Huangpi District of Wuhan, image features, including perimeter, line segment detector (LSD), Hough transform, gray-level cooccurrence matrix (GLCM), histogram of oriented gradients (HoG), and local binary patterns (LBP), are calculated, and four machine learning approaches and 25 variables are applied to identify urban poverty and relatively important variables. The results show that image features and machine learning approaches can be used to identify urban poverty with the best model performance with a coefficient of determination, R2, of 0.5341 and 0.5324 for Jiangxia and Huangpi, respectively, although some differences exist among the approaches and study areas. The importance of each variable differs for each approach and study area; however, the relatively important variables are similar. In particular, four variables achieved relatively satisfactory prediction results for all models and presented obvious differences in varying communities with different poverty levels. Housing inequality within low-income neighborhoods, which is a response to gaps in wealth, income, and housing affordability among social groups, is an important manifestation of urban poverty. Policy makers can implement these findings to rapidly identify urban poverty, and the findings have potential applications for addressing housing inequality and proving the rationality of urban planning for building a sustainable society.

A Fast Vision-inertial Odometer Based on Line Midpoint Descriptor

Visual Simultaneous Localization and Mapping are essential technologies to realize the autonomous movement of vehicles. Visual-Inertial Odometry (VIO) is often used as the front-end of VSLAM because of its rich information, lightweight, and robustness. This article proposes the FPL-VIO, an optimization-based Fast Vision-Inertial Odometer with Points and Lines. Traditional VIO mostly uses points as landmarks; meanwhile, most of the geometrical structure information is ignored. Therefore, the accuracy will be jeopardized under motion blur and texture-less area. Some researchers improve accuracy by adding lines as landmarks in the system. However, almost all of them use Line Segment Detector (LSD) and Line Band Descriptor (LBD) in line processing, which is very time-consuming. This article first proposes a fast line feature description and matching method based on the midpoint and compares the three line detection algorithms of LSD, FLD (Fast Line Detector), and EDLines (Edge Drawing Line). Then, the measurement model of the line is introduced in detail. Finally, FPL-VIO is proposed by adding the above method to VINS-Mono, an optimization-based Fast Vision-Inertial Odometer with lines described by midpoint and points. Compared with PL-VIO, the line processing efficiency of FPL-VIO is increased by 3-4 times while ensuring the same accuracy.

Learning Regional Attraction for Line Segment Detection.

This paper presents regional attraction of line segment maps, and hereby poses the problem of line segment detection (LSD) as a problem of region coloring. Given a line segment map, the proposed regional attraction first establishes the relationship between line segments and regions in the image lattice. Based on this, the line segment map is equivalently transformed to an attraction field map (AFM), which can be remapped to a set of line segments without loss of information. Accordingly, we develop an end-to-end framework to learn attraction field maps for raw input images, followed by a squeeze module to detect line segments. Apart from existing works, the proposed detector properly handles the local ambiguity and does not rely on the accurate identification of edge pixels. Comprehensive experiments on the Wireframe dataset and the YorkUrban dataset demonstrate the superiority of our method. In particular, we achieve an F-measure of 0.831 on the Wireframe dataset, advancing the state-of-the-art performance by 10.3 percent.

Airport Detection-Based Cosaliency on Remote Sensing Images

This paper proposes a contour extraction model based on cosaliency detection for remote sensing image airport detection and improves the traditional line segmentation detection (LSD) algorithm to make it more suitable for the goal of this paper. Our model consists of two parts, a cosaliency detection module and a contour extraction module. In the first part, the cosaliency detection module mainly uses the network framework of Visual Geometry Group-19 (VGG-19) to obtain the result maps of the interimage comparison and the intraimage consistency, and then the two result maps are multiplied pixel by pixel to obtain the cosaliency mask. In the second part, the contour extraction module uses superpixel segmentation and parallel line segment detection (PLSD) to refine the airport contour and runway information to obtain the preprocessed result map, and then we merge the result of cosaliency detection with the preprocessed result to obtain the final airport contour. We compared the model proposed in this article with four commonly used methods. The experimental results show that the accuracy of the model is 15% higher than that of the target detection result based on the saliency model, and the accuracy of the active contour model based on the saliency analysis is improved by 1%. This shows that the model proposed in this paper can extract a contour that closely matches the actual target.

An Improved Method for Stable Feature Points Selection in Structure-from-Motion Considering Image Semantic and Structural Characteristics.

Feature matching plays a crucial role in the process of 3D reconstruction based on the structure from motion (SfM) technique. For a large collection of oblique images, feature matching is one of the most time-consuming steps, and the matching result directly affects the accuracy of subsequent tasks. Therefore, how to extract the reasonable feature points robustly and efficiently to improve the matching speed and quality has received extensive attention from scholars worldwide. Most studies perform quantitative feature point selection based on image Difference-of-Gaussian (DoG) pyramids in practice. However, the stability and spatial distribution of feature points are not considered enough, resulting in selected feature points that may not adequately reflect the scene structures and cannot guarantee the matching rate and the aerial triangulation accuracy. To address these issues, an improved method for stable feature point selection in SfM considering image semantic and structural characteristics is proposed. First, the visible-band difference vegetation index is used to identify the vegetation areas from oblique images, and the line feature in the image is extracted by the optimized line segment detector algorithm. Second, the feature point two-tuple classification model is established, in which the vegetation area recognition result is used as the semantic constraint, the line feature extraction result is used as the structural constraint, and the feature points are divided into three types. Finally, a progressive selection algorithm for feature points is proposed, in which feature points in the DoG pyramid are selected by classes and levels until the number of feature points is satisfied. Oblique images of a 40-km2 area in Dongying city, China, were used for validation. The experimental results show that compared to the state-of-the-art method, the method proposed in this paper not only effectively reduces the number of feature points but also better reflects the scene structure. At the same time, the average reprojection error of the aerial triangulation decrease by 20%, the feature point matching rate increase by 3%, the selected feature points are more stable and reasonable.

A method of lining seam elimination with angle adaptation and rectangular mark for road tunnel concrete lining images

Road Tunnels are an important part of the current road transportation infrastructure. As the main form of tunnel lining diseases, cracks are easy to interact with other areas, which seriously affects the safe operation of the tunnel. Due to the similarity of brightness and linearity between surface cracks and lining cracks, the existing crack detection algorithms can not extract cracks accurately and quickly. An algorithm of lining seam crack elimination with rectangular mark is proposed here. First, the line segments in the image are detected by the Line Segment Detector algorithm based on the coarse percolation detection of the crack. Second, the distribution directions are calculated, and cracks from the lining seams are distinguished by the adaptive threshold judgment method. Third, by using the distribution characteristics of pixels, the line segments are extended to form rectangular marks perpendicular to the direction of lining seams. Finally, the marking information is used to remove the lining joints and obtain the real surface cracks of tunnel lining. Experimental results show that the algorithm can quickly and effectively remove any shape distribution of lining seam. The algorithm fills in the blank of concrete tunnel lining surface crack detection technology.

A method of detecting defects of smart meter LCD screen based on LSD and deep learning

In order to detect the defects of smart meter liquid crystal display (LCD) screen quickly and accurately, this paper proposes a method based on line segment detector (LSD) and deep learning. Characters of smart meter LCD screen can be divided into two types: ordinary characters and digital tube characters. Convolutional neural network (CNN) is used to locate and identify the ordinary characters. Since there may be certain missing segment in digital tube characters, conventional text detection methods are not applicable, and template matching method is used for detecting them. Compared with the whole screen, the digital tube area is very small, and the relative positions of the digital tube characters of smart meter of most manufacturers are nearly the same, so the detection rate of the template matching method is extremely high and the method can be commonly used. The whole detection process is as follows: First, horizontal straight lines got by LSD are used for tilt correction. Secondly, CNN is used to locate and identify an ordinary character of the meter screen to determine whether it is missing or not. Then, whether there are missing segments in the digital tube area is determined by template matching method. At last, the experimental results show that the accuracy of the method for detecting the character defects of LCD screen is about 96%, and the accuracy of the electric quantity identification function of the method is about 97%.

AG3line: Active grouping and geometry-gradient combined validation for fast line segment extraction

Line segment detectors based on local image domain passively fit a line segment from a set of pixels, but no constraint on line geometry is set in the grouping process. Therefore, unstable pixels, such as the pixels in grass, clouds, or weak gradient edges, may cause false positives and fractures. This paper proposes the detector named AG3line, which employs an efficient active grouping strategy. In AG3line, the pixel for the next grouping is calculated actively with the line geometry and it can even be accurate to one pixel. To reduce the fracture caused by unstable pixels, when the adjacent pixel cannot satisfy the grouping rules, the candidate pixels for the next grouping are expanded with the line geometry constraint. To furtherly control false positives, AG3line then validates and refines the line segments by exploiting both the line geometry and the alignment of gradient magnitude. When AG3line was evaluated utilizing the image dataset with the ground truth, it outperformed both the classical and the latest detectors.The implementation of AG3line is available at https://github.com/weidong-whu/AG3line.

LSD-based adaptive lane detection and tracking for ADAS in structured road environment

Lane recognition is important for safe driving in structured road environment; it is becoming an indispensable part of the advanced driver-assistance system (ADAS) for active security control. This paper proposes a novel lane detection and tracking approach for ADAS by using the line segment detector (LSD), adaptive angle filter and dual Kalman filter. In the lane detection process, the region of interest (ROI) within the inputted image is transformed into grayscale image, which is further preprocessed with median filtering, histogram equalization, image thresholding and perspective mapping. Then, the fast and robust LSD algorithm is applied on the ROI with the proposed adaptive angle filter to eliminate incorrect line segments more efficiently. In addition, a new three-level classifier based on the color, length and quantity of detected line segments is designed for lane classification. Finally, two Kalman filters are applied to track the detected lane and predict the following ROIs to improve the robustness and processing speed. The experimental results on four datasets show that the proposed method has robust performance in complex environments with the presence of shadows or other artifacts. It has average correct rate of lane detection higher than 94%, while the processing time is reduced by about 50% compared with a state-of-the-art method, and the average success rate of lane classification is above 85%.

Multiscale Detection of Circles, Ellipses and Line Segments, Robust to Noise and Blur

This paper proposes a basic taxonomy of image contours. Our goal is to classify smooth curves into five categories, namely, circles, ellipses, line segments, arcs of circles and arcs of ellipses. These geometrical structures have been chosen as they serve as input of many computer vision tasks. The proposed strategy is applied on a set of initial disjoint contours, which are grouped together to form the aforementioned structures. These, in turn, are validated using an a contrario approach that guarantees a reduced number of false detections. The use of a multiscale strategy permits the detection at different resolution levels, which makes the method robust to noise and blur.

A Sparse Branch and Bound Optimization of Noisy Weighted DAG Modification Under Constraints: A Method for Monocular Data Association to Multiple Laser Planes

The line segments of multiple (and nearly parallel) planar laser sources appearing in monocular video are objectively associated to their respective laser sources. Such association is a precursor to 3D surface digitization where each laser plane develops a 3D surface. In this work, the detected laser line segments are modeled as a weighted directed acyclic graph (wDAG), in which the graph nodes represent the line segments and their maximum path hopes from any source node represents the laser source label. However, due to noise and occlusions, the structure of the wDAG will normally vary from that expected in the ideal cases. Furthermore, it is essential to robustness to incorporate the association information of the previous frames in the new frame associations. The proposed work objectively modifies the current frame’s wDAG under these constraints so the resulting wDAG becomes in an appropriate structure for the labeling of the detected line segments. Branch and bound (BnB) optimization has been a good choice for global optimization of combinatorial problems. Rather than approaching the optimization as a pure (or full) combinatorial problem, the proposed prioritizations of the branch modification actions are nearly optimal. The high priority actions will very likely lead to an optimal modification. This aspect is exploited by the proposed sparse variant of BnB (SBnB) optimization. It significantly reduces the number of the branches to be considered, in addition to those contributed by the bounding part of the BnB optimization. The proposed approach was tested on numerous monocular videos and plausible association results were efficiently achieved.

Detection of Binary Square Fiducial Markers Using an Event Camera

Event cameras are a new type of image sensors that output changes in light intensity (events) instead of absolute intensity values. They have a very high temporal resolution and a high dynamic range. In this paper, we propose a method to detect and decode binary square markers using an event camera. We detect the edges of the markers by detecting line segments in an image created from events in the current packet. The line segments are combined to form marker candidates. The bit value of marker cells is decoded using the events on their borders. To the best of our knowledge, no other approach exists for detecting square binary markers directly from an event camera using only the CPU unit in real-time. Experimental results show that the performance of our proposal is much superior to the one from the RGB ArUco marker detector. The proposed method can achieve the real-time performance on a single CPU thread.

Airport Detection in SAR Images Via Salient Line Segment Detector and Edge-Oriented Region Growing

Airport detection in synthetic aperture radar (SAR) images has attracted much concern in the field of remote sensing. Affected by other salient objects with geometrical features similar to those of airports, traditional methods often generate false detections. In order to produce the geometrical features of airports and suppress the influence of irrelevant objects, we propose a novel method for airport detection in SAR images. First, a salient line segment detector is constructed to extract salient line segments in the SAR images. Second, we obtain the airport support regions by grouping these line segments according to the commonality of these geometrical features. Finally, we design an edge-oriented region growing (EORG) algorithm, where growing seeds are selected from the airport support regions with the help of edge information in SAR images. Using EORG, the airport region can be mapped by performing region growing with these seeds. We implement experiments on real radar images to validate the effectiveness of our method. The experimental results demonstrate that our method can acquire more accurate locations and contours of airports than several state-of-the-art airport detection algorithms.

Fast infrared horizon detection algorithm based on gradient directional filtration.

Infrared imaging has been widely used in the field of sea surface monitoring. Horizon detection is a key step before a target's detection, locating, and tracking in the sea-sky infrared scene. Reducing processing time while ensuring accuracy is the research focus of infrared horizon detection. This paper proposes a novel method of a line segment detector (LSD) algorithm with gradient direction filtering. First, the rough extraction of the sea-sky region is used to limit the size of the detected image, and then the potential horizon line segment is extracted, applying the improved LSD algorithm in the sea-sky region, which probably contains many false extraction results. Then, gradient direction filtering is designed to pick the horizon line segments in this step. Finally, the horizon line segments are stitched to obtain the whole horizon line based on random sample consensus. The results of the comparative experiments show that this novel method has high detection accuracy, and the processing time is significantly shortened; what is more, we can also conclude that this method has a good performance on the detection stability.

Rail Detection Based on LSD and the Least Square Curve Fitting

It is necessary to rely on the rail gauge to determine whether the object beside the track will affect train operation safety or not. A convenient and fast method based on line segment detector (LSD) and the least square curve fitting to identify the rail in the image is proposed in this paper. The image in front of the train can be obtained through the camera on-board. After preprocessing, it will be divided equally along the longitudinal axis. Utilizing the characteristics of the LSD algorithm, the edges are approximated into multiple line segments. After screening the terminals of the line segments, it can generate the mathematical model of the rail in the image based on the least square. Experiments show that the algorithm in this paper can fit the rail curve accurately and has good applicability and robustness.

Vanishing point detection using the teaching learning‐based optimisation algorithm

Within the computer vision field, estimating image vanishing points has many applications regarding robotic navigation, camera calibration, image understanding, visual measurement, 3D reconstruction, among others. Different methods for detecting vanishing points relies on accumulator space techniques, while others employ a heuristic approach such as RANSAC. Nevertheless, these types of methods suffer from low accuracy or high computational cost. To explore a different technique, this paper focuses on improving the efficiency of the metaheuristic search for vanishing points by using a recently proposed population-based method: The Teaching Learning Based Optimisation algorithm (TLBO). The TLBO algorithm is a metaheuristic technique inspired by the teaching–learning process. In our method, the TLBO algorithm is used after a line segment detection, to cluster line segments according to their more optimal vanishing point. Thus, our algorithm detects both orthogonal and nonorthogonal vanishing points in real images. To corroborate the performance of our proposed algorithm, different comparison and tests with other approaches were carried out. The results validate the accuracy and efficiency of our proposed method. Our approach had an average computational time of1.42 seconds and obtained a cumulative focal length error of 1 pixel, and cumulative angular error of 0.1°.