Line Segment Detector Algorithm Research Articles

Line segment detection algorithm in image extraction improvement study

In recent years, image processing technology has been developing and maturing, but due to the influence of many interfering factors in the acquisition process, there is a large amount of redundant information in the images obtained. The line segment detection algorithm in image extraction needs to be improved. This study utilizes computer technology to improve the line segment detection technology, and designs a line segment detection algorithm based on the linear detection improvement. Firstly, based on the basic principle of straight line detection algorithm, for the problems of line segment breakage and missing in straight line detection, RGB three-channel grayscale map is applied to detect line segments. Then the detected line segments are connected, merged and deleted. The test results show that the line segment detection algorithm improved based on straight line detection has the highest accuracy rate of 94.50 %, and the average processing time per image is also the lowest at 0.2 s. The algorithm runs faster at 0.25 s and has a higher F-value. It is able to detect the boundaries of a variety of rectangular targets, using the improved line segment detection algorithm has a wide range of applicability, lower error rate, and strong anti-interference ability. The improved line segment detection algorithm has a greater advantage in rectangular target extraction for document, text and book type images.

DeeplabV3+-based navigation line extraction for the sunlight robust combine harvester.

Visual navigation is widely used in intelligent combine harvesters, but the existing algorithms do not have sufficiently high accuracy of the visual navigation line recognition under different sunlight conditions. To address this problem, this article proposes a sunlight-robust DeepLabV3+-based navigation line extraction method for combine harvesters. The navigation lines are extracted by constructing a new dataset and predicting the boundaries of the areas that have been and have not been cut. To address the problem that DeeplabV3+ is not sufficient light in the DCNN part, improvement is proposed by incorporating the MobileNetV2 module. In image segmentation, the prediction time is 22.5 ms, and the mean intersection over union (FMIOU) is 0.79. After image segmentation, the navigation lines are drawn using the line segment detection algorithm for the harvester. The proposed method is compared with other mainstream networks, and the prediction results are compared using the line segment detection method. The results show that this method can more quickly identify the navigation lines under different conditions of sunlight with less labeled data than the improved U-Net and DeeplabV3+, which uses Xception as the backbone. Compared to the traditional method and the improved U-Net, this method achieves good results and improves the recognition speed by 27 and 9 ms, respectively.

DIG-SLAM: an accurate RGB-D SLAM based on instance segmentation and geometric clustering for dynamic indoor scenes

Simultaneous localization and mapping (SLAM) has emerged as a critical technology enabling robots to navigate in unknown environments, drawing extensive attention within the robotics research community. However, traditional visual SLAM ignores the presence of dynamic objects in indoor scenes, and dynamic point features of dynamic objects can lead to incorrect data correlation, making the traditional visual SLAM is difficult to accurately estimate the camera’s pose when the objects in the scenes are moving. Using only point features cannot fully extract geometric information in dynamic indoor scenes, reducing the system’s robustness. To solve this problem, we develop a RGB-D SLAM system called DIG-SLAM. Firstly, the objects’ contour regions are extracted using the YOLOv7 instance segmentation method, serving as a prerequisite for determining dynamic objects and constructing a semantic information map. Meanwhile, the line features are extracted using the line segment detector algorithm, and the redundant line features are optimized via K-means clustering. Secondly, moving consistency checks combined with instance partitioning determine dynamic regions, and the point and line features of the dynamic regions are removed. Finally, the combination of static line features and point features optimizes the camera pose. Meanwhile, a static semantic octree map is created to provide richer and higher-level scene understanding and perception capabilities for robots or autonomous systems. The experimental results on the Technische Universität München dataset show that the average absolute trajectory error of the developed DIG-SLAM is reduced by 28.68% compared with the dynamic semantic SLAM. Compared with other dynamic SLAM methods, the proposed system shows better camera pose estimation accuracy and system’s robustness in dynamic indoor environments and better map building in real indoor scenes.

An optimized design of the pointer meter image enhancement and automatic reading system in low illumination environment

As meter data becomes more and more important in the power industry, detection robots are led into substations for automatic collection of the pointer meters. However, the meter images captured in low illumination environments are unclear, resulting in poor recognition of the meter reading. A low-illumination image enhancement method based on virtual exposure is proposed in this paper, improving the dark and bright areas of low-illumination images, respectively. Then the image fusion was performed based on the Laplace pyramid to obtain clear meter images. In addition, the dial area was extracted using the Hough circle transform, and the pointer’s rotation center was fitted using the least squares approach. Finally, the straight line of the pointer was extracted, and the data reading was based on the line segment detector algorithm. Case studies show the above method has good robustness in low illumination environment, with high rate, and accuracy during the image enhancement and automatic reading of the pointer meter.

Fast Line Segment Detection and Large Scene Airport Detection for PolSAR

In this paper, we propose a fast Line Segment Detection algorithm for Polarimetric synthetic aperture radar (PolSAR) data (PLSD). We introduce the Constant False Alarm Rate (CFAR) edge detector to obtain the gradient map of the PolSAR image, which tests the equality of the covariance matrix using the test statistic in the complex Wishart distribution. A new filter configuration is applied here to save time. Then, the Statistical Region Merging (SRM) framework is utilized for the generation of line-support regions. As one of our main contributions, we propose a new Statistical Region Merging algorithm based on gradient Strength and Direction (SRMSD). It determines the merging predicate with consideration of both gradient strength and gradient direction. For the merging order, we set it by bucket sort based on the gradient strength. Furthermore, the pixels are restricted to belong to a unique region, making the algorithm linear in time cost. Finally, based on Markov chains and a contrario approach, the false alarm control of line segments is implemented. Moreover, a large scene airport detection method is designed based on the proposed line segment detection algorithm and scattering characteristics. The effectiveness and applicability of the two methods are demonstrated with PolSAR data provided by UAVSAR.

Multi-Angle Optical Image Automatic Registration by Combining Point and Line Features.

Image registration is an important basis of image processing, which is of great significance in image mosaicking, target recognition, and change detection. Aiming at the automatic registration problem of multi-angle optical images for ground scenes, a registration method combining point features and line features to register images is proposed. Firstly, the LSD (Line Segment Detector) algorithm is used to extract line features of images. The obtained line segments whose length are less than a given threshold are eliminated by a visual significant algorithm. Then, an affine transform model obtained by estimating a Gaussian mixture model (GMM) is applied to the image to be matched. Lastly, Harris point features are utilized in fine matching to overcome shortages of methods based on line features. In experiments, the proposed algorithm is compared with popular feature-based registration algorithms. The results indicate that the proposed algorithm in this work has obvious advantages in terms of registration accuracy and reliability for optical images acquired at different angles.

A Robust Method for Detecting Wind-Fallen Stems from Aerial RGB Images Using a Line Segment Detection Algorithm

Increased frequencies and windspeeds of storms may cause disproportionately high increases in windthrow damage. Storm-felled trees provide a surplus of breeding material for bark beetles, often resulting in calamities in the subsequent years. Thus, the timely removal of fallen trees is regarded as a good management practice that requires strategic planning of salvage harvesting. Precise information on the number of stems and their location and orientation are needed for the efficient planning of strip roads and/or cable yarding lines. An accurate assessment of these data using conventional field-based methods is very difficult and time-consuming; remote sensing techniques may be a cost-efficient alternative. In this research, a methodology for the automatic detection of fallen stems from aerial RGB images is presented. The presented methodology was based on a line segment detection algorithm and proved to be robust regarding image quality. It was shown that the method can detect frequency, position, spatial distribution and orientation of fallen stems with high accuracy, while stem lengths were systematically underestimated. The methodology can be used for the optimized planning of salvage harvesting in the future and may thus help to reduce consequential bark beetle calamities after storm events.

LSDNet: Trainable Modification of LSD Algorithm for Real-Time Line Segment Detection

As of today, the best accuracy in line segment detection (LSD) is achieved by algorithms based on convolutional neural networks – CNNs. Unfortunately, these methods utilize deep, heavy networks and are slower than traditional model-based detectors. In this paper we build an accurate yet fast CNN-based detector, LSDNet, by incorporating a lightweight CNN into a classical LSD detector. Specifically, we replace the first step of the original LSD algorithm – construction of line segments heatmap and tangent field from raw image gradients – with a lightweight CNN, which is able to calculate more complex and rich features. The second part of the LSD algorithm is used with only minor modifications. Compared with several modern line segment detectors on standard Wireframe dataset, the proposed LSDNet provides the highest speed (among CNN-based detectors) of 214 FPS with a competitive accuracy of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$78~F^{H}$ </tex-math></inline-formula> . Although the best-reported accuracy is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$83~F^{H}$ </tex-math></inline-formula> at 33 FPS, we speculate that the observed accuracy gap is caused by errors in annotations and the actual gap is significantly lower. We point out systematic inconsistencies in the annotations of popular line detection benchmarks – Wireframe and York Urban, carefully reannotate a subset of images and show that (i) existing detectors have improved quality on updated annotations without retraining, suggesting that new annotations correlate better with the notion of correct line segment detection; (ii) the gap between accuracies of our detector and others diminishes to negligible <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.2~F^{H}$ </tex-math></inline-formula> , with our method being the fastest.

FPGA-Based Linear Detection Algorithm of an Underground Inspection Robot

Conveyor belts are key pieces of equipment for bulk material transport, and they are of great significance to ensure safe operation. With the development of belt conveyors in the direction of long distances, large volumes, high speeds, and high reliability, the use of inspection robots to perform full inspections of belt conveyors has not only improved the efficiency and scope of the inspections but has also eliminated the dependence of the traditional method on the density of sensor arrangement. In this paper, relying on the wireless-power-supply orbital inspection robot independently developed by the laboratory, aimed at the problem of the deviation of the belt conveyor, the methods for the diagnosis of the deviation of the conveyor belt and FPGA (field-programmable gate array) parallel computing technology are studied. Based on the traditional LSD (line segment detection) algorithm, a straight-line extraction IP core, suitable for an FPGA computing platform, was constructed. This new hardware linear detection algorithm improves the real-time performance and flexibility of the belt conveyor diagnosis mechanism.

An Improved Method for Stable Feature Points Selection in Structure-from-Motion Considering Image Semantic and Structural Characteristics.

Feature matching plays a crucial role in the process of 3D reconstruction based on the structure from motion (SfM) technique. For a large collection of oblique images, feature matching is one of the most time-consuming steps, and the matching result directly affects the accuracy of subsequent tasks. Therefore, how to extract the reasonable feature points robustly and efficiently to improve the matching speed and quality has received extensive attention from scholars worldwide. Most studies perform quantitative feature point selection based on image Difference-of-Gaussian (DoG) pyramids in practice. However, the stability and spatial distribution of feature points are not considered enough, resulting in selected feature points that may not adequately reflect the scene structures and cannot guarantee the matching rate and the aerial triangulation accuracy. To address these issues, an improved method for stable feature point selection in SfM considering image semantic and structural characteristics is proposed. First, the visible-band difference vegetation index is used to identify the vegetation areas from oblique images, and the line feature in the image is extracted by the optimized line segment detector algorithm. Second, the feature point two-tuple classification model is established, in which the vegetation area recognition result is used as the semantic constraint, the line feature extraction result is used as the structural constraint, and the feature points are divided into three types. Finally, a progressive selection algorithm for feature points is proposed, in which feature points in the DoG pyramid are selected by classes and levels until the number of feature points is satisfied. Oblique images of a 40-km2 area in Dongying city, China, were used for validation. The experimental results show that compared to the state-of-the-art method, the method proposed in this paper not only effectively reduces the number of feature points but also better reflects the scene structure. At the same time, the average reprojection error of the aerial triangulation decrease by 20%, the feature point matching rate increase by 3%, the selected feature points are more stable and reasonable.

LSD-based adaptive lane detection and tracking for ADAS in structured road environment

Lane recognition is important for safe driving in structured road environment; it is becoming an indispensable part of the advanced driver-assistance system (ADAS) for active security control. This paper proposes a novel lane detection and tracking approach for ADAS by using the line segment detector (LSD), adaptive angle filter and dual Kalman filter. In the lane detection process, the region of interest (ROI) within the inputted image is transformed into grayscale image, which is further preprocessed with median filtering, histogram equalization, image thresholding and perspective mapping. Then, the fast and robust LSD algorithm is applied on the ROI with the proposed adaptive angle filter to eliminate incorrect line segments more efficiently. In addition, a new three-level classifier based on the color, length and quantity of detected line segments is designed for lane classification. Finally, two Kalman filters are applied to track the detected lane and predict the following ROIs to improve the robustness and processing speed. The experimental results on four datasets show that the proposed method has robust performance in complex environments with the presence of shadows or other artifacts. It has average correct rate of lane detection higher than 94%, while the processing time is reduced by about 50% compared with a state-of-the-art method, and the average success rate of lane classification is above 85%.

Fast infrared horizon detection algorithm based on gradient directional filtration.

Infrared imaging has been widely used in the field of sea surface monitoring. Horizon detection is a key step before a target's detection, locating, and tracking in the sea-sky infrared scene. Reducing processing time while ensuring accuracy is the research focus of infrared horizon detection. This paper proposes a novel method of a line segment detector (LSD) algorithm with gradient direction filtering. First, the rough extraction of the sea-sky region is used to limit the size of the detected image, and then the potential horizon line segment is extracted, applying the improved LSD algorithm in the sea-sky region, which probably contains many false extraction results. Then, gradient direction filtering is designed to pick the horizon line segments in this step. Finally, the horizon line segments are stitched to obtain the whole horizon line based on random sample consensus. The results of the comparative experiments show that this novel method has high detection accuracy, and the processing time is significantly shortened; what is more, we can also conclude that this method has a good performance on the detection stability.

Rail Detection Based on LSD and the Least Square Curve Fitting

It is necessary to rely on the rail gauge to determine whether the object beside the track will affect train operation safety or not. A convenient and fast method based on line segment detector (LSD) and the least square curve fitting to identify the rail in the image is proposed in this paper. The image in front of the train can be obtained through the camera on-board. After preprocessing, it will be divided equally along the longitudinal axis. Utilizing the characteristics of the LSD algorithm, the edges are approximated into multiple line segments. After screening the terminals of the line segments, it can generate the mathematical model of the rail in the image based on the least square. Experiments show that the algorithm in this paper can fit the rail curve accurately and has good applicability and robustness.

Persistence-based resolution-independent meshes of superpixels

The over-segmentation problem is to split a pixel-based image into a smaller number of superpixels that can be treated as indecompasable regions to speed up higher level image processing such as segmentation or object detection. A traditional superpixel is a potentially disconnected union of square pixels, which can have complicated topology (with holes) and geometry (highly zigzag boundaries). This paper contributes to new resolution-independent superpixels modeled as convex polygons with straight-line edges and vertices with real coordinates not restricted to a fixed pixel grid. Any such convex polygon can be rendered at any resolution higher than in original images, hence superpixels are resolution-independent.The key difficulty in obtaining resolution-independent superpixels is to find continuous straight-line edges, while classical edge detection focuses on extracting only discrete edge pixels. The recent Persistent Line Segment Detector (PLSD) avoids intersections and small angles between line segments, which are hard to fix before a proper polygonal mesh can be constructed. The key novelty is an automatic selection of strongest straight-line segments by using the concept of persistence from Topological Data Analysis, which allows to rank segments by their strength. The PLSD performed well in comparison with the only past Line Segment Detector Algorithm (LSDA) on the Berkeley Segmentation Database of 500 real-life images. The PLSD is now extended to the Persistent Resolution-Independent Mesh (PRIM).

A Survey on Airport Detection on Remote Sensing Images using Deep Convolutional Neural Network

This survey investigated the use of deep convolutional neural networks (CNNs) in providing a solution for the problem of airport detection in remote sensing images (RSIs). In recent years, Deep CNNs is mostly used in many applications undertaken in the area of computer vision. Researchers generally approach airport detection as a pattern recognition problem, in which first various distinctive features are extracted, and then one of the classifier is adopted to detect airports. As per the research in various field CNNs not only ensure a tuned feature vector, but also yield better classification accuracy. The method proposed in this study first detects various regions on RSIs using Line Segment Detection algorithm and then uses these candidate regions to train CNN architecture with Matconv-net tool. The CNN model used has five convolution and three fully connected layers. Normalization and dropout layers were employed in order to build efficient architecture. A data augmentation strategy was used to reduce overfitting. Several experiments were performed to evaluate the performance of CNNs.

Fast and Resource-Efficient Hardware Implementation of Modified Line Segment Detector

Lines are significant features enclosing high-level information in an image. The line segment Detector (LSD) Algorithm with low error rate is a widely used method to extract lines in images effectively and accurately. However, the algorithm on PC performs too costly both in time and resources for the real-time video processing. This paper provides a fast and resource-efficient hardware implementation solution for a modified LSD algorithm on Field Programmable Gate Arrays (FPGA) for real-time line detection. The task-level pipeline structures are exploited fully in a stream process mapped to the hardware architecture free of frame buffer. Our proposed hardware implementation processes in a stream-in–stream-out manner with little consumption of the on-chip block RAM to store intermediate values. We first employ hardware Gaussian filter and adjust Canny edge detection to obtain an edge map at single-pixel width. Then, a novel structure of region growing model based on dynamic rooted tree is used to detect line segment regions accurately with a latency of only a few rows of pixels. The low cost in time, on-chip resources, and power consumption makes our proposed algorithm suitable for portable real-time streaming video processing applications using line segment features, such as Lane departure warning systems. It can also be applied in real-time machine vision systems that use line segments information for further recognition or stereo correspondence and many others. The proposed algorithm is synthesized and tested on XC7Z020 FPGA with high reliability, accuracy speed, and low cost in both resources and energy.

Hyphae Detection in Fungal Keratitis Images With Adaptive Robust Binary Pattern

Fungal keratitis is an inflammation of the cornea that results from infection by fungal organisms. It has a high rate of blindness, which makes the accurate diagnosis of fungal keratitis important. Confocal microscopy is an optical imaging technique that assists doctors in diagnosing fungal keratitis, and cornea images obtained by confocal microscopy can be used to detect hyphae. The current challenges are how to classify normal cornea images with nerves and abnormal cornea images with hyphae and how to detect the hyphae in a complicated background. To address this problem, this paper proposes a novel automatic hyphae detection method that assists doctors in making diagnoses. It includes two primary steps: texture classification of images and hyphae detection. In texture classification step, first, after image enhancement using a subregional contrast stretching algorithm, an adaptive robust binary pattern (ARBP), which is an improvement on the adaptive median binary pattern (AMBP), is proposed and adopted to extract texture features; and a support vector machine model is used to classify the normal and abnormal images. In the hyphae detection step, binarization and a connected domain process are used to further enhance the targets, and a line segment detector algorithm is adopted to detect the hyphae; then, the hyphal density is defined to quantitatively evaluate the infection severity. The contributions of this study include the improvement of the AMBP and the design of a novel framework. ARBP can extract effective texture features of images with relatively bright and small targets. The experimental results demonstrate the effectiveness of the proposed framework.

Comparison of Pleiades and LiDAR Digital Elevation Models for Terraces Detection in Farmlands

Among the most evident anthropogenic modifications of the landscape, terraces related to agricultural activities are ubiquitous structures that constitute important investments worldwide, and they recently acquired a new relevance to modern concerns about land-use management and erosion control. Conservation agriculture and terraces management are an application with great potentialities for Satellite Earth observation and the derived high-resolution topography. Due to its high agility, the Pleiades satellite constellation provides new, high-resolution digital elevation models (DEMs) with a submetric resolution that could be potentially useful for this task, and their application in a farmland context is nowadays an open research line. This work provides a first analysis, performing an automatic terrace mapping from DEMs obtained from Pleiades images, as compared to LiDAR DEMs. Two existing methods are considered: 1) the fast line segment detector (LSD) algorithm and 2) a geomorphometric method based on surface curvature. Despite the lower performances of Pleiades DEMs with respect to that of the LiDAR models, the results indicate that the Pleiades models can be used to automatically detect terrace slopes greater than 2 m with a detection rate of more than 80% of the total length of the terraces. In addition, the results showed that when using noisy DEMs, the geomorphometric method is more robust, and it slightly outperforms the LSD algorithm. These results provide a first analysis on how effective Pleiades DEMs can be as an alternative to LiDAR DEMs, also highlighting the future challenges for monitoring large extents in a farmland context.

LSD Based Pose and Position Estimation for UAV Landing

Currently, there has been growing interest in unmanned aerial vehicle (UAV) during the landing. With the widespread use of the UAVs, a more precise estimation on pose and position in the process of landing is required to support the higher-level applications. In this paper, the estimation of pose and position based on the line segment detection (LSD) is proposed. By applying a vision camera, a landmark is detected using the effective LSD algorithm. Then a line-based vision model is built to calculate the pose and position of the UAV. Experimental results show that the state solutions of the proposed method are effective with different shape of landmarks, and the accuracy is minute-level in pose angle error and centimeter-level in position error.

Image processing technology of flaws within infrared transmitting glasses

We investigated image processing algorithms of the original infrared glass flaw image. Using the Laplacian edge enhancement following LSD (Line Segment Detector) algorithm, we can get a good flaw image very consistent with the original one. This study is very helpful to further enhance the infrared glass flaw inspection technique.