Line Segment Detector Research Articles

A High Precision Rotating Line Detection Method for the Rotation Angle Measurement Based on Machine Vision

The accuracy of the rotation angle measurement method based on machine vision with the line as the reference target is limited by line detection accuracy. The Line Segment Detector (LSD) method is a high precision and effective method, but there are detected false lines and discontinuous lines when the LSD method is actually used to detect the rotating lines in the images. In this paper, we adjust three parts of the LSD method: image scaling, gradient magnitude threshold, and aligned points density threshold to detect lines accurately. The high precision detection of the rotating line is obtained by our adjusted method. Besides, the rotation angle measurement accuracy with the lines detected by our adjusted LSD reaches the measurement error level of arcseconds, which proves its effective performance.

Automatic hotspots detection based on UAV infrared images for large‐scale PV plant

Owing to the significantly installed capacity of solar energy during the past decades, the operation and maintenance of the large photovoltaic power station is a big challenge, as the manual inspection is labour-intensive and expensive. This Letter presents a solution for the intelligent inspection of the hotspot with the unmanned aerial vehicle in the large-scale photovoltaic plant. First, a traditional image processing method is presented to eliminate the noise and crop the infrared image, which can make the defected feature more obvious. Then, the module is extracted by the line segments detection. Finally, considering the great advance in the realm of the computer vision, a deep-learning-based method for automatic hotspot detection is proposed for locating the hotspot with the data augmentation. The deep-learning-based model can extract the hotspot feature by the training process in the data set. In the end, the performance of the method proposed is extensively evaluated and the numerical results prove the accuracy and precision of the model.

Detection of Parking Slots Based on Mask R-CNN

Obtaining information on parking slots is a prerequisite for the development of automatic parking systems, which is an essential part of the automatic driving processes. In this paper, we proposed a parking-slot-marking detection approach based on deep learning. The detection process involves the generation of mask of the marking-points by using the Mask R-CNN algorithm, extracting parking guidelines and parallel lines on the mask using the line segment detection (LSD) to determine the candidate parking slots. The experimental results show that the proposed method works well under the condition of complex illumination and around-view images from different sources, with a precision of 94.5% and a recall of 92.7%. The results also indicate that it can be applied to diverse slot types, including vertical, parallel and slanted slots, which is superior to previous methods.

A Ship Draft Line Detection Method Based on Image Processing and Deep Learning

The traditional ship draft detection method mainly adopts the method of the human eye observation, which has the problems of large precision error and slow detection speed. Aiming at this problem, this paper proposes a ship draft line detection method based on image processing and deep learning. The color image is first transformed into a grayscale image, and the corner detection is carried out by Harris operator. Contour map is obtained using Canny operator for edge detection, and line segment detection is performed using Hough algorithm. The water surface is determined by calculating the lowest position of the corner point. Then the grayscale image is binarized to increase the contrast of the image, the digital area is determined by the corner positions of the highest and lowest points, and finally the convolutional neural network is used to perform character recognition and calculate the draft of the ship. Tested on videos of various scenes, the error of the proposed method is less than 1cm.

A Novel Water-Shore-Line Detection Method for USV Autonomous Navigation.

For the navigation of an unmanned surface vehicle (USV), detection and recognition of the water-shore-line (WSL) is an important part of its intellectualization. Current research on this issue mainly focuses on the straight WSL obtained by straight line fitting. However, the WSL in the image acquired by boat-borne vision is not always in a straight line, especially in an inland river waterway. In this paper, a novel three-step approach for WSL detection is therefore proposed to solve this problem through the information of an image sequence. Firstly, the initial line segment pool is built by the line segment detector (LSD) algorithm. Then, the coarse-to-fine strategy is used to obtain the onshore line segment pool, including the rough selection of water area instability and the fine selection of the epipolar constraint between image frames, both of which are demonstrated in detail in the text. Finally, the complete shore area is generated by an onshore line segment pool of multi-frame images, and the lower boundary of the area is the desired WSL. In order to verify the accuracy and robustness of the proposed method, field experiments were carried out in the inland river scene. Compared with other detection algorithms based on image processing, the results demonstrate that this method is more adaptable, and can detect not only the straight WSL, but also the curved WSL.

Persistence-based resolution-independent meshes of superpixels

The over-segmentation problem is to split a pixel-based image into a smaller number of superpixels that can be treated as indecompasable regions to speed up higher level image processing such as segmentation or object detection. A traditional superpixel is a potentially disconnected union of square pixels, which can have complicated topology (with holes) and geometry (highly zigzag boundaries). This paper contributes to new resolution-independent superpixels modeled as convex polygons with straight-line edges and vertices with real coordinates not restricted to a fixed pixel grid. Any such convex polygon can be rendered at any resolution higher than in original images, hence superpixels are resolution-independent.The key difficulty in obtaining resolution-independent superpixels is to find continuous straight-line edges, while classical edge detection focuses on extracting only discrete edge pixels. The recent Persistent Line Segment Detector (PLSD) avoids intersections and small angles between line segments, which are hard to fix before a proper polygonal mesh can be constructed. The key novelty is an automatic selection of strongest straight-line segments by using the concept of persistence from Topological Data Analysis, which allows to rank segments by their strength. The PLSD performed well in comparison with the only past Line Segment Detector Algorithm (LSDA) on the Berkeley Segmentation Database of 500 real-life images. The PLSD is now extended to the Persistent Resolution-Independent Mesh (PRIM).

Accurate Extraction Method for Structural Features of Building Facades Through Texture Fusion

Facade structural features can represent the overall framework of buildings. However, structural features extracted by the current methods contain a quantity of trivial unstructured information. In this study, we proposed an accurate extraction method for structural features of building facades through texture fusion. By performing texture fusion on building facade images, the interference of textural elements on structural feature extraction could be eliminated. After texture fusion, the line segment detector (LSD) algorithm is used to extract the structural features from the building facade images, and random sample consensus (RANSAC) is used to improve the continuity of structural features. The accuracy and effectiveness of the proposed method was demonstrated by comparing results with the state-of-the-art methods, such as LSD, MLSD, CannyLines, and MCMLSD. Value setting of three important parameters is discussed in detail. The imagery facade features extracted through the proposed method provide valuable support for many fields, such as image feature registration and 3D reconstruction of building surfaces.

PLSD: A Perceptually Accurate Line Segment Detection Approach

Most existing line segment detection methods suffer from the over-segmentation phenomenon. An improved line segment detection method is presented in this work, which can generate more and longer line segments, yet still accurately reflect the structural details of the image. Line segment grouping, line segment validation and a multiscale framework are adopted to reach this end. Specifically, smart grouping rules are introduced to locate potential homologous line segments (derived from the same boundaries). Novel merging criteria based on Helmholtz principle is then used to evaluate the meaningfulness between separate line segments and their merged ones. The improved multiscale framework facilitates line segments merging in detection and post-detection processes, yielding more high-quality line segments. Finally, the proposed method is compared with four leading methods on the famous public dataset, YorkUrban-LineSegment. Experimental results demonstrate that the method has good continuity and outperforms the leading methods in F-measure.

A NSRS Detection Method for Ablation Needles in CT Images

In CT image-guided interventional surgery, the rapid and accurate positioning of ablation needle plays a key role in the smooth operation. There is no universal and effective method for line segment detection, especially for the accurate extraction of feature points in image processing. Aiming at the problem of detecting and locating the ablation needle and its tip in CT image-guided interventional surgery, a method of NSRS line segment detection based on Randon transform and sliding window method is proposed. Experiments show that the method can accurately extract the end coordinates of the needle and accurately locate the ablation needle in CT image.

Airfield detection based on JPEG2000 compressed domain

With the rapid increase of optical remote sensing (RS) images, on-orbit specific target detection has been facing more and more challenges: limited processing environment, wireless bandwidth, space, power and storage. To resolve this issue, visual characteristics in the compressed domain are exploited to represent and recognise specific objects. In this study, a new saliency method and an airfield detection framework in the JPEG2000 compressed domain is proposed. First, the authors exploit the header information in raw JPEG2000 code streams to generate structural saliency map, and a great deal of non-structural regions in RS images are eliminated. Second, airfield candidates are extracted from low frequency wavelet sub-bands using line segment detection and the parallel density model. Finally, the geometrical properties of airfield candidates are represented by the speed-up robust features descriptor, and the support vector machine classifier is utilised to gain the final detection results. The proposed framework is evaluated on self-made dataset. Compared with the existing relevant state-of-the-art approaches, the proposed method reduces the processing time by 60% while guaranteeing the detection rate.

Automatic space object detection on all-sky images from a synoptic survey synthetic telescope array

Detection, classification, and characterization of Space Objects (SOs) are important tasks in many areas of research. Detection of SOs is important in predicting collisions with debris that could become hazards for satellites or space missions in Near Earth orbits. This paper describes a flexible pipeline able to detect sunlit SOs automatically in images acquired using an all-sky camera with a large field of view (FoV). The proposed pipeline includes the following main steps: image distortion correction, filtering for noise reduction, generation of a background model for subtraction, star elimination using a star catalog, local-based contrast enhancement, and, finally, for automatic SO detection, two methodologies were developed to detect line segments. The first one uses a Canny edge detector and a Progressive Probabilistic Hough Transform, and the second is based on the Radon Transform for detecting line segments. The method was applied to a dataset of 22 × 3 images obtained from the Omnidirectional Space Situational Awareness (OmniSSA) Array at the Georgia Institute of Technology in downtown Atlanta. The OmniSSA array has 3 sensors that capture high-resolution images simultaneously (3352 × 2532 pixels) using a wide FoV for each camera. An intensity scaled by noise (ISN) signal was defined and measured to show improvement in SO detection objectively. Fusing images from the three OmniSSA sensors after the background subtraction step improved both the ISN and visualization during the detection stage. Ground-truth data were extracted from a Space-Track catalog and marked by human experts to validate the results of the pipeline, considering information from Astrometry.net. Results showed that almost all the SOs were correctly detected by the pipeline.

LB-LSD: A length-based line segment detector for real-time applications

Line segments play an important role in the perception and representation of images by providing the geometry information about the scene. These segments can also be used as low-level features to analyze and detect more elaborated shapes. A length-based line segment detector is proposed in this paper. Based on the edge proportion statistics, an adaptive, robust and effective edge detection method is presented to extract edge segments from the image. Each of the segments is a clean, contiguous, 1-pixel wide chain of pixels. The line segment detector then approximates all the edge segments in a way that the curve satisfies a criterion of length condition. The detection result is a series of piecewise-linear segments with some dominant corners. Experimental results indicate that the proposed detector has a good detection accuracy and outperforms the state-of-the-art methods in terms of execution time.

LSDSAR, a Markovian a contrario framework for line segment detection in SAR images

In this paper, we propose a generic method for the detection of line segments in SAR images. The approach relies on an a contrario framework and is inspired by the state-of-the art LSD detector. As with all a contrarioapproaches, false detections are controlled through the use of a background model, whose development is especially challenging in the framework of SAR images. Indeed, statistical characteristics of SAR images strongly differ from those of optical images, making the use of existing background models intrinsically inadequate. In order to circumvent this problem, we proceed in two steps. First, the building blocks of the detector, namely the local orientations, are computed carefully to avoid any spatial bias. Second, we propose a new background model, in which the spatial dependency between local orientations are modeled with a Markov chain. This is in strong contrast with most existing a contrario methods who heavily rely on independence assumptions. We provide a complete and detailed algorithm for our line segment detector, and perform experiments on synthetic and real images demonstrating its efficiency.

A robust, real-time and calibration-free lane departure warning system

A real-time and calibration-free lane departure warning system algorithm is proposed. The pre-processing stage of the lane departure algorithm is carried out using Gaussian pyramid to smooth the image and reduce its dimensions, which decrease the unnecessary details in the image. A lane detection stage is then developed based on Edge Drawing Lines (EDLines) algorithm that is a real-time line segment detector, which has false detection control. The reference-counting technique is used to track the lane boundaries and predict the missing ones. Experimental results show that the proposed algorithm has accuracy of 99.36% and average processing time of 80 fps (frame per second). The proposed algorithm is efficient to be used in the self-driving systems in the Original Equipment Manufacturers (OEMs) cars.

Advanced Stereo Seam Carving by Considering Occlusions on Both Sides

Stereo image retargeting plays a significant role in the field of image processing, which aims at making major objects as prominent as possible when the resolution of an image is changed, including maintaining disparity and depth information at the same time. Some seam carving methods are proposed to preserve the geometric consistency of the images. However, the regions of occlusion on both sides are not considered properly. In this article, we propose a solution to solve this problem. A new strategy of seams finding is designed by considering occluded and occluding regions on both of the input images, and leaving geometric consistency in both images intact. We also introduced the method of line segment detection and superpixel segmentation to further improve the quality of the images. Imaging effects are optimized in the process and visual comfort, which is also influenced by other factors, can be boosted as well.

Automatic Building Outline Extraction from ALS Point Clouds by Ordered Points Aided Hough Transform

Many urban applications require building polygons as input. However, manual extraction from point cloud data is time- and labor-intensive. Hough transform is a well-known procedure to extract line features. Unfortunately, current Hough-based approaches lack flexibility to effectively extract outlines from arbitrary buildings. We found that available point order information is actually never used. Using ordered building edge points allows us to present a novel ordered points–aided Hough Transform (OHT) for extracting high quality building outlines from an airborne LiDAR point cloud. First, a Hough accumulator matrix is constructed based on a voting scheme in parametric line space (θ, r). The variance of angles in each column is used to determine dominant building directions. We propose a hierarchical filtering and clustering approach to obtain accurate line based on detected hotspots and ordered points. An Ordered Point List matrix consisting of ordered building edge points enables the detection of line segments of arbitrary direction, resulting in high-quality building roof polygons. We tested our method on three different datasets of different characteristics: one new dataset in Makassar, Indonesia, and two benchmark datasets in Vaihingen, Germany. To the best of our knowledge, our algorithm is the first Hough method that is highly adaptable since it works for buildings with edges of different lengths and arbitrary relative orientations. The results prove that our method delivers high completeness (between 90.1% and 96.4%) and correctness percentages (all over 96%). The positional accuracy of the building corners is between 0.2–0.57 m RMSE. The quality rate (89.6%) for the Vaihingen-B benchmark outperforms all existing state of the art methods. Other solutions for the challenging Vaihingen-A dataset are not yet available, while we achieve a quality score of 93.2%. Results with arbitrary directions are demonstrated on the complex buildings around the EYE museum in Amsterdam.

A Survey on Airport Detection on Remote Sensing Images using Deep Convolutional Neural Network

This survey investigated the use of deep convolutional neural networks (CNNs) in providing a solution for the problem of airport detection in remote sensing images (RSIs). In recent years, Deep CNNs is mostly used in many applications undertaken in the area of computer vision. Researchers generally approach airport detection as a pattern recognition problem, in which first various distinctive features are extracted, and then one of the classifier is adopted to detect airports. As per the research in various field CNNs not only ensure a tuned feature vector, but also yield better classification accuracy. The method proposed in this study first detects various regions on RSIs using Line Segment Detection algorithm and then uses these candidate regions to train CNN architecture with Matconv-net tool. The CNN model used has five convolution and three fully connected layers. Normalization and dropout layers were employed in order to build efficient architecture. A data augmentation strategy was used to reduce overfitting. Several experiments were performed to evaluate the performance of CNNs.

A SCENE-ASSISTED POINT-LINE FEATURE BASED VISUAL SLAM METHOD FOR AUTONOMOUS FLIGHT IN UNKNOWN INDOOR ENVIRONMENTS

Abstract. Since Global Navigation Satellite System may be unavailable in complex dynamic environments, visual SLAM systems have gained importance in robotics and its applications in recent years. The SLAM system based on point feature tracking shows strong robustness in many scenarios. Nevertheless, point features over images might be limited in quantity or not well distributed in low-textured scenes, which makes the behaviour of these approaches deteriorate. Compared with point features, line features as higher-dimensional features can provide more environmental information in complex scenes. As a matter of fact, line segments are usually sufficient in any human-made environment, which suggests that scene characteristics remarkably affect the performance of point-line feature based visual SLAM systems. Therefore, this paper develops a scene-assisted point-line feature based visual SLAM method for autonomous flight in unknown indoor environments. First, ORB point features and Line Segment Detector (LSD)-based line features are extracted and matched respectively to build two types of projection models. Second, in order to effectively combine point and line features, a Convolutional Neural Network (CNN)-based model is pre-trained based on the scene characteristics for weighting their associated projection errors. Finally, camera motion is estimated through non-linear minimization of the weighted projection errors between the correspondent observed features and those projected from previous frames. To evaluate the performance of the proposed method, experiments were conducted on the public EuRoc dataset. Experimental results indicate that the proposed method outperforms the conventional point-line feature based visual SLAM method in localization accuracy, especially in low-textured scenes.

Multi-Constrained Optimization Method of Line Segment Extraction Based on Multi-Scale Image Space

Image-based line segment extraction plays an important role in a wide range of applications. Traditional line segment extraction algorithms focus on the accuracy and efficiency, without considering the integrity. Serious line segmentation fracture problems caused by image quality will result in poor subsequent applications. To solve this problem, a multi-constrained line segment extraction method, based on multi-scale image space, is presented. Firstly, using Gaussian down-sampling with a classical line segment detection method, a multi-scale image space is constructed to extract line segments in each image scale and all line segments are projected onto the original image. Then, a new line segment optimization and purification strategy is proposed with the horizontal and vertical distances and angle geometric constraint relationships between line segments to merge fracture line segments and delete redundant line segments. Finally, line segments with adjacent positions are optimized using the grayscale constraint relationship, based on normalized cross-correlation similarity criterion for realizing the second optimization of fracture line segments. Compared with mainstream line segment detector and edge drawing lines methods, experimental results (i.e., indoor, outdoor, and aerial images) indicate the validity and superiority of our proposed methods which can extract longer and more complete line segments.

Line-CNN: End-to-End Traffic Line Detection With Line Proposal Unit

The task of traffic line detection is a fundamental yet challenging problem. Previous approaches usually conduct traffic line detection via a two-stage way, namely the line segment detection followed by a segment clustering, which is very likely to ignore the global semantic information of an entire line. To address the problem, we propose an end-to-end system called Line-CNN (L-CNN), in which the key component is a novel line proposal unit (LPU). The LPU utilizes line proposals as references to locate accurate traffic curves, which forces the system to learn the global feature representation of the entire traffic lines. We benchmark the proposed L-CNN on two public datasets including MIKKI and TuSimple, and the results suggest that L-CNN outperforms the state-of-the-art methods. In addition, L-CNN can run at approximately 30 f/s on a Titan X GPU, which indicates the practicability and effectiveness of L-CNN for real-time intelligent self-driving systems.