Line Segment Features Research Articles

A Vehicle Monocular Ranging Method Based on Camera Attitude Estimation and Distance Estimation Networks

A monocular ranging method for forward vehicles in intelligent driving is proposed. This method measures vehicle distance more accurately under the condition of a single camera and can estimate camera attitude in real-time. For the estimation of camera pitch and yaw angles, it is achieved using road vanishing points. The images collected by the camera are sequentially processed through the Roberts operator amplitude calculation, feature point extraction, feature line segment generation, road vanishing point voting, and estimation of camera attitude to obtain pitch and yaw angles. A distance estimation network was designed, which is divided into multiple levels based on image size and incorporates image feature, integrating vehicle grounding point, and vehicle width information, effectively improving ranging accuracy. Finally, validation was conducted on KITTI data, with a relative error (AbsRel) of 8.3%. Additionally, the TuSimple dataset and continuous driving scenarios were also validated, resulting in improved performance compared to previous algorithms.

The Use of Computer Vision Technology in the Inheritance of Intangible Cultural Heritage: The Case of Regional Cultural Characteristics

Abstract The inheritance of intangible cultural heritage generally faces the situation of having no successors and difficulties in dissemination. The use of digital technology can overcome time and space limitations, expanding the dissemination effect of intangible heritage. A panoramic exhibition model of intangible cultural heritage is constructed using computer vision technology in this paper. In order to ensure that every pixel point in the image has a unique position, the perspective projection algorithm and the chi-square coordinate system are designed according to phase-forming geometry. Secondly, the optical information of each point is reproduced in the form of the image by using spherical projection, and the image alignment and image fusion smooth transition is carried out by improving the minimum deviation matching method of feature line segments based on chromatic aberration chunking. To convert the spherical panoramic image into a view that the user can explore, the inverse projection calculation is carried out. The performance of the model is tested after its construction is completed, and it is used to reproduce the panoramic view of the non-heritage contents of Xijiang Thousand Households Miaozhai in Guizhou. It is found that the matching error of this paper’s algorithm in the 20-frame scene test is close to or slightly higher than that of the SVD algorithm only in the 11th to 16th frame bits, and the rest of the frame errors are lower than that of the SVD algorithm by more than 0.04m. This paper’s algorithm is superior to OpenCV and Xiong methods by at least 0.1 points or more in all five scenes when testing the OG-IQA parameter. It can be concluded that the online exhibition content of non-heritage in Xijiang Miaozhai, Guizhou, produced using the model of this paper, has achieved a better communication effect and meets expectations. This paper provides innovative ideas and effective examples to demonstrate the use of computer vision technology in expanding the dissemination effect of intangible cultural heritage and promoting inheritance.

Enriching Representation and Enhancing Nearest Neighbor Classification of Slope/Landslide Data Using Rectified Feature Line Segments and Hypersphere-Based Scaling: A Reproducible Experimental Comparison

Measuring geotechnical and natural hazard engineering features, along with pattern recognition algorithms, allows us to categorize the stability of slopes into two main classes of interest: stable or at risk of collapse. The problem of slope stability can be further generalized to that of assessing landslide susceptibility. Many different methods have been applied to these problems, ranging from simple to complex, and often with a scarcity of available data. Simple classification methods are preferred for the sake of both parsimony and interpretability, as well as to avoid drawbacks such as overtraining. In this paper, an experimental comparison was carried out for three simple but powerful existing variants of the well-known nearest neighbor rule for classifying slope/landslide data. One of the variants enhances the representational capacity of the data using so-called feature line segments, while all three consider the concept of a territorial hypersphere per prototype feature point. Additionally, this experimental comparison is entirely reproducible, as Python implementations are provided for all the methods and the main simulation, and the experiments are performed using three publicly available datasets: two related to slope stability and one for landslide susceptibility. Results show that the three variants are very competitive and easily applicable.

Laser Radar Data Registration Algorithm Based on DBSCAN Clustering

At present, the core of lidar data registration algorithms depends on search correspondence, which has become the core factor limiting the performance of this kind of algorithm. For point-based algorithms, the data coincidence rate is too low, and for line-based algorithms, the method of searching the correspondence is too complex and unstable. In this paper, a laser radar data registration algorithm based on DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering is proposed, which avoids the search and establishment of the corresponding relationship. Firstly, a ring band filter is designed to process the outliers with noise in a point cloud. Then, the adaptive threshold is used to extract the line segment features in the laser radar point cloud. For the point cloud to be registered, a DBSCAN density clustering algorithm is used to obtain the key clusters of the rotation angle and translation matrix. In order to evaluate the similarity of the two frames of the point cloud in the key clusters after data registration, a kernel density estimation method is proposed to describe the registered point cloud, and K-L divergence is used to find the optimal value in the key clusters. The experimental results show that the proposed algorithm avoids the direct search of the correspondence between points or lines in complex scenes with many outliers in laser point clouds, which can effectively improve the robustness of the algorithm and suppress the influence of outliers on the algorithm. The relative error between the registration result and the actual value is within 10%, and the accuracy is better than the ICP algorithm.

Robust Extraction of 3D Line Segment Features from Unorganized Building Point Clouds

As one of the most common features, 3D line segments provide visual information in scene surfaces and play an important role in many applications. However, due to the huge, unstructured, and non-uniform characteristics of building point clouds, 3D line segment extraction is a complicated task. This paper presents a novel method for extraction of 3D line segment features from an unorganized building point cloud. Given the input point cloud, three steps were performed to extract 3D line segment features. Firstly, we performed data pre-processing, including subsampling, filtering and projection. Secondly, a projection-based method was proposed to divide the input point cloud into vertical and horizontal planes. Finally, for each 3D plane, all points belonging to it were projected onto the fitting plane, and the α-shape algorithm was exploited to extract the boundary points of each plane. The 3D line segment structures were extracted from the boundary points, followed by a 3D line segment merging procedure. Corresponding experiments demonstrate that the proposed method works well in both high-quality TLS and low-quality RGB-D point clouds. Moreover, the robustness in the presence of a high degree of noise is also demonstrated. A comparison with state-of-the-art techniques demonstrates that our method is considerably faster and scales significantly better than previous ones. To further verify the effectiveness of the line segments extracted by the proposed method, we also present a line-based registration framework, which employs the extracted 2D-projected line segments for coarse registration of building point clouds.

An Adaptive Threshold Line Segment Feature Extraction Algorithm for Laser Radar Scanning Environments

An accurate map is needed for the autonomous navigation of mobile robots in unknown environments. The application of laser radars has the advantages of high ranging accuracy and long ranging distances. Due to the small amount of data on laser radars and the influence of noise on the sensor itself, these amount to causing problems such as low accuracies of map construction and large positioning errors. Currently, the feature extraction of environmental line segments based on radar scanning data generally adopts the idea of recursion. However, the amount of calculations for applying recursion is large, and the threshold of extracted feature points needs to be set manually. Moreover, the fixed segmentation threshold will cause under-segmentation or over-segmentation. In this paper, an adaptive threshold-based feature extraction method for environmental line segments is proposed. The method denoises the original data first, and then an adaptive threshold of the nearest neighbor algorithm is provided to improve the accuracy of breakpoint judgment; next, the slope difference between adjacent line segments is evaluated according to the line segment fitting error in order to obtain the optimal corner feature. Finally, the point set is segmented to fit line-segment features. Based on actual environment tests, the environmental similarity of the line segment features extracted by the new algorithm in this paper increases by 8.3% compared with the IEPF (Iterative End Point Fit) algorithm. The algorithm avoids recursive operations, improves the efficiency by four times, and meets the real-time requirements of line segment fitting.

A global path planning algorithm based on the feature map

The feature map is a characteristic of high computational efficiency, but it is seldom used in path planning due to its lack of expression of environmental details. To solve this problem, a global path planning algorithm based on the feature map is proposed based on the directionality of line segment features. First, the robot searches the path along the direction of the target position but turns to search in the direction parallel to the obstacle, which it approaches until the line between the robot and the target position does not intersect with obstacles. Then it turns to the target position, keep searching the path. Meanwhile, the problems of the direction selection of turning point, corner point and obstacle circumvention in the searching process are analysed and corresponding solutions are put forth. Finally, a path optimisation algorithm with variable parameters is proposed, making the optimised path shorter and smoother. Simulation experiments demonstrate that the proposed algorithm is superior to A* algorithm in terms of computation time and path length, especially of the computation efficiency.

LSPP: A Novel Path Planning Algorithm Based on Perceiving Line Segment Feature

Path planning in the automatic driving system generally uses point features to evaluate the influence of obstacles on the robot, leading to missing information or increasing computational burden. This paper proposes a novel path planning with line segment (LSPP) algorithm from the viewpoint of line segment, which is easy to be obtained directly from sensors. In LSPP, an artificial potential field (APF) based on the azimuth and distance (ADAPF) from obstacles to the robot is first constructed for implementation of integral calculation. Unlike the traditional APF which only considers the nearest distance from obstacles to the robot, ADAPF can effectively assess the influence of boundary points on the robot by fully utilizing the information of obstacles. Based on ADAPF, boundary points are integrated into lines as edges of the drivable area in a specialized coordinate system. Then, a multi-loss function is designed to guarantee that the robot is far away from obstacles to the optimized path, and thus solved by a modified Dogleg (MDL) algorithm combined with symmetric rank-1 (SR1) method based on adaptive initial trust-region radii. Finally, a smooth and safe path is provided in LSPP with the requirements of speed and quality being satisfied simultaneously. Real-world experimental results validate the robustness and the effectiveness of LSPP.

A Novel Self-Positioning Based on Feature Map Creation and Laser Location Method for RBPF-SLAM

Aiming at the high computational complexity of the traditional Rao-Blackwellized Particle Filtering (RBPF) method for simultaneous localization and Mapping (SLAM), an optimization method of RBPF-SLAM system is proposed, which is based on lidar and least square line segment feature extraction as well as raster, reliability mapping continuity. Validation test results show that less storage in constructing a map with this method is occupied, and the computational complexity is significantly reduced. The effect of noise data on feature data extraction results is effectively avoided. It also solves the problem of error accumulation caused by noninteger grid size movement of unmanned vehicle in time update stage based on Markov positioning scheme. The improved RBPF-SLAM method can enable the unmanned vehicle to construct raster map in real time, and the efficiency and accuracy of map construction are significantly improved.

An automatic method for matching salient structures in optical remote sensing images

ABSTRACT Feature matching in optical remote sensing images usually suffers from significant intensity differences and background variations. In this paper, an effective image matching approach is proposed to match salient structures in optical remote sensing images. The proposed method consists of three main steps. In the first step, EDLines algorithm is implemented to extract straight lines from two corresponding images, and these lines are further processed and divided into line segments for feature matching. In the second step, a robust descriptor namely line segment context (LSC) descriptor is computed for each line segment. An iterative procedure that combines line segment description and feature matching is presented for accurate matching. Finally, the consistency check is implemented to further refine the matching results. We conduct the experiments on pairs of optical remote sensing images, and experimental results show that the proposed method can automatically and robustly generate line segment features and achieve favourable performance compared to existing image matching approaches.

A Monocular Visual Odometry Method Based on Virtual-Real Hybrid Map in Low-Texture Outdoor Environment.

With the extensive application of robots, such as unmanned aerial vehicle (UAV) in exploring unknown environments, visual odometry (VO) algorithms have played an increasingly important role. The environments are diverse, not always textured, or low-textured with insufficient features, making them challenging for mainstream VO. However, for low-texture environment, due to the structural characteristics of man-made scene, the lines are usually abundant. In this paper, we propose a virtual-real hybrid map based monocular visual odometry algorithm. The core idea is that we reprocess line segment features to generate the virtual intersection matching points, which can be used to build the virtual map. Introducing virtual map can improve the stability of the visual odometry algorithm in low-texture environment. Specifically, we first combine unparallel matched line segments to generate virtual intersection matching points, then, based on the virtual intersection matching points, we triangulate to get a virtual map, combined with the real map built upon the ordinary point features to form a virtual-real hybrid 3D map. Finally, using the hybrid map, the continuous camera pose estimation can be solved. Extensive experimental results have demonstrated the robustness and effectiveness of the proposed method in various low-texture scenes.

WGLSM: An end-to-end line matching network based on graph convolution

Line matching plays an essential role in Structure from Motion (SFM) and Simultaneous Localization and Mapping (SLAM), especially in low-texture scenes, where feature points are hard to be detected. In this paper, we present a new method by combining Convolutional Neural Networks and Graph Convolutional Networks to match line segments in pairs of images. We design a graph-based method to predict the assignment matrix of two feature sets with solving a relaxed optimal transport problem. In contrast to handcrafted line matching algorithms, our approach learns the line segment features and performs matching simultaneously through end-to-end weakly supervised training. The experiment results show that our method outperforms the state-of-the-art techniques and is robust to various image transformations. Besides, the generalization experiment illustrates that our method has good generalization ability without fine-tuning. The code of our work is available athttps://github.com/mameng1/GraphLineMatching.

Line-Segment Feature Extraction Algorithm for Input Dimensionality Reduction in Multi-Layer Perceptron Model

Background/Objectives: Today, the deep neural network has more intermediate layers along with learning data and has gradually become larger to draw a lower result due to the bound conventional neural network. Methods/Statistical analysis: Therefore, in this paper, we proposed the line-segment feature extraction for input dimensionality reduction in pattern mining. The proposed algorithm extracts the line segment information, constituting the image of input data, and assigns a unique value to each segment using predefined filters. Using such unique values to identify the number of line segments, create a one-dimensional vector, in the size of 256 or 512. Findings: This vector is used as the input data for a multi-layer perceptron (MLP). For performance evaluation, LFA was compared with principal component analysis (PCA). As a result, LFA-256 (at 96.81% accuracy) had −0.1% lower performance but a faster speed than PCA (96.91% accuracy), and LFA-512 (at 97.25% accuracy) had 0.35% higher performance than PCA. Improvements/Applications: We will study the recognition service possible to use in industry (mobile devices, PC, edge computing, etc.) and real-life through this algorithm.

Slam algorithm for multi-robot communication in unknown environment based on particle filter

To solve the SLAM (simultaneous localization and map building) of mobile robots, a multi-robot SLAM algorithm based on particle filter for communication in unknown environment is proposed. In the standard particle filter, the incremental map construction method based on point-line consistency is introduced to preserve the hypothesis of the line segment feature map in each particle, and the importance function is introduced to the observation information. On this basis, sampling map fusion algorithm extracts ORB (oriented FAST and rotated BRIEF) features of local map constructed by a single robot and finds the optimal matching points, calculates the homography matrix of local map optimal matching point set, determines the optimal affine transformation parameters among point sets, so as to complete the local map fusion, and realize the multi-robot communication based on SLAM in unknown environment. The results show that the algorithm achieves high-precision map fusion and it is an effective SLAM construction algorithm for multiple machines.

Robust 3D reconstruction from uncalibrated small motion clips

Small motion can be induced from burst video clips captured by a handheld camera when the shutter button is pressed. Although uncalibrated burst video clip conveys valuable parallax information, it generally has small baseline between frames, making it difficult to reconstruct 3D scenes. Existing methods usually employ a simplified camera parameterization process with keypoint-based structure from small motion (SFSM), followed by a tailored dense reconstruction. However, such SFSM methods are sensitive to insufficient or unreliable keypoint features, and the subsequent dense reconstruction may fail to recover the detailed surface. In this paper, we propose a robust 3D reconstruction pipeline by leveraging both keypoint and line segment features from video clips to alleviate the uncertainty induced by small baseline. A joint feature-based structure from small motion method is first presented to improve the robustness of the self-calibration with line segment constraints, and then, a noise-aware PatchMatch stereo module is proposed to improve the accuracy of the dense reconstruction. Finally, a confidence weighted fusion process is utilized to further suppress depth noise and mitigate erroneous depth. The proposed method can reduce the failure cases of self-calibration when the keypoints are insufficient, while recovering the detailed 3D surfaces. In comparison with state of the arts, our method achieves more robust and accurate 3D reconstruction results for a variety of challenging scenes.

Line-segment Feature Analysis Algorithm Using Input Dimensionality Reduction for Handwritten Text Recognition

Recently, demand for handwriting recognition, such as automation of mail sorting, license plate recognition, and electronic memo pads, has exponentially increased in various industrial fields. In addition, in the image recognition field, methods using artificial convolutional neural networks, which show outstanding performance, have been applied to handwriting recognition. However, owing to the diversity of recognition application fields, the number of dimensions in the learning and reasoning processes is increasing. To solve this problem, a principal component analysis (PCA) technique is used for dimensionality reduction. However, PCA is likely to increase the accuracy loss due to data compression. Therefore, in this paper, we propose a line-segment feature analysis (LFA) algorithm for input dimensionality reduction in handwritten text recognition. This proposed algorithm extracts the line segment information, constituting the image of input data, and assigns a unique value to each segment using 3 × 3 and 5 × 5 filters. Using the unique values to identify the number of line segments and adding them up, a 1-D vector with a size of 512 is created. This vector is used as input to machine-learning. For the performance evaluation of the method, the Extending Modified National Institute of Standards and Technology (EMNIST) database was used. In the evaluation, PCA showed 96.6% and 93.86% accuracy with k-nearest neighbors (KNN) and support vector machine (SVM), respectively, while LFA showed 97.5% and 98.9% accuracy with KNN and SVM, respectively.

Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions

As people communicate with each other, they use gestures and facial expressions as a means to convey and understand emotional state. Non-verbal means of communication are essential to understanding, based on external clues to a person’s emotional state. Recently, active studies have been conducted on the lifecare service of analyzing users’ facial expressions. Yet, rather than a service necessary for everyday life, the service is currently provided only for health care centers or certain medical institutions. It is necessary to conduct studies to prevent accidents that suddenly occur in everyday life and to cope with emergencies. Thus, we propose facial expression analysis using line-segment feature analysis-convolutional recurrent neural network (LFA-CRNN) feature extraction for health-risk assessments of drivers. The purpose of such an analysis is to manage and monitor patients with chronic diseases who are rapidly increasing in number. To prevent automobile accidents and to respond to emergency situations due to acute diseases, we propose a service that monitors a driver’s facial expressions to assess health risks and alert the driver to risk-related matters while driving. To identify health risks, deep learning technology is used to recognize expressions of pain and to determine if a person is in pain while driving. Since the amount of input-image data is large, analyzing facial expressions accurately is difficult for a process with limited resources while providing the service on a real-time basis. Accordingly, a line-segment feature analysis algorithm is proposed to reduce the amount of data, and the LFA-CRNN model was designed for this purpose. Through this model, the severity of a driver’s pain is classified into one of nine types. The LFA-CRNN model consists of one convolution layer that is reshaped and delivered into two bidirectional gated recurrent unit layers. Finally, biometric data are classified through softmax. In addition, to evaluate the performance of LFA-CRNN, the performance was compared through the CRNN and AlexNet Models based on the University of Northern British Columbia and McMaster University (UNBC-McMaster) database.

Research on Collision Avoidance Control of Multi-Arm Medical Robot Based on C-Space

A multi-arm robot for mandible reconstruction assisted surgery is developed. It has three manipulators and eighteen degrees of freedom. In order to prevent the collision of the manipulator caused by the overlap of the motion space of the manipulator during the space positioning of the whole mandible reconstruction operation, a collision avoidance path planning method for manipulator based on C-space is proposed. In order to obtain the space non-interference posture of one manipulator, and the interference posture of the other manipulators is taken as the obstacle mapping in C-space to obtain the free space after removing obstacles. At the same time, combining with the space position of the system manipulator, the C-space mapping mathematical model of feature points and line segments is studied, and the mapping mathematical model of any point and line segment in the plane is obtained. By introducing admissibility proof lemma to weigh the evaluation function, the improved heuristic A* search algorithm is used to search the best path for collision avoidance. The collision avoidance experiment is carried out through simulation and robot platform, and finally the animal experiment is carried out. These experimental results verify the feasibility and validity of the collision avoidance path planning method for the space manipulator.

Real-Time Streaming Image Based PP2LFA-CRNN Model for Facial Sentiment Analysis

In modern society, the real-time emotion adaptive driving system for providing safety to drivers and emotion-based services are being researched. However, in the service process have problem of personal information might get leaked. Therefore, a robust personal information protection method is required for face recognition services based on real-time images. In this study, we propose a real-time streaming image based PingPong256 (PP2) algorithm, line-segment feature analysis (LFA), convolutional recurrent neural network (CRNN) model for facial sentiment analysis. The proposed method applied the PP2 algorithm to images for encryption and decryption for the security of the real-time images collected by image devices. For transmitting images to a server, LFA, as a dimensionality reduction algorithm, is used to extract facial information. PP2 encrypts and decrypts an image through a linear feedback shift register with a different length and sets a random value other than 0 so that inferring the initial value of encryption becomes difficult, and then executes the random operations approximately 1,000 times. The LFA analyzes the line segments of an image, assigns a different unique number depending on its type, and cumulatively adds them to generate a Line-Segment map (LS-map) with a size of 16 × 16. The LS-map is used as an input of the CRNN model designed in this study, and the facial expressions are classified. Performance evaluation compares the accuracy of face recognition by using the proposed method with the loss rate for other models. Performance evaluation renders excellence to the accuracy of face recognition and loss rate comparison.

Least-square Matching for Mobile Robot SLAM Based on Line-segment Model

This study proposes an efficient simultaneous localization and mapping (SLAM) algorithm for a mobile robot. The proposed algorithm consists of line-segment feature extraction from a set of points measured by a LIDAR, association and matching between the line-segments and a map database for position estimation, and the registration of the line-segments into the map database for the incremental construction of the map database. The line-segment features help reduce the amount of data required for map representation. The matching algorithm for position estimation is efficient in computation owing to the use of a number of inliers as the weights in the least-squares method. Experiments are conducted to demonstrate the performance of the proposed SLAM algorithm, and the results show that the proposed algorithm is effective in the map representation and the localization of a mobile robot.