Keypoint Extraction Research Articles

Multi-Scale PIIFD for Registration of Multi-Source Remote Sensing Images

This paper aims at providing multi-source remote sensing images registered in geometric space for image fusion. Focusing on the characteristics and differences of multi-source remote sensing images, a feature-based registration algorithm is implemented. The key technologies include image scale-space for implementing multi-scale properties, Harris corner detection for keypoints extraction, and partial intensity invariant feature descriptor (PIIFD) for keypoints description. Eventually, a multi-scale Harris-PIIFD image registration algorithm framework is proposed. The experimental results of fifteen sets of representative real data show that the algorithm has excellent, stable performance in multi-source remote sensing image registration, and can achieve accurate spatial alignment, which has strong practical application value and certain generalization ability.

Algorithmic support of the system of external observation and routing of autonomous mobile robots

This article presents the algorithmic support of the external monitoring and routing system of autonomous mobile robots. In some cases, the practical usage of mobile robots is related to the solution of navigation problems. In particular, the position of ground robots can be secured using unmanned aerial vehicles. In the proposed approach based on the video image obtained from an external video camera located above the working area of mobile robots, the location of both robots and nearby obstacles is recognized. The optimal route to the target point of the selected robot is built, and changes in its working area are monitored. Information about the allowed routes of the robot is transmitted to third-party applications via network communication channels. Primary image processing from the camera includes distortion correction, contouring and binarization, which allows to separate image fragments containing robots and obstacles from background surfaces and objects. Recognition of robots in a video frame is based on the use of a SURF detector. This technology extracts key points in the video frame and compares them with key points of reference images of robots. Trajectory planning is implemented using Dijkstra’s algorithm. The discreteness of the trajectories obtained using the algorithm for finding a path on the graph can be compensated for on board autonomous mobile robots by using spline approximation. Experimental studies have confirmed the efficiency of the proposed approach both in the problem of recognition and localization of mobile robots and in the problem of planning safe trajectories.

Visual odometer method based on improved ORB feature

The original ORB feature detection algorithm has many problems, such as uneven distribution of feature points, many feature mismatches and poor robustness. A visual odometer method based on improved ORB is proposed. First of all, in order to solve the problem of low extraction efficiency and uneven distribution in the process of key points extraction, an improved quadtree algorithm is proposed which set an appropriate quadtree depth for the image pyramid layer to improve calculation efficiency and filter key points. Secondly, aiming at the problem of mismatching in the feature matching process of adjacent frames, the PROSAC algorithm improved based on RANSAC is used to sort all the sampling points according to the matching quality, and the matching point pairs with good quality are selected to improve the matching accuracy. The experimental results shows that the improved visual odometer method has higher accuracy and less calculation time than the original algorithm.

Marker-Less UAV-LiDAR Strip Alignment in Plantation Forests Based on Topological Persistence Analysis of Clustered Canopy Cover

A holistic strategy is established for automated UAV-LiDAR strip adjustment for plantation forests, based on hierarchical density-based clustering analysis of the canopy cover. The method involves three key stages: keypoint extraction, feature similarity and correspondence, and rigid transformation estimation. Initially, the HDBSCAN algorithm is used to cluster the scanned canopy cover, and the keypoints are marked using topological persistence analysis of the individual clusters. Afterward, the feature similarity is calculated by considering the linear and angular relationships between each point and the pointset centroid. The one-to-one feature correspondence is retrieved by solving the assignment problem on the similarity score function using the Kuhn–Munkres algorithm, generating a set of matching pairs. Finally, 3D rigid transformation parameters are determined by permutations over all conceivable pair combinations within the correspondences, whereas the best pair combination is that which yields the maximum count of matched points achieving distance residuals within the specified tolerance. Experimental data covering eighteen subtropical forest plots acquired from the GreenValley and Riegl UAV-LiDAR platforms in two scan modes are used to validate the method. The results are extremely promising for redwood and poplar tree species from both the Velodyne and Riegl UAV-LiDAR datasets. The minimal mean distance residuals of 31 cm and 36 cm are achieved for the coniferous and deciduous plots of the Velodyne data, respectively, whereas their corresponding values are 32 cm and 38 cm for the Riegl plots. Moreover, the method achieves both higher matching percentages and lower mean distance residuals by up to 28% and 14 cm, respectively, compared to the baseline method, except in the case of plots with extremely low tree height. Nevertheless, the mean planimetric distance residual achieved by the proposed method is lower by 13 cm.

UAV-lidar aids automatic intelligent powerline inspection

In recent decades, a substantial increase in electricity demand has put pressure on powerline systems to ensure an uninterrupted power supply. In order to prevent power failures, timely and thorough powerline inspections are needed to detect possible anomalies in advance. In the past few years, the emerging unmanned aerial vehicle (UAV)-mounted sensors (e.g. light detection and ranging/lidar, optical cameras, infrared cameras, and ultraviolet cameras) have provided rich data sources for comprehensive and accurate powerline inspections. A challenge that still hinders the use of UAVs in powerline inspection is that their operation is highly dependent on the pilot’s experience, which may pose risks to the safety of the powerline system and reduce inspection efficiency. An intelligent automatic inspection solution could overcome the limitations of current UAV-based inspection solutions. The main objective of this paper is to provide a contemporary look at the current state-of-the-art UAV-based inspections as well as to discuss a potential lidar-supported intelligent powerline inspection concept. Overall, standardized protocols for lidar-supported intelligent powerline inspections include four data analysis steps, i.e., point cloud classification, key point extraction, route generation, and fault detection. To demonstrate the feasibility of the proposed concept, we implemented a workflow using a dataset of 3536 powerline spans, showing that the inspection of a single powerline span could be completed in 10 min with only one or two technicians. This demonstrates that lidar-supported intelligent inspection can be used to inspect a powerline system with extremely high efficiency and low costs.

A novel copy-move forgery detection algorithm via two-stage filtering

With the wide application of simple image editing software, forgery images have become severe social problems with extremely damaging effects. The copy-move forgery technique is commonly used to temper an image by copying some regions of an image and pasting them somewhere in the same image. In this paper, a novel algorithm is proposed for the detection of copy-move forgery. First, the keypoints of the input image are computed using a keypoint extraction algorithm. Second, a keypoint-matching algorithm is used to match similar keypoints as the candidate keypoint pairs. Third, a proposed novel Two-Stage Filtering algorithm, including the Grid-Based Filter and the Clustering-Based Filter, is applied to filter out most of the false matching keypoint pairs. Subsequently, image matting is achieved by the Delaunay triangulation algorithm so that the marked areas indicate the forgery regions. Three copy-move forgery datasets are used to compare the performances of the proposed algorithm with some state-of-the-art algorithms. The experimental results demonstrate that the proposed algorithm's overall performance is superior to other solutions for detecting copy-move forgery images.

Lightweight Cross-Fusion Network on Human Pose Estimation for Edge Device

The deployment of human pose estimation on edge devices are essential task in computer vision. Due to memory and storage space limitations, it is difficult for edge devices to maintain implementing Convolutional Neural Networks, which deployed large-scale terminal platforms with abundant computing resources. This paper proposed novel Lightweight Cross-fusion Network on Human Pose Estimation with information sharing. Using state-of-the-art efficient neural architecture, and Ghost Net, as the backbone, which are gradually applying a cross-information fusion network for key points extraction in the baseline and strengthen phases. As a result, the computational cost significantly reduces, while maintaining feature confidence more accurate and predicting key points heatmaps more precisely. Our network model can entirely execute on edge devices, and extensive self-comparison experiments have evaluated the architecture’s effectiveness. The MS COCO 2017 dataset proved that the cross-fusion network is superior than other lightweight structures for pose estimation.

A Deep-Learning-Based Smart Healthcare System for Patient’s Discomfort Detection at the Edge of Internet of Things

The Internet of Things (IoT) widely supports the smart healthcare field; combined with computer vision, machine, and deep learning techniques; it provides fast and accurate services for automated patient discomfort monitoring/detection systems. Traditional patient monitoring systems are commonly composed of wearable sensors and vision-based methods. In this article, an IoT-based noninvasive automated patient's discomfort monitoring/detection system is presented and implemented, using a deep-learning-based algorithm. The system is based on an IP camera device; the patient's body's movement and posture are detected without using any wearable devices. The Mask-RCNN method is employed for the extraction of different key points on the patient body. These detected key points are then transformed into six major body organs using association rules of data mining. Furthermore, for analyzing the patient's discomfort, detected key point coordinates information is measured. Finally, the distance and the temporal threshold are applied to classify movements as either associated with normal or discomfort conditions. These key points information is also used to determine the postures of the patient lying on the bed. The patient's body position and posture are continuously monitored, based on which comfort and discomfort level are discriminated. For experimental evaluation, different video sequences are recorded covering two patient's beds. The experimental results show the proposed system's worth by achieving a true-positive rate of 94% and a false-positive rate of 7%.

A Survey on Applications of Hybrid PV/T Panels

Photovoltaic-thermal (PV/T) collectors have gained a lot of attention in recent years due to their substantial advantages as compared to ST or PV systems alone and even to other non-solar technologies. However, PV/Ts are still not as popular in industry or construction and they are not even known to major players implementing solar energy installations. In this article, a general presentation of PV/Ts and a review of their applications are given. First, different heat extraction media (e.g., air, water, bi-fluid, etc.) and hybrid design configurations of hybrid PV/T collectors are addressed. Next, the main applications of PV/T collectors are discussed in order to highlight their feasibility and usefulness and to raise awareness for adoption in the industry and buildings sector. Applications include desalination, air-conditioning, drying, trigeneration, etc. This paper should be considered as a reference form of PV/Ts to extract key points for future research and development as well as for other applications.

Robust and effective multiple copy-move forgeries detection and localization

Copy-move (or copy-paste or cloning) is one of the most common image forgeries, wherein one or more region are copied and pasted within the same image. The motivations of such forgery include hiding an element in the image or emphasizing a particular object. Copy-move image forgery is more challenging to detect than other types, such as splicing and retouching. In recent years, keypoint based copy-move forgery detection, which extracts image keypoints and uses local visual features to identify duplicated regions, exhibits remarkable performance with respect to memory requirement and robustness against various attacks. However, these approaches usually have poor detection ability when copy-move forgeries only involve small or smooth regions. Moreover, they cannot always effectively deal with multiple copy-move forgeries. To tackle these challenges, we propose a robust and effective multiple copy-move forgeries detection and localization method through adaptive keypoint extraction, robust local feature representation, and offsets clustering based post-processing. Firstly, we develop a new image keypoint detector, named generic features from accelerated segment test, and extract adaptively the uniform distribution keypoints from the forged image by employing the adaptive-thresholding and non-maximum suppression. Then, we introduce fast quaternion polar complex exponential transform to describe the image keypoints compactly and distinctively, and utilize the KD tree based K-nearest neighbor matching to find possible correspondences. Finally, the falsely matched pairs are removed by employing the offsets information based candidate clustering, and the duplicated regions are localized using RANSAC and ZNCC algorithm. We conduct extensive experiments to evaluate the performance of the proposed approach, in which encouraging results validate the effectiveness of the proposed technique, especially for plain/multiple copy-move forgeries, in comparison with the state-of-the-art approaches recently proposed in the literature.

Pose Measurement for Unmanned Aerial Vehicle Based on Rigid Skeleton

Pose measurement is a necessary technology for UAV navigation. Accurate pose measurement is the most important guarantee for a UAV stable flight. UAV pose measurement methods mostly use image matching with aircraft models or 2D points corresponding with 3D points. These methods will lead to pose measurement errors due to inaccurate contour and key feature point extraction. In order to solve these problems, a pose measurement method based on the structural characteristics of aircraft rigid skeleton is proposed in this paper. The depth information is introduced to guide and label the 2D feature points to eliminate the feature mismatch and segment the region. The space points obtained from the marked feature points fit the space linear equation of the rigid skeleton, and the UAV attitude is calculated by combining with the geometric model. This method does not need cooperative identification of the aircraft model, and can stably measure the position and attitude of short-range UAV in various environments. The effectiveness and reliability of the proposed method are verified by experiments on a visual simulation platform. The method proposed can prevent aircraft collision and ensure the safety of UAV navigation in autonomous refueling or formation flight.

Efficient Feature Extraction from Multispectral Images for Face Recognition Applications: A Deep Learning Approach

In recent years many face recognition algorithms were used for the identification and authentication of a person to a system. However, still, feature extraction from multispectral images was considered to be a challenging task. Feature extraction, including highlight location and portrayal, assumes a significant job in real-time security-based applications. In this paper, a novel Geometric Algebra-based Multivariate Regression Feature Extraction (GA-MVRFE) algorithm was proposed to extract features from a huge dataset stored in the cloud efficiently. This proposed algorithm works with the supreme expedient deep learning approach - Convolutional Neural Network (CNN) for image classification. CNN will automatically detect significant features from the multispectral images without any human intrusion from a huge database. Real-time images were captured with three different cameras and applied filters over the images and were created as a dataset. To show the competence of the proposed algorithm, an exclusively created dataset with a set of 14,400 image data was applied in the proposed and other existing algorithms, and their efficiency and robustness were noted. Providentially, GA-MVRFE produced better accuracy in ‘Face Recognition’ with a less time fraction compared with former algorithms. Obtained accuracy % for Geometric Algebra Oriented fast and Rotated Brief (GA-ORB), Geometric Algebra Fast Retina key-point Extraction Algorithm (GA-FREAK), Trilateral Smooth Filtering (TRSF), Cross Regression Multiple View Features extraction (CRMVF) and GA-MVRFE was 87.81, 83.23, 90.72, 91.67 and 97.57 respectively.

Fusion of urban 3D point clouds with thermal attributes using MLS data and TIR image sequences

Buildings take a large proportion of the total energy consumption in the city area in winter, therefore Thermal Infrared (TIR) images are widely used to evaluate the energy consumption and leakage of the building. To overcome the difficulties of image interpretation and occlusion in TIR images, utilizing thermal information in 3D structures that fuse TIR images with usable georeferenced 3D coordinates of buildings, is mandatory. To accomplish this mission, we propose a strategy to generate a thermal point cloud combining Mobile Laser Scanning (MLS) point clouds and TIR images. At first, a key-points extraction method based on line-intersection is presented, which helps to detect key-points from highly distorted TIR images. Then, the semi-automatic and the automatic correspondence determination algorithm based on restricted RANSAC for 6DOF pose estimation is proposed and tested. Finally, a non-local mean strategy for data fusion is applied, which integrates the thermal attributes from image sequences and the geometric property from the point clouds, resulting in a smooth representation of thermal point clouds with comprehensive radiance property for thermal analysis. The fusion result qualitatively presents thermal radiation of elements of buildings with detailed geometric information. The generated thermal point clouds contribute to evaluating the energy efficiency of buildings, or building blocks, analyzing the invisible building structure or pipelines for retrofit, and monitoring the energy usage of buildings for long-term development.

Psychological analysis of classroom learning based on face recognition and neural network

With the rapid development of deep learning and parallel computing, deep learning neural network based on big data has been applied to the field of facial nerve recognition. This innovative operation has attracted extensive attention of scholars. The reason why the application of neural network is realized lies in deep learning, which can reduce the error and change the weight by means of back propagation and error optimization, so as to extract more key points and features. In spite of this, data collection and key points extraction is still a very complex problem. This paper mainly aims at the above problems, studies the way of deep learning and information extraction and its internal structure, and optimizes its application to classroom learning, so as to provide effective help for the realization of distance education.

Efficient and Accurate Start Point Guiding and Seam Tracking Method for Curve Weld Based on Structure Light

Start point guiding and seam tracking are important parts of the robot’s intelligent welding process. However, some start point guiding methods have a large amount of calculation and low accuracy. In addition, due to the large curvature of the curved weld, the problem of visual advance in seam tracking should be solved. Therefore, an efficient and accurate start point guiding and seam tracking method for curve weld is proposed in this article. First, according to the characteristics of structural light of different welds, more efficient and accurate start point detection algorithms are proposed for curve v-groove welds and curve lap welds. The extraction of key characteristic points and the judgment of the relationship between them could achieve efficient weld start point detection, and the reasonable robot speed planning during the start point searching process could improve the accuracy of the start point guiding. Besides, a sliding data queue method with cubic B-spline fitting is proposed to overcome the visual advance problem and achieve accurate seam tracking of curve welds. The sliding data queue method can plan the data queue automatically, and improve the computational efficiency by reducing the amount of data to be processed. Cubic B-spline fitting can reduce the measurement error and eliminate the data fluctuation, so as to obtain a smooth welding trajectory. The experimental results show that the proposed method could achieve efficient and accurate start point guiding and seam tracking of curve v-groove weld and lap weld.

Space Target Attitude Estimation From ISAR Image Sequences With Key Point Extraction Network

Attitude determination of space target from an inverse synthetic aperture radar (ISAR) image sequence is an important but difficult task, because of the complex electromagnetic scattering characteristics of the target. To autonomously estimate the attitude of the space target from the ISAR image sequence, this letter introduces the key point feature extraction network (KPEN), and proposes a novel method for attitude determination via the key point feature extraction. The geometric relationship between the target attitude parameters and the key point feature is established through range-Doppler (RD) projection matrix. With the help of the key point extraction network, key point feature is extracted from ISAR image. Then the target attitude and its component size are autonomously recovered through key point feature by solving a nonlinear optimization. Extensive experiments show the effectiveness and superiority of the proposal.

Learning to Detect Local Features Using Information Change

In this paper we propose that features extracted from deep convolutional neural networks have the structure and information necessary to detect location and scale of the local keypoints. Unlike the previous supervised and unsupervised methods, we define a local feature as an outcome of information change across different receptive fields around an image region. Exploiting the existent representation hierarchy in the deep convectional neural network, we propose a trainable information accumulation pyramid that allows us to relate the change in the receptive field with information change. The network is trained in an unsupervised fashion by applying random set of transformations over the images and minimizing the covariant loss. We demonstrate the efficacy of our proposed keypoint extractor by evaluating its performance for repeatability and matching scores. Our approach results in 3.7% and 5.2% higher over the state-of-the-art algorithms in repeatability and matching score, respectively.

Camera pose estimation based on structure from motion

This work isdevoted to estimate the position and the orientation of the robot (this robot carries a camera) based on the images captured by this camera. This method includes two stages: learning and navigation. At the learning stage, the environment is represented by a cloud of points (3D points) before navigation, each 3D point is associated with a vector of 32 bytes describes the keypoint using ORB descriptor. At the navigation stage, a matching 2D/3D is made between the keypoints extracted from the 2D query image and the cloud of 3D points to estimate the position and orientation of the camera. A new classification for methods using vision-based navigation is presented and an algorithm based on Structure from Motion (SFM) method to estimate location of camera is proposed. This algorithm uses FAST detector to extract keypoints and uses binary local feature ORB descriptor to describe keypoints. These descriptors are compared using hamming distance for matching between corresponding points, so this algorithm is suitable for real time application. With practical implementation in indoor environment, the obtained position error is less than 5cm, whereas the angle error is less than 1.5 degrees.

EXTRACTING FEATURES OF THE HUMAN FACE FROM RGB-D IMAGES TO PLAN FACIAL SURGERIES

Biometric identification of the human face is a pervasive subject which deals with a wide range of disciplines such as image processing, computer vision, pattern recognition, artificial intelligence, and cognitive psychology. Extracting key face points for developing software and commercial devices of face surgery analysis is one of the most challenging fields in computer image and vision processing. Many studies have developed a variety of techniques to extract facial features from color and gray images. In recent years, using depth information has opened up new approaches to researchers in the field of image processing. Hence, in this study, a statistical method is proposed to extract key nose points from color-depth images (RGB-D) of the face front view. In this study, the Microsoft Kinect sensor is used to produce the face RGB-D images. To assess the capability of the proposed method, this algorithm is applied to 20 RGB-D face images from the database collected in the ICT lab of Sahand University of Technology and promising results are achieved for extracting key points of the face. The results of this study indicated that using the available information in two different color-depth bands could make key points of the face more easily accessible and bring better results and we can conclude that the proposed algorithm provided a promising outcome for extracting the positions of key points.

Composite attacks‐based copy‐move image forgery detection using AKAZE and FAST with automatic contrast thresholding

Copy-move image forgery is one of the most popular image tampering technique which can be performed for vicious purposes. In this forgery technique, selected region is copied and pasted at different locations on the same image to produce a manipulated image. Such forgery is denigratory as it can alter the image content by hiding or appending visual information. In this study, the authors propose a novel keypoint-based technique to detect forged images sustaining composite attacks consisting of various combinations of geometrical and post-processing operations. In the authors' method, AKAZE and FAST techniques are used to extract keypoints from the image. Non-maximal value suppression with automatic contrast thresholding is performed during FAST keypoint extraction. SIFT and DAISY descriptors are computed corresponding to extracted keypoints. PCA is applied over SIFT and DAISY descriptors to discard lower components which are sensitive to distortions occurred in images. They apply a correlation-based nearest neighbour search technique to detect similarity among keypoint descriptors. HDBSCAN algorithm is applied to obtain matched keypoint clusters. Further, RANSAC algorithm is utilised for removal of keypoint outliers. In comparison to state-of-the-art techniques, their approach achieve high F-measure (%) and low FPR (%) for image-level as well as pixel-level copy-move forgery detection.