Feature Point Detection Research Articles

New Perspective to Isogeometric Analysis: Solving Isogeometric Analysis Problem by Fitting Load Function

Isogeometric analysis (IGA) is introduced to establish the direct link between computer-aided design and analysis. It is commonly implemented by Galerkin formulations (isogeometric Galerkin, IGA-G) through the use of nonuniform rational B-splines (NURBS) basis functions for geometric design and analysis. Another promising approach, isogeometric collocation (IGA-C), working directly with the strong form of the partial differential equation (PDE) over the physical domain defined by NURBS geometry, calculates the derivatives of the numerical solution at the chosen collocation points. In a typical IGA, the knot vector of the NURBS numerical solution is only determined by the physical domain. A new perspective on the IGA method is proposed in this study to improve the accuracy and convergence of the solution. Solving the PDE with IGA can be regarded as fitting the load function defined on the NURBS geometry (right-hand side) with derivatives of the NURBS numerical solution (left-hand side). Moreover, the design of the knot vector has a close relationship to the NURBS functions to be fitted in the area of data fitting in geometric design. Therefore, the detected feature points of the load function are integrated into the initial knot vector of the physical domain to construct the new knot vector of the numerical solution. Then, they are connected seamlessly with the IGA-C framework for its great potential combining the accuracy and smoothness merits with the computational efficiency, which we call isogeometric collocation by fitting load function (IGA-CL). In numerical experiments, we implement our method to solve 1D, 2D, and 3D PDEs and demonstrate the improvement in accuracy by comparing it with the standard IGA-C method. We also verify the superiority in the accuracy of our knot selection scheme when employed in the IGA-G method, which we call isogeometric Galerkin by fitting load function (IGA-GL).

Signal Feature Extraction of Music Melody Based on Deep Learning

In music composition, besides intended original orchestration, the main melody generally has a higher reproduction frequency. To better understand the content and idea of music works, this paper researches a novel method for extracting the features of music melody signals based on deep learning. At first, a supervised classification model is employed to select better features extracted from the raw data of music melody signals and create an optimal melody feature subset; then, the Temporal Convolution Network (TCN) is introduced to propose a new algorithm for detecting feature points of melody signals, and the detection principles are introduced in detail; after that, this paper elaborates on the melody signal feature point detection model built based on multi-branch and multi-task TCN, and gives the structures and work principles of the encoding module, decoding module, and mask estimation module of the TCN. At last, experimental results verify the validity of the proposed model.

Quantifying the Influence of Surface Texture and Shape on Structure from Motion 3D Reconstructions

In general, optical methods for geometrical measurements are influenced by the surface properties of the examined object. In Structure from Motion (SfM), local variations in surface color or topography are necessary for detecting feature points for point-cloud triangulation. Thus, the level of contrast or texture is important for an accurate reconstruction. However, quantitative studies of the influence of surface texture on geometrical reconstruction are largely missing. This study tries to remedy that by investigating the influence of object texture levels on reconstruction accuracy using a set of reference artifacts. The artifacts are designed with well-defined surface geometries, and quantitative metrics are introduced to evaluate the lateral resolution, vertical geometric variation, and spatial-frequency information of the reconstructions. The influence of texture level is compared to variations in capturing range. For the SfM measurements, the ContextCapture software solution and a 50 Mpx DSLR camera are used. The findings are compared to results using calibrated optical microscopes. The results show that the proposed pipeline can be used for investigating the influence of texture on SfM reconstructions. The introduced metrics allow for a quantitative comparison of the reconstructions at varying texture levels and ranges. Both range and texture level are seen to affect the reconstructed geometries although in different ways. While an increase in range at a fixed focal length reduces the spatial resolution, an insufficient texture level causes an increased noise level and may introduce errors in the reconstruction. The artifacts are designed to be easily replicable, and by providing a step-by-step procedure of our testing and comparison methodology, we hope that other researchers will make use of the proposed testing pipeline.

An Analysis of ML-Based Outlier Detection from Mobile Phone Trajectories

This paper provides an analysis of two machine learning algorithms, density-based spatial clustering of applications with noise (DBSCAN) and the local outlier factor (LOF), applied in the detection of outliers in the context of a continuous framework for the detection of points of interest (PoI). This framework has as input mobile trajectories of users that are continuously fed to the framework in close to real time. Such frameworks are today still in their infancy and highly required in large-scale sensing deployments, e.g., Smart City planning deployments, where individual anonymous trajectories of mobile users can be useful to better develop urban planning. The paper’s contributions are twofold. Firstly, the paper provides the functional design for the overall PoI detection framework. Secondly, the paper analyses the performance of DBSCAN and LOF for outlier detection considering two different datasets, a dense and large dataset with over 170 mobile phone-based trajectories and a smaller and sparser dataset, involving 3 users and 36 trajectories. Results achieved show that LOF exhibits the best performance across the different datasets, thus showing better suitability for outlier detection in the context of frameworks that perform PoI detection in close to real time.

Robust Keypoint Detection and Matching on Fisheye Images by Self-Supervised Learning.

Accurate image feature point detection and matching are essential to computer vision tasks such as panoramic image stitching and 3D reconstruction. However, ordinary feature point approaches cannot be directly applied to fisheye images due to their large distortion, which makes the ordinary camera model unable to adapt. To address such a problem, this paper proposes a self-supervised learning method for feature point detection and matching on fisheye images. This method utilizes a Siamese network to automatically learn the correspondence of feature points across transformed image pairs to avoid high annotation costs. Due to the scarcity of the fisheye image dataset, a two-stage viewpoint transform pipeline is also adopted for image augmentation to increase the data variety. Furthermore, this method adopts both deformable convolution and contrastive learning loss to improve the feature extraction and description of distorted image regions. Compared with traditional feature point detectors and matchers, this method has been demonstrated with superior performance on fisheye images.

Fast stitching method for multi-view images of cupping spots

Aiming at the problems of cupping machine camera with a small field of view, complete and high-definition cupping spot characteristics need to be monitored in real time, so as to effectively ensure that the cupping machine can accurately control the cupping time to avoid harm to the human body and extracting the complete back contour for automatic acupuncture point positioning, a fast stitching method for multi-view image of cupping spots is proposed. The present study uses linear transformation and Gaussian smoothing to preprocess the images to enhance image details, improve contrast, and reduce the influence of tank occlusion; Then, we combine the BRISK algorithm and the match factor to match the image and estimate the overlapping area through establishing a binary tree model so as to improve the efficiency of feature point detection and matching. Experimental results show that compared with the AutoStitch, the image stitched which is high definition has no obvious seams, ghosting, and distortion. What’s more, the algorithm in this paper is more efficient and the back contour is more complete. The algorithm in this paper has good real-time performance and the clarity of the cupping spots is high, which is conducive to the subsequent real-time detection of the cupping spots. At the same time, the back contour of the image stitched by the algorithm in this paper is complete, which is conducive to the subsequent automatic acupuncture point positioning.

Cloud-based cross-platform collaborative augmented reality in flutter

Augmented Reality (AR) as a technology in the business area is utilized in new frontiers such as collaborative real-time experiences and cloud-based solutions. However, there is still a strong tendency towards game engines, which hinders widespread adoption for businesses. We present a collaborative AR framework (Flutter plugin) aimed at lowering the entry barriers and operating expenses of AR applications. A cross-platform and cloud-based solution combined with a web-based content management system (cloud) is a powerful tool for non-technical staff to take over operational tasks such as providing 3D models or moderating community annotations. To achieve cross-platform support, the AR Flutter plugin builds upon ARCore (Android) and ARKit (iOS) and unifies the two frameworks using an abstraction layer written in Dart. In this extensive description we present an in-depth summary of the concepts to realize the framework and prove its performance being on the same level as the native AR frameworks. This includes application-level metrics like CPU and RAM consumption and tracking-level qualities such as keyframes per second used by the underlying SLAM algorithm, detected feature points, and area of tracked planes. Our contribution closes a gap in today’s technological landscape by providing an AR framework with the familiar development process of cross-platform apps. Building upon on a content management system (cloud) and AR can be a game changer to achieve business objectives, while being not restrained to stand-alone single-purpose apps. This will trigger a potential paradigm shift for previously complex-to-realize applications relying on AR, e.g., in production and planning. The AR Flutter plugin is fully open-source, the code can be found at: https://github.com/CariusLars/ar_flutter_plugin.

Vehicular Electronic Image Stabilization System Based on a Gasoline Model Car Platform

Noise, vibration and harshness (NVH) problems in vehicle engineering are always challenging in both traditional vehicles and intelligent vehicles. Although high accuracy manufacturing, modern structural roads and advanced suspension technology have already significantly reduced NVH problems and their impacts; off-road condition, obstacles and extreme operating condition could still trigger NVH problems unexpectedly. This paper proposes a vehicular electronic image stabilization (EIS) system to solve the vibration problem of the camera and ensure the environment perceptive function of vehicles. Firstly, feature point detection and matching based on an oriented FAST and rotated BRIEF (ORB) algorithm are implemented to match images in the process of EIS. Furthermore, a novel improved random sampling consensus algorithm (i-RANSAC) is proposed to eliminate mismatched feature points and increase the matching accuracy significantly. And an adaptive Kalman filter (AKF) is applied to improve the adaptability of the vehicular EIS. Finally, an experimental platform based on a gasoline model car was established to validate its performance. The experimental results show that the proposed EIS system can satisfy vehicular performance requirements even under off-road condition with obvious obstacles.

Performance Comparison of Feature Detectors on Various Layers of Underwater Acoustic Imagery

Image feature matching is essential in many computer vision applications, and the foundation of matching is feature detection, which is a crucial feature quantification process. This manuscript focused on detecting more features from underwater acoustic imageries for further ocean engineering applications of autonomous underwater vehicles (AUVs). Currently, the mainstream feature detection operators are developed for optical images, and there is not yet a feature detector oriented to underwater acoustic imagery. To better analyze the suitability of existing feature detectors for acoustic imagery and develop an operator that can robustly detect feature points in underwater imageries in the future, this manuscript compared the performance of well-established handcrafted feature detectors and that of the increasingly popular deep-learning-based detectors to fill the gap in the literature. The datasets tested are from the most commonly used side-scan sonars (SSSs) and forward-looking sonars (FLSs). Additionally, the detection idea of these detectors on the acoustic imagery phase congruency (PC) layer was innovatively proposed with the aim of finding a solution that balances detection accuracy and speed. The experimental results show that the ORB (Oriented FAST and Rotated BRIEF) and BRISK (Binary Robust Invariant Scalable Keypoints) detectors achieve the best overall performance, the FAST detector is the fastest, and the PC and Sobel layers are the most favorable for implementing feature detection.

A Fast Point Clouds Registration Algorithm Based on ISS-USC Feature for the 3D Laser Scanner

The point clouds registration is a key step in data processing for the 3D laser scanner to obtain complete information of the object surface, and there are many algorithms. In order to overcome the disadvantages of slow calculation speed and low accuracy of existing point clouds registration algorithms, a fast point clouds registration algorithm based on the improved voxel filter and ISS-USC feature is proposed. Firstly, the improved voxel filter is used for down-sampling to reduce the size of the original point clouds data. Secondly, the intrinsic shape signature (ISS) feature point detection algorithm is used to extra feature points from the down-sampled point clouds data, and then the unique shape context (USC) descriptor is calculated to describe the extracted feature points. Next, the improved random sampling consensus (RANSAC) algorithm is used for coarse registration to obtain the initial position. Finally, the iterative closest point (ICP) algorithm based on KD tree is used for fine registration, which realizes the transform from the point clouds scanned by the 3D laser scanner at different angles to the same coordinate system. Through comparing with other algorithms and the registration experiment of the VGA connector for monitor, the experimental results verify the effectiveness and feasibility of the proposed algorithm, and it has fastest registration speed while maintaining high registration accuracy.

An Improved ASIFT Image Feature Matching Algorithm Based on POS Information.

The affine scale-invariant feature transform (ASIFT) algorithm is a feature extraction algorithm with affinity and scale invariance, which is suitable for image feature matching using unmanned aerial vehicles (UAVs). However, there are many problems in the matching process, such as the low efficiency and mismatching. In order to improve the matching efficiency, this algorithm firstly simulates image distortion based on the position and orientation system (POS) information from real-time UAV measurements to reduce the number of simulated images. Then, the scale-invariant feature transform (SIFT) algorithm is used for feature point detection, and the extracted feature points are combined with the binary robust invariant scalable keypoints (BRISK) descriptor to generate the binary feature descriptor, which is matched using the Hamming distance. Finally, in order to improve the matching accuracy of the UAV images, based on the random sample consensus (RANSAC) a false matching eliminated algorithm is proposed. Through four groups of experiments, the proposed algorithm is compared with the SIFT and ASIFT. The results show that the algorithm can optimize the matching effect and improve the matching speed.

Joint Detection and Matching of Feature Points in Multimodal Images.

In this work, we propose a novel Convolutional Neural Network (CNN) architecture for the joint detection and matching of feature points in images acquired by different sensors using a single forward pass. The resulting feature detector is tightly coupled with the feature descriptor, in contrast to classical approaches (SIFT, etc.), where the detection phase precedes and differs from computing the descriptor. Our approach utilizes two CNN subnetworks, the first being a Siamese CNN and the second, consisting of dual non-weight-sharing CNNs. This allows simultaneous processing and fusion of the joint and disjoint cues in the multimodal image patches. The proposed approach is experimentally shown to outperform contemporary state-of-the-art schemes when applied to multiple datasets of multimodal images. It is also shown to provide repeatable feature points detections across multi-sensor images, outperforming state-of-the-art detectors. To the best of our knowledge, it is the first unified approach for the detection and matching of such images.

68 Face Feature Points Detection Based on Cascading Convolutional Neural Network with Small Filter

Facial detection has received more and more attention in the past two decades. Due to the pose, occlusion, and illumination changes in the photo, the detection task is quite a challenge in an unconstrained environment. This paper proposed a cascaded convolutional neural network DCNNSF-CFC (Deep Convolution Neural Network with Small Filter-with Coarse-to-Fine Cascade) to localize large facial landmarks to improve the accuracy and robustness of network prediction which based on the original small filter deep convolutional neural network. Each network is trained separately to locally refine a subset of facial landmarks generated at the previous network level, and each geometric network is explicitly constrained to modify the input information of the current network. AFW, LFPW and Helen datasets were used in this study and it was proved to be more accurate and robust than DRMF, Opm, CNN and RCPR.

An effective feature detection approach for image stitching of near-uniform scenes

The first step to accomplish panoramic or omnidirectional image stitching often rely on how well the feature detector extract sufficient number of matchable corresponding interest points between images. This step remains a challenging problem to many renowned feature detectors, especially when the image content is fairly near-uniform. The awful impacts are often reflected in obvious misaligned and projective distortion. To address this problem, we propose an effective feature detection method based on Log-Gabor feature transform. Instead of depending on image intensity for feature point detection, the proposed method utilizes log-Gabor function to construct a number of scaled and oriented log-Gabor scale spaces from which more distinctive and matchable features can be extracted effectively. In addition, we also introduce new evaluation metrics which are more relevant for evaluating the performance of image stitching. Our method has a better feature matching performance than other state-of-the-art feature extraction methods, particularly for near-uniform scenes.

Pulse Signal Analysis Based on Deep Learning Network.

Pulse signal is one of the most important physiological features of human body, which is caused by the cyclical contraction and diastole. It has great research value and broad application prospect in the detection of physiological parameters, the development of medical equipment, and the study of cardiovascular diseases and pulse diagnosis objective. In recent years, with the development of the sensor, measuring and saving of pulse signal has become very convenient. Now the pulse signal feature analysis is a hotspot and difficulty in the signal processing field. Therefore, to realize pulse signal automatic analysis and recognition is vital significance in the aspects of the noninvasive diagnosis and remote monitoring, etc. In this article, we combined the pulse signal feature extraction in time and frequency domain and convolution neural network to analyze the pulse signal. Firstly, a theory of wavelet transform and the ensemble empirical mode decomposition (EEMD) which is gradually developed in recent years have been used to remove the noises in the pulse signal. Moreover, a method of feature point detection based on differential threshold method is proposed which realized the accurate positioning and extraction time-domain values. Finally, a deep learning method based on one-dimensional CNN has been utilized to make the classification of multiple pulse signals in the article. In conclusion, a deep learning method is proposed for the pulse signal classification combined with the feature extraction in time and frequency domain in this article.

DeepHistory: A convolutional neural network for automatic animation of museum paintings

AbstractDeep learning models have shown that it is possible to train neural networks to dispense, to a lesser or greater extent, with the need for human intervention for the task of image animation, which helps to reduce not only the production time of these audiovisual pieces, but also presents benefits with respect to the economic investment they require to be made. However, these models suffer from two common problems: the animations they generate are of very low resolution and they require large amounts of training data to generate good results. To deal with these issues, this article introduces the architectural modification of a state‐of‐the‐art image animation model integrated with a video super‐resolution model to make the generated videos more visually pleasing to viewers. Although it is possible to train the animation models with higher resolution images, the time it would take to train them would be much longer, which does not necessarily benefit the quality of the animation, so it is more efficient to complement it with another model focused on improving the animation resolution of the generated video as we demonstrate in our results. We present the design and implementation of a convolutional neural network based on an state‐of‐art model focused on the image animation task, which is trained with a set of facial data from videos extracted from the YouTube platform. To determine which of all the modifications to the selected state‐of‐the‐art model architecture is better, the results are compared with different metrics that evaluate the performance in image animation and video quality enhancement tasks. The results show that modifying the architecture of the model focused on the detection of characteristic points significantly helps to generate more anatomically and visually attractive videos. In addition, perceptual testing with users shows that using a super‐resolution video model as a plugin helps generate more visually appealing videos.

ECFRNet: Effective corner feature representations network for image corner detection

Interest points (corners and blobs) play an important role in computer vision tasks such as image matching, image retrieval, and 3D reconstruction. Existing deep learning based interest point detection methods mainly focus on the interest point detection with high repeatability under image affine transformations while neglecting the importance of the characteristics of interest points. This will affect the detection and localization accuracy of interest points. In this paper, we design an effective corner feature representations network based on the characteristics of corners. The designed network has the ability to effectively learn corner feature information from images. A novel loss function is proposed to minimize the localization error between the corner positions of the original image block and the transformed image blocks. Furthermore, a novel corner detection architecture is proposed. The criteria on detection accuracy, localization accuracy, average repeatability, region repeatability, and image matching score are used to evaluate the proposed method against fourteen state-of-the-art methods. The experimental results show that the proposed performs significantly better than the state-of-the-arts.

Improvement of Image Stitching Using Binocular Camera Calibration Model

Image stitching is the process of stitching several images that overlap each other into a single, larger image. The traditional image stitching algorithm searches the feature points of the image, performs alignments, and constructs the projection transformation relationship. The traditional algorithm has a strong dependence on feature points; as such, if feature points are sparse or unevenly distributed in the scene, the stitching will be misaligned or even fail completely. In scenes with obvious parallaxes, the global homography projection transformation relationship cannot be used for image alignment. To address these problems, this paper proposes a method of image stitching based on fixed camera positions and a hierarchical projection method based on depth information. The method does not depend on the number and distribution of feature points, so it avoids the complexity of feature point detection. Additionally, the effect of parallax on stitching is eliminated to a certain extent. Our experiments showed that the proposed method based on the camera calibration model can achieve more robust stitching results when a scene has few feature points, uneven feature point distribution, or significant parallax.

A method for sperm activity analysis based on feature point detection network in deep learning

Sperm motility is an important index to evaluate semen quality. Computer-assisted sperm analysis (CASA) is based on the sperm image, through the image-processing algorithm to detect the position of the sperm target and track tracking, so as to judge the sperm activity. Because of the small and dense sperm targets in sperm images, traditional image-processing algorithms take a long time to detect sperm targets, while target-detection algorithms based on the deep learning have a lot of missed detection problems in the process of sperm target detection. In order to accurately and efficiently analyze sperm activity in the sperm image sequence, this article proposes a sperm activity analysis method based on the deep learning. First, the sperm position is detected through the deep learning feature point detection network based on the improved SuperPoint, then the multi-sperm target tracking is carried out through SORT and the sperm motion trajectory is drawn, and at last the sperm survival is judged through the sperm trajectory to realize the analysis of sperm activity. The experimental results show that this method can effectively analyze the sperm activity in the sperm image sequence. At the same time, the average detection speed of the sperm target detection method in the detection process is 65fps, and the detection accuracy is 92%.

Recognition and Depth Estimation of Ships Based on Binocular Stereo Vision

To improve the navigation safety of inland river ships and enrich the methods of environmental perception, this paper studies the recognition and depth estimation of inland river ships based on binocular stereo vision (BSV). In the stage of ship recognition, considering the computational pressure brought by the huge network parameters of the classic YOLOv4 model, the MobileNetV1 network was proposed as the feature extraction module of the YOLOv4 model. The results indicate that the mAP value of the MobileNetV1-YOLOv4 model reaches 89.25%, the weight size of the backbone network was only 47.6 M, which greatly reduced the amount of computation while ensuring the recognition accuracy. In the stage of depth estimation, this paper proposes a feature point detection and matching algorithm based on the ORB algorithm at sub-pixel level, that is, firstly, the FSRCNN algorithm was used to perform super-resolution reconstruction of the original image, to further increase the density of image feature points and detection accuracy, which was more conducive to the calculation of the image parallax value. The ships’ depth estimation results indicate that when the distance to the target is about 300 m, the depth estimation error is less than 3%, which meets the depth estimation needs of inland ships. The ship target recognition and depth estimation technology based on BSV proposed in this paper makes up for the shortcomings of the existing environmental perception methods, improves the navigation safety of ships to a certain extent, and greatly promotes the development of intelligent ships in the future.