Local Feature Points Research Articles

Residual 3-D Scene Flow Learning With Context-Aware Feature Extraction

Scene flow estimation is the task to predict the point-wise or pixel-wise 3-D displacement vector between two consecutive frames of point clouds or images, which has important application in fields such as service robots and autonomous driving. Although many previous works have explored greatly on scene flow estimation based on point clouds, there are two problems that have not been noticed or well solved before: 1) points of adjacent frames in repetitive patterns tend to be wrongly associated due to similar spatial structure in their neighborhoods and 2) scene flow between adjacent frames of point clouds with long-distance movement tends to be inaccurately estimated. To solve the first problem, a novel context-aware set convolution layer is proposed in this article to exploit contextual structure information of Euclidean space and learn soft aggregation weights for local point features. This design is inspired by human perception of contextual structure information during scene understanding with repetitive patterns. For the second problem, an explicit residual flow learning structure is proposed in the residual flow refinement layer to cope with long-distance movement. The experiments and ablation study on FlyingThings3D and Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) scene flow datasets demonstrate the effectiveness of each proposed component. The qualitative results show that the problems of ambiguous interframe association and long-distance movement estimation are well handled. Quantitative results on both FlyingThings3D and KITTI scene flow datasets show that the proposed method achieves state-of-the-art performance, surpassing all previous works to the best of our knowledge by at least 26.22% on FlyingThings3D dataset and 25.42% on KITTI scene flow dataset for 3-D endpoint error (EPE3D) metric. The source codes are available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/IRMVLab/CARFlow</uri> .

An intelligent detection method of local feature points in computer vision image

An intelligent detection method of local feature points in computer vision image

LOCAL FEATURE EXTRACTION IN IMAGES

The methods of the local feature point extraction are analyzed. The analysis shows that the most effective detectors are based on the brightness gradient determination. They usually use the Harris angle detector, which is complex in calcu­la­tions. The algorithm complexity minimization contradicts both the detector effective­ness and to the high dynamic range of the analyzed image. As a result, the high-speed methods could not recognize the feature points in the heavy luminance conditions. The modification of the high dynamic range (HDR) image compression algorithm based on the Retinex method is proposed. It contains an adaptive filter, which preserves the image edges. The filter is based on a set of feature detectors perfor­ming the Harris-Laplace transform which is much simpler than the Harris angle detector. A prototype of the HDR video camera is designed which provides sharp images. Its structure simplifies the design of the artificial intelligence engine, which is implemented in FPGA of medium or large size.

Artificial Intelligence-Based Echocardiographic Left Atrial Volume Measurement with Pulmonary Vein Comparison.

This paper combines echocardiographic signal processing and artificial intelligence technology to propose a deep neural network model adapted to echocardiographic signals to achieve left atrial volume measurement and automatic assessment of pulmonary veins efficiently and quickly. Based on the echocardiographic signal generation mechanism and detection method, an experimental scheme for the echocardiographic signal acquisition was designed. The echocardiographic signal data of healthy subjects were measured in four different experimental states, and a database of left atrial volume measurements and pulmonary veins was constructed. Combining the correspondence between ECG signals and echocardiographic signals in the time domain, a series of preprocessing such as denoising, feature point localization, and segmentation of the cardiac cycle was realized by wavelet transform and threshold method to complete the data collection. This paper proposes a comparative model based on artificial intelligence, adapts to the characteristics of one-dimensional time-series echocardiographic signals, automatically extracts the deep features of echocardiographic signals, effectively reduces the subjective influence of manual feature selection, and realizes the automatic classification and evaluation of human left atrial volume measurement and pulmonary veins under different states. The experimental results show that the proposed BP neural network model has good adaptability and classification performance in the tasks of LV volume measurement and pulmonary vein automatic classification evaluation and achieves an average test accuracy of over 96.58%. The average root-mean-square error percentage of signal compression is only 0.65% by extracting the coding features of the original echocardiographic signal through the convolutional autoencoder, which completes the signal compression with low loss. Comparing the training time and classification accuracy of the LSTM network with the original signal and encoded features, the experimental results show that the AI model can greatly reduce the model training time cost and achieve an average accuracy of 97.97% in the test set and increase the real-time performance of the left atrial volume measurement and pulmonary vein evaluation as well as the security of the data transmission process, which is very important for the comparison of left atrial volume measurement and pulmonary vein. It is of great practical importance to compare left atrial volume measurements with pulmonary veins.

Progressive Point Cloud Upsampling via Differentiable Rendering

In this paper, we propose one novel progressive point cloud upsampling framework to tackle the non-uniform distribution issue during the point cloud upsampling process. Specifically, we design an Up-UNet feature expansion module which is capable of learning the local and global point features via a down-feature operator and an up-feature operator, respectively, to alleviate the non-uniform distribution issue and remove the outliers. Moreover, we design a hybrid loss function considering both the multi-scale reconstruction loss and the rendering loss. The multi-scale reconstruction loss enables each upsampling module to generate a denser point cloud, while the rendering loss via point-based differentiable rendering ensures that the proposed model preserves the point cloud structures. Extensive experimental results demonstrate that our proposed model achieves state-of-the-art performance in terms of both qualitative and quantitative evaluations.

Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images.

Despite the rapid technical advancement of augmented reality (AR) and mixed reality (MR) in minimally invasive surgery (MIS) in recent years, monocular-based 2D/3D reconstruction still remains technically challenging in AR/MR guided surgery navigation nowadays. In principle, soft tissue surface is smooth and watery with sparse texture, specular reflection, and frequent deformation. As a result, we frequently obtain only sparse feature points that give rise to incorrect matching results with conventional image processing methods. To ameliorate, in this paper we enunciate an accurate and robust description and matching method for dense feature points in endoscopic videos. Our new method first extracts contours of the low-rank image sequences based on the adaptive robust principal component analysis (RPCA) decomposition. Then we propose a multi-scale dense geometric feature description approach, which simultaneously extracts dense feature descriptors of the contours in the original Euclidean coordinate space, the accompanying 3D color coordinate space, and the derived curvature-gradient coordinate space. Finally, we devise a new algorithm for both global and local point-wise matching based on feature fusion. For global matching, we employ the fast Fourier transform (FFT) to reduce the dimension of the dense feature descriptors. For local feature point matching, in order to enhance the robustness and accuracy of the matching, we cluster multiple contour points to form "super-point" based on dense feature descriptors and their spatio-temporal continuity. The comprehensive experimental results confirm that our novel approach can overcome the highlight influence, and robustly describe contours from image sequences of soft tissue surfaces. Compared with the state-of-the-art feature point description and matching methods, our analysis framework shows the key advantages of both robustness and accuracy in dense point-wise matching, even when the severe soft tissue deformation occurs. Our new approach is expected to have high potential in 2D/3D reconstruction in endoscopy.

Laser vision seam tracking system based on proximal policy optimization

PurposeThis paper aims to propose a weld seam tracking method based on proximal policy optimization (PPO).Design/methodology/approachBy constructing a neural network based on PPO and using the reference image block and the image block to be detected as the dual-channel input of the network, the method predicts the translation relation between the two images and corrects the location of feature points in the weld image. The localization accuracy estimation network (LAE-Net) is built to update the reference image block during the welding process, which is helpful to reduce the tracking error.FindingsOff-line simulation results show that the proposed algorithm has strong robustness and performs well on the test set of curved seam images with strong noise. In the welding experiment, the movement of welding torch is stable, the molten material is uniform and smooth and the welding error is small, which can meet the requirements of industrial production.Originality/valueThe idea of image registration is applied to weld seam tracking, and the weld seam tracking network is built on the basis of PPO. In order to further improve the tracking accuracy, the LAE-Net is constructed and the reference images can be updated.

Research on the SOH Prediction Based on the Feature Points of Incremental Capacity Curve

The accurate prediction of the state of health (SOH) is an important basis for ensuring the normal operation of the lithium-ion battery (LIB). The accurate SOH can extend the life-span, ensure safety, and improve the performance of LIBs. The charging voltage curve and incremental capacity (IC) curve of the LIB in different SOH are obtained through experiments. The location parameters of each feature point on IC curve are closely related to battery aging, to characterize the SOH of the LIB with the location of feature points. To solve the difficulty in identifying feature points due to the oscillation in solving IC curves with a traditional numerical analytic method, the piecewise polynomial fitting method is adopted to smooth IC. To discuss the law between the location change of all feature points on the IC curve and the capacity attenuation, a capacity prediction regression model is established after the dimensionality reduction of the coordinate data of feature points on the IC curve with the principal component analysis method. The proposed method can rapidly estimate the online SOH of LIBs during the charging process of electric vehicles and the results show the maximum error is 0.63AH (3.15%).

Effective solution for underwater image enhancement.

Degradation of underwater images severely limits people to exploring and understanding underwater world, which has become a fundamental but vital issue needing to be addressed in underwater optics. In this paper, we develop an effective solution for underwater image enhancement. We first employ an adaptive-adjusted artificial multi-exposure fusion (A-AMEF) and a parameter adaptive-adjusted local color correction (PAL-CC) to generate a contrast-enhanced version and a color-corrected version from the input respectively. Then we put the contrast enhanced version into the famous guided filter to generate a smooth base-layer and a detail-information containing detail-layer. After that, we utilize the color channel transfer operation to transfer color information from the color-corrected version to the base-layer. Finally, the color-corrected base-layer and the detail-layer are added together simply to reconstruct the final enhanced output. Enhanced results obtained from the proposed solution performs better in visual quality, than those dehazed by some current techniques through our comprehensive validation both in quantitative and qualitative evaluations. In addition, this solution can be also utilized for dehazing fogged images or improving accuracy of other optical applications such as image segmentation and local feature points matching.

An Efficient and Precise Remote Sensing Optical Image Matching Technique Using Binary-Based Feature Points.

Matching local feature points is an important but crucial step for various optical image processing applications, such as image registration, image mosaicking, and structure-from-motion (SfM). Three significant issues associated with this subject have been the focus for years, including the robustness of the image features detected, the number of matches obtained, and the efficiency of the data processing. This paper proposes a systematic algorithm that incorporates the synthetic-colored enhanced accelerated binary robust invariant scalar keypoints (SC-EABRISK) method and the affine transformation with bounding box (ATBB) procedure to address these three issues. The SC-EABRISK approach selects the most representative feature points from an image and rearranges their descriptors by adding color information for more precise image matching. The ATBB procedure, meanwhile, is an outreach that implements geometric mapping to retrieve more matches from the feature points ignored during SC-EABRISK processing. The experimental results obtained using benchmark imagery datasets, close-range photos (CRPs), and aerial and satellite images indicate that the developed algorithm can perform up to 20 times faster than the previous EABRISK method, achieve thousands of matches, and improve the matching precision by more than 90%. Consequently, SC-EABRISK with the ATBB algorithm can address image matching efficiently and precisely.

Robotic seam tracking system combining convolution filter and deep reinforcement learning

To perform automatic, real-time seam tracking tasks effectively, a robust and accurate seam tracking system must be designed. In this paper, we solve the seam tracking issue using a six-axis welding robot, a line laser sensor and an industrial computer. The processing of welding images is the core of the seam tracking system, which aims to determine the weld feature point in each image. We propose a two-stage weld feature point localization method that combines convolution filter and deep reinforcement learning (CF-DRL) to localize the weld feature point in each welding image robustly and accurately. In the first stage, the weld feature point is roughly tracked using a convolution filter tracker. But the position given by the convolution tracker is sometimes not accurate enough due to the natural gap between visual tracking and seam tracking. Consequently, in the second stage, the weld feature point should be further refined using our trained policy network. Using our two-stage weld feature point localization method, the weld feature points can be determined from noisy images in real time during the welding process. The 3D coordinate values of these points are obtained according to the structured light measurement principle to control the movement of the robot and the torch in real time. A robotic seam tracking system is established based on the equipment and methods mentioned above. Experimental results show that the welding torch runs smoothly with a strong arc light and splash interference. The mean tracking error of our experiments reaches 0.189 mm, which can fulfill actual welding requirements. Several comparison tests have been performed to illustrate the robustness and accuracy of our seam tracking system using our welding image dataset.

Vehicle behavior analysis using reconstructed 3D parameters for road safety

Road traffic safety is a very important issue in the field of intelligent transportation system (ITS). Vehicle segmentation and behavior analysis are an significant part for solving this problem. However, 2D image detection technology is difficult to reconstruct missing information of damaged image. In this paper, a bottom-up analysis method is employed to study the related technical problems, and it will provide a strong data foundation for road traffic safety. Firstly, the M-BRISK descriptor algorithm is proposed to describe the local feature points. Secondly, we propose a 3D feature analysis method based on rigid motion constraints for vehicle trajectory. Thirdly, a similarity measure method is proposed for trajectory clustering. Finally, we used the obtained 3D information of vehicles to analyze the vehicle behavior to find the abnormal vehicles for road traffic safety. The experimental results confirm that the M-BRISK descriptor performs well comparing with the state-of-the-art feature descriptors, and the proposed clustering method improves the accuracy of the trajectory clustering. Moreover, the vehicle motion information contained in the trajectory data can be analyzed to recognize vehicle behavior. The presented work in this paper provides an important foundation for vehicle abnormal behavior detection for road traffic safety.

Personality Trait Detection Based on ASM Localization and Deep Learning

Global competition is the competition of human resources, the social demand for high-quality talents is increasing, and the demand for all kinds of talents is increasing. Therefore, how to scientifically and efficiently complete the preliminary screening of college students’ mental health, so as to provide services for them, has become an important task. In order to solve the above problems, by combining the relevant professional knowledge of psychology, statistics, image processing, and artificial intelligence technology, a personality trait detection method based on active shape model (ASM) localization and deep learning is proposed. Firstly, the traditional ASM algorithm is improved and applied to facial feature point location, which provides training basis for further deep learning. It mainly includes three aspects of improvement: (1) 2D texture model based on Gabor wavelet and gradient features; (2) new multiresolution pyramid decomposition method; and (3) improved multiresolution pyramid search strategy. Secondly, the deep belief network model is used to train and classify the students’ four personality traits and facial features, so as to dig out the relationship between the four personality traits and facial features. The experimental results show that the localization effect of the improved ASM algorithm is obviously better than that of the traditional algorithm, and the classifier after learning and training has a good effect in analyzing the four personality traits.

Automated Mosaicking of Medical x-ray Images using SIFT Algorithm

Image registration is a process of joining any number of images that have similar overlapping regions of the same scene in order to make a panoramic image. In the field of medical, multimedia, and image processing applications image registration process stands challenging. The work presented here on medical images can be applicable for long limb operations and scoliosis operations. Traditional x-ray machines produce a single frame of x-ray image containing a portion of the body part, but they can not generate a large view of body x-ray image in a single frame. This problem can be solved by creating panoramic images by combining multiple images. The work proposed in this paper can automatically produce panoramic x-ray images by stitching multiple x-ray images. The proposed work uses scale-invariant feature transform (SIFT) for mosaicking x-ray images as a local feature point extractor which uses the difference of Gaussian (DOG) and invariant to orientation and scale. Based on the location relationship of x-ray images, random sample consensus (RANSAC) is incorporated to remove the effect of mismatched point pairs in x-ray images and to generate the panoramic view. The performance of the system is computed by using structural and time constraint parameters and is compared with different feature detection techniques. The experimental results show that combing SIFT and RANSAC yields less processing time with an increase in similarity measures.

A Dual-Watermarking Scheme Robust to Affine Transformation

This paper proposes a robust watermarking algorithm based on dual watermarking and robust affine transformation. First, the color image is divided into three channels, feature point extraction, watermark embedding, and location watermark extraction will be performed respectively. Feature points are used for image reconstruction when extracting watermarks. The watermark embedding needs to select a square area not at the edge of the image as the watermark embedding position, and then embed the watermark in the frequency domain of the corresponding area. According to the watermark embedding position, extract the zero watermark in another channel as the location watermark. Because the zero watermark does not modify the original image information, it has good robustness and can assist the attacked image to locate the watermark embedding position.

Research on Volleyball Image Classification Based on Artificial Intelligence and SIFT Algorithm

Due to the application scenarios of image matching, different scenarios have different requirements for matching performance. Faced with this situation, people cannot accurately and timely find the information they need. Therefore, the research of image classification technology is very important. Image classification technology is one of the important research directions of computer vision and pattern recognition, but there are still few researches on volleyball image classification. The selected databases are the general database ImageNet library and COCO library. First, the color image is converted into a gray image through gray scale transformation, and then the scale space theory is integrated into the image feature point extraction process through the SIFT algorithm. Extract local feature points from the volleyball image, and then combine them with the Random Sample Consensus (RANSAC) algorithm to eliminate the resulting mismatch. Analyze the characteristic data to obtain the data that best reflects the image characteristics, and use the data to classify existing volleyball images. The algorithm can effectively reduce the amount of data and has high classification performance. It aims to improve the accuracy of image matching or reduce the time cost. This research has very important use value in practical applications.

Analysis of Feature Extraction and Anti-Interference of Face Image under Deep Reconstruction Network Algorithm

In face recognition systems, highly robust facial feature representation and good classification algorithm performance can affect the effect of face recognition under unrestricted conditions. To explore the anti-interference performance of convolutional neural network (CNN) reconstructed by deep learning (DL) framework in face image feature extraction (FE) and recognition, in the paper, first, the inception structure in the GoogleNet network and the residual error in the ResNet network structure are combined to construct a new deep reconstruction network algorithm, with the random gradient descent (SGD) and triplet loss functions as the model optimizer and classifier, respectively, and it is applied to the face recognition in Labeled Faces in the Wild (LFW) face database. Then, the portrait pyramid segmentation and local feature point segmentation are applied to extract the features of face images, and the matching of face feature points is achieved using Euclidean distance and joint Bayesian method. Finally, Matlab software is used to simulate the algorithm proposed in this paper and compare it with other algorithms. The results show that the proposed algorithm has the best face recognition effect when the learning rate is 0.0004, the attenuation coefficient is 0.0001, the training method is SGD, and dropout is 0.1 (accuracy: 99.03%, loss: 0.0047, training time: 352 s, and overfitting rate: 1.006), and the algorithm proposed in this paper has the largest mean average precision compared to other CNN algorithms. The correct rate of face feature matching of the algorithm proposed in this paper is 84.72%, which is higher than LetNet-5, VGG-16, and VGG-19 algorithms, the correct rates of which are 6.94%, 2.5%, and 1.11%, respectively, but lower than GoogleNet, AlexNet, and ResNet algorithms. At the same time, the algorithm proposed in this paper has a faster matching time (206.44 s) and a higher correct matching rate (88.75%) than the joint Bayesian method, indicating that the deep reconstruction network algorithm proposed in this paper can be used in face image recognition, FE, and matching, and it has strong anti-interference.

A PHD-SLAM Method for Mixed Birth Map Information Based on Amplitude Information

The Simultaneous Localization and Mapping (SLAM) method of mobile robots has the problem of low accuracy in complex environments with dense clutter and various map features, such as complex indoor environments and underwater environments. This problem is mainly embodied in estimating the location and number of feature points on the map and the position of the robot itself. In order to solve this problem, a new method based on the probability hypothesis density (PHD) SLAM is proposed in this paper, a PHD-SLAM Method for Mixed Birth Map Information Based on Amplitude Information (AI-MBMI-PHD-SLAM). Firstly, this paper proposes a PHD-SLAM method based on amplitude information (AI-PHD-SLAM). The method uses the amplitude information of map features to obtain more precise map features. Then, the clutter likelihood function is used to improve the estimation accuracy of the feature map in the SLAM process. Meanwhile, this paper studies the performance of the PHD-SLAM method with the amplitude information under the condition of the known signal-to-noise ratio or the unknown signal-to-noise ratio. Secondly, aiming at the problem that PHD-SLAM lacks a priori information in the prediction stage, an AI-PHD-SLAM-based mixed birth map information method is added. In this method, map information that has been detected before the previous moment is added to the observation information in the map prediction phase as a new map information set in the prediction phase. This can increase the prior information and improve the problem of insufficient prior information in the prediction stage. The results of the experiments show that the proposed method and the improved method outperform the RB-PHD-SLAM method in estimating the number and location accuracy of map features and have higher computational efficiency.

Fast Aerial Image Geolocalization Using the Projective-Invariant Contour Feature

We address the problem of aerial image geolocalization over an area as large as a whole city through road network matching, which is modeled as a 2D point set registration problem under the 2D projective transformation and solved in a two-stage manner. In the first stage, all the potential transformations aligning the query road point set to the reference road point set are found by local point feature matching. A local geometric feature, called the Projective-Invariant Contour Feature (PICF), which consists of a road intersection and the closest points to it in each direction, is specifically designed. We prove that the proposed PICF is equivariant under the 2D projective transformation group. We then encode the PICF with a projective-invariant descriptor to enable the fast search of potential correspondences. The bad correspondences are then removed by a geometric consistency check with the graph-cut algorithm effectively. In the second stage, a flexible strategy is developed to recover the homography transformation with all the PICF correspondences with the Random Sample Consensus (RANSAC) method or to recover the transformation with only one correspondence and then refine it with the local-to-global Iterative Closest Point (ICP) algorithm when only a few correspondences exist. The strategy makes our method efficient to deal with both scenes where roads are sparse and scenes where roads are dense. The refined transformations are then verified with alignment accuracy to determine whether they are accepted as correct. Experimental results show that our method runs faster and greatly improves the recall compared with the state-of-the-art methods.

Localization Method Based on Image Processing for Autonomous Driving of Mobile Robot in the Linear Infrastructure

In recent years, the deterioration of infrastructure facilities such as bridges has become a problem. Precautionary measures such as visual inspection and repair by humans are in place as countermeasures for aging; however, there are issues with cost and safety in such inspections. If inspection by robots becomes possible, both these aspects will be improved, which will significantly contribute to the maintenance of infrastructure facilities. In this paper, we propose a complex image processing technique to specify the location of feature points as coordinates through smartphone cameras to obtain the location information of feature points needed for positioning BIREM-IV-P developed to support bridge inspection. The corners located in the bridge inspection environment are used as feature points, and the corners are specified using Harris corner detection, which is a conventional corner detection method, to obtain the position of the feature points. In addition, to compensate for the shortcomings of Harris corner detection, a line segment in the image is detected using the Hough transform, and the intersection points of the line segments are recognized as corners. By combining the results of the two detection methods in this manner, the target feature points can be accurately specified. Then, the position of the feature points of the specified image coordinate system can be changed to the world coordinate system. As a result, it was possible to detect the location of the target feature points in a three-dimensional coordinate system.