SURF Detector Research Articles

A Method for Tracking an Underwater Pipeline from Stereo Images Using an Autonomous Underwater Vehicle

The problem of tracking an underwater pipeline (UP) is considered in the context of an inspection mission. It is assumed that the diameter of the cylindrical pipe is known. To solve this problem, a tracking method is proposed, based on searching and calculating the center line of the UP, and determining the relative position of the autonomous underwater robot (AUV) and UP in the coordinate system of the AUV camera. Unlike well-known analogues, in which the solution is based on recognizing and constructing the boundaries of the UP in images, the proposed method searches for the true position of the center line of the UP on a set of possible options by checking their veracity. Possible options for the spatial position of the search centerline of the current UP section are generated by varying the direction of the beam in the horizontal and vertical plane. The starting point of the ray is the end point of the centerline of the previous section. The veracity criterion is to check that the 3D point features constructed in the scene belong to the cylindrical surface of the UP. The generation and matching of pointfeatures in images of a stereo pair is carried out using the SURF detector. The choice of the correct direction of the center line of the current UP section is ultimately made by voting. The end point of the center line of the current section is determined taking into account the calculated common visibility area with the previous section of the UP. To evaluate the effectiveness of the method, computational experiments were carried out on virtual scenes. The effectiveness was assessed by the accuracy of UP localization (in the AUV coordinate system) and by the speed of calculations in comparison with: a) the first version of this method, based on the use of a vectorized form of images; b) analogues using the Canny and Hough Transform detectors. The stability of the method to the accumulation of navigation accuracy errors during long-term tracking of UPs was also assessed.

Ship Hull Inspection Using Autonomous an Underwater Vehicle with a Stereo Camera

Regular visual inspection of the underwater surface of the ship’s hull to check its integrity and the degree of biofouling is necessary to ensure the safety and efficient operation of the ship. The article proposes a method for the automated solution of this problem using an autonomous uninhabited underwater vehicle (AUV) equipped with a stereo camera controlled by a tilt actuator. The movement of the AUV is carried out along equidistant lines along the ship’s hull with simultaneous video filming of the hull surface. The AUV trajectory is calculated using the visual navigation method (visual odometry). Estimated data on the localization of the AUV relative to the ship’s hull are used in the method of controlling the movement of the vehicle to ensure the stable movement of the vehicle at a given distance from the inspected surface. Direct calculation of AUV localization data is done using the proposed original algorithm, which is implemented in the form of a software tool "stereo rangefinder". The operation of the "stereo rangefinder" is based on the matching of features in images using the SURF detector, followed by the construction of a 3D point cloud. The technique for performing automatic inspection of the ship as a whole is described. Fixed stereo images are also used to build a 3D model of the surface being examined. The applied method of constructing a global spatial model of the surface is based on the union of 3D point clouds obtained for local views. The construction of a 3D point cloud for a particular view is based on a matching of 2D point features on images of a stereo pair (SURF detector / correlation calculation), followed by the use of the ray triangulation method to obtain the spatial coordinates of the points. The presence of a 3D model makes it possible to conduct a detailed visual analysis of the state of the ship’s hull. Simulation modeling of the functioning of the developed tools on virtual scenes was carried out. Quantitative and qualitative performance evaluations obtained as a result of testing showed the acceptability of the proposed method for automatic inspection of the underwater part of the ship’s hull.

ISVD-Based Advanced Simultaneous Localization and Mapping (SLAM) Algorithm for Mobile Robots

In the case of simultaneous localization and mapping, route planning and navigation are based on data captured by multiple sensors, including built-in cameras. Nowadays, mobile devices frequently have more than one camera with overlapping fields of view, leading to solutions where depth information can also be gathered along with ordinary RGB color data. Using these RGB-D sensors, two- and three-dimensional point clouds can be recorded from the mobile devices, which provide additional information for localization and mapping. The method of matching point clouds during the movement of the device is essential: reducing noise while having an acceptable processing time is crucial for a real-life application. In this paper, we present a novel ISVD-based method for displacement estimation, using key points detected by SURF and ORB feature detectors. The ISVD algorithm is a fitting procedure based on SVD resolution, which removes outliers from the point clouds to be fitted in several steps. The developed method removes these outlying points in several steps, in each iteration examining the relative error of the point pairs and then progressively reducing the maximum error for the next matching step. An advantage over relevant methods is that this method always gives the same result, as no random steps are included.

Tensor-based keypoint detection and switching regression model for relative radiometric normalization of bitemporal multispectral images

ABSTRACT In some remote sensing applications, such as unsupervised change detection, bitemporal multispectral images must be first aligned/harmonized radiometrically. For doing so, Many Relative Radiometric Normalization (RRN) algorithms exist; however, most suffer from misregistration problems and can only operate on geo/co-registered image pairs, while unregistered multispectral pairs are required. To tackle this situation, keypoint-based RRN methods were introduced, which can radiometrically calibrate unregistered/registered image pairs using keypoint matching algorithms. However, they ignore the spatial and spectral characteristics of spectral bands of input images, resulting in potential RRN errors. They also employ a linear mapping function for RRN modelling, which can not handle non-linear radiometric distortions. To address these limitations, this paper proposes a robust algorithm for RRN of bitemporal multispectral images, using a new extension of SURF detector for multispectral images, namely the Weighted Spectral Structure Tensor SURF (WSST-SURF), and a flexible Switching Regression (SR) model. Taking advantage of the tensor theory, WSST-SURF efficiently preserves both spatial and spectral information distributed over all bands of multispectral images for keypoint detection, resulting in extracting reliable inliers (or keypoints) for RRN. An adaptive SR model is introduced based on the normalized mutual information, accurately approximating a linear/non-linear relationship between inliers in multispectral images. Six unregistered multispectral image pairs captured by inter/intra remote sensing sensors were employed to validate the efficacy of the proposed method. The results indicate that adopting spectral tensor-based SURF methods in the RRN process exhibits better local and global performance than using the original SURF. Furthermore, the proposed method outperforms the existing conventional RRN methods in terms of accuracy and visual quality, indicating its competence for RRN of bitemporal multispectral images with high illumination, viewpoint, and scale differences.

Image Processing for Improvement of Facial Keypoints Detector

This paper discusses the improvement of facial detection algorithms using the DCT algorithm and image processing. Face key point is very needed in the face recognition system. Some important factors that have effects to detect its result are noise and illuminations. These two factors can be overcome by eliminating some DCT coefficients, both high and low. However, after handling the problem, most likely the image quality will become decrease, which will adversely influence the performance of the feature detector algorithm. Therefore, it is very important to test the performance of the feature detector algorithm on images that are implemented noise and illumination handling and how to improve the quality again. This research implemented Discrete Cosine Transform (DCT), by eliminating the high and low coefficient because there is noise and illumination. However, it is not known at what coefficient level is the most effective, so testing in this study was carried out. Four deblurring algorithms are tested in this research, Blind Deconvolution, Wiener Filter Deconvolution, Lucy-Richardson Deconvolution, and Regularization Deconvolution. And tested the CLAHE algorithm to overcome the effect of removing low coefficient DCT. The best coefficient value to be removed at the DCT frequency is 0.75 with the best SURF algorithm, without the use of other algorithms. Also, the highest F-score is produced by the SURF detector at removing DCT low frequency in combination with the CLAHE algorithm. With the most ideal coefficient of 0.25.

Transfer learning for image classification using VGG19: Caltech-101 image data set.

Image classification is getting more attention in the area of computer vision. During the past few years, a lot of research has been done on image classification using classical machine learning and deep learning techniques. Presently, deep learning-based techniques have given stupendous results. The performance of a classification system depends on the quality of features extracted from an image. The better is the quality of extracted features, the more the accuracy will be. Although, numerous deep learning-based methods have shown enormous performance in image classification, still due to various challenges deep learning methods are not able to extract all the important information from the image. This results in a reduction in overall classification accuracy. The goal of the present research is to improve the image classification performance by combining the deep features extracted using popular deep convolutional neural network, VGG19, and various handcrafted feature extraction methods, i.e., SIFT, SURF, ORB, and Shi-Tomasi corner detector algorithm. Further, the extracted features from these methods are classified using various machine learning classification methods, i.e., Gaussian Naïve Bayes, Decision Tree, Random Forest, and eXtreme Gradient Boosting (XGBClassifier) classifier. The experiment is carried out on a benchmark dataset Caltech-101. The experimental results indicate that Random Forest using the combined features give 93.73% accuracy and outperforms other classifiers and methods proposed by other authors. The paper concludes that a single feature extractor whether shallow or deep is not enough to achieve satisfactory results. So, a combined approach using deep learning features and traditional handcrafted features is better for image classification.

Algorithmic support of the system of external observation and routing of autonomous mobile robots

This article presents the algorithmic support of the external monitoring and routing system of autonomous mobile robots. In some cases, the practical usage of mobile robots is related to the solution of navigation problems. In particular, the position of ground robots can be secured using unmanned aerial vehicles. In the proposed approach based on the video image obtained from an external video camera located above the working area of mobile robots, the location of both robots and nearby obstacles is recognized. The optimal route to the target point of the selected robot is built, and changes in its working area are monitored. Information about the allowed routes of the robot is transmitted to third-party applications via network communication channels. Primary image processing from the camera includes distortion correction, contouring and binarization, which allows to separate image fragments containing robots and obstacles from background surfaces and objects. Recognition of robots in a video frame is based on the use of a SURF detector. This technology extracts key points in the video frame and compares them with key points of reference images of robots. Trajectory planning is implemented using Dijkstra’s algorithm. The discreteness of the trajectories obtained using the algorithm for finding a path on the graph can be compensated for on board autonomous mobile robots by using spline approximation. Experimental studies have confirmed the efficiency of the proposed approach both in the problem of recognition and localization of mobile robots and in the problem of planning safe trajectories.

Reliable object recognition system for cloud video data based on LDP features

Object recognition is one of the research areas with good scope in most of the applications. However, the object recognition on cloud stored data is very limited and the video based object recognition systems are minimal. Taking this into account, the videos are processed for recognizing the objects of interest by incorporating advanced image processing activities. The video frames are extracted from the videos for recognizing the objects. In order to recognize the objects, the objects have to be detected first. The objects are detected by means of SURF detector and the combination of local and global LDP features is extracted. Finally, the objects present in the videos are matched with the objects of interest. The performance of the proposed object recognition system for cloud video data is tested in three rounds. Initially, the proposed work is tested with different videos and then the proposed work is evaluated by varying the feature extractors such as Local Binary Pattern (LBP), Local LDP, Global LDP. Finally, the video processing time is calculated in terms of both CPU and GPU. All the performance evaluations are carried out in terms of accuracy, sensitivity, specificity and time consumption. The performance of the proposed approach is proven to be satisfactory.

Vocabulary Trees with OSS Detectors

In today’s diversified world, with every person having their own idiosyncratic identity, the field of face detection and recognition has received significant scope. Since birth, humans develop and harvest similar facial features, making their recognition a challenging problem. Thus, the development of faster, more accurate and robust algorithms is of prime importance.The most critical requirements in developing a reliable face recognition algorithm is a large database of facial images and a well-developed systematic procedure to evaluate the system. In this paper, we bring out an approach to implement quick facial recognition using the concept of vocabulary trees in accordance with ORB, SIFT and SURF detectors and compare their performance. The model is trained using the Grimace, Face95 and ORL databases. The experiments are conducted by employing various state-of-the-art clustering algorithms and the results are compared for accuracy. The results and comparisons obtained on the databases show that the ORB detector clustered using Spectral clustering algorithm to construct the Vocabulary Tree outperforms the other techniques.

Multiple Descriptors for Visual Odometry Trajectory Estimation

Visual Simultaneous Localization and Mapping (VSLAM) systems are widely used in mobile robots for autonomous navigation. One important part in VSLAM is trajectory estimation. Trajectory estimation is a part of the localisation task in VSLAM where a robot needs to estimate the camera pose in order to precisely align the real visited image locations. The poses are estimated using Visual Odometry Trajectory Estimation (VOTE) by extracting distinctive and trackable keypoints from sequence image locations having been visited by a robot. In the visual trajectory estimation, one of the most popular solutions is arguably PnP-RANSCA function. PnP-RANSAC is a common approach used for estimating the VOTE which uses a feature descriptor such as SURF to extract key-points and match them in pairs based on their descriptors. However, due to the sensor noise and the high fluctuating scenes constitute an inevitable shortcoming that reduces the single visual descriptor performance in extracting the distinctive and trackable keypoints. Thus, this paper proposes a method that uses a random sampling scheme to combine the result of multiple key-points descriptors. The scheme extracts the best keypoints from SIFT, SURF and ORB key-point detectors based on their key-point response value. These keypoints are combined and refined based on Euclidean distances. This combination of keypoints with their corresponding visual descriptors are used in VOTE which reduces the trajectory estimation errors. The proposed algorithm is evaluated on the widely used benchmark dataset KITTI where the three longest sequences are selected, 00 with 4541 images, 02 with 2761 images and 05 with 1101 images. In trajectory estimation experiment, the proposed algorithm can reduce the trajectory error of 44%, 8% and 13% on KITTI dataset for the sequence 00, 02 and 05 respectively based on translational and rotational errors. Also, the proposed algorithm succeeded in reducing the number of keypoints used in VOTE as combined with the state-of-the-art RTAB-Map.

A comparative evaluation of combined feature detectors and descriptors in different color spaces for stereo image matching of tree

Abstract Tree canopy geometric characteristics are directly related to tree growth and productivity, and this information has been used to predict yield, fertilizer application in citrus crops, water consumption or biomass. So a 3D model and a depth map of the tree can be useful. One method of creating 3D model is stereo vision technique. The comparison and performance assessment of different combinations of feature detectors and descriptors is very important in this technique. In this study, the performance of 12 combinations of well-known detectors and descriptors including BRISK with SURF, BRISK with BRISK, BRISK with FREAK, Harris with SURF, Harris with BRISK, Harris with FREAK, SURF with SURF, SURF with BRISK, SURF with FREAK, MSER with SURF, MSER with BRISK, and MSER with FREAK was evaluated. The included color spaces were HSV, H, YCbCr, Y, NTSC and RGB. The performance comparison for each combination was carried out in terms of precision and recall values using stereo image pairs of tree. It was observed that the largest number of keypoints were detected by MSER and SURF detectors almost in all possible spaces. Best combinations used from SURF detector and descriptor when the precision and recall results were considered. RGB and Y spaces were the best spaces to implement combinations. The combination of SURF with SURF, SURF with FREAK, and HARRIS-SURF were found to be preferable.

On-Board Detection and Matching of Feature Points

This paper presents a FPGA-based method for on-board detection and matching of the feature points. With the proposed method, a parallel processing model and a pipeline structure are presented to ensure a high frame rate at processing speed, but with a low power consumption. To save the FPGA resources and increase the processing speed, a model which combines the modified SURF detector and a BRIEF descriptor, is presented as well. Three pairs of images with different land coverages are used to evaluate the performance of FPGA-based implementation. The experiment results demonstrate that (1) when the image pairs with artificial features (such as buildings and roads), the performance of FPGA-based implementation is better than those image pairs with natural features (such as woods); (2) the proposed FPGA-based method is capable of ensuring the processing speed at a high frame rate, such as the speed of can achieve 304 fps under a 100 MHz clock frequency. The speedup of the proposed implementation is about 27 times higher than that when using the PC-based implementation.

Automatic quantification of multi-modal rigid registration accuracy using feature detectors

In radiotherapy, the use of multi-modal images can improve tumor and target volume delineation. Images acquired at different times by different modalities need to be aligned into a single coordinate system by 3D/3D registration. State of the art methods for validation of registration are visual inspection by experts and fiducial-based evaluation. Visual inspection is a qualitative, subjective measure, while fiducial markers sometimes suffer from limited clinical acceptance. In this paper we present an automatic, non-invasive method for assessing the quality of intensity-based multi-modal rigid registration using feature detectors.After registration, interest points are identified on both image data sets using either speeded-up robust features or Harris feature detectors. The quality of the registration is defined by the mean Euclidean distance between matching interest point pairs. The method was evaluated on three multi-modal datasets: an ex vivo porcine skull (CT, CBCT, MR), seven in vivo brain cases (CT, MR) and 25 in vivo lung cases (CT, CBCT). Both a qualitative (visual inspection by radiation oncologist) and a quantitative (mean target registration error—mTRE—based on selected markers) method were employed.In the porcine skull dataset, the manual and Harris detectors give comparable results but both overestimated the gold standard mTRE based on fiducial markers. For instance, for CT-MR-T1 registration, the mTREman (based on manually annotated landmarks) was 2.2 mm whereas mTREHarris (based on landmarks found by the Harris detector) was 4.1 mm, and mTRESURF (based on landmarks found by the SURF detector) was 8 mm. In lung cases, the difference between mTREman and mTREHarris was less than 1 mm, while the difference between mTREman and mTRESURF was up to 3 mm.The Harris detector performed better than the SURF detector with a resulting estimated registration error close to the gold standard. Therefore the Harris detector was shown to be the more suitable method to automatically quantify the geometric accuracy of multimodal rigid registration.

ARTAR-Artistic Augmented Reality

Augmented reality is a technique which adds computer generated virtual objects into the real world scene. ARTAR proposes a method to enhance the experience of paintings or artistic works by adding an extra level of perception through the inclusion of sound, music, and animations. It contains two layers of perception, the physical appearance of the paintings perceived by naked eye and an augmented layer containing animations and sounds which can be perceived by a mobile device. When an artwork is scanned using a predesigned mobile application certain image reference portions get animated along with some music and sound. In this work, a reference area is found in the input video frame using SURF detector and BRISK descriptor and the virtual object is placed in the particular position. Experimental result shows that SURF-BRISK combination provides better result when compared with other detector descriptor combinations in case of ARTAR.

Two Novel Detector-Descriptor Based Approaches for Face Recognition Using SIFT and SURF

Abstract A plethora of promising detectors and descriptors are available in Computer Vision for carrying out Face Recognition (FR) and although these techniques that form the backbone of FR have yielded reasonable efficacy, they are yet to advance to those levels where they demonstrate robust performance in unconstrained scenarios. In our deliberations, we employ the popular SIFT and SURF algorithms that are ubiquitously implemented in FR due to their remarkable potency in handling a variety of FR tasks. In this paper, we proffer two novel detector-descriptor variants to augment the proficiency of contemporary FR systems: (1) SURF detector with SIFT descriptor and (2) SIFT detector with SURF descriptor. We demonstrate the proficiency of the proposed techniques by utilizing pertinent mathematical arguments and performing comprehensive comparisons with the classical SIFT and SURF algorithms by employing a number of standard metrics over the benchmark LFW and Face94 datasets.

Fast catheter segmentation from echocardiographic sequences based on segmentation from corresponding X-ray fluoroscopy for cardiac catheterization interventions.

Echocardiography is a potential alternative to X-ray fluoroscopy in cardiac catheterization given its richness in soft tissue information and its lack of ionizing radiation. However, its small field of view and acoustic artifacts make direct automatic segmentation of the catheters very challenging. In this study, a fast catheter segmentation framework for echocardiographic imaging guided by the segmentation of corresponding X-ray fluoroscopic imaging is proposed. The complete framework consists of: 1) catheter initialization in the first X-ray frame; 2) catheter tracking in the rest of the X-ray sequence; 3) fast registration of corresponding X-ray and ultrasound frames; and 4) catheter segmentation in ultrasound images guided by the results of both X-ray tracking and fast registration. The main contributions include: 1) a Kalman filter-based growing strategy with more clinical data evalution; 2) a SURF detector applied in a constrained search space for catheter segmentation in ultrasound images; 3) a two layer hierarchical graph model to integrate and smooth catheter fragments into a complete catheter; and 4) the integration of these components into a system for clinical applications. This framework is evaluated on five sequences of porcine data and four sequences of patient data comprising more than 3000 X-ray frames and more than 1000 ultrasound frames. The results show that our algorithm is able to track the catheter in ultrasound images at 1.3 s per frame, with an error of less than 2 mm. However, although this may satisfy the accuracy for visualization purposes and is also fast, the algorithm still needs to be further accelerated for real-time clinical applications.

DETECÇÃO E RECONHECIMENTO DE OBJETOS EM IMAGENS UTILIZANDO ALGORITMOS DE EXTRAÇÃO DE PONTOS CHAVE

This paper presentsa comparative studyof descriptors keypoint algorithms doinga combinationof detectors anddescriptors. We combined the SIFT, SURF and Harris detector algorithms with the SIFT and SURF descriptor algorithms.Three experimentswith images withdifferent characteristicswere performed,intwo of theseexperimentswere usedimages of realobjects, in athird experiment, some degradationin an imagesuch asscale reduction, rotation, blurring, darkening andaddition ofnoisehave beenperformed, to analyze the behaviorof the algorithms. Experiments were also performed withalldegradationstogether.Toanalyze theproposedcombination, we extractedprocessing timeand the number ofinliers. This workcontributes toapplicationsthat requireusing keypoint descriptors algorithms in detection andrecognitionof objects inimages.

A Fast and Robust Binary Feature Descriptor Based on Tuning Sampling Pattern

Based on two sampling patterns’ characteristics of BRISK and FREAK, a new sampling pattern which results in a better descriptor is proposed. Both sampling-point density and size of overlapping receptive fields on sampling pattern have a significantly effect on the performance of descriptor through analyzing both sampling patterns of BRISK and FREAK. Optima of both factors are got from respective tests of describing local area to be more discriminative and enable the sampling pattern to produce a descriptor with better performance. Through a comparing performance test with state-of-the-art descriptors on benchmark datasets, the proposed descriptor in combination with the SURF detector has been proved to be a high-performance one to capture the most discriminative information in the detected salient region.

SIFT and SURF Performance Evaluation for Mobile Robot-Monocular Visual Odometry

Visual odometry is the process of estimating the motion of mobile through the camera attached to it, by matching point features between pairs of consecutive image frames. For mobile robots, a reliable method for comparing images can constitute a key component for localization and motion estimation tasks. In this paper, we study and compare the SIFT and SURF detector/ descriptor in terms of accurate motion determination and runtime efficiency in context the mobile robot-monocular visual odometry. We evaluate the performance of these detectors/ descriptors from the repeatability, recall, precision and cost of computation. To estimate the relative pose of camera from outlier-contaminated feature correspondences, the essential matrix and inlier set is estimated using RANSAC. Experimental results demonstrate that SURF, outperform the SIFT, in both accuracy and speed. 

Comparing Effectiveness of Feature Detectors in Obstacles Detection from a Video

We have already proposed an obstacles detection method using a video taken by a vehicle-mounted monocular camera. In this method, correct obstacles detection depends on whether we can accurately detect and match feature points. In order to improve the accuracy of obstacles detection, in this paper, we make comparison among four most commonly used feature detectors; Harris, SIFT, SURF and FAST detectors. The experiments are done using our obstacles detection method. The experimental results are compared and discussed, and then we find the most suitable feature point detector for our obstacles detection method.