Image Feature Matching Research Articles

PatchMatch Filter‐Census: A slanted‐plane stereo matching method for slope modelling application

Image matching is a well-studied problem in computer vision. Conventional image matching is solved using image feature matching algorithms, and later deep learning techniques are also applied to tackle the problem. Here, a slope-modelling framework is proposed by adopting the image matching techniques. First, image pairs of a slope scene are captured and camera calibration as well as image rectification are performed. Then, PatchMatch Filter (PMF-S) and PWC-Net techniques are adapted to solve the matching of image pairs. In the proposed PatchMatch Filter-Census (PMF-Census), slanted-plane modelling, image census transform and gradient difference are employed in matching cost formulation. Later, nine matching points are manually selected from an image pair. Matching point pairs are further used in fitting a transformation matrix to relate the matching between the image pair. Then, the transformation matrix is applied to obtain a ground truth matching image for algorithm evaluation. The challenges in this matching problem are that the slope is of a homogenous region and it has a slanted-surface geometric structure. In this work, it is found out that the error rate of the proposed PMF-Census is significantly lower as compared with the PWC-Net method and is more suitable in this slope-modelling task. In addition, to show the robustness of the proposed PMF-Census against the original PMF-S, further experiments on some image pairs from Middlebury Stereo 2006 dataset are conducted. It is demonstrated that the error percentage by the proposed PMF-Census is reduced significantly especially in the low-texture and photometric distorted region, in comparison to the original PMF-S algorithm. This further verifies the suitability of the PMF-Census in modelling the outdoor low-texture slope scene.

Real-Time Nonrigid Mosaicking of Laparoscopy Images.

The ability to extend the field of view of laparoscopy images can help the surgeons to obtain a better understanding of the anatomical context. However, due to tissue deformation, complex camera motion and significant three-dimensional (3D) anatomical surface, image pixels may have non-rigid deformation and traditional mosaicking methods cannot work robustly for laparoscopy images in real-time. To solve this problem, a novel two-dimensional (2D) non-rigid simultaneous localization and mapping (SLAM) system is proposed in this paper, which is able to compensate for the deformation of pixels and perform image mosaicking in real-time. The key algorithm of this 2D non-rigid SLAM system is the expectation maximization and dual quaternion (EMDQ) algorithm, which can generate smooth and dense deformation field from sparse and noisy image feature matches in real-time. An uncertainty-based loop closing method has been proposed to reduce the accumulative errors. To achieve real-time performance, both CPU and GPU parallel computation technologies are used for dense mosaicking of all pixels. Experimental results on in vivo and synthetic data demonstrate the feasibility and accuracy of our non-rigid mosaicking method.

Image Feature Synthesis and Matching in Content-Based Image Retrieval System – A Review

One of the important concepts in information & data analytics is the content-based image retrieval process. We are living in the information age. In the modern-day digital information technology imaging world, this is playing a predominant role in different sectors ranging from defense to research fields. Content-based image retrieval, also known as query by image content is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching for digital images or textual matters in large databases. The usage of digital images has been increased enormously from the last decade due to the drastic growth in storage & network technology. These technological changes have led professional users to use, store and manipulate remotely stored images. Information Retrieval (IR) deals with the location and retrieval of related documents or images based on user inputs such as keywords or examples as a query from the repository. This has motivated us to take up the research work on the CBIR concepts. Hence, to throw light into this chosen research topic, we are carrying out extensive research on the image feature synthesis and matching in content-based image retrieval systems using the concepts of AI, ML & Fuzzy Logic schemes, which could be used to improve the retrieval system's performance in CBIRs. A brief survey, i.e., an insight into the chosen research area in the field of content-based image retrievals was made & the same is being presented w.r.t. the work done by various researchers across the globe in the form of an extensive literature review. The work done by them was studied, lacunas observed & the problem was defined with a couple of good objectives to be solved four objectives were proposed as O1 - Investigation of the effectiveness of evolutionary computation in generating composite operator vectors for image, so that feature dimensionality is reduced to improve retrieval performances; O2 - Construction of the image-leve

A Hybrid Radio-Optical Wireless System With Efficient Sub-Centimeter Localization for Full-Coverage Indoor Services

Wireless transmission capacity demands are continuously growing. Infrared optical wireless communication (IR-OWC) is considered as a promising solution for the future high-speed indoor applications because it can easily offer more than 10 GHz bandwidth with commercially available devices of the fiber-optic communication eco-systems. An IR-OWC cell employing narrow beam widths requires powerful beam-steering and localization functions to dynamically pinpoint the optical beams to different user devices (UDs), and typically has a limited service area. Powerful localization functions with high accuracy typically require sophisticated hardware and resource- and time-consuming signal processing, which may cause serious economical burdens to the communication system. Besides, in regard of a system with extended service area, it may be comprehensive and thus expensive to fully cover the whole space by densely spaced IR-OWC cells. In this article, we introduce a hybrid radio-optical wireless system, which offers high speed IR-OWC services at specific hotspots, and covers spatial gaps among the IR-OWC cells through broadcasting of Wi-Fi signals. This balances the requirements among capacity, coverage, and cost. For each IR-OWC cell, image feature matching based on processing with additional differential operations is proposed to more efficiently process the signals. The images are captured by IP cameras, and can be transparently transported over the communication system through Wi-Fi. Based on our proposal, a 10 Gb/s automatically beam-steered IR-OWC system integrated with ∼80 Mb/s WiFi and with localization function is designed and discussed. An independent test showing 0.24-cm localization accuracy for a 1-cm UD aperture over 180-cm distance is performed. Our processing time shows ∼50% efficiency improvement to the speeded up robust features (SURF) method, and has less recognition uncertainties.

Multidimensional Local Binary Pattern for Hyperspectral Image Classification

For the large amount of spatial and spectral information contained in hyperspectral image (HSI), feature description of HSI has attracted widespread concern in recent years. Existing deep learning-based HSI feature description algorithms require a large number of training samples and have poor interpretability. Therefore, it is necessary to develop an efficient HSI features description algorithm with interpretability based on machine learning. Local binary pattern (LBP) is a classical descriptor used to extract the local spatial texture features of images, which has been widely applied to image feature description and matching. However, the existing LBP algorithms for HSI are based on the single-dimensional description, which leads to the limitations on the expression of spatial–spectral information. Therefore, a multidimensional LBP (MDLBP) based on Clifford algebra for HSI is proposed in this article, which is able to extract spatial–spectral feature from multiple dimensions. First, with the theory of the Clifford algebra, a new representation of HSI including spatial and spectral information is built. Second, the geometric relationship between the local geometry of HSI in Clifford algebra space is calculated to realize the local multidimensional description of the local spatial–spectral information. Finally, a novel LBP coding algorithm for HSI is implemented based on the local multidimensional description to calculate the feature descriptor of HSI. The experimental results on HSI classification show that our proposed MDLBP algorithm can achieve higher accuracy than the representative spatial–spectral features and the existing LBP algorithms, especially in the scenery of small-scale training samples.

Beach Topography Surveying Using Unmanned Aerial Vehicle Photogrammetry Technology

The mechanisms of in situ topography changes must be identified for spatial planning in coastal areas. In this study, the authors used in situ direct measurement methods and indirect remote-sensing technologies to develop a method for beach topography surveillance. Unmanned aerial vehicles (UAVs) have advantages of high portability and mobility and low operating altitude. Therefore, a UAV was equipped with a lightweight camera and a positioning system that comprised a global positioning system and an inertial measurement unit. Aerial photogrammetry and aerial triangu-lation methods were adopted for matching image feature points to obtain the corresponding topographic points in the sand. A virtual base station with real-time kinematic positioning functionality was used to measure the coordinates of the ground control points for correcting the actual coordinates to obtain actual topographic points of the beach sand. The results obtained using image-matching point clouds and direct measurements were compared. The ground sample distance of a UAV at the operational altitude of 70 m was 3.26 cm. Moreover, the average elevation error was 3.20 cm, and the root mean square error was 0.169 m. The measurement error was ±25 cm, which was within the acceptance criteria of±50 cm set by Water Resources Agency, Ministry of Economic Affairs, Taiwan. UAV imaging technology can increase the efficiency of conventional manual sampling and reduce the cost of indirect observations, thus minimizing the measurement errors and field measurement costs.

FSD-BRIEF: A Distorted BRIEF Descriptor for Fisheye Image Based on Spherical Perspective Model.

Fisheye images with a far larger Field of View (FOV) have severe radial distortion, with the result that the associated image feature matching process cannot achieve the best performance if the traditional feature descriptors are used. To address this challenge, this paper reports a novel distorted Binary Robust Independent Elementary Feature (BRIEF) descriptor for fisheye images based on a spherical perspective model. Firstly, the 3D gray centroid of feature points is designed, and the position and direction of the feature points on the spherical image are described by a constructed feature point attitude matrix. Then, based on the attitude matrix of feature points, the coordinate mapping relationship between the BRIEF descriptor template and the fisheye image is established to realize the computation associated with the distorted BRIEF descriptor. Four experiments are provided to test and verify the invariance and matching performance of the proposed descriptor for a fisheye image. The experimental results show that the proposed descriptor works well for distortion invariance and can significantly improve the matching performance in fisheye images.

Location and Accuracy Validation of Lunar Landing Point Based on Multi-source Images

High-precision positioning of the landing site is a critical step in the mission and an important prerequisite for surface operations of the lander and rover. In this paper, a high-precision positioning method of the landing site was designed based on image feature matching and multiple image localization methods. And the positioning experiment and accuracy validation were carried out in ChangE-3 landing site. On the basis of high-precision image matching and geometric transformation, the matching precision between the sequent descent images reached sub-pixel, and the matching accuracy between LRO NAC image and medium-resolution descent image was better than 1 pixel. The landing position was calculated to be (44.1196^°N, 19.5148^°W). This method comprehensively utilized multi-source images, and didnt entirely depend on the mapping accuracy of the landing base map. It was a further refinement of the matching results between the images and landing base map. So, this high-precision positioning method can be applied in future deep space exploration missions.

Panoramic Visual SLAM Technology for Spherical Images.

Simultaneous Localization and Mapping (SLAM) technology is one of the best methods for fast 3D reconstruction and mapping. However, the accuracy of SLAM is not always high enough, which is currently the subject of much research interest. Panoramic vision can provide us with a wide range of angles of view, many feature points, and rich information. The panoramic multi-view cross-imaging feature can be used to realize instantaneous omnidirectional spatial information acquisition and improve the positioning accuracy of SLAM. In this study, we investigated panoramic visual SLAM positioning technology, including three core research points: (1) the spherical imaging model; (2) spherical image feature extraction and matching methods, including the Spherical Oriented FAST and Rotated BRIEF (SPHORB) and ternary scale-invariant feature transform (SIFT) algorithms; and (3) the panoramic visual SLAM algorithm. The experimental results show that the method of panoramic visual SLAM can improve the robustness and accuracy of a SLAM system.

Measurement of Three-Dimensional Characteristics of Slug Flow

Gas-liquid slug flow in horizontal pipes occurs frequently in industrial applications and is harmful to the piping systems owing to its intermittent structures. Therefore, it is essential to accurately describe the flow structure and investigate the flow mechanism. A non-intrusive fluorescence imaging method is employed to quantitatively reconstruct the slug bubble in a 15mm ID horizontal pipe in this article. Optical distortion correction method is firstly developed to improve the measurement accuracy of the instantaneous gas holdup, and then the time-average axial bubble velocity is simultaneously measured according to the feature matching of cross-sectional images. Additionally, the influence of phase superficial velocity on the interface stability is investigated through the phase holdup and the circumferential curvature distribution analysis. The typical three-dimensional structures of slug bubble are finally reconstructed through the velocity-based bicubic interpolation and median fusion algorithm. Previous correlations and reference instrumentation are used to evaluate the proposed method. Comparison results show that the method can effectively measure the flow parameters and reconstruct slug bubbles.

基于改进ORB的图像特征匹配算法研究

基于改进ORB的图像特征匹配算法研究

Disparity image feature matching algorithm based on AKAZE and adaptive local affine matching

针对常用的图像特征匹配算法对具有视差的图像在图像特征匹配阶段会产生大量误匹配点的问题，提出了一种AKAZE（accelerated-KAZE）算法结合自适应局部仿射匹配的特征匹配算法。首先，采用AKAZE算法提取特征点；接着，采用二进制描述符M-LDB(modified-local difference binary)进行描述并进行暴力匹配产生粗匹配点对；最后，基于图像的仿射变换可以提供较强的几何约束这一特性，采用自适应局部仿射匹配完成精匹配。实验结果表明，该算法针对具有旋转变化、尺度变化、视角变化的图像匹配，具有提取特征点均匀、匹配准确等效果，提取的正确特征点数量分别平均相对于SIFT算法提升了1.66倍、SURF算法提升了1.08倍、ORB算法提升了6.92倍、GMS算法提升了1.23倍，能够满足具有较大视差图像匹配的需求。

GNC architecture for an optimal rendezvous to an uncooperative tumbling target using passive monocular camera

Many recent researches have been studying techniques to handle the problem of pose and shape reconstruction in the framework of space proximity operations. The most challenging scenario is when the target satellite is uncooperative, i.e., out of control, and possibly of unknown shape because of damages occurred to its structure. In such a situation, complete awareness of the target’s configuration is mandatory and can be achieved through optical hardware. Moreover, recent studies have shown how passive optical sensors are so far the best choice when limitations of mass and power consumption on board the chaser satellite are considered. Monocular cameras, if complemented with a single depth information (e.g., provided by a range-finder), can be used to extract meaningful information of the observed satellite. When paired with purposely built filtering algorithms, the accuracy reached by such systems can increase considerably. In this work, a navigation algorithm exploiting images acquired by a camera is included in a GNC architecture, aimed to perform a rendezvous to the target satellite. The navigation algorithm consists of the following blocks: an image features extraction and matching block, tuned to track features for as long as possible; a features sorting and storing routine, used to rearrange features in a usable way; a UKF filter, which is fed up with features’ location on the image plane and range distance, and returns the estimates of relative position, velocity, attitude, angular velocity along with the target reconstructed shape. From the operational point of view, firstly the possible docking area is identified from the acquired images. Then, after an initial phase in which the navigation blocks estimates the relative pose and shape, the reference approaching trajectory is computed through an optimization procedure carried out by means of an inverse method. The trajectory is parameterized through a polynomial basis and a gradient method is applied to seek for a minimum of the fuel consumption, under constraints such as collision avoidance and continuous visibility. The main contribution of this research resides in the integration of visual navigation with optimal guidance. In fact, iterative successful calculation of the optimal trajectory is exclusively possible if an accurate estimate of target satellite’s relative pose and shape is available. The GNC architecture is tested with a co-simulation approach, in which the orbital dynamics, navigation filtering and control laws are developed in MATLAB, while synthetic images are rendered and imported in closed-loop from Blender. Different dynamical states of the target are considered to analyze the robustness of the proposed architecture and to highlight its operational limitations.

Research on Binocular Visual System of Robotic Arm Based on Improved SURF Algorithm

With the increasingly extensive application of robots in industrial production, visual of the robotic arm has become an important research direction in the field of robots, which is also recognized as believed to widely considered to the most vital directions of modern high-tech development. In the robot vision system, the binocular visual system is most widely applied. When the robotic arm based on binocular vision captures the object, SURF algorithm is adopted to realize feature point extraction and matching of images. Because of the influence of the objective factors, there is false match in the process of SURF matching of feature points. In this paper, SURF algorithm is improved and reverse feature vectors matching strategy is used to initialize the matching points set. Then the RANSAC algorithm is adopted to select the initial matching points set in accordance of the polar geometric model to get the final good match set. The experiment results show that the improved SURF algorithm has higher registration accuracy and faster speed than SURF.

Registration of multimodal images with edge features and scale invariant PIIFD

For the past few decades, due to the extensive application of multi-sensor vision systems, the technology of multimodal image registration has been continuously developed, especially the registration of infrared and visible images. However, due to the difference of grayscale distribution, the results of most existing methods for infrared and visible image registration are commonly not satisfactory. Therefore, this paper proposes a method for visible and infrared image registration using edge features and partial intensity invariant feature descriptors (PIIFD) with scale invariance. This method extracts conspicuous edge features of the source image pair through a window grayscale weight algorithm, and calculates the feature descriptions of the edge images with scale invariant PIIFD. And then Gaussian field estimator and affine transform are used to realize image feature matching and alignment transformation. The quality and quantity comparisons with 5 other most advanced methods reveal that our method can attain an ascendant performance, can accurately and efficiently realize the registration of infrared and visible images.

A high-speed feature matching method of high-resolution aerial images

This paper presents a novel corner detection and scale estimation algorithm for image feature description and matching. Inspired by Adaboost’s weak classifier, a series of sub-detectors is elaborately designed to obtain reliable corner pixels. The new corner detection algorithm is more robust than the FAST and HARRIS algorithm, and it is especially suitable for the implementation in FPGA. The new scale estimation method can be directly implemented in the original image without building Gaussian pyramid and searching max response value in each level, which not only increase computational efficiency but also greatly reduces memory requirement. Based on the proposed algorithm, a CPU-FPGA cooperative parallel processing architecture is presented. The architecture overcomes the memory space limitation of FPGA and achieves high-speed feature matching for massive high-resolution aerial images. The speed of the CPU-FPGA cooperative process is hundred times faster than SIFT algorithm running on CPU, and dozens of times faster than SIFT running in CPU + GPU system.

Design and Implementation of Marine Automatic Target Recognition System Based on Visible Remote Sensing Images

Gao, X.B., 2020. Design and implementation of marine automatic target recognition system based on visible remote sensing images. In: Bai, X. and Zhou, H. (eds.), Advances in Water Resources, Environmental Protection, and Sustainable Development. Journal of Coastal Research, Special Issue No. 115, pp. 277-279. Coconut Creek (Florida), ISSN 0749-0208.In the process of visible light remote sensing imaging, different sea surface wave conditions have different reflection ability to illumination, which makes the brightness, contrast and other information of visible light remote sensing images have great changes. Marine ship target recognition is an important application of image processing in remote sensing field. Based on pixel-level fusion of visible and infrared bispectral images, this paper extracts features from fusion results, and recognizes targets in images according to the matching of target image features and template image features. Comprehensive utilization of image preprocessing, image smoothing and anti-cloud interference algorithms can quickly and accurately realize ship target detection under the complex background of land and sea. The experimental results show that the algorithm is effective both in physical mechanism and in actual image effect.

A passive technique for detecting copy-move forgeries by image feature matching

Due to the recent evolutions in the technologies various digital devices and image processing tools are available in the market. Consequently, crime rates are also proliferating in the developed and developing regions of the world. One such crime is the manipulation of digital image contents that can be achieved by using commercial and open-source image manipulation tools. The most widespread approach for image contents manipulation is the copy-move forgery. While crafting the copy-move forgeries (CMFs) it is often required to conceal undesired regions or duplicate desired regions in the image. Thus, forensic applications are needed to certify the contents of an image and to expose the manipulated areas. In this study, we are presenting a technique for detecting CMFs in digital images by image feature matching. The technique segments the suspected image into overlapping blocks and Tchebichef moments are computed for every block to characterize the manipulated regions of the image. Tchebichef moments are adopted because of their ability to represent image features more effectively as compared to other moments such as Legendre and Zernike moments. Each block of Tchebichef moments is further segmented into non-overlapping blocks and processed through singular values decomposition (SVD). To obtain a reduced size feature vector largest singular values of each sub-block are used that also enhanced the performance in the feature matching process. In the experiments standard databases namely, the DVMM Columbia University dataset, COVERAGE, and CoMoFoD are utilized to assess the performance of the proposed approach. The results conclusively demonstrate that the presented technique based on Tchebichef moments and SVD transcends the other methods.

Research on image feature matching algorithm based on feature optical flow and corner feature

Image feature matching is an important part of visual odometry. In order to improve the accuracy of feature point matching for visual odometry, an accurate algorithm combining the pyramid feature optical flow and corner features based on image feature matching is proposed. Firstly, the Oriented Features from Accelerated Segment Test and Rotated BRIEF (ORB) algorithm is used to extract the image feature points quickly. Secondly, the local feature window is utilised to calculate the displacement vector of the image feature points and the pyramid Lucas–Kanade feature optical flow is used to track feature points. Then, to solve the problem of matching alignment and feature loss, the K nearest neighbour radius search is used as feature filter to remove the confused matching. Finally, the Random Sample Consensus algorithm is introduced to eliminate redundant mismatch points and improved the matching rate. The comparison of experimental data shows that the proposed algorithm can get high matching rate. Compared with the traditional ORB feature matching algorithm, the algorithm had a significant improvement in real time and image feature matching accuracy.

New Fusion Techniques to Extract Image Features and Recognition of Palm Vascular Pattern

A strong and efficient Feature extraction algorithm is highly recommended for individual recognition in human authentication systems. This paper presents the work carried on palm vein image to extract features of the person vein for recognition and classification using improved canny edge detector. This paper describes a novel method to extract valuable features of the people’s vein pattern and achieving high recognition rate. The experiments carried using two algorithms 1) PCACE (principal component analysis with canny edge) algorithm and 2) LDACE (linear discriminant analysis with canny edge) algorithm. These two methods are analyzed on palm vein image and found LDACE algorithm is a best extractor compare to PCACE method. An Equal Error Rate (EER) is applied to evaluate two algorithms. Hidden Markova Model (HMM) is utilized for image feature classification and matching using contactless Palm Under Test (PUT) palm vein database. The percentage of recognition is measured by False Acceptance Ratio (FAR) and False Rejection Ratio (FRR). This method shows robust response with respect to human palm vein identification process computation time and improved recognition rate.