Image Feature Detection Research Articles

An intelligent framework for transmission map estimation in image dehazing using total variation regularized low-rank approximation

The presence of haze affects a multitude of applications that require detection of image features, such as target tracking, object recognition and camera-based advanced driving assistance systems. In this paper, an optimization framework is proposed to efficiently estimate the scene transmission map which aids the dehazing process in an effective manner. In the formulated optimization model, low-rank approximation using weighted nuclear norm minimization is introduced to smoothen the coarse transmission map obtained from hazy data in order to avoid the visual artifacts in the dehazed image. Total variation regularization is employed to preserve the prominent edges and salient structural details in the transmission map. Moreover, the inclusion of $$l_1$$ norm minimization helps to obtain a finer transmission map by enhancing the minute sparse structural details, thereby providing good dehazing results. The beauty of the proposed model is confined in the efficient formulation of a unified optimization model for the estimation of transmission map with fine-tuned regularization terms which is not reported until now in the direction of image dehazing. The extensive experiments prove that the proposed method surpasses the state-of-the-art methods in image dehazing.

Optimization and Innovation of SIFT Feature Matching Algorithm in Static Image Stitching Scheme

With the development of computer technology, people can obtain electronic images by a variety of means, but these images often need to obtain a panoramic view of a large field of view from multiple limited field of view images because the field of view is smaller than that of humans, and image stitching technology can be very good Solve the contradiction between vision and resolution. This paper studies the extraction and registration of image feature points and its related technologies, including image feature detection and extraction, point feature-based matching, and image fusion. According to the requirements of matching and image splicing, the feature points with good robustness to image deformation and occlusion are used as image features to indicate that it meets the requirements of image splicing registration. This paper studies and designs an improved algorithm to improve the accuracy of feature points extraction. Further research and analysis of the SIFT operator based on the characteristics of the invariant technology realize that the feature points extracted from the image can be used for reliable matching of an object or scene from different perspectives.

A Full-Featured FPGA-Based Pipelined Architecture for SIFT Extraction

Image feature detection is a key task in computer vision. Scale Invariant Feature Transform (SIFT) is a prevalent and well known algorithm for robust feature detection. However, it is computationally demanding and software implementations are not applicable for real-time performance. In this paper, a versatile and pipelined hardware implementation is proposed, that is capable of computing keypoints and rotation invariant descriptors on-chip. All computations are performed in single precision floating-point format which makes it possible to implement the original algorithm with little alteration. Various rotation resolutions and filter kernel sizes are supported for images of any resolution up to ultra-high definition. For full high definition images, 84 fps can be processed. Ultra high definition images can be processed at 21 fps.

An Efficient Method for Generating UAV-Based Hyperspectral Mosaics Using Push-Broom Sensors

Hyperspectral sensors mounted in unmanned aerial vehicles offer new opportunities to explore high-resolution multitemporal spectral analysis in remote sensing applications. Nevertheless, the use of hyperspectral data still poses challenges mainly in postprocessing to correct from high geometric deformation of images. In general, the acquisition of high-quality hyperspectral imagery is achieved through a time-consuming and complex processing workflow. However, this effort is mandatory when using hyperspectral imagery in a multisensor data fusion perspective, such as with thermal infrared imagery or photogrammetric point clouds. Push-broom hyperspectral sensors provide high spectral resolution data, but its scanning acquisition architecture imposes more challenges to create geometrically accurate mosaics from multiple hyperspectral swaths. In this article, an efficient method is presented to correct geometrical distortions on hyperspectral swaths from push-broom sensors by aligning them with an RGB photogrammetric orthophoto mosaic. The proposed method is based on an iterative approach to align hyperspectral swaths with an RGB photogrammetric orthophoto mosaic. Using as input preprocessed hyperspectral swaths, apart from the need of introducing some control points, the workflow is fully automatic and consists of: adaptive swath subdivision into multiple fragments; detection of significant image features; estimation of valid matches between individual swaths and the RGB orthophoto mosaic; and calculation of the best geometric transformation model to the retrieved matches. As a result, geometrical distortions of hyperspectral swaths are corrected and an orthomosaic is generated. This methodology provides an expedite solution able to produce a hyperspectral mosaic with an accuracy ranging from two to five times the ground sampling distance of the high-resolution RGB orthophoto mosaic, enabling the hyperspectral data integration with data from other sensors for multiple applications.

Automatically Detecting Image–Text Mismatch on Instagram with Deep Learning

Visual social media have emerged as an essential brand communication channel for advertisers and brands. The active use of hashtags has enabled advertisers to identify customers interested in their brands and better understand their consumers. However, some users post brand-incongruent content—for example, posts composed of brand-irrelevant images with brand-relevant hashtags. Such visual information mismatch can be problematic because it hinders other consumers’ information search processes and advertisers’ insight generation from consumer-initiated social media data. This study aims to characterize visually mismatched content in brand-related posts on Instagram and builds a visual information mismatch detection model using computer vision. We propose a machine-learning model based on three cues: image, text, and metadata. Our analysis shows the effectiveness of deep learning and the importance of combining text and image features for mismatch detection. We discuss the advantages of machine-learning methods as a novel research tool for advertising research and conclude with implications of our findings.

Recent advances in local feature detector and descriptor: a literature survey

The computer vision system is the technology that deals with identifying and detecting the objects of a particular class in digital images and videos. Local feature detection and description play an essential role in many computer vision applications like object detection, object classification, etc. The accuracy of these applications depends on the performance of local feature detectors and descriptors used in the methods. Over the past decades, new algorithms and techniques have been introduced with the development of machine learning and deep learning techniques. The machine learning techniques can lead the work to the next level when sufficient data is provided. Deep learning algorithms can handle a large amount of data efficiently. However, this may raise questions in a researcher’s mind about selecting the best algorithm and best method for a particular application to increase the performance. The selection of the algorithms highly depends on the type of application and amount of data to be handled. This encouraged us to write a comprehensive survey of local image feature detectors and descriptors from state-of-the-art to the recent ones. This paper presents feature detection and description methods in the visible band with their advantages and disadvantages. We also gave an overview of current performance evaluations and benchmark datasets. Besides, the methods and algorithms are described to find the features beyond the visible band. Finally, we concluded the survey with future directions. This survey may help researchers and serve as a reference in the field of the computer vision system.

Eye tracking reveals expertise-related differences in the time-course of medical image inspection and diagnosis.

Purpose: Physicians' eye movements provide insights into relative reliance on different visual features during medical image review and diagnosis. Current theories posit that increasing expertise is associated with relatively holistic viewing strategies activated early in the image viewing experience. This study examined whether early image viewing behavior is associated with experience level and diagnostic accuracy when pathologists and trainees interpreted breast biopsies. Approach: Ninety-two residents in training and experienced pathologists at nine major U.S. medical centers interpreted digitized whole slide images of breast biopsy cases while eye movements were monitored. The breadth of visual attention and frequency and duration of eye fixations on critical image regions were recorded. We dissociated eye movements occurring early during initial viewing (prior to first zoom) versus later viewing, examining seven viewing behaviors of interest. Results: Residents and faculty pathologists were similarly likely to detect critical image regions during early image viewing, but faculty members showed more and longer duration eye fixations in these regions. Among pathology residents, year of residency predicted increasingly higher odds of fixating on critical image regions during early viewing. No viewing behavior was significantly associated with diagnostic accuracy. Conclusions: Results suggest early detection and recognition of critical image features by experienced pathologists, with relatively directed and efficient search behavior. The results also suggest that the immediate distribution of eye movements over medical images warrants further exploration as a potential metric for the objective monitoring and evaluation of progress during medical training.

Robustness evaluation of remote-sensing image feature detectors with TH priori-information data set

In this paper, we build a remote-sensing satellite imagery priori-information data set, and propose an approach to evaluate the robustness of remote-sensing image feature detectors. The building TH Priori-Information (TPI) data set with 2297 remote sensing images serves as a standardized high-resolution data set for studies related to remote-sensing image features. The TPI contains 1) raw and calibrated remote-sensing images with high spatial and temporal resolutions (up to 2 m and 7 days, respectively), and 2) a built-in 3-D target area model that supports view position, view angle, lighting, shadowing, and other transformations. Based on TPI, we further present a quantized approach, including the feature recurrence rate, the feature match score, and the weighted feature robustness score, to evaluate the robustness of remote-sensing image feature detectors. The quantized approach gives general and objective assessments of the robustness of feature detectors under complex remote-sensing circumstances. Three remote-sensing image feature detectors, including scale-invariant feature transform (SIFT), speeded up robust features (SURF), and priori information based robust features (PIRF), are evaluated using the proposed approach on the TPI data set. Experimental results show that the robustness of PIRF outperforms others by over 6.2%.

Toward a High-Definition Remote Sensing Approach to the Study of Deserted Medieval Cities in the Near East

The variability of currently available remote sensing datasets raises the question of which specific processing methods should be used for feature detection and feature extraction in both large and small-scale overhead images. In some cases, particular analyses allow us to carry out feature detection much more easily and effectively. The high-definition approach enables enhanced analysis of remote sensing data using all the spectral and graphical potential of multi-temporal ordered components. The deserted urban site of Kona Makhmūr, Iraqi Kurdistan, is taken as a case study to demonstrate this fine-grained approach. The analysis described in this paper is based on the complementary use of a variety of overlayed imagery, augmented by data from terrestrial surveys. The resulting map substantially enhances our knowledge of the built environment and the economic and environmental conditions of this early Islamic-period site. Spectral analysis of raster images allowed us to detect the real shapes of features, and with the addition of digital elevation model (DEM) (created via unmanned aerial system (UAV)) we were also able to interpret the state of preservation of the overground relics and the diachronic dynamics of their degradation.

Real-Time Video Stitching for Mine Surveillance Using a Hybrid Image Registration Method

Video stitching technology provides an effective solution for a wide viewing angle monitoring mode for industrial applications. At present, the observation angle of a single camera is limited, and the monitoring network composed of multiple cameras will have many overlapping images captured. Monitoring surveillance cameras can cause the problems of viewing fatigue and low video utilization rate of involved personnel. In addition, current video stitching technology has poor adaptability and real-time performance. We propose an effective hybrid image feature detection method for fast video stitching of mine surveillance video using the effective information of the surveillance video captured from multiple cameras in the actual conditions in the industrial coal mine. The method integrates the Moravec corner point detection and the scale-invariant feature transform (SIFT) feature extractor. After feature extraction, the nearest neighbor method and the random sampling consistency (RANSAC) algorithm are used to register the video frames. The proposed method reduces the image stitching time and solves the problem of feature re-extraction due to the change of observation angle, thus optimizing the entire video stitching process. The experimental results on the real-world underground mine videos show that the optimized stitching method can stitch videos at a speed of 21 fps, effectively meeting the real-time requirement, while the stitching effect has a good stability and applicability in real-world conditions.

Improved photometric acceptance testing in image feature extraction tasks

Image correspondence is established by “matching” the feature descriptors of the interest points in the target image to that of the reference image. By acceptance testing, we refer to a postmatching hypothesis test used to screen out potential false matches—conventionally using descriptor test statistics. We propose a new acceptance testing strategy that does not rely on the descriptor test statistics exclusively. The contribution we bring is to demonstrate that, unlike feature matching, acceptance testing may incorporate additional photometric values of the scene to improve the recall rate. We show experimentally that the acceptance testing strategy that incorporates image feature detection statistics we refer to as detector response-ratio thresholding that are usually excluded from the feature descriptor vectors has a superior recall–precision performance compared to the state-of-the-art feature extraction techniques.

Feature Point Registration Model of Farmland Surface and Its Application Based on a Monocular Camera.

In this study, an image registration algorithm was applied to calculate the rotation angle of objects when matching images. Some commonly used image feature detection algorithms such as features from accelerated segment test (FAST), speeded up robust features (SURF) and maximally stable extremal regions (MSER) algorithms were chosen as feature extraction components. Comparing the running time and accuracy, the image registration algorithm based on SURF has better performance than the other algorithms. Accurately obtaining the roll angle is one of the key technologies to improve the positioning accuracy and operation quality of agricultural equipment. To acquire the roll angle of agriculture machinery, a roll angle acquisition model based on the image registration algorithm was built. Then, the performance of the model with a monocular camera was tested in the field. The field test showed that the average error of the rolling angle was 0.61°, while the minimum error was 0.08°. The field test indicated that the model could accurately obtain the attitude change trend of agricultural machinery when it was working in irregular farmlands. The model described in this paper could provide a foundation for agricultural equipment navigation and autonomous driving.

Deployment of Artificial Intelligence in Real-World Practice: Opportunity and Challenge.

Artificial intelligence has rapidly evolved from the experimental phase to the implementation phase in many image-driven clinical disciplines, including ophthalmology. A combination of the increasing availability of large datasets and computing power with revolutionary progress in deep learning has created unprecedented opportunities for major breakthrough improvements in the performance and accuracy of automated diagnoses that primarily focus on image recognition and feature detection. Such an automated disease classification would significantly improve the accessibility, efficiency, and cost-effectiveness of eye care systems where it is less dependent on human input, potentially enabling diagnosis to be cheaper, quicker, and more consistent. Although this technology will have a profound impact on clinical flow and practice patterns sooner or later, translating such a technology into clinical practice is challenging and requires similar levels of accountability and effectiveness as any new medication or medical device due to the potential problems of bias, and ethical, medical, and legal issues that might arise. The objective of this review is to summarize the opportunities and challenges of this transition and to facilitate the integration of artificial intelligence (AI) into routine clinical practice based on our best understanding and experience in this area.

A 3D reconstruction method of image sequence based on deep learning

Camera calibration, image feature detection, matching and other aspects have become barriers that traditional 3D reconstruction methods are difficult to break through. The important role of deep learning in data detection and classification has a deep impact on the real world, and has become a research hotspot at home and abroad to deal with this problem. In this paper, a 3D reconstruction method of image sequence based on depth learning is proposed. Firstly, the principle and method of 3D reconstruction are introduced. Then, the new method of 3D reconstruction based on image is studied and discussed in combination with deep learning theory. Finally, the conclusion and prospect are given.

Lesion Feature Detection of Retinopathy Images

In recent years, Diabetic Retinopathy (DR) is fast growing health problem in the society. Ophthalmologists use exudates, microaneurysms and hemorrhages features to diagnose the DR. Detection of these features plays important role in diagnosis and further treatment. This paper proposes automatic lesion detection and analysis methods so as to identify all three features without human intervention. The algorithm developed includes pre-processing as well as post processing, which is very important in DR images. Extraction of minute details in lesions requires highly enhanced images. Contrast enhancement and Morphological operations were applied on green channel DR images. Further, the segmentation is performed on enhanced DR images to capture required region of interest. The Kirsch algorithm has been applied to capture exudates candidate from DR image. Further Top-hat method on preprocessed image extracts microaneurysms and Thresholding technique has been applied for hemorrhage detection. The performance evaluation of algorithms is carried out through the publicly available database. The proposed algorithms depict better results in detection of three major parameters from digital retinopathy images with respect to toolkit evaluation adapted. This automatic detection method helps in reducing screening DR image time as well. The development of proposed algorithm leads to accurate DR detection by helping Ophthalmologists in the diagnosis process.

A smart surface inspection system using faster R-CNN in cloud-edge computing environment

Automated surface inspection has become a hot topic with the rapid development of machine vision technologies. Traditional machine vision methods need experts to carefully craft image features for defect detection. This limits their applications to wider areas. The emerging convolutional neural networks (CNN) can automatically extract features and yield good results in many cases. However, the CNN-based image classification methods are more suitable for flat surface texture inspection. It is difficult to accurately locate small defects in geometrically complex products. Furthermore, the computational power required in CNN algorithms is usually high and it is not efficient to be implemented on embedded hardware. To solve these problems, a smart surface inspection system is proposed using faster R-CNN algorithm in the cloud-edge computing environment. The faster R-CNN as a CNN-based object detection method can efficiently identify defects in complex product images and the cloud-edge computing framework can provide fast computation speed and evolving algorithm models. A real industrial case study is presented to illustrate the effectiveness of the proposed method. The results show that the proposed method can provide high detection accuracy within a short time.

Two-Stream RGB-D Human Detection Algorithm Based on RFB Network

In order to effectively combine RGB image features with depth image features for human detection, this paper proposes a two-stream RGB-D human detection algorithm based on RFB network. The proposed algorithm mainly contains three parts: RGB-stream, Depth-stream and Channel Weight Fusion (CWF) strategy. (1) The RGB-stream extracts RGB image features using RFB-Net as the backbone network. (2) By analyzing the results of depth features visualization, we build the Depth-stream, which can effectively extract the depth image features. (3) The improved CWF strategy can enhance the effectiveness of important channels in RGB-D fusion features and improve the capability of the network expression. The experimental results show that the proposed algorithm has a significant improvement compared with other algorithms on two common datasets.

Boosting Feature Matching Accuracy with Pairwise Affine Estimation.

Local image feature matching lies in the heart of many computer vision applications. Achieving high matching accuracy is challenging when significant geometric difference exists between the source and target images. The traditional matching pipeline addresses the geometric difference by introducing the concept of support region. Around each feature point, the support region defines a neighboring area characterized by estimated attributes like scale, orientation, affine shape, etc. To correctly assign support region is not an easy job, especially when each feature is processed individually. In this paper, we propose to estimate the relative affine transformation for every pair of to-be-compared features. This "tailored" measurement of geometric difference is more precise and helps improve the matching accuracy. Our pipeline can be incorporated into most existing 2D local image feature detectors and descriptors. We comprehensively evaluate its performance with various experiments on a diversified selection of benchmark datasets. The results show that the majority of tested detectors/descriptors gain additional matching accuracy with proposed pipeline.

Monocular VO Scale Ambiguity Resolution Using an Ultra Low-Cost Spike Rangefinder

Monocular visual odometry (VO) is the process of determining a user’s trajectory through a series of consecutive images taken by a single camera. A major problem that affects the accuracy of monocular visual odometry, however, is the scale ambiguity. This research proposes an innovative augmentation technique, which resolves the scale ambiguity problem of monocular visual odometry. The proposed technique augments the camera images with range measurements taken by an ultra-low-cost laser device known as the Spike. The size of the Spike laser rangefinder is small and can be mounted on a smartphone. Two datasets were collected along precisely surveyed tracks, both outdoor and indoor, to assess the effectiveness of the proposed technique. The coordinates of both tracks were determined using a total station to serve as a ground truth. In order to calibrate the smartphone’s camera, seven images of a checkerboard were taken from different positions and angles and then processed using a MATLAB-based camera calibration toolbox. Subsequently, the speeded-up robust features (SURF) method was used for image feature detection and matching. The random sample consensus (RANSAC) algorithm was then used to remove the outliers in the matched points between the sequential images. The relative orientation and translation between the frames were computed and then scaled using the spike measurements in order to obtain the scaled trajectory. Subsequently, the obtained scaled trajectory was used to construct the surrounding scene using the structure from motion (SfM) technique. Finally, both of the computed camera trajectory and the constructed scene were compared with ground truth. It is shown that the proposed technique allows for achieving centimeter-level accuracy in monocular VO scale recovery, which in turn leads to an enhanced mapping accuracy.

Feature Detection and Matching With Linear Adjustment and Adaptive Thresholding

Image feature detection and matching technologies are crucial aspects in machine vision. However it is still facing the dilemma between fast operation for real-time application and robust matching. To address this issue, we propose a robust and relatively fast method for image feature extraction and matching with linear adjustment and adaptive thresholding (LAAT) in this paper. The major challenge of this method is reducing the sensitivity to the brightness. To solve this problem, we adopt brightness and contrast adjustment for image pairs processed by Gaussian filtering. An adaptive thresholding FAST approach is applied for feature selection to improve the performance. The proposed method is compared with the traditional and state-of-the-art extraction methods on public dataset. Particularly, this paper focuses on the illumination change, image blur, and image rotation aspects. Experiments show that our proposed algorithm is superior to other algorithms in the comprehensive evaluation of various parameters, especially for illumination and blur transformations.