Image Descriptors Research Articles

Weather Forecast Based on Color Cloud Image Recognition under the Combination of Local Image Descriptor and Histogram Selection

Numerous researchers have used machine vision in recent years to identify and categorize clouds according to their volume, shape, thickness, height, and coverage. Due to the significant variations in illumination, climate, and distortion that frequently characterize cloud images as a type of naturally striated structure, the Local Binary Patterns (LBP) descriptor and its variants have been proposed as feature extraction methods for characterizing natural texture images. Rotation invariance, low processing complexity, and resistance to monotonous brightness variations are characteristics of LBP. The disadvantage of LBP is that it produces binary data that are extremely noise-sensitive and it struggles on regions of the image that are “flat” because it depends on intensity differences. This paper considers the Local Ternary Patterns (LTP) feature to overcome the drawbacks of the LBP feature. We also propose the fusion of color characteristics, LBP features, and LTP features for the classification of cloud/sky images. Morover, this study proposes to apply the Intra-Class Similarity (ICS) technique, a histogram selection approach, with the goal of minimizing the number of histograms for characterizing images. The proposed approach achieves better performance of recognition with less features in use by fusing LBP and LTP features and using the ICS technique to choose potential histograms.

Tensor Affinity Learning for Hyperorder Graph Matching

Hypergraph matching has been attractive in the application of computer vision in recent years. The interference of external factors, such as squeezing, pulling, occlusion, and noise, results in the same target displaying different image characteristics under different influencing factors. After extracting the image feature point description, the traditional method directly measures the feature description using distance measurement methods such as Euclidean distance, cosine distance, and Manhattan distance, which lack a sufficient generalization ability and negatively impact the accuracy and effectiveness of matching. This paper proposes a metric-learning-based hypergraph matching (MLGM) approach that employs metric learning to express the similarity relationship between high-order image descriptors and learns a new metric function based on scene requirements and target characteristics. The experimental results show that our proposed method performs better than state-of-the-art algorithms on both synthetic and natural images.

Deep metric learning with mirror attention and fine triplet loss for fundus image retrieval in ophthalmology

Fundus image retrieval can help ophthalmologists make evidence-based medico-decision by providing similar cases. Its basic task is to learn highly discriminative visual descriptors from image space, in which lesion features are the main differentiating clue. Lesions in fundus images appear small in size, similar in textures, and scatter around vessels, such as microaneurysms and hemorrhages. Hence, although a single small lesion has a saliently visual manifestation, its discriminative information is hard to reserve in the last image descriptors. For fundus images, the optic disc of the left and right eyes are symmetric, and the macular area lies in the central axis from the vertical view. Based on such spatial structure and lesion characteristics, we present a novel deep metric learning framework equipped with mirror attention to enhance the discriminative features of small and scattering lesions and encode them into image descriptors. The mirror attention can give lesions high attention scores by capturing spatial dependency of vertical and horizontal views, especially the relations between lesions and vessels. Based on the mirror attention, we further propose a new fine triplet loss to confine distances of positive pairs by exploiting the learned relevant degrees of positive pairs in a self-supervised manner. The fine triplet loss can help detect the subtle differences of positive pairs to improve the ranking performance of hit items. To demonstrate the effectiveness of improving retrieval performance, we conduct comprehensive experiments on the largest fundus dataset of diabetic retinopathy (DR) detection and achieve the best precision compared to counterparts. The experiments show that our method produces significant performance improvements for fundus image retrieval, especially the ranking quality of DR grades containing microaneurysms and hemorrhages. Our proposed mirror attention can be applied to off-the-shelf backbones and trained efficiently in an end-to-end manner for other medical images to obtain highly discriminative image descriptors.

Deep Feature Space: A Geometrical Perspective.

One of the most prominent attributes of Neural Networks (NNs) constitutes their capability of learning to extract robust and descriptive features from high dimensional data, like images. Hence, such an ability renders their exploitation as feature extractors particularly frequent in an abundance of modern reasoning systems. Their application scope mainly includes complex cascade tasks, like multi-modal recognition and deep Reinforcement Learning (RL). However, NNs induce implicit biases that are difficult to avoid or to deal with and are not met in traditional image descriptors. Moreover, the lack of knowledge for describing the intra-layer properties -and thus their general behavior- restricts the further applicability of the extracted features. With the paper at hand, a novel way of visualizing and understanding the vector space before the NNs' output layer is presented, aiming to enlighten the deep feature vectors' properties under classification tasks. Main attention is paid to the nature of overfitting in the feature space and its adverse effect on further exploitation. We present the findings that can be derived from our model's formulation and we evaluate them on realistic recognition scenarios, proving its prominence by improving the obtained results.

Forest: A Lightweight Semantic Image Descriptor for Robust Visual Place Recognition

Visual place recognition (VPR) is the process of identifying previously visited places using visual information. It is crucial for a robot to achieve fast and accurate VPR under appearance and viewpoint changes. Inspired by human perception intuition, this letter proposes a lightweight semantic image descriptor called Forest for robust VPR. Specifically, for each semantic object in the image, three most recognizable reference objects are first found and ranked in descending order according to a customized measure. And then, categories, position and area of the semantic object itself, along with the categories of the three recognized reference objects, are used to encode the local descriptor of the object. As a result, the set of local descriptors is used as the global descriptor of the image. Performance of the proposed descriptor is evaluated and compared with six state-of-the-art VPR methods, Ibow-LCD, CoHOG, NetVLAD, LOST-X, Region VLAD, Patch-NetVLAD, over four public datasets of urban scenes, Extended-CMU Season, RobotCar Seasons v2, SYNTHIA and Kitti, using three Precision-Recall based metrics, AUC(Area Under Curve), Recall@100%Precision and Precision@100%Recall. The results show that our method maintains competitive accuracy in scenes with appearance and viewpoint changes and exhibits high robustness. At the same time, our method is robust to the noise of semantic segmentation and exhibits high real-time performance.

Efficient feature descriptor selection for improved Arabic handwritten words recognition

<span lang="EN-US">Arabic handwritten text recognition has long been a difficult subject, owing to the similarity of its characters and the wide range of writing styles. However, due to the intricacy of Arabic handwriting morphology, solving the challenge of cursive handwriting recognition remains difficult. In this paper, we propose a new efficient based image processing approach that combines three image descriptors for the feature extraction phase. To prepare the training and testing datasets, we applied a series of preprocessing techniques to 100 classes selected from the handwritten Arabic database of the </span><em><span lang="EN-US">Institut Für Nachrichtentechnik/Ecole Nationale d'Ingénieurs de Tunis</span></em><span lang="EN-US"> (IFN/ENIT). Then, we trained the k-nearest neighbor’s algorithm (k-NN) algorithm to generate the best model for each feature extraction descriptor. The best k-NN model, according to common performance evaluation metrics, is used to classify Arabic handwritten images according to their classes. Based on the performance evaluation results of the three k-NN generated models, the majority-voting algorithm is used to combine the prediction results. A high recognition rate of up to 99.88% is achieved, far exceeding the state-of-the-art results using the IFN/ENIT dataset. The obtained results highlight the reliability of the proposed system for the recognition of handwritten Arabic words.</span>

Images indexing and matched assessment of semantics and visuals similarities applied to a medical learning X-ray image base

Medical diagnostic support systems are important tools in the field of radiology. However, the precision obtained, during the exploitation of high homogeneity image datasets, needs to be improved. To develop a new learning system dedicated to public health practitioners. This study presents an upgraded version dedicated to radiology experts for better clinical decision-making when diagnosing and treating the patient (CAD approach). Our system is a hybrid approach based on a matching of semantic and visual attributes of images. It is a combination of two complementary subsystems to form the intermodal system. The first one named α based on semantic attributes. Indexing and image retrieval based on specific keywords. The second system named β based on low-level attributes. Vectors characterizing the digital content of the image (color, texture and shape) represent images. Our image database consists of 930 X-ray images including 320 mammograms acquired from the mini-MIAS database of mammograms and 610 X-rays acquired from the Public Hospital Establishment (EPH-Rouiba Algeria). The combination of two subsystems gives rise to the intermodal system: α-subsystem offers an overall result (based on visual descriptors), then β-subsystem (low level descriptors) refines the result and increases relevance. Our system can perform a specific image search (in a database of images with very high homogeneity) with an accuracy of around 90% for a recall of 25%. The average (overall) accuracy of the system exceeds 70%. The results obtained are very encouraging, and demonstrate efficiency of our approach, particularly for the intermodal system.

LiDAR-Based Global Localization Using Histogram of Orientations of Principal Normals

Global localization on LiDAR point cloud maps is a challenging task because of the sparse nature of point clouds and the large size difference between LiDAR scans and the maps. In this paper, we solve the LiDAR-based global localization problem based upon the plane-motion assumption. We first project the clouds into Bird’s-eye View (BV) images and transform the problem into a BV image matching problem. We then introduce a novel local descriptor, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , Histogram of Orientations of Principal Normals (HOPN), to perform matching. The HOPN descriptor encodes the point normals of the clouds, and is more effective in matching BV images than the common image descriptors. In addition, we present the consensus set maximization algorithm to robustly estimate a rigid pose from the HOPN matches in the case of the low inlier ratio. The experimental results on three large-scale datasets show that our method achieves state-of-the-art global localization performance when using either single LiDAR scans or local maps.

Dimension Reduction of Digital Image Descriptors in Neural Identification of Damaged Malting Barley Grains.

The paper covers the problem of determination of defects and contamination in malting barley grains. The analysis of the problem indicated that although several attempts have been made, there are still no effective methods of identification of the quality of barley grains, such as the use of information technology, including intelligent sensors (currently, quality assessment of grain is performed manually). The aim of the study was the construction of a reduced set of the most important graphic descriptors from machine-collected digital images, important in the process of neural evaluation of the quality of BOJOS variety malting barley. Grains were sorted into three size fractions and seed images were collected. As a large number of graphic descriptors implied difficulties in the development and operation of neural classifiers, a PCA (Principal Component Analysis) statistical method of reducing empirical data contained in the analyzed set was applied. The grain quality expressed by an optimal set of transformed descriptors was modelled using artificial neural networks (ANN). The input layer consisted of eight neurons with a linear Postsynaptic Function (PSP) and a linear activation function. The one hidden layer was composed of sigmoid neurons having a linear PSP function and a logistic activation function. One sigmoid neuron was the output of the network. The results obtained show that neural identification of digital images with application of Principal Component Analysis (PCA) combined with neural classification is an effective tool supporting the process of rapid and reliable quality assessment of BOJOS malting barley grains.

Determination of a Similar Anatomical Area on a Chest CT Image Using Traditional Image Feature Extraction Methods

The traditional image descriptor definition algorithms are considered, such as SIFT, ORB, LBP, GLSM. With the help of them, the searching task for a similar anatomical area on the CT images of the lungs is solved. The article proposes a methodology for performing a comparative traditional algorithms for determining images descriptors analysis and optimal anatomical features. Algorithms are tested when searching for a similar anatomical layer in the framework of the computer tomography images layers of of light patient, as part of the search for similar anatomical form on the layer among the computer tomography images of light two patients, and among the images of computed tomography of light hundred patients. As a result, it is determined that GLSM shows the best results when solving the task of classifying an image anatomical area (averaged error of determining the anatomical layer is 5 %). It is determined that the optimal signs on the lungs correspond to the presence of organs: heart, liver and top edge of the lung. Conclusions are fomulated about the need to use neural network methods to improve the error in determining the similar layer containing the necessary anatomical structure.

An enhancement for image-based malware classification using machine learning with low dimension normalized input images

This paper proposes a simple and effective model applied for image-based malware classification using machine learning in which malware images (converted from malware binary files) are directly fed into the classifiers, i.e. k nearest neighbour (k-NN), support vector machine (SVM) and convolution neural networks (CNN). The proposed model does not use the normalized fixed-size square images (e.g. 64 × 64 pixels) or features extracted by image descriptor (e.g. GIST) for training classifiers as existing models do in the literature. Instead, the input images are normalized and horizontally sized down (the width of the image) to a lower dimension of 32 × 64, 16 × 64 or even 8 × 64 than square ones (e.g. 64 × 64 pixels) to reduce the complexity and training time of the model. It is based on the fact that the texture of the malware image is mainly vertically distributed as analysed in this paper. This finding is significant for training those devices which have limited computational resources such as IoT devices. The experiment was conducted on the Malimg, Malheur datasets which contains 9339 (25 malware families) and 3133 variant samples (24 malware families) using k-NN, SVM and CNN classifiers. The achieved results show that it is possible to reduce the dimension of the input images (i.e. 32 × 64, 16 × 64 or even 8 × 64) while still retaining the accuracy of classification as the same as the accuracy obtained by classifier feeding by the fixed-size square image (i.e. 64 × 64 pixels). As a result, training time of the propose model reduces by a half, a quarter, and one-eighth compared to training time taken by the same machine learning-based classifier (i.e. k-NN, SVM and CNN) feeding by fixed-sized square images, i.e. 64 × 64, respectively.

Local binary pattern‐based reversible data hiding

A novel local binary pattern-based reversible data hiding (LBP-RDH) technique has been suggested to maintain a fair symmetry between the perceptual transparency and hiding capacity. During embedding, the image is divided into various 3×3 blocks. Then, using the LBP-based image descriptor, the LBP codes for each block are computed. Next, the obtained LBP codes are XORed with the embedding bits and are concealed in the respective blocks using the proposed pixel readjustment process. Further, each cover image (CI) pixel produces two different stego-image pixels. Likewise, during extraction, the CI pixels are restored without the loss of a single bit of information. The outcome of the proposed technique with respect to perceptual transparency measures, such as peak signal-to-noise ratio and structural similarity index, is found to be superior to that of some of the recent and state-of-the-art techniques. In addition, the proposed technique has shown excellent resilience to various stego-attacks, such as pixel difference histogram as well as regular and singular analysis. Besides, the out-off boundary pixel problem, which endures in most of the contemporary data hiding techniques, has been successfully addressed.

VLSG-SANet: A feature matching algorithm for remote sensing image registration

Finding a reliable correspondence between two feature point sets efficiently and accurately is the first step in many computer vision applications. It is a crucial prerequisite for many remote sensings, target recognition, and photogrammetry applications. However, in different application scenarios, images may undergo different degrees of deformation, which makes the predefined geometric model no longer appropriate. This article attempts to remove mismatches between the set of assumed matching points. In order to achieve this goal, we propose an effective matching method, called ”visualized local structure generation-Siamese attention” network (VLSG-SANet), which transform the task of eliminating mismatching of feature points into a dynamic visual similarity assessment. The VLSG-SANet can generate visual descriptors by using the local structure information about each feature point. We embed the attention module based on the Siamese network for the generated feature descriptors, allowing the network to adjust the descriptors dynamically. In order to prove the robustness and versatility of the VLSG-SANet algorithm, we have performed good experiments on open remote sensing data images and got good experimental results.

DRCP: Dimensionality Reduced Chess Pattern for Person Independent Facial Expression Recognition

Automatic Facial Expression Recognition (FER) has become essential today as it has many applications in real time such as animation, driver mood detection, lie detection, and clinical psychology. The effectiveness of FER systems mainly depends on the extracted features. For extracting distinctive features with low dimensions, a new local texture-based image descriptor named Dimensionality Reduced Chess Pattern (DRCP) is proposed for recognizing facial expressions in a person independent scenario. DRCP, an improvement over Chess Pattern (CP), is mainly proposed for effectively reducing the feature vector length of CP. For feature extraction, DRCP also considers the movements of chessmen in a [Formula: see text] neighborhood, as like CP. As a part of feature extraction through DRCP, apart from the center pixel, the remaining 24 pixels are arranged into four groups in such a manner that each group contains the pixels corresponding to three chessmen. From each group, one feature is extracted and thus corresponding to four groups, four features are extracted in a [Formula: see text] neighborhood. The extracted features are fed into multi-class Support Vector Machine (SVM) for expression recognition. The experiments are performed on five “in the lab” datasets (MUG, TFEID, JAFFE, CK[Formula: see text] and KDEF) and on two “in the wild” datasets (RAF and SFEW) in person independent setup to simulate a real world scenario.

The role of 4-dimensional flow in the assessment of bicuspid aortic valve and its valvulo-aortopathies.

Bicuspid aortic valve is the most common congenital cardiac malformation and the leading cause of aortopathy and aortic stenosis in younger patients. Aortic wall remodelling secondary to altered haemodynamic flow patterns, changes in peak velocity, and wall shear stress may be implicated in the development of aortopathy in the presence of bicuspid aortic valve and dysfunction. Assessment of these parameters as potential predictors of disease severity and progression is thus desirable. The anatomic and functional information acquired from 4D flow MRI can allow simultaneous visualisation and quantification of the pathological geometric and haemodynamic changes of the aorta. We review the current clinical utility of haemodynamic quantities including velocity, wall sheer stress and energy losses, as well as visual descriptors such as vorticity and helicity, and flow direction in assessing the aortic valve and associated aortopathies.

Egg Pattern Mimicry in Avian Brood Parasitism Assessed Using Local Image Descriptors and Human-Eyes

Egg pattern plays a critical role in avian brood parasitism where a brood parasite exploits a host's parental care by laying eggs in host nests relying on the host to rear the parasitic egg(s). It has been demonstrated that many hosts have evolved an ability to recognize and reject parasitic eggs, based on differences in egg patterns, as a defense against brood parasitism, while some parasites have evolved egg mimicry to counter the host defense. Egg pattern is a complex phenotype and its quantitative evaluation has been a focus of much research. In this paper, we propose a novel method for quantifying an egg pattern and assessing the degree of egg pattern mimicry—a measure of the similarity of a parasite egg to host eggs. Our approach is based on image analysis focused on local binary pattern (LBP) and its variant completed local binary pattern (CLBP) that captures the local structure of a pattern in an image. We compare the results obtained by LBP and CLBP with human-eye evaluation, a classical method widely used in previous studies. Both LBP and CLBP can successfully assess the similarity in egg pattern that is positively correlated with human-eye scores with a high accuracy rate. LBP tends to perform better than CLBP at small scales while CLBP performs better for a wide range of larger scales. Our method can be an effective alternative means of assessing the degree of egg pattern mimicry objectively, providing a useful tool for biologists studying avian brood parasitism. Many bird species have specific eggshell patterns (e.g. spots, blotches, lines) and it has been suggested that these patterns are an important functional trait, such as for camouflage against predators. We propose our method as a useful and objective tool for assessing egg patterns.

Self-Supervised Keypoint Detection and Cross-Fusion Matching Networks for Multimodal Remote Sensing Image Registration

Remote sensing image matching is the basis upon which to obtain integrated observations and complementary information representation of the same scene from multiple source sensors, which is a prerequisite for remote sensing tasks such as remote sensing image fusion and change detection. However, the intricate geometric and radiometric differences between the multimodal images render the registration quite challenging. Although multimodal remote sensing image matching methods have been developed in recent decades, most classical and deep learning based techniques cannot effectively extract high repeatable keypoints and discriminative descriptors for multimodal images. Therefore, we propose a two-step “detection + matching” framework in this paper, where each step consists of a deep neural network. A self-supervised detection network is first designed to generate similar keypoint feature maps between multimodal images, which is used to detect highly repeatable keypoints. We then propose a cross-fusion matching network, which aims to exploit global optimization and fusion information for cross-modal feature descriptors and matching. The experiments show that the proposed method has superior feature detection and matching performance compared with current state-of-the-art methods. Specifically, the keypoint repetition rate of the detection network and the NN mAP of the matching network are 0.435 and 0.712 on test datasets, respectively. The proposed whole pipeline framework is evaluated, which achieves an average M.S. and RMSE of 0.298 and 3.41, respectively. This provides a novel solution for the joint use of multimodal remote sensing images for observation and localization.

Scale-selective and noise-robust extended local binary pattern for texture classification

As one of the most successful local feature descriptors, the local binary pattern (LBP) estimates the texture distribution rule of an image based on the signs of differences between neighboring pixels to obtain intensity- and rotation- invariance. In this paper, we propose a novel image descriptor to address scale transformation and noise interference simultaneously. We name it scale-selective and noise-robust extended LBP (SNELBP). First, each image in training sets is transformed into different scale spaces by a Gaussian filter. Second, noise-robust pattern histograms are obtained from each scale space by using our previously proposed median robust extended LBP (MRELBP). Then, scale-invariant histograms are determined by selecting the maximum among all scale levels for a certain image. Finally, the most informative patterns are selected from the dictionary pretrained by the two-stage compact dominant feature selection method (CDFS), maintaining the descriptor more lightweight with sufficiently low time cost. Extensive experiments on five public databases (Outex_TC_00011, TC_00012, KTH-TIPS, UMD and NEU) and one fresh texture database (JoJo) under two kinds of interferences (Gaussian and salt pepper) indicate that our SNELBP yields more competitive results than thirty classical LPB variants as well as eight typical deep learning methods.

On the use of multi-objective evolutionary classifiers for breast cancer detection.

PurposeBreast cancer is one of the most common tumours in women, nevertheless, it is also one of the cancers that is most usually treated. As a result, early detection is critical, which can be accomplished by routine mammograms. This paper aims to describe, analyze, compare and evaluate three image descriptors involved in classifying breast cancer images from four databases.ApproachMulti–Objective Evolutionary Algorithms (MOEAs) prove themselves as being efficient methods for selection and classification problems. This paper aims to study combinations of well–known classification objectives in order to compare the results of their application in solving very specific learning problems. The experimental results undergo empirical analysis which is supported by a statistical approach. The results are illustrated on a collection of medical image databases, but with a focus on the MOEAs’ performance in terms of several well–known measures. The databases were chosen specifically to feature reliable human annotations, so as to measure the correlation between the gold standard classifications and the various MOEA classifications.ResultsWe have seen how different statistical tests rank one algorithm over the others in our set as being better. These findings are unsurprising, revealing that there is no single gold standard for comparing diverse techniques or evolutionary algorithms. Furthermore, building meta-classifiers and evaluating them using a single, favorable metric is both extremely unwise and unsatisfactory, as the impact is to skew the results.ConclusionsThe best method to address these flaws is to select the right set of objectives and criteria. Using accuracy-related objectives, for example, is directly linked to maximizing the number of true positives. If, on the other hand, accuracy is chosen as the generic metric, the primary classification goal is shifted to increasing the positively categorized data points.

Novel nested patch-based feature extraction model for automated Parkinson's Disease symptom classification using MRI images

ObjectiveParkinson's disease (PD) is a common neurological disorder with variable clinical manifestations and magnetic resonance imaging (MRI) findings. We propose a handcrafted image classification model that can accurately (i) classify different PD stages, (ii) detect comorbid dementia, and (iii) discriminate PD-related motor symptoms. MethodsSelected image datasets from three PD studies were used to develop the classification model. Our proposed novel automated system was developed in four phases: (i) texture features are extracted from the non-fixed size patches. In the feature extraction phase, a pyramid histogram-oriented gradient (PHOG) image descriptor is used. (ii) In the feature selection phase, four feature selectors: neighborhood component analysis (NCA), Chi2, minimum redundancy maximum relevancy (mRMR), and ReliefF are used to generate four feature vectors. (iii) Two classifiers: k-nearest neighbor (kNN) and support vector machine (SVM) are used in the classification step. A ten-fold cross-validation technique is used to validate the results. (iv) Eight predicted vectors are generated using four selected feature vectors and two classifiers. Finally, iterative majority voting (IMV) is used to attain general classification results. Therefore, this model is named nested patch-PHOG-multiple feature selectors and multiple classifiers-IMV (NP-PHOG-MFSMCIMV). ResultsOur presented NP-PHOG-MFSMCIMV model achieved 99.22, 98.70, and 99.53% accuracies for the collected PD stages, PD dementia, and PD symptoms classification datasets, respectively. SignificanceThe obtained accuracies (over 98% for all states) demonstrated the performance of developed NP-PHOG-MFSMCIMV model in automated PD state classification.