Global Feature Descriptors Research Articles

PND-Net: plant nutrition deficiency and disease classification using graph convolutional network

Crop yield production could be enhanced for agricultural growth if various plant nutrition deficiencies, and diseases are identified and detected at early stages. Hence, continuous health monitoring of plant is very crucial for handling plant stress. The deep learning methods have proven its superior performances in the automated detection of plant diseases and nutrition deficiencies from visual symptoms in leaves. This article proposes a new deep learning method for plant nutrition deficiencies and disease classification using a graph convolutional network (GNN), added upon a base convolutional neural network (CNN). Sometimes, a global feature descriptor might fail to capture the vital region of a diseased leaf, which causes inaccurate classification of disease. To address this issue, regional feature learning is crucial for a holistic feature aggregation. In this work, region-based feature summarization at multi-scales is explored using spatial pyramidal pooling for discriminative feature representation. Furthermore, a GCN is developed to capacitate learning of finer details for classifying plant diseases and insufficiency of nutrients. The proposed method, called Plant Nutrition Deficiency and Disease Network (PND-Net), has been evaluated on two public datasets for nutrition deficiency, and two for disease classification using four backbone CNNs. The best classification performances of the proposed PND-Net are as follows: (a) 90.00% Banana and 90.54% Coffee nutrition deficiency; and (b) 96.18% Potato diseases and 84.30% on PlantDoc datasets using Xception backbone. Furthermore, additional experiments have been carried out for generalization, and the proposed method has achieved state-of-the-art performances on two public datasets, namely the Breast Cancer Histopathology Image Classification (BreakHis 40×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes $$\\end{document}: 95.50%, and BreakHis 100×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes $$\\end{document}: 96.79% accuracy) and Single cells in Pap smear images for cervical cancer classification (SIPaKMeD: 99.18% accuracy). Also, the proposed method has been evaluated using five-fold cross validation and achieved improved performances on these datasets. Clearly, the proposed PND-Net effectively boosts the performances of automated health analysis of various plants in real and intricate field environments, implying PND-Net’s aptness for agricultural growth as well as human cancer classification.

Quaternion fast and accurate polar harmonic Fourier moments for color image analysis and object recognition

Image moments, as a kind of global feature descriptor of images, have become a valuable tool for pattern recognition and image analysis. However, traditional methods are mainly used to deal with grayscale images. In this paper, we apply quaternions to fast and accurate polar harmonic Fourier moments, proposing a kind of quaternion fast and accurate polar harmonic Fourier moment (QFAPHFM) capable of handling color images. Furthermore, this paper provides a detailed analysis of the invariance of QFAPHFMs under rotation, scaling, and translation transformations. The experimental results show that QFAPHFMs exhibit excellent performance in both image reconstruction and object recognition tasks. QFAPHFMs achieve accurate image reconstruction under noiseless and noisy conditions, and demonstrate excellent recognition performance in the color-based object recognition tasks.

Feature Relation Guided Cross-View Image Based Geo-Localization

The goal of cross-view image based geo-localization is to determine the location of a given street-view image by matching it with a collection of geo-tagged aerial images, which has important applications in the fields of remote sensing information utilization and augmented reality. Most current cross-view image based geo-localization methods focus on the image content and ignore the relations between feature nodes, resulting in insufficient mining of effective information. To address this problem, this study proposes feature relation guided cross-view image based geo-localization. This method first processes aerial remote sensing images using a polar transform to achieve the geometric coarse alignment of ground-to-aerial images, and then realizes local contextual feature concern and global feature correlation modeling of the images through the feature relation guided attention generation module designed in this study. Specifically, the module includes two branches of deformable convolution based multiscale contextual feature extraction and global spatial relations mining, which effectively capture global structural information between feature nodes at different locations while correlating contextual features and guiding global feature attention generation. Finally, a novel feature aggregation module, MixVPR, is introduced to aggregate global feature descriptors to accomplish image matching and localization. After experimental validation, the cross-view image based geo-localization algorithm proposed in this study yields results of 92.08%, 97.70%, and 98.66% for the top 1, top 5, and top 10 metrics, respectively, in CVUSA, a popular public cross-view dataset, and exhibits superior performance compared to algorithms of the same type.

Cross-domain fashion cloth retrieval via novel attention-guided cascade neural network and clothing parsing

Cross-domain fashion retrieval is a feasible method for finding the required clothing among a large number of products. Computer vision-based retrieval algorithms have been developed by many researchers. However, the precise retrieval process still faces challenges such as corrupted query images, ambiguous patterns, and an array of attributes. In this work, a novel attention-guided cascaded network has been proposed for efficient cloth retrieval by overcoming the aforementioned challenges. Initially, the input images are pre-processed with Wavelet integrated Retinex algorithm (WRA) to enhance the query cloth image without diminishing the important information. The attention-guided cascaded network is designed with global feature descriptor (GFD) and local feature descriptor (LFD) modules for extracting the domain-enriched features. These two modules are integrated with an attention block to reduce complexity and improve performance using selected features. In the LFD module, the Clothing Parsing Encoder-Decoder network (CPED-Net) is used to separate the bottom and top regions and sends them into the respective network for fine-grained feature analysis. Finally, global and local features from the modules are combined in similarity analysis for cloth retrieval. The proposed model achieves an overall accuracy of 98.54% based on the fashion retrieval benchmarks dataset. From the experimental results, the proposed model is superior to the state-of-the-art models.

Development of a deep stacked ensemble with process based volatile memory forensics for platform independent malware detection and classification

Malware has become more complicated in its purpose and abilities over time, demanding continuous progress in detection and defense technologies. Malware designers use anti-analysis obfuscation techniques, including packing and encryption, to evade detection and hinder the analysis process. Current malware detection methods have shortcomings; thus, an alternative dynamic platform-independent scheme is proposed to extract harmful hardware impressions. This scheme includes extracting and converting a file from process memory dumps into an image. A combined structural and statistical image textural analysis is performed by designing a hybrid local and global feature descriptor. The hybrid feature descriptor helps to improve the data training ability of the proposed deep-stacked ensemble model by reducing input dimensions. A deep-stacked ensemble model is developed by combining prediction outputs from weak learners (CNNs) and feeding them into a meta-learner (MLP) as learning input. An explainable artificial intelligence-based approach is employed to interpret and validate the final results of the proposed scheme. Evaluations are conducted using three datasets: the publicly available Dumpware10 dataset, which contains 3686 samples from 10 different malware families; the publicly available CIC-MalMem-2022 dataset, which includes 2,916 samples from 15 different obfuscated malware families; and a real-world dataset, which contains 2375 samples of both malware and benign android apps. Experimental outcomes show that the proposed scheme achieved 99.1 % accuracy in analyzing Windows malware memory dumps, 94.3 % accuracy in analyzing Android malware memory dumps, and 99.8 % accuracy in analyzing Windows obfuscated malware memory dumps. The final results indicate that our vision-based system provides an excellent defense against malicious programs.

Loop Closure Detection Method of Laser SLAM Based on Global Feature Descriptor

Loop Closure Detection Method of Laser SLAM Based on Global Feature Descriptor

Unifying Deep ConvNet and Semantic Edge Features for Loop Closure Detection

Loop closure detection is an important component of Simultaneous Localization and Mapping (SLAM). In this paper, a novel two-branch loop closure detection algorithm unifying deep Convolutional Neural Network (ConvNet) features and semantic edge features is proposed. In detail, we use one feature extraction module to extract both ConvNet and semantic edge features simultaneously. The deep ConvNet features are subjected to a Context Feature Enhancement (CFE) module in the global feature ranking branch to generate a representative global feature descriptor. Concurrently, to reduce the interference of dynamic features, the extracted semantic edge information of landmarks is encoded through the Vector of Locally Aggregated Descriptors (VLAD) framework in the semantic edge feature ranking branch to form semantic edge descriptors. Finally, semantic, visual, and geometric information is integrated by the similarity score fusion calculation. Extensive experiments on six public datasets show that the proposed approach can achieve competitive recall rates at 100% precision compared to other state-of-the-art methods.

Loop Closure Detection Based on Residual Network and Capsule Network for Mobile Robot.

Loop closure detection based on a residual network (ResNet) and a capsule network (CapsNet) is proposed to address the problems of low accuracy and poor robustness for mobile robot simultaneous localization and mapping (SLAM) in complex scenes. First, the residual network of a feature coding strategy is introduced to extract the shallow geometric features and deep semantic features of images, reduce the amount of image noise information, accelerate the convergence speed of the model, and solve the problems of gradient disappearance and network degradation of deep neural networks. Then, the dynamic routing mechanism of the capsule network is optimized through the entropy peak density, and a vector is used to represent the spatial position relationship between features, which can improve the ability of image feature extraction and expression to optimize the overall performance of networks. Finally, the optimized residual network and capsule network are fused to retain the differences and correlations between features, and the global feature descriptors and feature vectors are combined to calculate the similarity of image features for loop closure detection. The experimental results show that the proposed method can achieve loop closure detection for mobile robots in complex scenes, such as view changes, illumination changes, and dynamic objects, and improve the accuracy and robustness of mobile robot SLAM.

A Deep Convolutional Neural Network Stacked Ensemble for Malware Threat Classification in Internet of Things

Malicious attacks to software applications are on the rise as more people use Internet of things (IoT) devices and high-speed internet. When a software system crash happens caused by malicious action, a malware imaging method can examine the application. In this study, we present a novel malware classification method that captures suspected operations in a variety of discrete size image features, allowing us to identify such IoT device malware families. To decrease deep neural network training time, essential local and global image features are selected using a combined local and global feature descriptor (LBP-GLCM). The classification performance of the proposed deep learning model is improved by combining the predictions of weak learners (CNNs) and using them as knowledge input to a multi-layer perceptron meta learner. This is a neural network ensemble with stacked generalization that is used to improve network generalization ability. The public dataset used for performance evaluation contains 5472 samples from 11 different malware families. In order to compare the proposed methodology to current malware detection systems, we developed a baseline experiment. The proposed approach improved malware classification results to 98.5% accuracy and 98.4% accuracy when using [Formula: see text] and [Formula: see text] image sizes, respectively. Overall, the results showed that the stacked generalization ensemble with multi-step extracting features is a more effective method for classification performance and response time.

Android Device Malware Classification Framework Using Multistep Image Feature Extraction and Multihead Deep Neural Ensemble

The incidence of malicious threats to computer systems has increased with the increasing use of Android devices and high-speed Internet. Malware visualization mechanism can analyze a computer whenever a software or system crash occurs because of malicious activity. This paper presents a new malware classification approach to recognize such Android device malware families by capturing suspicious processes in the form of different size color images. Important local and global characteristics of color images are extracted through a combined local and global feature descriptor (structure based local and statistical based global combined texture analysis) to reduce the training complexity of neural networks. A multihead ensemble of neural networks is proposed to increase network classification performance by merging prediction results from weak learners (convolutional neural network + gated recurrent unit) and using them as learning input to a multi-layer perceptron meta learner. Two public datasets of Android device malware are used to evaluate the classification and detection performance of the proposed approach. A baseline is established to compare the classification performance of the proposed approach with those of state-of-the-art and previous malware detection approaches. The proposed multihead ensemble improved the malware classification performance, with up to 97.8%, accuracy with the R2-D2 dataset and 94.1% accuracy with the MalNet dataset. The overall results show that a multihead ensemble with multi-step feature extraction is a practical approach to classify and detect Android malware.

Fast and accurate face recognition system using MORSCMs-LBP on embedded circuits.

Because of the current COVID-19 circumstances in the world and the tremendous technological developments, it has become necessary to use this technology to combat the spread of the new coronavirus. The systems that depend on using hands, such as fingerprint systems and PINs in ATM systems, could lead to infection, so they have become undesirable and we can replace them by using facial recognition instead. With the development of technology and the availability of nano devices like the Raspberry Pi, such applications can be implemented easily. This study presents an efficient face recognition system in which the face image is taken by a standalone camera and then passed to the Raspberry Pi to extract the face features and then compare them with the database. This approach is named MORSCMs-LBP by combining two algorithms for feature extraction: Local Binary Pattern (LBP) as a local feature descriptor and radial substituted Chebyshev moments (MORSCMs) as a global feature descriptor. The significant advantage of this method is that it combines the local and global features into a single feature vector from the detected faces. The proposed approach MORSCMs-LBP has been implemented on the Raspberry Pi 4 computer model B with 1 GB of RAM using C++ OpenCV. We assessed our method on various benchmark datasets: face95 with an accuracy of 99.0278%, face96 with an accuracy of 99.4375%, and grimace with 100% accuracy. We evaluated the proposed MORSCMs-LBP technique against other recently published approaches; the comparison shows a significant improvement in favour of the proposed approach.

SAGA-net

In this paper, we propose a real-time shape-assisted graph attention neural network to perform local pointcloud repairment. The orderless pointclouds require an effective shape encoder to distill local and global geometric feature descriptors. Previous work has attempted to convert pointcloud representation into a voxelized shape or perform grid-transformations. While these approaches can subsequently allow common convolution operations on the structured data, they either pose additional computational cost or disrupt the local geometric information. We present SAGA-Net, an efficient graph attention neural network framework with a prior shape inquiry protocol that effectively extracts local geometric information, locates the descriptor for the missing region and accurately reconstructs the local region in a real-time manner. Our framework has been validated on a benchmark dataset, ShapeNet. We demonstrate that our framework can repair each partial pointcloud with accuracy surpassing other frameworks in most object categories, and improve the computational efficiency by orders of magnitude in terms of time cost.

Enhancing Scalability of Image Retrieval Using Visual Fusion of Feature Descriptors

Content-Based Image Retrieval (CBIR) is an approach of retrieving similar images from a large image database. Recently CBIR poses new challenges in semantic categorization of the images. Different feature extraction technique have been proposed to overcome the semantic breach problems, however these methods suffer from several shortcomings. This paper contributes an image retrieval system to extract the local features based on the fusion of scale-invariant feature transform (SIFT) and KAZE. The strength of local feature descriptor SIFT complements global feature descriptor KAZE. SIFT concentrates on the complete region of an image using high fine points of features and KAZE ponders on details of a boundary. The fusion of local feature descriptor and global feature descriptor boost the retrieval of images having diverse semantic classification and also helps in achieving the better results in large scale retrieval. To enhance the scalability of image retrieval bag of visual words (BoVW) is mainly used. The fusion of local and global feature representations are selected for image retrieval for the reason that SIFT effectively captures shape and texture and robust towards the change in scale and rotation, while KAZE have strong response towards boundary and changes in illumination. Experiments conducted on two image collections, namely, Caltech-256 and Corel 10k demonstrate the proposed scheme appreciably enhanced the performance of the CBIR compared to state-of-the-art image retrieval techniques.

Modal Frequencies Based Human Action Recognition Using Silhouettes And Simplicial Elements

Human action recognition has been a pioneer research problem among the researchers. Feature descriptors are categorized into two categories: global and local. The disadvantage of global feature descriptors is that global features only give the structural information of the action whereas disadvantage of local descriptor is they give only motion information of the action. As a result, the recognition rate gets affected. To improve the recognition rate, hybrid descriptors are also used. But the disadvantage of hybrid descriptors is that they increase the complexity of the descriptor as both global and local features have to be fused. To overcome both the issues we proposed a new local feature descriptor in terms of modal frequency using silhouette and simplicial elements of a silhouette with the help of Finite Element Analysis (FEA). This local descriptor represents the distinctive human poses in the form of modal frequency. These modal frequencies are subject to the stiffness matrix of the body that is associated with the displacement. The silhouettes of the human body are used for the analysis. These silhouettes are represented into simplicial elements. The modal frequencies of silhouettes are calculated using simplicial elements. These modal frequencies of the silhouette are used as the feature vectors that are given to the Radial Basis Function-Support Vector Machine (RBF-SVM) classifier. The challenging datasets Weizmann, KTH and IXMAS are used for validation of the proposed methodology

A heterogeneous 3D map-based place recognition solution using virtual LiDAR and a polar grid height coding image descriptor

Place recognition is widely used for global localization technology. However, the existing place recognition solutions are limited by the requirement for the same type of sensors to be used in both the localization process and the mapping process. Therefore, the existing heterogeneous 3D map cannot be used for place recognition directly with the existing methods, leading to underutilization of information. In addition, most of the existing global feature descriptors used in place recognition solutions are still not highly descriptive and perform poorly under changed viewpoint scenes. To resolve these challenges, this paper presents a place recognition solution using virtual light detection and ranging (LiDAR) and polar grid height coding image (PGHCI) descriptors in the existing heterogeneous 3D map. First, virtual LiDAR is proposed to generate a series of virtual scans that are similar to the real scan of the localization sensor from the existing map, overcoming the limitation of the existing place recognition methods. Next, a novel PGHCI descriptor for place recognition is generated, and a method that overcomes the recognition difficulty of changed viewpoints in the same scene is presented. Two weighted distances for similarity estimation are analyzed, and the performance of the PGHCI descriptor with different parameters is evaluated. Finally, the performance of the PGHCI descriptor and the solution proposed in the paper is evaluated on several popular benchmark datasets and our own dataset. Comprehensive experiments demonstrated that the PGHCI descriptor has higher descriptiveness and is robust with respect to the changed viewpoint scene, as shown by the comparison of precision-recall (PR) curves using datasets with multiple scenes. The proposed place recognition solution has 100% success rates in the evaluation exclude under occluded conditions, showing that it is feasible to achieve robust place recognition using a heterogeneous 3D map.

Audio-based spam call detection

Spam communications are organized attempts of falsified claims with the purpose of marketing, spreading false information and deceiving the end recipient. Phone spam is an international nuisance, with the U. S. among the most spammed countries in the world in 2020. In addition to the agitating nature of these calls, criminal scams are defrauding subscribers of billions of dollars every year. Therefore, it is necessary to develop automated systems for the identification of spam calls to minimize fraud and reduce the displeasure of receiving them. The call origin, call duration and other Call Detail Records can be used to assess whether a call is fraudulent or not, but the actual audio content is overlooked. This work focuses on extracting acoustic features from voicemail recordings containing speech, which are used to train Machine Learning models that identify spam calls. Both local and global feature descriptors are used, including Mel-Frequency Cepstral Coefficients and Log-Mel Spectrum, and their efficacy for distinguishing spam from non-spam calls is explored. We demonstrate that a spam voice call can be detected while relying only on the acoustic information of the call. A further analysis of the temporal and spectral features that are most informative for the task is also presented.

The Research on Rejoining of the Oracle Bone Rubbings Based on Curve Matching

The rejoining of oracle bone rubbings is a fundamental topic for oracle research. However, it is a tough task to reassemble severely broken oracle bone rubbings because of detail loss in manual labeling, the great time consumption of rejoining, and the low accuracy of results. To overcome the challenges, we introduce a novel CFDA&CAP algorithm that consists of the Curve Fitting Degree Analysis (CFDA) algorithm and the Correlation Analysis of Pearson (CAP) algorithm. First, the orthogonalization system is constructed to extract local features based on the curve features analysis. Second, the global feature descriptor is depicted by using coordinate points sequences. Third, we screen candidate curves based on the features as well as the CFDA algorithm, so the search range of the candidates is narrowed down. Finally, image recommendation libraries for target curves are generated by adopting the CAP algorithm, and the rank for each target matching curve generates simultaneously for result evaluation. With experiments, the proposed method shows a good effect in rejoining oracle bone rubbings automatically: (1) it improves the average accuracy rate of curve matching up to 84%, and (2) for a low-resource task, the accuracy of our method has 25% higher accuracy than that of other methods.

Patch-SIFT: Enhanced feature descriptor to learn human facial emotions using an Ensemble approach

Background: Having experienced more than a year of pandemic, a variety of applications such as online classrooms, virtual office meetings, conferences, online games, Social media & Networks, Mobile applications, and many other infotainment areas have made humans live with gadgets and respond to them. However, all these applications have an impact on human behavioral transformation. It is very significant for employers to understand the emotions of their employees in the era of online office & work from home concept to increase productivity. Learning and identifying emotions from the human face has its application in all online portals when physical contact could not be achieved. Ojbective: Human Facial emotions can be learned using enormous feature descriptors that extract image features. While local feature descriptors retrieve pixel-level information, global feature descriptors extract the overall image information. Both of the feature descriptors quantify the image information, however, they don’t provide complete and relevant information. Hence, this research work aims to improve the existing local feature descriptor to perform globally for emotion recognition. Method: Our proposed feature descriptor, Patch-SIFT collects features from multiple patches within an image. This strategy is evolved to globally apply the local feature descriptor as a hybridization paradigm. The extracted features are trained and tested on an ensemble model. Findings: The Proposed Feature descriptor (Patch-SIFT) performance with ensemble model is found to produce an improved accuracy of 98% compared with existing feature descriptors and Machine learning classifiers. Novelty: This research work tries to evolve a new Feature descriptor algorithm based on SIFT algorithm for an efficient emotion recognition system that works without the need for any additional GPU or huge dataset. Keywords Classification, Ensemble, Feature descriptor, Patch­SIFT

Flexible Body Partition-Based Adversarial Learning for Visible Infrared Person Re-Identification.

Person re-identification (Re-ID) aims to retrieve images of the same person across disjoint camera views. Most Re-ID studies focus on pedestrian images captured by visible cameras, without considering the infrared images obtained in the dark scenarios. Person retrieval between visible and infrared modalities is of great significance to public security. Current methods usually train a model to extract global feature descriptors and obtain discriminative representations for visible infrared person Re-ID (VI-REID). Nevertheless, they ignore the detailed information of heterogeneous pedestrian images, which affects the performance of Re-ID. In this article, we propose a flexible body partition (FBP) model-based adversarial learning method (FBP-AL) for VI-REID. To learn more fine-grained information, FBP model is exploited to automatically distinguish part representations according to the feature maps of pedestrian images. Specially, we design a modality classifier and introduce adversarial learning which attempts to discriminate features between visible and infrared modality. Adaptive weighting-based representation learning and threefold triplet loss-based metric learning compete with modality classification to obtain more effective modality-sharable features, thus shrinking the cross-modality gap and enhancing the feature discriminability. Extensive experimental results on two cross-modality person Re-ID data sets, i.e., SYSU-MM01 and RegDB, exhibit the superiority of the proposed method compared with the state-of-the-art solutions.

Localization and classification of human facial emotions using local intensity order pattern and shape-based texture features

Facial emotion recognition system (FERS) recognize the person’s emotions based on various image processing stages including feature extraction as one of the major processing steps. In this study, we presented a hybrid approach for recognizing facial expressions by performing the feature level fusion of a local and a global feature descriptor that is classified by a support vector machine (SVM) classifier. Histogram of oriented gradients (HoG) is selected for the extraction of global facial features and local intensity order pattern (LIOP) to extract the local features. As HoG is a shape-based descriptor, with the help of edge information, it can extract the deformations caused in facial muscles due to changing emotions. On the contrary, LIOP works based on the information of pixels intensity order and is invariant to change in image viewpoint, illumination conditions, JPEG compression, and image blurring as well. Thus both the descriptors proved useful to recognize the emotions effectively in the images captured in both constrained and realistic scenarios. The performance of the proposed model is evaluated based on the lab-constrained datasets including CK+, TFEID, JAFFE as well as on realistic datasets including SFEW, RaF, and FER-2013 dataset. The optimal recognition accuracy of 99.8%, 98.2%, 93.5%, 78.1%, 63.0%, 56.0% achieved respectively for CK+, JAFFE, TFEID, RaF, FER-2013 and SFEW datasets respectively.