Feature Extraction Fusion Research Articles

Deep Fusion Feature Extraction for Caries Detection on Dental Panoramic Radiographs

Caries is the most well-known disease and relates to the oral health of billions of people around the world. Despite the importance and necessity of a well-designed detection method, studies in caries detection are still limited and show a restriction in performance. In this paper, we proposed a computer-aided diagnosis (CAD) method to detect caries among normal patients using dental radiographs. The proposed method mainly consists of two processes: feature extraction and classification. In the feature extraction phase, the chosen 2D tooth image was employed to extract deep activated features using a deep pre-trained model and geometric features using mathematic formulas. Both feature sets were then combined, called fusion feature, to complement each other defects. Then, the optimal fusion feature set was fed into well-known classification models such as support vector machine (SVM), k-nearest neighbor (KNN), decision tree (DT), Naïve Bayes (NB), and random forest (RF) to determine the best classification model that fit the fusion features set and perform the most preeminent result. The results show 91.70%, 90.43%, and 92.67% for accuracy, sensitivity, and specificity, respectively. The proposed method has outperformed the previous state-of-the-art and shows promising results when none of the measured factors is less than 90%; therefore, the method is promising for dentists and capable of wide-scale implementation caries detection in hospitals.

Detection Method Based on Automatic Visual Shape Clustering for Pin-Missing Defect in Transmission Lines

Bolts are the most numerous fasteners in transmission lines and are prone to losing their split pins. How to realize the automatic pin-missing defect detection for bolts in transmission lines so as to achieve timely and efficient troubleshooting is a difficult problem and the long-term research target of power systems. In this article, an automatic detection model called automatic visual shape clustering network (AVSCNet) for the pin-missing defect is constructed. First, an unsupervised clustering method for the visual shapes of bolts is proposed and applied to construct a defect detection model that can learn the difference in visual shape. Next, three deep convolutional neural network optimization methods are used in the model: the feature enhancement, feature fusion, and region feature extraction. The defect detection results are obtained by applying the regression calculation and classification to the regional features. In this article, the object detection model of different networks is used to test the data set of pin-missing defect constructed by the aerial images of transmission lines from multiple locations, and it is evaluated by various indicators and is fully verified. The results show that our method can achieve considerably satisfactory detection effect.

Research on Discriminative Skeleton-Based Action Recognition in Spatiotemporal Fusion and Human-Robot Interaction

A novel posture motion-based spatiotemporal fused graph convolutional network (PM-STGCN) is presented for skeleton-based action recognition. Existing methods on skeleton-based action recognition focus on independently calculating the joint information in single frame and motion information of joints between adjacent frames from the human body skeleton structure and then combine the classification results. However, that does not take into consideration of the complicated temporal and spatial relationship of the human body action sequence, so they are not very efficient in distinguishing similar actions. In this work, we enhance the ability of distinguishing similar actions by focusing on spatiotemporal fusion and adaptive feature extraction for high discrimination information. Firstly, the local posture motion-based attention (LPM-TAM) module is proposed for the purpose of suppressing the skeleton sequence data with a low amount of motion in the temporal domain, and the representation of motion posture features is concentrated. Besides, the local posture motion-based channel attention module (LPM-CAM) is introduced to make use of the strongly discriminative representation between different action classes of similarity. Finally, the posture motion-based spatiotemporal fusion (PM-STF) module is constructed which fuses the spatiotemporal skeleton data by filtering out the low-information sequence and enhances the posture motion features adaptively with high discrimination. Extensive experiments have been conducted, and the results demonstrate that the proposed model is superior to the commonly used action recognition methods. The designed human-robot interaction system based on action recognition has competitive performance compared with the speech interaction system.

EEG based emotion recognition using fusion feature extraction method

As a high-level function of the human brain, emotion is the external manifestation of people’s psychological characteristics. The emotion has a great impact on people’s personality and mental health. At the same time, emotion classification from electroencephalogram (EEG) signals have attracted much attention. To improve the precision of EEG-based emotion recognition, we proposed a fused feature extraction method to complete the classification of three emotions (neutral, happiness, and sadness). The standardized movie clips were selected to induce the corresponding emotion and the EEG response of 10 participants is collected by Emotiv EPOC. This paper systematically compared two kinds of EEG features (power spectrum and wavelet energy entropy) and their fusion for emotion classification. To reduce the dimension of fused features, we used principal component analysis (PCA) for dimensionality reduction and feature selection. The support vector machine (SVM) classifier and the relevance vector machines (RVM) classifier were utilized for emotion recognition respectively. From experimental results, we found that the fusion of two kinds of features outperformed a single feature for emotion classification by both the SVM classifier and the RVM classifier, and the averaged classification accuracy was 89.17% and 91.18%, respectively.

SSD-TSEFFM: New SSD Using Trident Feature and Squeeze and Extraction Feature Fusion.

The single shot multi-box detector (SSD) exhibits low accuracy in small-object detection; this is because it does not consider the scale contextual information between its layers, and the shallow layers lack adequate semantic information. To improve the accuracy of the original SSD, this paper proposes a new single shot multi-box detector using trident feature and squeeze and extraction feature fusion (SSD-TSEFFM); this detector employs the trident network and the squeeze and excitation feature fusion module. Furthermore, a trident feature module (TFM) is developed, inspired by the trident network, to consider the scale contextual information. The use of this module makes the proposed model robust to scale changes owing to the application of dilated convolution. Further, the squeeze and excitation block feature fusion module (SEFFM) is used to provide more semantic information to the model. The SSD-TSEFFM is compared with the faster regions with convolution neural network features (RCNN) (2015), SSD (2016), and DF-SSD (2020) on the PASCAL VOC 2007 and 2012 datasets. The experimental results demonstrate the high accuracy of the proposed model in small-object detection, in addition to a good overall accuracy. The SSD-TSEFFM achieved 80.4% mAP and 80.2% mAP on the 2007 and 2012 datasets, respectively. This indicates an average improvement of approximately 2% over other models.

A New Framework for Automatic Airports Extraction from SAR Images Using Multi-Level Dual Attention Mechanism

The detection of airports from Synthetic Aperture Radar (SAR) images is of great significance in various research fields. However, it is challenging to distinguish the airport from surrounding objects in SAR images. In this paper, a new framework, multi-level and densely dual attention (MDDA) network is proposed to extract airport runway areas (runways, taxiways, and parking lots) in SAR images to achieve automatic airport detection. The framework consists of three parts: down-sampling of original SAR images, MDDA network for feature extraction and classification, and up-sampling of airports extraction results. First, down-sampling is employed to obtain a medium-resolution SAR image from the high-resolution SAR images to ensure the samples (500 × 500) can contain adequate information about airports. The dataset is then input to the MDDA network, which contains an encoder and a decoder. The encoder uses ResNet_101 to extract four-level features with different resolutions, and the decoder performs fusion and further feature extraction on these features. The decoder integrates the chained residual pooling network (CRP_Net) and the dual attention fusion and extraction (DAFE) module. The CRP_Net module mainly uses chained residual pooling and multi-feature fusion to extract advanced semantic features. In the DAFE module, position attention module (PAM) and channel attention mechanism (CAM) are combined with weighted filtering. The entire decoding network is constructed in a densely connected manner to enhance the gradient transmission among features and take full advantage of them. Finally, the airport results extracted by the decoding network were up-sampled by bilinear interpolation to accomplish airport extraction from high-resolution SAR images. To verify the proposed framework, experiments were performed using Gaofen-3 SAR images with 1 m resolution, and three different airports were selected for accuracy evaluation. The results showed that the mean pixels accuracy (MPA) and mean intersection over union (MIoU) of the MDDA network was 0.98 and 0.97, respectively, which is much higher than RefineNet and DeepLabV3. Therefore, MDDA can achieve automatic airport extraction from high-resolution SAR images with satisfying accuracy.

Classification of stomach infections: A paradigm of convolutional neural network along with classical features fusion and selection.

Automated detection and classification of gastric infections (i.e., ulcer, polyp, esophagitis, and bleeding) through wireless capsule endoscopy (WCE) is still a key challenge. Doctors can identify these endoscopic diseases by using the computer-aided diagnostic (CAD) systems. In this article, a new fully automated system is proposed for the recognition of gastric infections through multi-type features extraction, fusion, and robust features selection. Five key steps are performed-database creation, handcrafted and convolutional neural network (CNN) deep features extraction, a fusion of extracted features, selection of best features using a genetic algorithm (GA), and recognition. In the features extraction step, discrete cosine transform, discrete wavelet transform strong color feature, and VGG16-based CNN features are extracted. Later, these features are fused by simple array concatenation and GA is performed through which best features are selected based on K-Nearest Neighbor fitness function. In the last, best selected features are provided to Ensemble classifier for recognition of gastric diseases. A database is prepared using four datasets-Kvasir, CVC-ClinicDB, Private, and ETIS-LaribPolypDB with four types of gastric infections such as ulcer, polyp, esophagitis, and bleeding. Using this database, proposed technique performs better as compared to existing methods and achieves an accuracy of 96.5%.

Intervention of Hepatocellular Carcinoma in Fibrosis Staging Based on Multi-Parameter Magnetic Resonance Image Depth Learning Method

This paper mainly discusses the deep learning solution for non-invasive evaluation of the differentiation degree of hepatocellular carcinoma based on multi-parameter nuclear magnetic resonance images, combined with the clinical diagnosis experience of radiologists and the characteristics of nuclear magnetic resonance images. The method of multimodal data fusion is studied based on multi-parameter nuclear magnetic resonance imaging data. Multi-channel three-dimensional convolution neural network and multi-scale depth residual network are proposed to extract the features of three-dimensional medical image data and two-dimensional fusion medical image data, and to solve the problem of insufficient cases in clinical image data of hepatocellular carcinoma (HCC). We examine the role of transition learning and metric learning in medical image classification. In this study, we use a method of data fusion, transition learning and multi scale feature extraction to construct a deep learning model for medical image aided diagnosis. Multiple modal fusion decisions for finding complementary modal data fusion for the complementarity of multimodal images in diagnostic decisions can effectively improve diagnostic effects. Although there is a clear difference between natural and medical images through experiments, a model trained with a natural image dataset as an initialization of the network can ensure and converge the training. At the same time, improve the performance of the model on the test set. The multi-scale feature extraction model proposed in this paper enhances the robustness of the model and further improves the effect of medical image classification.

Deep Fusion Feature Extraction and Classification of Pellet Phase

Pellet quality including chemical composition, physical properties and metallurgical performance of three parts, its quality and mineral composition, properties and structure of the pellets has the close relation, studies show that the mineralogical micro structure and distribution of pellets had significant effects on the metallurgical properties, so the analysis and determination of pellets of mineral composition and micro structure is very important to improve the quality of pellets. Paper to pellets micro mineral as the research object, mainly studies the CNN and PCA two kinds of image processing algorithm, in the heart of the traditional model of CNN structure characteristics obtained by convolution PCA dimension reduction at a time, will be the main features of PCA to extract into the depth of the CNN learning, realize the mineral phase of shallow and deep information of the image to do effective fusion, in a larger extent, reflects the mineralogical characteristics, thus more intuitive response metallurgical properties. The location and alkalinity of the ore phase were identified by the extracted deep fusion feature, and the results were compared with those of the traditional CNN algorithm. It was found that the accuracy of location and alkalinity recognition of PCA and CNN coupling algorithm was 93.82% and 91.26%, respectively, which were higher than the accuracy of traditional CNN algorithm 92.73% and 88.93%, which verified the accuracy of PCA and CNN coupling model and its applicability in mineral phase recognition.

Writer-independent signature verification based on feature extraction fusion

In this work, with the aim of separating the genuine and forgery samples of the signature, we developed an automatic handwritten signature verification (AH-SV) based on the informative feature. The large intra-class variation is an important challenge among specific writer samples in SV. Thus, the proposed method is focused on feature extraction fusion (FEF) to increase the accuracy and efficiency of the system. We use features such as extracting intersection points based on neighboring intersection pixels relative to the central pixel to achieve the most informative feature, which can be regarded as a model for estimating the transitional probabilities of the signature strokes. Discriminative features extraction for SV relies on the domain of graphology and texture description. Therefore, we extracted parametric features and local binary pattern (LBP) from the signature images, and then we applied fusion using canonical correlation analysis (CCA) to improve the discriminative features. Hence, the measurement of the proximity between genuine and forgery signatures can be computed by their corresponding feature vectors. We used MCYT, GPDSsynthetic and CEDAR datasets, with the classification of writer-independent structure on k-nearest neighbour (K-NN) classifier. Moreover, in the experiment phase, we selected random number of samples in datasets. The experimental results show that the accuracy in the CEDAR dataset has achieved the best value. Also, the results of the average error rate, false acceptance rate, and false rejection rate criteria have improved compared to well-known methods.

Online Signature Verification Based on a Single Template via Elastic Curve Matching.

Person verification using online handwritten signatures is one of the most widely researched behavior-biometrics. Many signature verification systems typically require five, ten, or even more signatures for an enrolled user to provide an accurate verification of the claimed identity. To mitigate this drawback, this paper proposes a new elastic curve matching using only one reference signature, which we have named the curve similarity model (CSM). In the CSM, we give a new definition of curve similarity and its calculation method. We use evolutionary computation (EC) to search for the optimal matching between two curves under different similarity transformations, so as to obtain the similarity distance between two curves. Referring to the geometric similarity property, curve similarity can realize translation, stretching and rotation transformation between curves, thus adapting to the inconsistency of signature size, position and rotation angle in signature curves. In the matching process of signature curves, we design a sectional optimal matching algorithm. On this basis, for each section, we develop a new consistent and discriminative fusion feature extraction for identifying the similarity of signature curves. The experimental results show that our system achieves the same performance with five samples assessed with multiple state-of-the-art automatic signature verifiers and multiple datasets. Furthermore, it suggests that our system, with a single reference signature, is capable of achieving a similar performance to other systems with up to five signatures trained.

Proposed a Content-Based Image Retrieval System Based on the Shape and Texture Features

In this paper, we proposed a fusion feature extraction method for content based image retrieval. The feature is extracted by focusing on the texture and shape features of the visual image by using the Local Binary Pattern (LBP – texture feature) and Edge Histogram Descriptor (EHD – shape feature). The SVD is used for decreasing the number of the feature vector of images. The Kd-tree is used for reducing the retrieval time. The input to this system is a query image and Database (the reference images) and the output is the top n most similar images for the query image. The proposed system is evaluated by using (precision and recall) to measure the retrieval effectiveness. The values of the recall are between [43% –93%] and the average recall is 64.3%. The values of precision are between [30%-100%] and the average is 72.86% for the entire system and for both databases.

Multimodal biometric system combining left and right palmprints

PurposePalmprint recognition is a very interesting and promising area of research. Much work has already been done in this area, but much more needs to be done to make the systems more efficient. In this paper, a multimodal biometrics system based on fusion of left and right palmprints of a person is proposed to overcome limitations of unimodal systems.Design/methodology/approachFeatures are extracted using some proposed multi-block local descriptors in addition to MBLBP. Fusion of extracted features is done at feature level by a simple concatenation of feature vectors. Then, feature selection is performed on the resulting global feature vector using evolutionary algorithms such as genetic algorithms and backtracking search algorithm for a comparison purpose. The benefits of such step selecting the relevant features are known in the literature, such as increasing the recognition accuracy and reducing the feature set size, which results in runtime saving. In matching step, Chi-square similarity measure is used.FindingsThe resulting feature vector length representing a person is compact and the runtime is reduced.Originality/valueIntensive experiments were done on the publicly available IITD database. Experimental results show a recognition accuracy of 99.17 which prove the effectiveness and robustness of the proposed multimodal biometrics system than other unimodal and multimodal biometrics systems.

Infrared and visible image fusion with ResNet and zero-phase component analysis

In image fusion approaches, feature extraction and processing are key tasks, and the fusion performance is directly affected by the different features and processing methods undertaken. However, most of deep learning-based methods use deep features directly without them. This leads to the fusion performance degradation in some cases. To solve these drawbacks, in our paper, a deep features and zero-phase component analysis (ZCA) based novel fusion framework is proposed. Firstly, the residual network (ResNet) is used to extract deep features from source images. Then ZCA and l1-norm are utilized to normalize the deep features and obtain initial weight maps. The final weight maps are obtained by employing a soft-max operation in association with the initial weight maps. Finally, the fused image is reconstructed using a weighted-averaging strategy. Compared with the existing fusion methods, experimental results demonstrate that the proposed framework achieves better performance in both objective assessment and visual quality. The code of our fusion algorithm is available at https://github.com/hli1221/imagefusion_resnet50.

A novel convolutional transfer feature discrimination network for unbalanced fault diagnosis under variable rotational speeds

Deep learning has been widely used in the field of fault diagnosis due to its excellent performance in feature extraction and has been gradually applied to solve various problems in fault diagnosis. Convolutional neural networks and transfer learning networks have been gradually employed to solve the problems of sample imbalance and domain adaptation under variable rotational speeds in fault diagnosis. However, there are still some weaknesses in current research. Firstly, sample imbalance in fault diagnosis is accompanied by the domain-adaptive problem in practice. Secondly, transfer learning cannot extract domain-invariant features that do not change with the rotational speed, which leads to poor diagnosis results when it is applied to other working conditions without transferring the rotating speed. To solve the above problems, an architecture named a convolutional transfer feature discrimination network (CTFDN) is proposed in this paper. It uses the scaled exponential linear unit activation function, and the main body adopts a two-branch network architecture of weight sharing, which mainly includes a deep feature extraction network for fusion feature extraction, a transfer learning network for domain-adaptive problems, and an unbalanced-sample feature discrimination network for feature similarity determination. Finally, two fault datasets are used to test the validity of the proposed model. The results show that the CTFDN model can extract domain-invariant features well in unbalanced fault sample information, and still has good diagnostic accuracy under variable rotational speeds.

Classification of gastrointestinal diseases of stomach from WCE using improved saliency-based method and discriminant features selection

Wireless capsule endoscopy (WCE) is a new imaging procedure that is used to record internal conditions of gastrointestinal tract for medical diagnosis. However, due to the presence of bulk of WCE image data, it becomes difficult for the physician to investigate it thoroughly. Therefore, considering aforementioned constraint, lately gastrointestinal diseases are identified by computer-aided methods and with better classification accuracy. In this research, a new computer-based diagnosis method is proposed for the detection and classification of gastrointestinal diseases from WCE images. The proposed approach comprises of four fundamentalsteps:1) HSI color transformation before implementing automatic active contour segmentation; 2) implementation of a novel saliency-based method in YIQ color space; 3) fusion of images using proposed maximizing a posterior probability method; 4) fusion of extracted features, calculated using SVD, LBP, and GLCM, prior to final classification step. We perform our simulations on our own collected dataset – containing total 9000 samples of ulcer, bleeding and healthy. To prove the authenticity of proposed work, list of statistical measures is considered including classification accuracy, FNR, sensitivity, AUC, and Time. Further, a fair comparison of state-of-the-art classifiers is also provided which will be giving readers a deep inside of classifier’s selection for this application. Simulation results clearly reveal that the proposed method shows improved performance in terms of segmentation and classification accuracy.

A Novel Sketch-Based Medical 3D Model Retrieval Approach by Multi-Scale Weighted Gabor Feature Fusion and IP-HOG Feature Extraction

A Novel Sketch-Based Medical 3D Model Retrieval Approach by Multi-Scale Weighted Gabor Feature Fusion and IP-HOG Feature Extraction

A Study of Speech Feature Extraction Based on Manifold Learning

Manifold learning is a nonlinear data dimension reduction method. It can look for the essence of things from the observed phenomena, and find the inherent law of data. Traditional MFCC feature will lead a slower learning speed on account of it has high dimension and useless noise. Therefore, a speech feature extraction method based on manifold learning is proposed. Firstly, we use the manifold learning dimension reduction algorithm for the dimension reduction of Mel features and then for vowels classification. In order to further demonstrate the effectiveness of manifold learning feature in speech recognition, we propose a fusion speech feature extraction method and apply it to the identification of Chinese isolated words. Experiments prove that the fusion feature extraction method has achieved a better result than that of traditional MFCC feature extraction method.

Deep Fusion Feature Based Object Detection Method for High Resolution Optical Remote Sensing Images

With the rapid growth of high-resolution remote sensing image-based applications, one of the fundamental problems in managing the increasing number of remote sensing images is automatic object detection. In this paper, we present a fusion feature-based deep learning approach to detect objects in high-resolution remote sensing images. It employs fine-tuning from ImageNet as a pre-training model to address the challenge of it lacking a large amount of training datasets in remote sensing. Besides, we improve the binarized normed gradients algorithm by multiple weak feature scoring models for candidate window selection and design a deep fusion feature extraction method with the context feature and object feature. Experiments are performed on different sizes of high-resolution optical remote sensing images. The results show that our model is better than regular models, and the average detection accuracy is 8.86% higher than objNet.

Multiple classifiers fusion and CNN feature extraction for handwritten digits recognition

Handwritten digits recognition has been treated as a multi-class classification problem in the machine learning context, where each of the ten digits (0–9) is viewed as a class and the machine learning task is essentially to train a classifier that can effectively discriminate the ten classes. In practice, it is very usual that the performance of a single classifier trained using a standard learning algorithm is varied on different datasets, which indicates that the same learning algorithm may train strong classifiers on some datasets but weak classifiers may be trained on other datasets. It is also possible that the same classifier shows different performance on different test sets, especially when considering the case that image instances can be highly diverse due to the different handwriting styles of different people on the same digits. To address the above issue, development of ensemble learning approaches have been very necessary to improve the overall performance and make the performance more stable on different datasets. In this paper, we propose a framework that involves CNN-based feature extraction from the MINST dataset and algebraic fusion of multiple classifiers trained on different feature sets, which are prepared through feature selection applied to the original feature set extracted using CNN. The experimental results show that the classifiers fusion can achieve the classification accuracy of ≥ 98%.