Fusion Convolutional Neural Network Research Articles

An incremental learning method with hybrid data over/down-sampling for sEMG-based gesture classification

Surface electromyography(sEMG)-based gesture classification methods have been widely developed in neural decoding. However, these decoding methods are usually constrained to a fixed set of gestures, which hinders flexibility in practical application. This paper contributes to an incremental learning framework to make classifiers learn different gesture sets (new tasks) gradually without catastrophic forgetting. First, this study analyzes the existing neural decoding methods with deep learning, and introduces an early and late fusion convolutional neural network (ELFCNN) structure based on frequency spectrum. Then, a sEMG-based gesture classification with incremental learning is demonstrated, which modifies the end to end learning method with hybrid data over/down-sampling (HDOD) method. By combining ELFCNN and the HDOD progress, the incremental learning method can make a comparable performance without data dimension reduction, and mitigate catastrophic forgetting while reducing data storage. Experimental results on both small and large size situations show consistent classification accuracy improvement from 0.47% to 0.71% compared with other popular incremental learning methods.

A double-layer feature fusion convolutional neural network for infrared small target detection

ABSTRACT Infrared small target detection is critical in remote sensing, military, and other fields. However, the low resolution of most infrared images and the lack of texture and detailed information could cause the target to be lost in a relatively noisy background. Therefore, in recent years, researchers have paid particular attention to the problem of small infrared target detection. In this paper, we propose a double-layer feature fusion convolutional neural network for infrared small target detection (DLFF), consisting of a simultaneous upsampling two-layer network module and a ‘T’-type fusion structure. First, the upsampling double-layer network module shares detection information while synchronizing detection, suppressing the background noise and enhancing the detection of the target. In addition, for the small target detection task, since the direct fusion of shallow spatial information and deep semantic information may lose only some small target features, we propose a ‘T’-type fusion structure to solve this problem. Furthermore, we collate an infrared small target dataset (MDFA_SIRIST) and design a pre-processing method for pre-detection images. The experimental results show that our network outperforms the other six state-of-the-art methods in combined evaluation metrics ( -score) and mean intersection ratio (mIou).

MMViT-Seg: A lightweight transformer and CNN fusion network for COVID-19 segmentation

Background and objective: COVID-19 is a serious threat to human health. Traditional convolutional neural networks (CNNs) can realize medical image segmentation, whilst transformers can be used to perform machine vision tasks, because they have a better ability to capture long-range relationships than CNNs. The combination of CNN and transformers to complete the task of semantic segmentation has attracted intense research. Currently, it is challenging to segment medical images on limited data sets like that on COVID-19.Methods: This study proposes a lightweight transformer+CNN model, in which the encoder sub-network is a two-path design that enables both the global dependence of image features and the low layer spatial details to be effectively captured. Using CNN and MobileViT to jointly extract image features reduces the amount of computation and complexity of the model as well as improves the segmentation performance. So this model is titled Mini-MobileViT-Seg (MMViT-Seg). In addition, a multi query attention (MQA) module is proposed to fuse the multi-scale features from different levels of decoder sub-network, further improving the performance of the model. MQA can simultaneously fuse multi-input, multi-scale low-level feature maps and high-level feature maps as well as conduct end-to-end supervised learning guided by ground truth.Results: The two-class infection labeling experiments were conducted based on three datasets. The final results show that the proposed model has the best performance and the minimum number of parameters among five popular semantic segmentation algorithms. In multi-class infection labeling results, the proposed model also achieved competitive performance.Conclusions: The proposed MMViT-Seg is tested on three COVID-19 segmentation datasets, with results showing that this model has better performance than other models. In addition, the proposed MQA module, which can effectively fuse multi-scale features of different levels further improves the segmentation accuracy.

Identification of Chinese red wine origins based on Raman spectroscopy and deep learning

In this study, we combined Raman spectroscopy with deep learning for the first time to establish an accurate, simple, and fast method to identify the origin of red wines. We collected Raman spectra from 200 red wine samples of the Cabernet Sauvignon variety from four different origins with a portable Raman spectrometer. The red wine samples, made in 2021, were from the same producer in China. Differences were found by analyzing the Raman spectra of red wine samples. These differences are mainly caused by ethanol, carboxylic acids, and polyphenols. After further analysis, for different origins, the different performances of these substances on the Raman spectrum are related to the climate and geographical conditions of the origin. The Raman spectra were analyzed by principal component analysis (PCA). The data with PCA dimensionality reduction were imported into an artificial neural network (ANN), multifeature fusion convolutional neural network (MCNN), GoogLeNet, and residual neural network (ResNet) to establish red wine origin identification models. The classification results of the model prove that climate, geography, and other conditions can provide support for the classification of red wine origin. The experiments showed that all four models performed well, among which MCNN performed the best with 93.2% classification accuracy, and the area under the curve (AUC) was 0.987. This study provides a new means to classify the origin of red wine and opens up new ideas for identifying origins in the food field.

Defect identification in adhesive structures using multi-Feature fusion convolutional neural network

The interface-debonding defects of adhesive bonding structures may cause a reduction in bonding strength, which in turn affects the bonding quality of adhesive bonding samples. Hence, defect recognition in adhesive bonding structures is particularly important. In this study, a terahertz (THz) wave was used to analyze bonded structure samples, and a multi-feature fusion convolutional neural network (CNN) was used to identify the defect waveforms. The pooling method of the squeeze-and-excitation (SE) attention mechanism was optimized, defect feature weights were adaptively assigned, and feature fusion was conducted using automatic label net-works to segment the THz waveforms in the adhesive bonding area with fine granularity waveforms as an input to the multi-channel CNN. The results revealed that the speed of the THz waveform labeling with the automatic labeling network was 10 times higher than that with traditional methods, and the defect-recognition accuracy of the defect-recognition network constructed in this study was up to 99.28%. The F1-score was 99.73%, and the lowest pre-embedded defect recognition error rate of the generalization experiment samples was 0.27%.

Integrated CNN and Waterwheel Plant Algorithm for Enhanced Global Traffic Detection

Traffic detection is critical in ensuring road safety and efficient traffic management, demanding deploying accurate and practical algorithms. This research explores the fusion of Convolutional Neural Networks (CNNs) and the Waterwheel Plant Algorithm to augment global traffic detection capabilities, utilizing a diverse dataset primarily collected from Turkey. A comprehensive evaluation of prominent CNN architectures, such as VGG19Net, AlexNet, ResNet-50, GoogLeNet, and a generic CNN, underscores substantial efficacy, with the CNN achieving an accuracy of 92.14%. Introducing the Waterwheel Plant Algorithm (WWPA) further enhances performance, as exemplified by the hybrid WWPA-CNN model, exhibiting an impressive accuracy of 97.28%. These findings highlight the promising synergies between traditional optimization algorithms and advanced neural networks, showcasing the potential for innovative developments in traffic monitoring systems and broader applications within computer vision. The statistical analyses, encompassing ANOVA and the Wilcoxon Signed Rank Test, robustly underscore the significance of this integrated approach. As the research contributes to the evolution of traffic monitoring systems, these insights provide a solid foundation for advancements in the field, fostering innovation and shaping the future landscape of computer vision applications.

Intelligent fault diagnosis algorithm of rolling bearing based on optimization algorithm fusion convolutional neural network.

As an essential component of mechanical equipment, the fault diagnosis of rolling bearings may not only guarantee the systematic operation of the equipment, but also minimize any financial losses caused by equipment shutdowns. Fault diagnosis algorithms based on convolutional neural networks (CNN) have been widely used. However, traditional CNNs have limited feature representation capabilities, thereby making it challenging to determine their hyperparameters. This paper proposes a fault diagnosis method that combines a 1D-CNN with an attention mechanism and hyperparameter optimization to overcome the aforementioned limitations; this method improves the search speed for optimal hyperparameters of CNN models, improves the diagnostic accuracy, and enhances the representation of fault feature information in CNNs. First, the 1D-CNN is improved by combining it with an attention mechanism to enhance the fault feature information. Second, a swarm intelligence algorithm based on Differential Evolution (DE) and Grey Wolf Optimization (GWO) is proposed, which not only improves the convergence accuracy, but also increases the search efficiency. Finally, the improved 1D-CNN alongside hyperparameters optimization are used to diagnose the faults of rolling bearings. By using the Case Western Reserve University (CWRU) and Jiangnan University (JNU) datasets, when compared to other common diagnosis models, the results demonstrate the usefulness and dependability of the DE-GWO-CNN algorithm in fault diagnosis applications by demonstrating the increased diagnostic accuracy and superior anti-noise capabilities of the proposed method. The fault diagnosis methodology presented in this paper can accurately identify faults and provide dependable fault classification, thereby assisting technicians in promptly resolving faults and minimizing equipment failures and operational instabilities.

A Deep Learning-Based Target Recognition Method for Entangled Optical Quantum Imaging System

Quantum imaging has the characteristics of non-locality and strong anti-interference ability, and it has received much attention. However, the target after entangled optical quantum imaging loses its local appearance structure, and it is difficult to be recognized accurately. Therefore, this paper proposes a denoise and target recognition method for entangled optical quantum imaging system. Specifically, we design the restore conditional generative adversarial network (RestoreCGAN) to restore and reconstruct the missing edge contour structure of the target. Then, we design the two-stream feature fusion convolutional neural network (TSFFCNet) to extract deep semantic features and shallow features for target recognition. The experimental results show that RestoreCGAN outperforms the state-of-the-art methods in terms of both PSNR and SSIM. Moreover, the recognition accuracy of the RestoreCGAN in combination with the TSFFCNet reaches 97.42%. This proves that the deep learning method is effective for denoise and target recognition of entangled optical quantum imaging system.

Sternum age estimation with dual channel fusion CNN model

Although age determination by radiographs of the hand and wrist before the age of 18 is an area where there is a lot of radiological knowledge and many studies are carried out, studies on age determination for adults are limited. Studies on adult age determination through sternum multidetector computed tomography (MDCT) images using artificial intelligence algorithms are much fewer. The reason for the very few studies on adult age determination is that most of the changes observed in the human skeleton with age are outside the limits of what can be perceived by the human eye. In this context, with the dual-channel Convolutional Neural Network (CNN) we developed, we were able to predict the age groups defined as 20-35, 35-50, 51-65, and over 65 with 73% accuracy over sternum MDCT images. Our study shows that fusion modeling with dual-channel convolutional neural networks and using more than one image from the same patient is more successful. Fusion models will make adult age determination, which is often a problem in forensic medicine, more accurate.

Solid waste mapping based on very high resolution remote sensing imagery and a novel deep learning approach

The urbanization worldwide leads to the rapid increase of solid waste, posing a threat to environment and people’s wellbeing. However, it is challenging to detect solid waste sites with high accuracy due to complex landscape, and very few studies considered solid waste mapping across multi-cities and in large areas. To tackle this issue, this study proposes a novel deep learning model for solid waste mapping from very high resolution remote sensing imagery. By integrating a multi-scale dilated convolutional neural network (CNN) and a Swin-Transformer, both local and global features are aggregated. Experiments in China, India and Mexico indicate that the proposed model achieves high performance with an average accuracy of 90.62%. The novelty lies in the fusion of CNN and Transformer for solid waste mapping in multi-cities without the need for pixel-wise labelled data. Future work would consider more sophisticated methods such as semantic segmentation for fine-grained solid waste classification.

Dual-Branch Fusion of Convolutional Neural Network and Graph Convolutional Network for PolSAR Image Classification

Polarimetric synthetic aperture radar (PolSAR) images contain useful information, which can lead to extensive land cover interpretation and a variety of output products. In contrast to optical imagery, there are several challenges in extracting beneficial features from PolSAR data. Deep learning (DL) methods can provide solutions to address PolSAR feature extraction challenges. The convolutional neural networks (CNNs) and graph convolutional networks (GCNs) can drive PolSAR image characteristics by deploying kernel abilities in considering neighborhood (local) information and graphs in considering long-range similarities. A novel dual-branch fusion of CNN and mini-GCN is proposed in this study for PolSAR image classification. To fully utilize the PolSAR image capacity, different spatial-based and polarimetric-based features are incorporated into CNN and mini-GCN branches of the proposed model. The performance of the proposed method is verified by comparing the classification results to multiple state-of-the-art approaches on the airborne synthetic aperture radar (AIRSAR) dataset of Flevoland and San Francisco. The proposed approach showed 1.3% and 2.7% improvements in overall accuracy compared to conventional methods with these AIRSAR datasets. Meanwhile, it enhanced its one-branch version by 0.73% and 1.82%. Analyses over Flevoland data further indicated the effectiveness of the dual-branch model using varied training sampling ratios, leading to a promising overall accuracy of 99.9% with a 10% sampling ratio.

Research on cassava disease classification using the multi-scale fusion model based on EfficientNet and attention mechanism.

Cassava disease is one of the leading causes to the serious decline of cassava yield. Because it is difficult to identify the characteristics of cassava disease, if not professional cassava growers, it will be prone to misjudgment. In order to strengthen the judgment of cassava diseases, the identification characteristics of cassava diseases such as different color of cassava leaf disease spots, abnormal leaf shape and disease spot area were studied. In this paper, deep convolutional neural network was used to classify cassava leaf diseases, and image classification technology was used to recognize and classify cassava leaf diseases. A lightweight module Multi-scale fusion model (MSFM) based on attention mechanism was proposed to extract disease features of cassava leaves to enhance the classification of disease features. The resulting feature map contained key disease identification information. The study used 22,000 cassava disease leaf images as a data set, including four different cassava leaf disease categories and healthy cassava leaves. The experimental results show that the cassava leaf disease classification model based on multi-scale fusion Convolutional Neural Network (CNN) improves EfficientNet compared with the original model, with the average recognition rate increased by nearly 4% and the average recognition rate up to 88.1%. It provides theoretical support and practical tools for the recognition and early diagnosis of plant disease leaves.

Fusion of multivariate EEG signals for schizophrenia detection using CNN and machine learning techniques

Schizophrenia is a severe mental disorder that has adverse effects on the behavior of an individual such as disorganized speech and delusions. Electroencephalography (EEG) signals are widely used for its identification as they are non-invasive and have high temporal resolution. EEG signals may be captured using wearable devices but transmission of complete data from all channels is both battery and data consuming. Several studies on Schizophrenia have either used all channels or relied on sophisticated feature extraction algorithms to find the most relevant EEG channels for further processing. That too, however, needs data from all channels beforehand to identify the most relevant features. In this study, a publicly available multi-channel EEG signals dataset from the institute of Psychiatry and Neurology in Warsaw, Poland is studied for an automated identification of Schizophrenia using only a subset of data from selected channels. To achieve this, we device a channel selection mechanism based on a rigorous performance analysis of the Convolutional Neural Network (CNN) while considering the individual EEG channels at different brain regions. The selected channels are combined, and we use a fusion of CNN and different machine learning (ML) classifiers to train the classification model. Our experiments show that a combination of three channels namely, T4, T3, and Cz achieves 90% and 98% accuracies on subject-based and non-subject based testing, respectively, using a hybridization of CNN and logistic regression (LR).

A convolutional neural network model of multi-scale feature fusion: MFF-Net

MFF-Net (a multi-scale feature fusion convolutional neural network) was designed to improve the recognition rate of handwritten digits. The low-level, middle-level and high-level features of the image were first extracted through the convolution operation, and then the low-level and intermediate features were further extracted through different convolutional layers, later directly fused with the high-level features of the image with a certain weight, and then processed by the full connection layer. By adding a batch normalization layer before the activation layer, and a dropout layer between the full connection layers, the accuracy and generalization capacity of the network are improved. At the same time, a dynamic learning rate algorithm was designed, with which, the trained network accuracy was significantly improved as shown in the experiments on the MNIST data set. The accurate rate could reach 99.66% through only 30 epochs training. The comparison indicated that the accuracy of the network model is significantly higher than that of others.

Bearing fault diagnosis based on particle swarm optimization fusion convolutional neural network.

Bearings are the most basic and important mechanical parts. The stable and safe operation of the equipment requires bearing fault diagnosis in advance. So, bearing fault diagnosis is an important technology. However, the feature extraction quality of the traditional convolutional neural network bearing fault diagnosis is not high and the recognition accuracy will decline under different working conditions. In response to these questions, a bearing fault model based on particle swarm optimization (PSO) fusion convolution neural network is proposed in this paper. The model first adaptively adjusts the hyperparameters of the model through PSO, then introduces residual connections to prevent the gradient from disappearing, uses global average pooling to replace the fully connected layer to reduce the training parameters of the model, and finally adds a dropout layer to prevent network overfitting. The experimental results show that the model is under four conditions, two of which can achieve 100% recognition, and the other two can also achieve more than 98% accuracy. And compared with the traditional diagnosis method, the model has higher accuracy under variable working conditions. This research has important research significance and economic value in the field of the intelligent machinery industry.

TC-Net: Dual coding network of Transformer and CNN for skin lesion segmentation.

Skin lesion segmentation has become an essential recent direction in machine learning for medical applications. In a deep learning segmentation network, the convolutional neural network (CNN) uses convolution to capture local information for modeling. However, it ignores the relationship between pixels and still can not meet the precise segmentation requirements of some complex low contrast datasets. Transformer performs well in modeling global feature information, but their ability to extract fine-grained local feature patterns is weak. In this work, The dual coding fusion network architecture Transformer and CNN (TC-Net), as an architecture that can more accurately combine local feature information and global feature information, can improve the segmentation performance of skin images. The results of this work demonstrate that the combination of CNN and Transformer brings very significant improvement in global segmentation performance and allows outperformance as compared to the pure single network model. The experimental results and visual analysis of these three datasets quantitatively and qualitatively illustrate the robustness of TC-Net. Compared with Swin UNet, on the ISIC2018 dataset, it has increased by 2.46% in the dice index and about 4% in the JA index. On the ISBI2017 dataset, the dice and JA indices rose by about 4%.

Multi-View Information Fusion Fault Diagnosis Method Based on Attention Mechanism and Convolutional Neural Network

Multi-view information fusion can provide more accurate, complete and reliable data descriptions for monitoring objects, effectively improve the limitations and unreliability of single-view data. Existing multi-view information fusion based on deep learning mostly focuses on the feature level and decision level, with large information loss, and does not distinguish the view weight in the fusion process. To this end, a multi-view data level information fusion model CAM_MCFCNN with view weight was proposed based on a channel attention mechanism and convolutional neural network. The model used the channel characteristics to implement multi-view information fusion at the data level stage, which made the fusion position and mode more natural and reduced the loss of information. A multi-channel fusion convolutional neural network was used for feature learning. In addition, the channel attention mechanism was used to learn the view weight, so that the algorithm could pay more attention to the views that contribute more to the fault identification task during the training process, and more reasonably integrate the information of different views. The proposed method was verified by the data of the planetary gearbox experimental platform. The multi-view data and single-view data were used as the input of the CAM_MCFCNN model and single-channel CNN model respectively for comparison. The average accuracy of CAM_MCFCNN on three constant-speed datasets reached 99.95%, 99.87% and 99.92%, which was an improvement of 0.95%, 2.25%, and 0.04%, compared with the single view with the highest diagnostic accuracy, respectively. When facing limited samples, CAM_MCFCNN had similar performance. Finally, compared with different multi-view information fusion algorithms, CAM_MCFCNN showed better stability and higher accuracy. The experimental results showed that the proposed method had better performance, higher diagnostic accuracy and was more reliable, compared with other methods.

Convolutional neural network method for damage detection of CFRP in electrical impedance tomography

Damage detection is vitally important for carbon fiber reinforced polymer (CFRP) laminates. When CFRP laminates are damaged, its impedance property is changed. Based on the electrical properties of CFRP laminates, the changed conductivity distribution can be reconstructed with the electrical impedance tomography (EIT) method. The detection method is attractive due to its simple equipment, low cost, and easy operation. However, image reconstruction of EIT faces a serious ill-conditioned nonlinear inverse problem. In order to solve this problem, a feature fusion convolutional neural network based on the dense connection (FF-D) method is applied in EIT to establish the mapping relationship between voltage measurement and conductivity distribution in this paper. The optimization can extract and utilize features to a greater degree and improve reconstruction accuracy and robustness. For the purpose of simulating the electrical properties of CFRP better, the conductivity values measured by an impedance analyzer are used as the data set. The correlation coefficient (CC) and root mean square error (RMSE) are used as indicators to evaluate the quality of image reconstruction. The simulation and experimental results suggest that the FF-D method can reconstruct images better than typical algorithms based on deep learning and conventional algorithms of EIT.

Bearing fault diagnosis method using a spatio-temporal neural network based on feature transfer learning

An intelligent bearing fault diagnosis method based requires a large quantity of labeled data. However, in an actual engineering environment, only a tiny amount of unlabeled data can be collected. To solve this problem, we construct a spatio-temporal neural network (STN) model by multi-layer fusion of convolutional neural network (CNN) and long-term memory network features. Then, a model based on feature migration is constructed and a STN is applied as the feature extractor of the network. Finally, the Case Western Reserve University bearing dataset is employed to verify the performance of our proposed model, and the influence of different neural network feature extractors (CNN, recurrent neural network, long- and short-term memory network, STN) and several feature transfer measures [correlation alignment, multiple kernel maximum mean discrepancy, joint maximum mean discrepancy, discriminative joint probability maximum mean discrepancy (DJP-MMD) on the accuracy of the model were compared. The results show that the diagnostic accuracy of the proposed method is over 98%, and the diagnostic accuracy can be maintained at around 99% in most cases when the signal to noise ratio (SNR) is 10 dB. When the SNR is lower than 2 dB, the accuracy of the STN-DJPMMD model is still over 88%.

Dexterous Identification of Carcinoma through ColoRectalCADx with Dichotomous Fusion CNN and UNet Semantic Segmentation.

Human colorectal disorders in the digestive tract are recognized by reference colonoscopy. The current system recognizes cancer through a three-stage system that utilizes two sets of colonoscopy data. However, identifying polyps by visualization has not been addressed. The proposed system is a five-stage system called ColoRectalCADx, which provides three publicly accessible datasets as input data for cancer detection. The three main datasets are CVC Clinic DB, Kvasir2, and Hyper Kvasir. After the image preprocessing stages, system experiments were performed with the seven prominent convolutional neural networks (CNNs) (end-to-end) and nine fusion CNN models to extract the spatial features. Afterwards, the end-to-end CNN and fusion features are executed. These features are derived from Discrete Wavelet Transform (DWT) and Vector Support Machine (SVM) classification, that was used to retrieve time and spatial frequency features. Experimentally, the results were obtained for five stages. For each of the three datasets, from stage 1 to stage 3, end-to-end CNN, DenseNet-201 obtained the best testing accuracy (98%, 87%, 84%), ((98%, 97%), (87%, 87%), (84%, 84%)), ((99.03%, 99%), (88.45%, 88%), (83.61%, 84%)). For each of the three datasets, from stage 2, CNN DaRD-22 fusion obtained the optimal test accuracy ((93%, 97%) (82%, 84%), (69%, 57%)). And for stage 4, ADaRDEV2-22 fusion achieved the best test accuracy ((95.73%, 94%), (81.20%, 81%), (72.56%, 58%)). For the input image segmentation datasets CVC Clinc-Seg, KvasirSeg, and Hyper Kvasir, malignant polyps were identified with the UNet CNN model. Here, the loss score datasets (CVC clinic DB was 0.7842, Kvasir2 was 0.6977, and Hyper Kvasir was 0.6910) were obtained.