Fusion Convolutional Neural Network Research Articles

A Client-Cloud-Chain Data Annotation System of Internet of Things for Semi-Supervised Missing Data

With continuous progress in science and technology, a large amount of data are produced in all fields of the world at anytime and anywhere. These data are unmarked and lack marking information, while manual marking is time-consuming and laborious. Herein, this paper introduces a distributed semi-supervised labeling framework. This framework addresses the issue of missing data by proposing an attribute-filling method based on subspace learning. Furthermore, this paper presents a distributed semi-supervised learning strategy that trains sub-models (private models) within each sub-system. Finally, this paper develops a distributed graph convolutional neural network fusion technique with enhanced interpretability grounded on the attention mechanism. This paper assigns weights of importance to the edges of each layer in the graph neural network based on sub-models and public data, thereby enabling distributed and interpretable graph convolutional attention. Extensive experimentation using public datasets demonstrates the superiority of the proposed scheme over other state-of-the-art baselines, achieving a reduction in loss of 50% compared to the original approach.

Optimal Deep Convolutional Neural Network based Fusion Model for Soil Nutrient Analysis

The vast majority of people in India, agriculture is their main line of work, and it has a large economic impact.. Soil is important for supplying vital nutrients to crops for better yield. Determining soil nutrients is certainly essential for selecting appropriate crops and monitoring growth. Common methods used by agriculturalists are inadequate to satisfy increasing demands and have to obstruct cultivating soil. For a better crop yield, agriculturalists must have an awareness regarding the soil nutrients for a specific crop. There comes a need for using Deep learning methods in soil analysis that would help farmers in the domain. This study introduces an Optimal Deep Convolutional Neural Network Fusion Model for Soil nutrient Type Classification (ODCNNF-STC) technique. The presented ODCNNF-STC technique examines the input soil images to classify them into different nutrients present in the soil. In this approach, the noise present in the soil images are initially filtered using a bilateral filter (BF) followed by contrast enhancement. The preprocessed soil images are fed to the model formed by the fusion of DenseNet201 and InceptionResNetV2 models extracting the soil images that can successfully differentiate soil nutrients. Finally, classification of soil nutrients were performed by three classifiers namely extreme learning machine (ELM), RMSProp optimizer-based 1DCNN, and RMSProp optimizer-based Stacked Auto Encoder (SAE). The experimental validation of ODCNNF-STC method is examined on real-time dataset of soil images with a maximum accuracy of 99.39% over recent methods.

A Convolutional Neural Network Based on Soft Attention Mechanism and Multi-Scale Fusion for Skin Cancer Classification

The seven most common skin diseases are melanocytic nevus, melanoma, benign keratosis, basal cell carcinoma, actinic keratosis, vascular lesions, and dermatofibroma. Among them, melanoma has been identified as one of the deadliest cancers based on medical studies and research. The current trend in disease detection revolves around the use of machine learning and deep learning models. Regardless of the model used, the crucial aspect is achieving accurate classification for these diseases. With the emergence of powerful convolutional neural networks (CNNs), significant progress has been made in classification of skin cancer lesions in recent years. However, various challenges hinder the development of practical and effective solutions. First, due to the specific nature of skin cancer lesion images, the current deep neural network architectures and training strategies have poor adaptability to medical images. They are also prone to gradient vanishing issues during network iteration, which hinders the construction of high-performance deep learning models that leverage distinctive characteristics of skin lesion images. Second, there exists a discordance between skin lesion images and deep learning network structures. To address these issues, this study introduces a soft attention mechanism to enhance adaptability to skin cancer lesion images and improve the extraction of informative features from medical images. Additionally, a novel multi-scale fusion convolutional neural network model is proposed to overcome the mismatch between deep learning CNN architectures and skin lesion images. This model autonomously extracts appearance features from raw dermatological medical images. Comparisons with other popular techniques demonstrate the effectiveness of the proposed model, which can achieve an accuracy of 93.9% on HAM10000 dataset. There is ongoing research to overcome the remaining challenges and further enhance the performance of skin cancer classification algorithms.

Integrating Temporal Fluctuations in Crop Growth with Stacked Bidirectional LSTM and 3D CNN Fusion for Enhanced Crop Yield Prediction

Optimizing farming methods and guaranteeing a steady supply of food depend critically on accurate predictions of crop yields. The dynamic temporal changes that occur during crop growth are generally ignored by conventional crop growth models, resulting in less precise projections. Using a stacked bidirectional Long Short-Term Memory (LSTM) structure and a 3D Convolutional Neural Network (CNN) fusion, we offer a novel neural network model that accounts for temporal oscillations in the crop growth process. The 3D CNN efficiently recovers spatial and temporal features from the crop development data, while the bidirectional LSTM cells capture the sequential dependencies and allow the model to learn from both past and future temporal information. Our model's prediction accuracy is improved by combining the LSTM and 3D CNN layers at the top, which better captures temporal and spatial patterns. We also provide a novel label-related loss function that is optimized for agricultural yield forecasting. Because of the relevance of temporal oscillations in crop development and the dynamic character of crop growth, a new loss function has been developed. This loss function encourages our model to learn and take advantage of the temporal trends, which improves our ability to estimate crop yield. We perform comprehensive experiments on real-world crop growth datasets to verify the efficacy of our suggested approach. The outcomes prove that our unified strategy performs far better than both baseline crop growth prediction algorithms and cutting-edge applications of deep learning. Improved crop yield prediction accuracy is achieved with the integration of temporal variations via the merging of bidirectional LSTM and 3D CNN and a unique loss function. This study helps move the science of estimating crop yields forward, which is important for informing agricultural policy and ensuring a steady supply of food.

Online monitoring of local defects in robotic laser additive manufacturing process based on a dynamic mapping strategy and multibranch fusion convolutional neural network

Online monitoring is essential for laser additive manufacturing (AM) to improve in-process quality control. Currently, accurate monitoring of local defects in the laser AM process remains a challenge. This paper proposes a method for predicting local defects in the laser AM process based on a dynamic mapping strategy and the multibranch fusion convolutional neural network (MBFCNN). In-situ sensing of the laser-material interaction zone is achieved using a camera integrated coaxially with the printing system. Experiment-based datasets are constructed, in which the in-process images were sampled and matched to the extracted local defect information based on their temporal-spatial correspondence. A dynamic mapping strategy using a sliding sampling window is introduced to achieve continuous monitoring. Considering the cyclic and layer-by-layer processing principle of laser AM, we propose MBFCNN to map the in-process images to local defect information. A multibranch feature extraction module is designed based on the deposited layers of the target region to be monitored, in which each branch extracts high-dimensional representations from in-process images corresponding to a certain layer. We further introduce an attention mechanism to distinguish the importance of each branch and a feature fusion module to fuse the high-level information. Experimental results and comparison with traditional convolutional neural networks demonstrate the effectiveness of our method.

Deep transfer learning rolling bearing fault diagnosis method based on convolutional neural network feature fusion

Current deep-learning methods are often based on significantly large quantities of labeled fault data for supervised training. In practice, it is difficult to obtain samples of rolling bearing failures. In this paper, a transfer learning-based feature fusion convolutional neural network approach for bearing fault diagnosis is proposed. Specifically, the raw vibration signal features and the corresponding time-frequency image features of the input data are extracted by a one-dimensional convolutional neural network and a pre-trained ConvNeXt, respectively, and connected by a feature fusion strategy. Then, the fine-tuning method based on transfer learning can effectively reduce the reliance on labeled samples in the target domain. A wide convolution kernel is introduced in the time-domain signal feature extraction to increase the receptive field, which is combined with the channel attention mechanism to further optimize the feature quality. Finally, two common bearing datasets are utilized for fault diagnosis experiments. The experimental results show that the proposed model achieves an average accuracy of more than 98.63% in both cross-working conditions and cross-device diagnosis tasks. Meanwhile, anti-noise experiments and ablation experiments further validate the accuracy and robustness of the proposed method.

Image processing based on neural networks

This article integrates convolutional neural network (CNN) and graph convolutional network (GCN) techniques. Performance is improved by the suggested architecture's use of crucial methods such as dropout, batch normalization, and rank-based random pooling. The network was trained and tested on a sizable amount of breast lesion imaging data, and its precision was evaluated in comparison to other methods. The findings showed a significant increase in accuracy, resulting in high rates of malignant lesion diagnosis with few false positives. The effective fusion of CNN and GCN methods highlights the potential for improving the detection of malignant breast lesions and provides a viable path for further study in this area.

Application of serum SERS technology combined with deep learning algorithm in the rapid diagnosis of immune diseases and chronic kidney disease

Surface-enhanced Raman spectroscopy (SERS), as a rapid, non-invasive and reliable spectroscopic detection technique, has promising applications in disease screening and diagnosis. In this paper, an annealed silver nanoparticles/porous silicon Bragg reflector (AgNPs/PSB) composite SERS substrate with high sensitivity and strong stability was prepared by immersion plating and heat treatment using porous silicon Bragg reflector (PSB) as the substrate. The substrate combines the five deep learning algorithms of the improved AlexNet, ResNet, SqueezeNet, temporal convolutional network (TCN) and multiscale fusion convolutional neural network (MCNN). We constructed rapid screening models for patients with primary Sjögren’s syndrome (pSS) and healthy controls (HC), diabetic nephropathy patients (DN) and healthy controls (HC), respectively. The results showed that the annealed AgNPs/PSB composite SERS substrates performed well in diagnosing. Among them, the MCNN model had the best classification effect in the two groups of experiments, with an accuracy rate of 94.7% and 92.0%, respectively. Previous studies have indicated that the AgNPs/PSB composite SERS substrate, combined with machine learning algorithms, has achieved promising classification results in disease diagnosis. This study shows that SERS technology based on annealed AgNPs/PSB composite substrate combined with deep learning algorithm has a greater developmental prospect and research value in the early identification and screening of immune diseases and chronic kidney disease, providing reference ideas for non-invasive and rapid clinical medical diagnosis of patients.

Deep learning-based end-to-end 3D depth recovery from a single-frame fringe pattern with the MSUNet++ network.

Deep learning (DL)-based single-frame fringe pattern to 3D depth reconstruction methods have aroused extensive research interest. The goal is to estimate high-precision 3D shape from a single frame of fringe pattern with limited information. Therefore, the purpose of this work attempts to propose an end-to-end DL-based 3D reconstruction method from the single fringe pattern with excellent capability of achieving high accuracy depth recovery and geometry details preservation of tested objects. We construct a multi-scale feature fusion convolutional neural network (CNN) called MSUNet++, which incorporates discrete wavelet transform (DWT) in data preprocessing for extracting high-frequency signals of fringe patterns as input of the network. Additionally, a loss function that combines structural similarity with edge perception is established. Through these measures, high-frequency geometry details of the reconstruction results can be obviously enhanced, while the geometric shape can be effectively maintained. Ablation experiments are involved in validating the effectiveness of our proposed solution. 3D reconstructed results and analysis of generalization experiments on different tested samples imply that the proposed method in this research enjoys capabilities of higher accuracy, better detail preservation, and robustness in comparison with the compared methods.

Classification of hyperspectral images using fusion of CNN and MiniGCN with SVM

ABSTRACT Convolutional neural networks (CNNs) have gained popularity for categorizing hyperspectral (HS) images due to their ability to capture representations of spatial-spectral features. However, their ability to model relationships between data is limited. Graph convolutional networks (GCNs) have been introduced as an alternative, as they are effective in representing and analyzing irregular data beyond grid sampling constraints. While GCNs have traditionally been computationally intensive, minibatch GCNs (miniGCNs) enable minibatch training of large-scale GCNs. We have improved the classification performance by using miniGCNs to infer out-of-sample data without retraining the network. In addition, fuzing the capabilities of CNNs and GCNs, through concatenative fusion has been shown to improve performance compared to using CNNs or GCNs individually. Finally, support vector machine (SVM) is employed instead of softmax in the classification stage. These techniques were tested on two HS datasets and achieved an average accuracy of 92.80 using Indian Pines dataset, demonstrating the effectiveness of miniGCNs and fusion strategies.

Acoustic emission for surface quality monitoring in laser shock peening via dual-feature fusion convolution neural network

Acoustic emission for surface quality monitoring in laser shock peening via dual-feature fusion convolution neural network

Brain tumor detection model based on CNN and threshold segmentation

Brain tumours pose a substantial global health issue, emphasising the criticality of timely and precise identification to ensure optimal treatment outcomes. This research introduces an innovative methodology for the identification of brain tumours by employing a fusion of Convolutional Neural Networks (CNNs) with threshold segmentation techniques. The objective of the suggested model is to improve the precision and effectiveness of brain tumour identification in medical imaging, namely Magnetic Resonance Imaging (MRI) scans. Fast and precise diagnosis is necessary in the medical profession for effective treatment, but current technologies lack this capability. For successful therapy, it is therefore necessary to develop an effective diagnosis application. Global threshold segmentation for pre-processing is used in this study. Image capture and de-noising were completed in the first stage, while classification and regression were completed in the second stage using ML approaches. A computer-aided automated identification method is the computational technique used in this study. This investigation uses one hundred twenty (120) brain scans from a real-time MRI brain database, of them 15 normal and 105 abnormal. According to performance metrics, the accuracy of training and testing pictures was 99.46%. Comparing this method to recently published methods, it is determined that LR-ML with the threshold segmentation has a rapid, precise brain diagnostic system.

Facial feature fusion convolutional neural network for driver fatigue detection

Driver fatigue detection is critical for preventing traffic accidents. However, in real driving scenarios, low-quality input situations often occur due to occlusion, large-turn head deformation, and inconsistent eye states, which introduce noise and degrade the performance of single-stream networks. To address this, we propose a multi-stream facial feature fusion convolution neural network (FFF–CNN) that enhances performance on low-quality inputs. The FFF-CNN includes a global face stream and two local eye streams, with feature fusion used for eye state classification. To facilitate feature fusion, we implemented two preprocessing modules, the feature attention module (FAM) and the stream interaction module (SIM). FAM emphasizes important features in the global face stream, while SIM corrects information from the two local eye streams. The proposed FFF-CNN achieved good performance on both our dataset and the public Closed Eyes in the Wild (CEW) dataset. Compared to single-stream networks, the FFF-CNN improved performance by 4.95% and 4.27% on our dataset and the CEW dataset, respectively. Furthermore, it achieved state-of-the-art performance on the CEW dataset with an accuracy of 98.35%. The effectiveness of the proposed FAM and SIM has been verified by experiments. Notably, our experimental results show that the proposed FFF-CNN effectively improved fatigue detection performance on low-quality input, approaching the performance on normal situations.

CoTrFuse: a novel framework by fusing CNN and transformer for medical image segmentation

Medical image segmentation is a crucial and intricate process in medical image processing and analysis. With the advancements in artificial intelligence, deep learning techniques have been widely used in recent years for medical image segmentation. One such technique is the U-Net framework based on the U-shaped convolutional neural networks (CNN) and its variants. However, these methods have limitations in simultaneously capturing both the global and the remote semantic information due to the restricted receptive domain caused by the convolution operation’s intrinsic features. Transformers are attention-based models with excellent global modeling capabilities, but their ability to acquire local information is limited. To address this, we propose a network that combines the strengths of both CNN and Transformer, called CoTrFuse. The proposed CoTrFuse network uses EfficientNet and Swin Transformer as dual encoders. The Swin Transformer and CNN Fusion module are combined to fuse the features of both branches before the skip connection structure. We evaluated the proposed network on two datasets: the ISIC-2017 challenge dataset and the COVID-QU-Ex dataset. Our experimental results demonstrate that the proposed CoTrFuse outperforms several state-of-the-art segmentation methods, indicating its superiority in medical image segmentation. The codes are available at https://github.com/BinYCn/CoTrFuse.

A histopathological image classification method for cholangiocarcinoma based on spatial-channel feature fusion convolution neural network.

Histopathological image analysis plays an important role in the diagnosis and treatment of cholangiocarcinoma. This time-consuming and complex process is currently performed manually by pathologists. To reduce the burden on pathologists, this paper proposes a histopathological image classification method for cholangiocarcinoma based on spatial-channel feature fusion convolutional neural networks. Specifically, the proposed model consists of a spatial branch and a channel branch. In the spatial branch, residual structural blocks are used to extract deep spatial features. In the channel branch, a multi-scale feature extraction module and some multi-level feature extraction modules are designed to extract channel features in order to increase the representational ability of the model. The experimental results of the Multidimensional Choledoch Database show that the proposed method performs better than other classical CNN classification methods.

Quadruplet depth-wise separable fusion convolution neural network for ballistic target recognition with limited samples

Radar high-resolution range profile (HRRP) plays a crucial role in ballistic target recognition due to its simplicity and fast computation. Convolutional neural networks (CNNs), a popular deep learning approach, are widely used in radar automatic target recognition (RATR) based on HRRP, thanks to their excellent feature extraction capabilities. However, current research mainly focuses on integrating branch networks like recurrent neural networks and attention mechanisms into vanilla CNNs to enhance feature extraction. Limited research exists on reducing the computational complexity of vanilla CNNs when applied to HRRP-based RATR, as well as target recognition problems with limited samples. To address these issues, this paper proposes a lightweight depth-wise separable fusion CNN (DSFCNN) for ballistic target HRRP recognition. The DSFCNN reduces the computational complexity of vanilla CNNs while improving recognition accuracy. Furthermore, we introduce the sample-level fitting and class-level distinguishability (SFCD) loss for limited-sample ballistic target HRRP recognition. The SFCD loss ensures better fitting performance of all samples, as well as higher intra-class compactness and inter-class separability, enhancing the distinguishability of embedded features in limited-sample conditions. During the training phase, the SFCD loss is equally applied to four DSFCNNs with shared structures and parameters, resulting in a model known as Quadruplet DSFCNN (QDSFCNN). However, during the testing phase, only one DSFCNN is utilized. Additionally, we propose the parameter quantity shifting-fitting performance (PQS-FP) coordinate system to elucidate the performance disparities between DSFCNN and vanilla CNNs in our experiments. PQS-FP offers inspirational insights into the relations among parameter quantity, fitting performance, and recognition performance. Extensive experimental results demonstrate the robustness and effectiveness of QDSFCNN for ballistic target recognition in various limited-sample scenarios.

TBi-YOLOv5: A surface defect detection model for crane wire with Bottleneck Transformer and small target detection layer

Surface defect detection of crane wire rope is very important for safe operations of crane. However, the surface defect characteristics of crane wire rope are varied, there is no standardized data and tiny-size, which makes the universal surface defect detection models ineffective. Therefore, a surface defect detection model named TBi-YOLOv5 is proposed in this article. First, a random augment method is used to search the optimal data augment strategy automatically and enforce the generalization ability of the surface defect detection model. Then, in order to enhance the feature extraction ability on both the local and global views, a Bottleneck Transformer (BOT) module is added in YOLOv5, which makes effective fusion of convolutional neural network (CNN) and multi-head self-attention mechanism (MHSA). After that, inspired by the idea of bidirectional feature pyramid network (BiFPN), a small target detection layer (STPL) is added in the neck part of YOLOv5 to detect the tiny-size surface defect on the wire rope. Experiments on the surface defect detection of crane wire rope show the effectiveness of the proposed TBi-YOLOv5. Compared with YOLOv5s, the mean average precision (mAP) of TBi-YOLOv5 has been increased by nearly 4%.

Multi-Scale Feature Fusion Convolutional Neural Networks for Fault Diagnosis of Electromechanical Actuator

Airborne electromechanical actuators (EMAs) play a key role in the flight control system, and their health condition has a considerable impact on the flight status and safety of aircraft. Considering the multi-scale feature of fault signals and the fault diagnosis reliability for EMAs under complex working conditions, a novel fault diagnosis method of multi-scale feature fusion convolutional neural network (MSFFCNN) is proposed. Leveraging the multiple different scales’ learning structure and attention mechanism-based feature fusion, the fault-related information can be effectively captured and learned, thereby improving the recognition ability and diagnostic performance of the network. The proposed method was evaluated by experiments and compared with the other three fault-diagnosis algorithms. The results show that the proposed MSFFCNN approach has a better diagnostic performance compared with the state-of-the-art fault diagnosis methods, which demonstrates the effectiveness and superiority of the proposed method.

Simultaneous analysis of mildew degree and aflatoxin B1 of wheat by a multi-task deep learning strategy based on microwave detection technology

This study proposes a novel qualitative and quantitative analysis method for wheat mildew degree and aflatoxin B1 (AFB1) by microwave detection technology combined with multi-task learning strategy. The team developed a miniaturized microwave detection device with double heterodyne mixing structure, and used this device to obtain the transmission indexes (amplitude and phase) of moldy wheat samples. A multi-task learning model based on convolutional neural network (CNN) structure was designed to realize the self-learning and model calibration of the transmission indexes, and to complete the classification and regression tasks of wheat mildew degree recognition and AFB1 content detection. The obtained results found that in the qualitative analysis, the accuracy, precision, recall, and F1-score of the fusion CNN model were all 100% during the prediction. In the quantitative detection, the RMSEP, RP2 and RPD of the fusion CNN model respectively were 2.0138 μg kg−1, 0.9807 and 7.2849. The overall results reveal that it is feasible to realize simultaneous analysis of the contamination level of wheat mycotoxins by microwave detection technology combined with deep learning, and the application of multi-task learning strategy can effectively avoid the waste of resources in the process of model calibration.

GuidedNet: A General CNN Fusion Framework via High-Resolution Guidance for Hyperspectral Image Super-Resolution

Hyperspectral image super-resolution (HISR) is about fusing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI) to generate a high-resolution hyperspectral image (HR-HSI). Recently, convolutional neural network (CNN)-based techniques have been extensively investigated for HISR yielding competitive outcomes. However, existing CNN-based methods often require a huge amount of network parameters leading to a heavy computational burden, thus, limiting the generalization ability. In this article, we fully consider the characteristic of the HISR, proposing a general CNN fusion framework with high-resolution guidance, called GuidedNet. This framework consists of two branches, including 1) the high-resolution guidance branch (HGB) that can decompose the high-resolution guidance image into several scales and 2) the feature reconstruction branch (FRB) that takes the low-resolution image and the multiscaled high-resolution guidance images from the HGB to reconstruct the high-resolution fused image. GuidedNet can effectively predict the high-resolution residual details that are added to the upsampled HSI to simultaneously improve spatial quality and preserve spectral information. The proposed framework is implemented using recursive and progressive strategies, which can promote high performance with a significant network parameter reduction, even ensuring network stability by supervising several intermediate outputs. Additionally, the proposed approach is also suitable for other resolution enhancement tasks, such as remote sensing pansharpening and single-image super-resolution (SISR). Extensive experiments on simulated and real datasets demonstrate that the proposed framework generates state-of-the-art outcomes for several applications (i.e., HISR, pansharpening, and SISR). Finally, an ablation study and more discussions assessing, for example, the network generalization, the low computational cost, and the fewer network parameters, are provided to the readers. The code link is: https://github.com/Evangelion09/GuidedNet.