Multi-Scale Fusion Convolutional Neural Network Research Articles

Multi-View Video Quality Enhancement Method Based on Multi-Scale Fusion Convolutional Neural Network and Visual Saliency

Multi-View Video Quality Enhancement Method Based on Multi-Scale Fusion Convolutional Neural Network and Visual Saliency

Load prediction of integrated energy systems for energy saving and carbon emission based on novel multi-scale fusion convolutional neural network

The integrated energy system plays an important role in the energy conservation, emission reduction and the resource-efficient utilization. Accurate load forecasting is a significant basis for the optimal scheduling of the integrated energy system. The integrated energy system has coupling interaction between different energy sources in production, distribution and storage. However, the traditional method cannot effectively extract multi-scale features and utilize the coupling information between the multivariate energy sources. In this paper, a novel multi-scale fusion convolutional neural network integrating the bi-directional long short-term memory network and multi-domains hierarchical decoding is proposed to extract and analyze multivariate load data coupling in the integrated energy system data. The multi-scale fusion convolutional neural network is constructed by the multi-dimension convolution layer to obtain multi-scale feature of the integrated energy system data. Meanwhile, the bi-directional long short-term memory network is applied to extract the time dependencies of the integrated energy system data. Finally, the multi-domains hierarchical decoding extracts the coupling characteristics of different domains to predict multiple domains values. Compared with the backpropagation neural network, the support vector machine, the short and long-time memory network, the bi-directional long short-term memory network, and the convolutional neural network - bi-directional long short-term memory network, the proposed method achieves state-of-the-art results in terms of the mean absolute percentage error with 0.365 %, the explained variance score with 99.987 % and the R2-score with 99.984 %, which proves the effectiveness of the proposed model in the load prediction of the integrated energy system. In addition, the proposed method can provide operational guidance for energy production and storage. Through the operation guidance of the proposed method, the carbon emissions are reduced by 238791.58 kg every week.

A Convolutional Neural Network Based on Soft Attention Mechanism and Multi-Scale Fusion for Skin Cancer Classification

The seven most common skin diseases are melanocytic nevus, melanoma, benign keratosis, basal cell carcinoma, actinic keratosis, vascular lesions, and dermatofibroma. Among them, melanoma has been identified as one of the deadliest cancers based on medical studies and research. The current trend in disease detection revolves around the use of machine learning and deep learning models. Regardless of the model used, the crucial aspect is achieving accurate classification for these diseases. With the emergence of powerful convolutional neural networks (CNNs), significant progress has been made in classification of skin cancer lesions in recent years. However, various challenges hinder the development of practical and effective solutions. First, due to the specific nature of skin cancer lesion images, the current deep neural network architectures and training strategies have poor adaptability to medical images. They are also prone to gradient vanishing issues during network iteration, which hinders the construction of high-performance deep learning models that leverage distinctive characteristics of skin lesion images. Second, there exists a discordance between skin lesion images and deep learning network structures. To address these issues, this study introduces a soft attention mechanism to enhance adaptability to skin cancer lesion images and improve the extraction of informative features from medical images. Additionally, a novel multi-scale fusion convolutional neural network model is proposed to overcome the mismatch between deep learning CNN architectures and skin lesion images. This model autonomously extracts appearance features from raw dermatological medical images. Comparisons with other popular techniques demonstrate the effectiveness of the proposed model, which can achieve an accuracy of 93.9% on HAM10000 dataset. There is ongoing research to overcome the remaining challenges and further enhance the performance of skin cancer classification algorithms.

Application of serum SERS technology combined with deep learning algorithm in the rapid diagnosis of immune diseases and chronic kidney disease

Surface-enhanced Raman spectroscopy (SERS), as a rapid, non-invasive and reliable spectroscopic detection technique, has promising applications in disease screening and diagnosis. In this paper, an annealed silver nanoparticles/porous silicon Bragg reflector (AgNPs/PSB) composite SERS substrate with high sensitivity and strong stability was prepared by immersion plating and heat treatment using porous silicon Bragg reflector (PSB) as the substrate. The substrate combines the five deep learning algorithms of the improved AlexNet, ResNet, SqueezeNet, temporal convolutional network (TCN) and multiscale fusion convolutional neural network (MCNN). We constructed rapid screening models for patients with primary Sjögren’s syndrome (pSS) and healthy controls (HC), diabetic nephropathy patients (DN) and healthy controls (HC), respectively. The results showed that the annealed AgNPs/PSB composite SERS substrates performed well in diagnosing. Among them, the MCNN model had the best classification effect in the two groups of experiments, with an accuracy rate of 94.7% and 92.0%, respectively. Previous studies have indicated that the AgNPs/PSB composite SERS substrate, combined with machine learning algorithms, has achieved promising classification results in disease diagnosis. This study shows that SERS technology based on annealed AgNPs/PSB composite substrate combined with deep learning algorithm has a greater developmental prospect and research value in the early identification and screening of immune diseases and chronic kidney disease, providing reference ideas for non-invasive and rapid clinical medical diagnosis of patients.

Research on cassava disease classification using the multi-scale fusion model based on EfficientNet and attention mechanism.

Cassava disease is one of the leading causes to the serious decline of cassava yield. Because it is difficult to identify the characteristics of cassava disease, if not professional cassava growers, it will be prone to misjudgment. In order to strengthen the judgment of cassava diseases, the identification characteristics of cassava diseases such as different color of cassava leaf disease spots, abnormal leaf shape and disease spot area were studied. In this paper, deep convolutional neural network was used to classify cassava leaf diseases, and image classification technology was used to recognize and classify cassava leaf diseases. A lightweight module Multi-scale fusion model (MSFM) based on attention mechanism was proposed to extract disease features of cassava leaves to enhance the classification of disease features. The resulting feature map contained key disease identification information. The study used 22,000 cassava disease leaf images as a data set, including four different cassava leaf disease categories and healthy cassava leaves. The experimental results show that the cassava leaf disease classification model based on multi-scale fusion Convolutional Neural Network (CNN) improves EfficientNet compared with the original model, with the average recognition rate increased by nearly 4% and the average recognition rate up to 88.1%. It provides theoretical support and practical tools for the recognition and early diagnosis of plant disease leaves.

Near-infrared spectroscopy combined with pattern recognition algorithms to quickly classify raisins

With the development of commodity economy, the emergence of fake and shoddy raisin has seriously harmed the interests of consumers and enterprises. To deal with this problem, a classification method combining near-infrared spectroscopy and pattern recognition algorithms were proposed for adulterated raisins. In this study, the experiment was performed by three kinds of raisins in Xinjiang (Hongxiangfei, Manaiti, Munage). After collecting and normalizing the spectral data, we compared the spectra of three kinds of raisins. Next the principal component analysis (PCA) was preformed to compress the dimension of the spectral data, and then classification models including support vector machine (SVM), multiscale fusion convolutional neural network (MCNN) and improved AlexNet were established to identify raisins. The accuracy of SVM, MCNN, and improved AlexNet is 100%, 92.83%, and 97.78% respectively. This study proves that near-infrared spectroscopy combined with pattern recognition is feasible for the raisin inspection.

Automated Classification of Retinal OCT Images Using a Deep Multi-Scale Fusion CNN

The automated analysis of optical coherence tomography (OCT) images can play a crucial role in the diagnosis and management of retinal diseases. The wide variations of the retinal disease manifestations in terms of shape, size, texture and spatial location pose a huge challenge in designing reliable and efficient automated methods. Existing methods mostly use single-scale deep frameworks for encoding features from the OCT images to make a diagnosis decision. Such approaches potentially ignore the useful discriminative information in different scales. Therefore, in this paper, we propose a Deep Multi-scale Fusion Convolutional Neural Network (DMF-CNN) that can encode the multi-scaled disease characteristics and effectively combine them for reliable classification. Specifically, multiple CNNs with different receptive fields are used to obtain scale-specific feature representations from the OCT images. These representations are fused to mine the cross-scale powerful discriminative features for classification. A joint multi-loss optimization strategy is designed to collectively learn the scale-specific and cross-scale complementary information during training. The method is evaluated on two publicly available OCT databases (the UCSD and the NEH) and delivers state-of-the-art performance. An impressive overall accuracy of 96.03% and 99.60% is obtained for the UCSD and the NEH datasets, respectively. The outstanding performance and the improved generalization makes the method a reliable diagnostic aid for medical practitioners.

Fast detection of cumin and fennel using NIR spectroscopy combined with deep learning algorithms

Cumin and fennel are two types of food raw materials that have similar appearances but great differences in efficacy. Therefore, the efficient identification of these two types of food is very important to the related food processing industry and the development of food detection technology. In this study, a deep learning algorithm was designed to process Fourier NIR spectral data for the classification of cumin and fennel. The method was based on the difference in the signal intensity in NIR spectra between cumin and fennel, and the identification was performed by combining the improved multiscale fusion convolutional neural network (MCNN) and the bidirectional long-short-term memory network (BILSTM) based on NIR spectra. The classification accuracies of the MCNN and BILSTM models were 100% and 98.57%, respectively. The models have high sensitivity and fast analysis speed, which directly promote the development of real-time identification technology of cumin and fennel and verify the feasibility of deep learning algorithms in the field of food detection.

A Multi-Scale Fusion Convolutional Neural Network Based on Attention Mechanism for the Visualization Analysis of EEG Signals Decoding.

Brain-computer interface (BCI) based on motor imagery (MI) electroencephalogram (EEG) decoding helps motor-disabled patients to communicate with external devices directly, which can achieve the purpose of human-computer interaction and assisted living. MI EEG decoding has a core problem which is extracting as many multiple types of features as possible from the multi-channel time series of EEG to understand brain activity accurately. Recently, deep learning technology has been widely used in EEG decoding. However, the variability of the simple network framework is insufficient to satisfy the complex task of EEG decoding. A multi-scale fusion convolutional neural network based on the attention mechanism (MS-AMF) is proposed in this paper. The network extracts spatio temporal multi-scale features from multi-brain regions representation signals and is supplemented by a dense fusion strategy to retain the maximum information flow. The attention mechanism we added to the network has improved the sensitivity of the network. The experimental results show that the network has a better classification effect compared with the baseline method in the BCI Competition IV-2a dataset. We conducted visualization analysis in multiple parts of the network, and the results show that the attention mechanism is also convenient for analyzing the underlying information flow of EEG decoding, which verifies the effectiveness of the MS-AMF method.

Spatiotemporal Fusion of Land Surface Temperature Based on a Convolutional Neural Network

Due to the tradeoff between spatial and temporal resolutions commonly encountered in remote sensing, no single satellite sensor can provide fine spatial resolution land surface temperature (LST) products with frequent coverage. This situation greatly limits applications that require LST data with fine spatiotemporal resolution. Here, a deep learning-based spatiotemporal temperature fusion network (STTFN) method for the generation of fine spatiotemporal resolution LST products is proposed. In STTFN, a multiscale fusion convolutional neural network is employed to build the complex nonlinear relationship between input and output LSTs. Thus, unlike other LST spatiotemporal fusion approaches, STTFN is able to form the potentially complicated relationships through the use of training data without manually designed mathematical rules making it is more flexible and intelligent than other methods. In addition, two target fine spatial resolution LST images are predicted and then integrated by a spatiotemporal-consistency (STC)-weighting function to take advantage of STC of LST data. A set of analyses using two real LST data sets obtained from Landsat and moderate resolution imaging spectroradiometer (MODIS) were undertaken to evaluate the ability of STTFN to generate fine spatiotemporal resolution LST products. The results show that, compared with three classic fusion methods [the enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM), the spatiotemporal integrated temperature fusion model (STITFM), and the two-stream convolutional neural network for spatiotemporal image fusion (StfNet)], the proposed network produced the most accurate outputs [average root mean square error (RMSE) <; 1.40 °C and average structural similarity (SSIM) > 0.971].

Deep Multi-Scale Fusion Neural Network for Multi-Class Arrhythmia Detection.

Automated electrocardiogram (ECG) analysis for arrhythmia detection plays a critical role in early prevention and diagnosis of cardiovascular diseases. Extracting powerful features from raw ECG signals for fine-grained diseases classification is still a challenging problem today due to variable abnormal rhythms and noise distribution. For ECG analysis, the previous research works depend mostly on heartbeat or single scale signal segments, which ignores underlying complementary information of different scales. In this paper, we formulate a novel end-to-end Deep Multi-Scale Fusion convolutional neural network (DMSFNet) architecture for multi-class arrhythmia detection. Our proposed approach can effectively capture abnormal patterns of diseases and suppress noise interference by multi-scale feature extraction and cross-scale information complementarity of ECG signals. The proposed method implements feature extraction for signal segments with different sizes by integrating multiple convolution kernels with different receptive fields. Meanwhile, joint optimization strategy with multiple losses of different scales is designed, which not only learns scale-specific features, but also realizes cumulatively multi-scale complementary feature learning during the learning process. In our work, we demonstrate our DMSFNet on two open datasets (CPSC_2018 and PhysioNet/CinC_2017) and deliver the state-of-art performance on them. Among them, CPSC_2018 is a 12-lead ECG dataset and CinC_2017 is a single-lead dataset. For these two datasets, we achieve the F1 score [Formula: see text] and [Formula: see text] which are higher than previous state-of-art approaches respectively. The results demonstrate that our end-to-end DMSFNet has outstanding performance for feature extraction from a broad range of distinct arrhythmias and elegant generalization ability for effectively handling ECG signals with different leads.

Identification of hepatitis B using Raman spectroscopy combined with gated recurrent unit and multiscale fusion convolutional neural network

This study presents a novel method for the diagnosis of hepatitis B virus infection using human blood serum Raman spectroscopy combined with a deep learning model. The sera of 499 people infected with the hepatitis B virus and 435 healthy controls were measured in this experiment. The data were subjected to a dimensionality reduction by principal component analysis. Then, the features of multiple scales were preserved and fused by a multiscale fusion convolution operation. The gated recurrent unit network was added to extract time series features and finally output the result of the classification through a softmax layer. A diagnostic model based on a gated recurrent unit and multiscale fusion convolutional neural network was constructed and evaluated by a 10-fold cross-validation method. Compared to existing analysis methods for serum Raman spectroscopy, the proposed model achieved the best performance. The combination of Raman spectroscopy and deep learning models is expected to be applied well in the early screening of hepatitis B and is a promising screening method.

Contourlet Transform based Multi-scale Fusion Network for Face Hallucination

Recently, Convolutional neural network (CNN) has achieved great success in the field of face hallucination. However, such approaches usually generate blurry and over-smoothed Super Resolution (SR) results, and the performance suffers from degradation when super-resolve a very Low Resolution (LR) face image. To solve these problems, this paper proposes an contourlet transform based accurate CNN architecture, namely Multi-scale Fusion CNN (MSFC), which are able to reconstruct a High Resolution (HR) face image from a very low resolution input. First of all, we present multi-scale fusion CNN (MSFC) to fully detect and exploit features from LR inputs. And then, we formulate the SR problem as the prediction of contourlet transform coefficients, which is able to make MSFC further capture the texture details for super-resolve face images. Extensive qualitative and quantitative experiments show that the proposed method is capable of preserving details and achieves superior SR performance compared to the state-of-the-art methods.

A Multiscale Fusion Convolutional Neural Network for Plant Leaf Recognition

Plant leaf recognition is a computer vision task used to automatically recognize plant species. It is very challenging since rich plant leaf morphological variations, such as sizes, textures, shapes, venation, and so on. Most existing plant leaf methods typically normalize all plant leaf images to the same size and recognize them at one scale, resulting in unsatisfactory performances. In this letter, a multiscale fusion convolutional neural network (MSF-CNN) is proposed for plant leaf recognition at multiple scales. First, an input image is down-sampled into multiples low resolution images with a list of bilinear interpolation operations. Then, these input images with different scales are step-by-step fed into the MSF-CNN architecture to learn discriminative features at different depths. At this stage, the feature fusion between two different scales is realized by a concatenation operation, which concatenates feature maps learned on different scale images from a channel view. Along with the depth of the MSF-CNN, multiscale images are progressively handled and the corresponding features are fused. Third, the last layer of the MSF-CNN aggregates all discriminative information to obtain the final feature for predicting the plant species of the input image. Experiments show the proposed MSF-CNN method is superior to multiple state-of-the art plant leaf recognition methods on the MalayaKew Leaf dataset and the LeafSnap Plant Leaf dataset.