Multilayer Convolutional Neural Network Research Articles

An enhanced CNN-LSTM remaining useful life prediction model for aircraft engine with attention mechanism

Remaining useful life (RUL) prediction is one of the key technologies of aircraft prognosis and health management (PHM) which could provide better maintenance decisions. In order to improve the accuracy of aircraft engine RUL prediction under real flight conditions and better meet the needs of PHM system, we put forward an improved CNN-LSTM model based on the convolutional block attention module (CBAM). First, the features of aircraft engine operation data are extracted by multi-layer CNN network, and then the attention mechanism is processed by CBAM in channel and spatial dimensions to find key variables related to RUL. Finally, the hidden relationship between features and service time is learned by LSTM and the predicted RUL is output. Experiments were conducted using C-MPASS dataset. Experimental results indicate that our prediction model has feasibility. Compared with other state-of-the-art methods, the RMSE of our method decreased by 17.4%, and the score of the prediction model was improved by 25.9%.

Hand Activity Recognition From Automatic Estimated Egocentric Skeletons Combining Slow Fast and Graphical Neural Networks

In this paper, we present an unified framework for understanding hand action from the first-person video. The proposed framework composes two main components. The first component estimates three-dimensional (3D) hand joints from RGB images. Two network structures derived from the baseline HopeNet network are proposed: convolutional neural networks (CNNs) which are traditional multi-layer CNN and CNN combining with GraphCNN to perform 3D hand pose estimation, without the use of GraphUNet as in baseline HopeNet method. The second component of the framework recognizes hand action from skeleton stream. We first deploy two recent advanced neuronal networks that are PA-ResGCN and Double-feature Double-motion (DDNet). To focus more on the hand pose changes, we improve DDNet with two normalization strategies of the hand joints. Finally, we fuse PA-ResGCN with our improved DDNet to still boost the recognition performance. We evaluate our proposed methods on First-Person Hand Action Benchmark dataset. Experiments show that our model for 3D hand joints estimation achieves the best precision (36.6 mm). Our hand joint normalization strategies improve the original DDNet from 0.71% to 4.05% of accuracy with the ground-truth hand pose while the improvement is significantly larger (from 2.96% to 10.98%) with the estimated hand pose. The late fusion schemes outperform different state-of-the-art methods for the hand action recognition with the highest accuracy of 86.67%. These experimental results show potential and extendable possibilities for developing practical first-person vision applications.

Automatic BASED scoring on scalp EEG in children with infantile spasms using convolutional neural network.

In recent years, the Burden of Amplitudes and Epileptiform Discharges (BASED) score has been used as a reliable, accurate, and feasible electroencephalogram (EEG) grading scale for infantile spasms. However, manual EEG annotation is, in general, very time-consuming, and BASED scoring is no exception. Convolutional neural networks (CNNs) have proven their great potential in many EEG classification problems. However, very few research studies have focused on the use of CNNs for BASED scoring, a challenging but vital task in the diagnosis and treatment of infantile spasms. This study proposes an automatic BASED scoring framework using EEG and a deep CNN. The feasibility of using CNN for automatic BASED scoring was investigated in 36 patients with infantile spasms by annotating their long-term EEG data with four levels of the BASED score (scores 5, 4, 3, and ≤2). In the validation set, the accuracy was 96.9% by applying a multi-layer CNN to classify the EEG data as a 4-label problem. The extensive experiments have demonstrated that our proposed approach offers high accuracy and, hence, is an important step toward an automatic BASED scoring algorithm. To the best of our knowledge, this is the first attempt to use a CNN to construct a BASED-based scoring model.

DCSE:Double-Channel-Siamese-Ensemble model for protein protein interaction prediction

BackgroundProtein-protein interaction (PPI) is very important for many biochemical processes. Therefore, accurate prediction of PPI can help us better understand the role of proteins in biochemical processes. Although there are many methods to predict PPI in biology, they are time-consuming and lack accuracy, so it is necessary to build an efficiently and accurately computational model in the field of PPI prediction.ResultsWe present a novel sequence-based computational approach called DCSE (Double-Channel-Siamese-Ensemble) to predict potential PPI. In the encoding layer, we treat each amino acid as a word, and map it into an N-dimensional vector. In the feature extraction layer, we extract features from local and global perspectives by Multilayer Convolutional Neural Network (MCN) and Multilayer Bidirectional Gated Recurrent Unit with Convolutional Neural Networks (MBC). Finally, the output of the feature extraction layer is then fed into the prediction layer to output whether the input protein pair will interact each other. The MCN and MBC are siamese and ensemble based network, which can effectively improve the performance of the model. In order to demonstrate our model’s performance, we compare it with four machine learning based and three deep learning based models. The results show that our method outperforms other models in all evaluation criteria. The Accuracy, Precision, F_{1}, Recall and MCC of our model are 0.9303, 0.9091, 0.9268, 0.9452, 0.8609. For the other seven models, the highest Accuracy, Precision, F_{1}, Recall and MCC are 0.9288, 0.9243, 0.9246, 0.9250, 0.8572. We also test our model in the imbalanced dataset and transfer our model to another species. The results show our model is excellent.ConclusionOur model achieves the best performance by comparing it with seven other models. NLP-based coding method has a good effect on PPI prediction task. MCN and MBC extract protein sequence features from local and global perspectives and these two feature extraction layers are based on siamese and ensemble network structures. Siamese-based network structure can keep the features consistent and ensemble based network structure can effectively improve the accuracy of the model.

Measuring Waste Recyclability Level Using Convolutional Neural Network and Fuzzy Inference System

This paper presents a hybrid model that is used to measure the waste recyclability level using a convolutional neural network (CNN) and fuzzy inference system (FIS; WRL-CNNFIS). The proposed system uses waste images to train a multilayer convolutional neural network to extract the most relevant features that were used in a rule-based fuzzy system to give an accurate percentage of the recyclability level of these images. The proposed model did overcome many challenges in transfer learning models alone, like overfitting and low accuracy. The use of fuzzy rules improved the performance even with a small data set. Results have shown the effectiveness of the proposed model in terms of all four metrics: accuracy, precision, recall, and F1 score. The performance was measured under two testing scenarios. For all evaluation measurements in all experiments, the validation was conducted using the cross validation in the last step. The proposed approach is a robust and consistent approach for classifying organic and recyclable waste types. WRL-CNNFIS has achieved an accuracy rate of more than 98%.

High-Speed Multi-Layer Convolutional Neural Network Based on Free-Space Optics

Convolutional neural networks (CNNs) are at the heart of several machine learning applications, while they suffer from computational complexity due to their large number of parameters and operations. Recently, all-optical implementation of the CNNs has achieved many attentions, however, the recently proposed optical architectures for CNNs cannot fully utilize the tremendous capabilities of optical processing, due to the required electro-optical conversions in-between successive layers. To implement an all-optical multi-layer CNN, it is essential to optically implement all required operations, namely convolution, summation of channels’ output for each convolutional kernel feeding the nonlinear unit, nonlinear activation function, and finally, pooling operations. Considering the lack of multi-layer photonic CNN implementation, in this paper, we explore a fully-optical design for implementing successive convolutional layers in an optical CNN. As a proof of concept, and without loss of generality, we considered two successive optical layers in the proposed network, named as 2L-OPCNN, for comparative studies against electrical counterpart and single optical layer CNN. Our simulation results confirm nearly the same accuracies for classifying images of Kaggle Cats and Dogs challenge, CIFAR-10, and MNIST datasets, compared to the electrical counterpart, as well as improved accuracies compared to single optical layer CNN.

Image Captioning - A Deep Learning Approach

Abstract: Image captioning is a brand-new study area in the science of computer vision. The primary goal of picture captioning is to create a natural language description for the input image. In recent years, research on natural language processing and computer vision has become increasingly interested in the problem of automatically synthesising descriptive phrases for photos. Image captioning is a crucial task that demands both the ability to create precise and accurate description phrases as well as a semantic understanding of the images. Long Short Term Memory (LSTM) is used to precisely organise data using the available keywords to form meaningful sentences. The authors of this research propose a hybrid system based on multilayer Convolutional Neural Networks to create a lexicon for characterising the visuals. The convolutional neural network employs trained captions to deliver an accurate description after comparing the target image to a sizable dataset of training images. We demonstrate the effectiveness of our suggested methodology using the Flickr 8K datasets.

Towards a Low-Cost Solution for Gait Analysis Using Millimeter Wave Sensor and Machine Learning.

Human Activity Recognition (HAR) that includes gait analysis may be useful for various rehabilitation and telemonitoring applications. Current gait analysis methods, such as wearables or cameras, have privacy and operational constraints, especially when used with older adults. Millimeter-Wave (MMW) radar is a promising solution for gait applications because of its low-cost, better privacy, and resilience to ambient light and climate conditions. This paper presents a novel human gait analysis method that combines the micro-Doppler spectrogram and skeletal pose estimation using MMW radar for HAR. In our approach, we used the Texas Instruments IWR6843ISK-ODS MMW radar to obtain the micro-Doppler spectrogram and point clouds for 19 human joints. We developed a multilayer Convolutional Neural Network (CNN) to recognize and classify five different gait patterns with an accuracy of 95.7 to 98.8% using MMW radar data. During training of the CNN algorithm, we used the extracted 3D coordinates of 25 joints using the Kinect V2 sensor and compared them with the point clouds data to improve the estimation. Finally, we performed a real-time simulation to observe the point cloud behavior for different activities and validated our system against the ground truth values. The proposed method demonstrates the ability to distinguish between different human activities to obtain clinically relevant gait information.

Classification and Reconstruction of Biomedical Signals Based on Convolutional Neural Network.

The efficient biological signal processing method can effectively improve the efficiency of researchers to explore the work of life mechanism, so as to better reveal the relationship between physiological structure and function, thus promoting the generation of major biological discoveries; high-precision medical signal analysis strategy can, to a certain extent, share the pressure of doctors' clinical diagnosis and assist them to formulate more favorable plans for disease prevention and treatment, so as to alleviate patients' physical and mental pain and improve the overall health level of the society. This article in biomedical signal is very representative of the two types of signals: mammary gland molybdenum target X-ray image (mammography) and the EEG signal as the research object, combined with the deep learning field of CNN; the most representative model is two kinds of biomedical signal classification, and reconstruction methods conducted a series of research: (1) a new classification method of breast masses based on multi-layer CNN is proposed. The method includes a CNN feature representation network for breast masses and a feature decision mechanism that simulates the physician's diagnosis process. By comparing with the objective classification accuracy of other methods for the identification of benign and malignant breast masses, the method achieved the highest classification accuracy of 97.0% under different values of c and gamma, which further verified the effectiveness of the proposed method in the identification of breast masses based on molybdenum target X-ray images. (2) An EEG signal classification method based on spatiotemporal fusion CNN is proposed. This method includes a multi-channel input classification network focusing on spatial information of EEG signals, a single-channel input classification network focusing on temporal information of EEG signals, and a spatial-temporal fusion strategy. Through comparative experiments on EEG signal classification tasks, the effectiveness of the proposed method was verified from the aspects of objective classification accuracy, number of model parameters, and subjective evaluation of CNN feature representation validity. It can be seen that the method proposed in this paper not only has high accuracy, but also can be well applied to the classification and reconstruction of biomedical signals.

Identifying the Fingerprint of a Volcano in the Background Seismic Noise from Machine Learning-Based Approach

This work is devoted to the analysis of the background seismic noise acquired at the volcanoes (Campi Flegrei caldera, Ischia island, and Vesuvius) belonging to the Neapolitan volcanic district (Italy), and at the Colima volcano (Mexico). Continuous seismic acquisition is a complex mixture of volcanic transients and persistent volcanic and/or hydrothermal tremor, anthropogenic/ambient noise, oceanic loading, and meteo-marine contributions. The analysis of the background noise in a stationary volcanic phase could facilitate the identification of relevant waveforms often masked by microseisms and ambient noise. To address this issue, our approach proposes a machine learning (ML) modeling to recognize the “fingerprint” of a specific volcano by analyzing the background seismic noise from the continuous seismic acquisition. Specifically, two ML models, namely multi-layer perceptrons and convolutional neural network were trained to recognize one volcano from another based on the acquisition noise. Experimental results demonstrate the effectiveness of the two models in recognizing the noisy background signal, with promising performance in terms of accuracy, precision, recall, and F1 score. These results suggest that persistent volcanic signals share the same source information, as well as transient events, revealing a common generation mechanism but in different regimes. Moreover, assessing the dynamic state of a volcano through its background noise and promptly identifying any anomalies, which may indicate a change in its dynamics, can be a practical tool for real-time monitoring.

A Network Intrusion Detection System Using Hybrid Multilayer Deep Learning Model.

An intrusion detection system (IDS) is designed to detect and analyze network traffic for suspicious activity. Several methods have been introduced in the literature for IDSs; however, due to a large amount of data, these models have failed to achieve high accuracy. A statistical approach is proposed in this research due to the unsatisfactory results of traditional intrusion detection methods. The features are extracted and selected using a multilayer convolutional neural network, and a softmax classifier is employed to classify the network intrusions. To perform further analysis, a multilayer deep neural network is also applied to classify network intrusions. Furthermore, the experiments are performed using two commonly used benchmark intrusion detection datasets: NSL-KDD and KDDCUP'99. The performance of the proposed model is evaluated using four performance metrics: accuracy, recall, F1-score, and precision. The experimental results show that the proposed approach achieved better accuracy (99%) compared with other IDSs.

Noise reduction technique using deep learning for ultrasound imaging during high-intensity focused ultrasound treatment

One of the problems with ultrasound imaging during high-intensity focused ultrasound (HIFU) treatment is that the therapeutic ultrasound components interfere with the diagnostic ultrasound components, making it impossible to monitor the tissue changes during HIFU exposure. In this study, a convolutional neural network (CNN) framework was applied to the reconstructed ultrasound images with HIFU noise to remove the therapeutic ultrasound components while the diagnostic ultrasound components remain intact. In the experiments, the chicken breast was used as a tissue sample and exposed to HIFU in the water tank. The ultrasound images with and without noise were acquired during an intermission period of HIFU exposure and the noise-reduced images was predicted using the proposed multi-layer regression CNN model through the training process. As a result, ultrasound images with sufficient spatial resolution to detect the thermal lesion were acquired.

Dynamic Gesture Recognition Model Based on Millimeter-Wave Radar With ResNet-18 and LSTM.

In this article, a multi-layer convolutional neural network (ResNet-18) and Long Short-Term Memory Networks (LSTM) model is proposed for dynamic gesture recognition. The Soli dataset is based on the dynamic gesture signals collected by millimeter-wave radar. As a gesture sensor radar, Soli radar has high positional accuracy and can recognize small movements, to achieve the ultimate goal of Human-Computer Interaction (HCI). A set of velocity-range Doppler images transformed from the original signal is used as the input of the model. Especially, ResNet-18 is used to extract deeper spatial features and solve the problem of gradient extinction or gradient explosion. LSTM is used to extract temporal features and solve the problem of long-time dependence. The model was implemented on the Soli dataset for the dynamic gesture recognition experiment, where the accuracy of gesture recognition obtained 92.55%. Finally, compare the model with the traditional methods. The result shows that the model proposed in this paper achieves higher accuracy in dynamic gesture recognition. The validity of the model is verified by experiments.

Apple Leaf Disease Classification Using Image Dataset: a Multilayer Convolutional Neural Network Approach

Agriculture is one of the prime sources of economic growth in Russia; the global apple production in 2019 was 87 million tons. Apple leaf diseases are the main reason for annual decreases in apple production, which creates huge economic losses. Automated methods for detecting apple leaf diseases are beneficial in reducing the laborious work of monitoring apple gardens and early detection of disease symptoms. This article proposes a multilayer convolutional neural network (MCNN), which is able to classify apple leaves into one of the following categories: apple scab, black rot, and apple cedar rust diseases using a newly created dataset. In this method, we used affine transformation and perspective transformation techniques to increase the size of the dataset. After that, OpenCV crop and histogram equalization method-based preprocessing operations were used to improve the proposed image dataset. The experimental results show that the system achieves 98.40% training accuracy and 98.47% validation accuracy on the proposed image dataset with a smaller number of training parameters. The results envisage a higher classification accuracy of the proposed MCNN model when compared with the other well-known state-of-the-art approaches. This proposed model can be used to detect and classify other types of apple diseases from different image datasets.

Graph Triple-Attention Network for Disease-Related LncRNA Prediction.

Abnormal expressions of long non-coding RNAs (lncRNAs) are associated with various human diseases. Identifying disease-related lncRNAs can help clarify complex disease pathogeneses. The latest methods for lncRNA-disease association prediction rely on diverse data about lncRNAs and diseases. These methods, however, cannot adequately integrate the neighbour topological information of lncRNA and disease nodes. Moreover, more intrinsic features of lncRNA-disease node pairs can be explored to better predict their latent associations. We developed a novel method, named GTAN, to predict the association propensities between lncRNAs and diseases. GTAN integrates various information about lncRNAs and diseases, and exploits neighbour topology and attribute representations of a pair of lncRNA-disease nodes. We adopted in GTAN a graph neural network architecture with three attention mechanisms and multi-layer convolutional neural networks. First, a neighbour-level self-attention mechanism is constructed to learn the importance of each neighbour for an interested lncRNA or disease node. Second, topology-level attention is proposed to enhance contextual dependencies among multiple local topology representations. An attention-enhanced graph neural network framework is then established to learn a topology representation of top-ranked neighbours. GTAN also has attribute-level attention to distinguish various contributions of attributes of the lncRNA-disease pair. Finally, attribute representation is learned by multi-layer CNN to integrate detailed features and representative features of the pair. Extensive experimental results demonstrated that GTAN outperformed state-of-the-art methods. The ablation studies confirmed the important contributions of three attention mechanisms. Case studies on three cancers further showed GTAN's ability in discovering potential lncRNA candidates related to diseases.

IoT-Inspired Monitoring Framework for Real-Time Stereotypic Movement Analysis

Physical activity recognition has become one of the prominent domain of research in the field of smart healthcare. The identification of physical activities through wearable sensors provide significant information about the functional ability and lifestyle of an individual. In this article, a smart physical activity-assisted behavior prediction framework is presented to recognize different stereotypical motor movements of the individual suffering from autism spectrum disorder in real time. This article employs the data acquisition efficiency of Internet of Things (IoT) assisted smart wearable sensors to capture the physical activities of an individual. To predict the physical activities, a deep learning-inspired multilayered convolutional neural network (CNN) + long short-term memory network is proposed. Furthermore, the utility of the framework is increased by presenting an effective fog-assisted alert-based decision-making module. The decision-making module notifies the concerned caretaker or medical specialist about the performed irregular physical activity in real time. Moreover, the process of cloud-based record generation also enhances the significance of the proposed solution. The proposed approach is validated by comparing the prediction performance with traditional handcraft-based and modern deep learning-based state-of-the-art approaches such as <formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex>$k$</tex></formula> -nearest neighbors, hidden Markov model, and CNN, where it outperformed by achieving the overall accuracy of 91.88%.

Nanoscale Memristor‐Based Spike Timing‐Dependent Plasticity Learning in a Radix‐X Quantized Retinal Neural Network

The human retina sends visual signals to the brain's visual cortex from photoreceptors (rod and cone cells) through various synaptic pathways and performs crucial early vision processing before signals are passed to higher brain regions. Herein, an artificial retina system implemented based on the leaky integrate‐and‐fire spiking neuron model is presented. The architecture of the proposed retina system consists of a multilayer convolutional neural network (CNN), and the system uses spike timing‐dependent plasticity (STDP) as a feedforward learning rule. In addition, the system integrates a feedback plasticity learning rule to expedite learning convergence. The system weights are implemented using nanoscale memristor arrays, taking on a constrained (radix‐X) range of conductance states. The proposed system produces an output image of 25 × 25 pixels, corresponding to the output retina ganglion cells that act as the interface between the retina and the visual cortex, using an input image of 100 × 100 pixels.

Question and Answer Techniques for Financial Audits in Universities Based on Deep Learning

Financial auditing in universities is highly specialized, with a huge knowledge system and rapid updates. Auditors will encounter various problems and situations in their work and need to acquire domain knowledge efficiently and accurately to solve the difficulties they encounter. The existing audit information software, however, is mostly aimed at the management of audit affairs and lacks the relevant functions to acquire and retrieve knowledge of specific audit domains. In this study, we use deep learning theory as support to conduct an in-depth study on the key technologies of question and answer systems in the field of financial auditing in universities. In the question-answer retrieval stage, the local information and the global information of the sentence are first modelled using a two-way coding model based on the attentional mechanism, and then, an interactive text matching model is used to interact directly at the input layer, and a multilayer convolutional neural network model cable news network (CNN) is used to extract the fine-grained matching features from the interaction matrix; this study adopts two matching methods. We have conducted comparative experiments to verify the effectiveness and application value of the entity recognition algorithm based on this study’s algorithm and the question-answer retrieval model based on multi-granularity text matching in the university financial audit domain.

A Deep Learning Method Approach for Sleep Stage Classification with EEG Spectrogram.

The classification of sleep stages is an important process. However, this process is time-consuming, subjective, and error-prone. Many automated classification methods use electroencephalogram (EEG) signals for classification. These methods do not classify well enough and perform poorly in the N1 due to unbalanced data. In this paper, we propose a sleep stage classification method using EEG spectrogram. We have designed a deep learning model called EEGSNet based on multi-layer convolutional neural networks (CNNs) to extract time and frequency features from the EEG spectrogram, and two-layer bi-directional long short-term memory networks (Bi-LSTMs) to learn the transition rules between features from adjacent epochs and to perform the classification of sleep stages. In addition, to improve the generalization ability of the model, we have used Gaussian error linear units (GELUs) as the activation function of CNN. The proposed method was evaluated by four public databases, the Sleep-EDFX-8, Sleep-EDFX-20, Sleep-EDFX-78, and SHHS. The accuracy of the method is 94.17%, 86.82%, 83.02% and 85.12%, respectively, for the four datasets, the MF1 is 87.78%, 81.57%, 77.26% and 78.54%, respectively, and the Kappa is 0.91, 0.82, 0.77 and 0.79, respectively. In addition, our proposed method achieved better classification results on N1, with an F1-score of 70.16%, 52.41%, 50.03% and 47.26% for the four datasets.

A DDoS-Attack Detection Method Oriented to the Blockchain Network Layer

By nature, a traditional attack method, denial-of-service (DDoS) attack poses a considerable threat to the security of the blockchain network layer. This paper proposes a distributed DDoS-attack traffic detection method based on a cross multilayer convolutional neural network model in the blockchain network layer. The method resolves the low generalisation, high misreporting rate, and low detection efficiency problems of the existing detection methods, which are caused by nondistinctive core features and the high complexity of robust features when detecting DDoS attacks transmitted by mixed protocols on a blockchain network layer. First, the model performs a convolution operation on preprocessed traffic on the blockchain network layer using a cross-layer method based on L2 regularisation. After this operation, the model can perceive the detailed features of attack traffic from multiple levels while enhancing the representational performance of key features; specifically, the parameters with high-variance terms are penalised to limit changes in the model’s weight parameters. The highly robust abstract features of attack traffic are extracted, thereby increasing the generalisation ability and reducing the misreporting rate of the model. Second, parametric encoding of the abstract features is performed by a stacked sparse autoencoder based on Kullback–Leibler divergence, and the sparsity of the model is adjusted to reduce the redundant data and the coupling between abstract features. The outputs of the encoded features are then effectively categorised. Finally, the global optimisation of parameters is performed by an improved random gradient-descent algorithm, which prevents oscillation of the training parameters and accelerates the model convergence. In an experimental evaluation, the proposed method achieved satisfactory binary- and multiclass detection of DDoS-attack traffic on both CSE-CIC-IDS 2018 on the AWS dataset and on the real mixed data of a blockchain network layer.