Parallel Deep Convolutional Neural Network Research Articles

A Novel Method for Bearing Fault Diagnosis Based on a Parallel Deep Convolutional Neural Network

The symmetry of vibration signals collected from healthy machinery, which gradually degenerates with the development of faults, must be detected for timely diagnosis and prognosis. However, conventional methods may miss spatiotemporal relationships, struggle with varying sampling rates, and lack adaptability to changing loads and conditions, affecting diagnostic accuracy. A novel bearing fault diagnosis approach is proposed to address these issues, which integrates the Gramian angular field (GAF) transformation with a parallel deep convolutional neural network (DCNN). The crux of this method lies in the preprocessing of input signals, where sampling rate normalization is employed to minimize the effects of varying sampling rates on diagnostic outcomes. Subsequently, the processed signals undergo GAF transformation, converting them into an image format that effectively represents their spatiotemporal relationships in a two-dimensional space. These images serve as inputs to the parallel DCNN, facilitating feature extraction and fault classification through deep learning techniques and leading to improved generalization capabilities on test data. The proposed method achieves an overall accuracy of 96.96%, even in the absence of training data within the test set. Discussions are also conducted to quantify the effects of sampling rate normalization and model structures on diagnostic accuracy.

Telugu letters dataset and parallel deep convolutional neural network with a SGD optimizer model for TCR

<p>Because of the rapid growth in technology breakthroughs, including multimedia and cell phones, Telugu character recognition (TCR) has recently become a popular study area. It is still necessary to construct automated and intelligent online TCR models, even if many studies have focused on offline TCR models. The Telugu character dataset construction and validation using an Inception and ResNet-based model are presented. The collection of 645 letters in the dataset includes 18 Achus, 38 Hallus, 35 Othulu, 34×16 Guninthamulu, and 10 Ankelu. The proposed technique aims to efficiently recognize and identify distinctive Telugu characters online. This model's main pre-processing steps to achieve its goals include normalization, smoothing, and interpolation. Improved recognition performance can be attained by using stochastic gradient descent (SGD) to optimize the model's hyperparameters.</p>

One level deep convolutional neural network for facial key points detection

Facial landmark detection has a lot of applications in face recognition, face alignment, facial expression recognition, video surveillance and security systems. In the existing literature, there are multiple methods utilizing convolutional neural networks (CNNs) that address this problem in various ways. In many cases, the models use a tree-like structure of CNNs to achieve better results. This paper proposes a combination of three parallel deep convolutional neural networks (DCNNs) to estimate the accurate localization of each keypoint. The first one focuses on the whole face to outperform five points, including the eyes, nose, and mouth corners. The second one focuses on the eyes-nose parts to outperform three points, specifically the eyes and nose. The last one focuses on the nose-mouth parts to outperform three points, namely the nose and mouth corners. Further, we combine all outputs of the three DCNNs and take the average value of each detected key point as the final output. In the first step, we improvthe the parameter efficiency and accuracy of each DCNNs through a set of experiments using the labeled face parts in-the-wild database (LFPW) and the helen facial feature dataset (Helen). Then, we demonstrate that our approach yields more accurate estimations of facial key points than two state-of-the-art methods and commercial software in terms of accuracy.

Arrhythmia classification based on multi-feature multi-path parallel deep convolutional neural networks and improved focal loss

<abstract> <p>Early diagnosis of abnormal electrocardiogram (ECG) signals can provide useful information for the prevention and detection of arrhythmia diseases. Due to the similarities in Normal beat (<italic>N</italic>) and Supraventricular Premature Beat (<italic>S</italic>) categories and imbalance of ECG categories, arrhythmia classification cannot achieve satisfactory classification results under the inter-patient assessment paradigm. In this paper, a multi-path parallel deep convolutional neural network was proposed for arrhythmia classification. Furthermore, a global average RR interval was introduced to address the issue of similarities between <italic>N</italic> vs. <italic>S</italic> categories, and a weighted loss function was developed to solve the imbalance problem using the dynamically adjusted weights based on the proportion of each class in the input batch. The MIT-BIH arrhythmia dataset was used to validate the classification performances of the proposed method. Experimental results under the intra-patient evaluation paradigm and inter-patient evaluation paradigm showed that the proposed method could achieve better classification results than other methods. Among them, the accuracy, average sensitivity, average precision, and average specificity under the intra-patient paradigm were 98.73%, 94.89%, 89.38%, and 98.24%, respectively. The accuracy, average sensitivity, average precision, and average specificity under the inter-patient paradigm were 91.22%, 89.91%, 68.23%, and 95.23%, respectively.</p> </abstract>

Automatic detection of artifacts and improved classification models for emotional activity detection from multimodal physiological data

This manuscript proposes an automated artifacts detection and multimodal classification system for human emotion analysis from human physiological signals. First, multimodal physiological data, including the Electrodermal Activity (EDA), electrocardiogram (ECG), Blood Volume Pulse (BVP) and respiration rate signals are collected. Second, a Modified Compressed Sensing-based Decomposition (MCSD) is used to extract the informative Skin Conductance Response (SCR) events of the EDA signal. Third, raw features (edge and sharp variations), statistical and wavelet coefficient features of EDA, ECG, BVP, respiration and SCR signals are obtained. Fourth, the extracted raw features, statistical and wavelet coefficient features from all physiological signals are fed into the parallel Deep Convolutional Neural Network (DCNN) to reduce the dimensionality of feature space by removing artifacts. Fifth, the fused artifact-free feature vector is obtained for neutral, stress and pleasure emotion classes. Sixth, an artifact-free feature vector is used to train the Random Forest Deep Neural Network (RFDNN) classifier. Then, a trained RFDNN classifier is applied to classify the test signals into different emotion classes. Thus, leveraging the strengths of both RF and DNN algorithms, more comprehensive feature learning using multimodal psychological data is achieved, resulting in robust and accurate classification of human emotional activities. Finally, an extensive experiment using the Wearable Stress and Affect Detection (WESAD) dataset shows that the proposed system outperforms other existing human emotion classification systems using physiological data.

P‐DNN: Parallel DNN based IDS framework for the detection of IoT vulnerabilities

SummaryThe rapid growth of the Internet of Things (IoT) in our daily life has recently received attention from hackers in releasing novel attacks. This is because the existing traditional Intrusion Detection System (IDS) uses an alert‐based approach that cannot detect new emerging attacks, making it unfeasible for devices with limited resources. The study presents the deployment of a Parallel Deep Neural Network (P‐DNN) IDS framework to improve the detection rate and resource constraint issues. To validate the framework for detecting the latest IoT vulnerabilities, we generated a proprietary dataset created under the IoT environment. The experimental results reveal that the P‐DNN IDS framework proposed in this study outperformed the K‐nearest Neighbour (KNN), Support Vector Machine (SVM), and Naive Bayes algorithms with respect to performance metrics. In addition, a comparative analysis of the P‐DNN framework with Snort and Suricata IDS results demonstrates a significant reduction in CPU consumption, memory utilization, and processing time. The results of this investigation indicate that the deployment of the P‐DNN framework has the capability to detect vulnerabilities in resource‐constraint IoT environments effectively.

Detection of leaf disease in tomato plants using a lightweight parallel deep convolutional neural network

Plant diseases and poisonous insects are major threats to agriculture. As a result, detecting and diagnosing these illnesses as soon as feasible is critical. The ongoing development of major deep learning techniques has substantially aided in the diagnosis of plant leaf diseases, providing a potent instrument with incredibly exact results. Deep learning algorithms, on the other hand, are dependent on the quality and quantity of labelled data used for training. The lightweight parallel deep convolutional neural network is described in this study for detecting plant leaf disease. In addition, the Generative Adversarial Neural Network is introduced for creating synthetic data in order to overcome the data scarcity problem caused by uneven dataset size. The experimental results for two-class, six-class and ten-class disease identification of tomato plant samples from the Plant Village dataset are provided. The effectiveness of the proposed model is assessed using numerous performance measures, including accuracy, recall, precision and F1-score, and compared to known state-of-the-art approaches for tomato plant leaf disease detection. The proposed system provides better accuracy (99.14%, 99.05%, 98.11% accuracy for the 2-class, 6-class and 10-class) for tomato leaf disease detection compared with traditional existing approaches.

MRI brain tumor detection and classification using parallel deep convolutional neural networks

Convolutional neural network (CNN) is widely used to classify brain tumors with high accuracy. Since CNN collects features randomly without knowing the local and global features and causes overfitting problems, this research proposes a novel parallel deep convolutional neural network (PDCNN) topology to extract both global and local features from the two parallel stages and deal with the over-fitting problem by utilizing dropout regularizer alongside batch normalization. To begin, input images are resized and grayscale transformation is conducted, which helps to reduce complexity. After that, data augmentation has been used to maximize the number of datasets. The benefits of parallel pathways are provided by combining two simultaneous deep convolutional neural networks with two different window sizes, allowing this model to learn local and global information. Three forms of MRI datasets are used to determine the effectiveness of the proposed method. The binary tumor identification dataset-I, Figshare dataset-II, and Multiclass Kaggle dataset-III provide accuracy of 97.33%, 97.60%, and 98.12%, respectively. The proposed structure is not only accurate but also efficient, as the proposed method extracts both low-level and high-level features, improving results compared to state-of-the-art techniques.

Detection of Plant Leaf Disease Using a Lightweight Parallel Deep Convolutional Neural Network

Detection of Plant Leaf Disease Using a Lightweight Parallel Deep Convolutional Neural Network

Anterior Cruciate Ligament Tear Detection Based on Deep Convolutional Neural Network.

Anterior cruciate ligament (ACL) tear is very common in football players, volleyball players, sprinters, runners, etc. It occurs frequently due to extra stretching and sudden movement and causes extreme pain to the patient. Various computer vision-based techniques have been employed for ACL tear detection, but the performance of most of these systems is challenging because of the complex structure of knee ligaments. This paper presents a three-layered compact parallel deep convolutional neural network (CPDCNN) to enhance the feature distinctiveness of the knee MRI images for anterior cruciate ligament (ACL) tear detection in knee MRI images. The performance of the proposed approach is evaluated for the MRNet knee images dataset using accuracy, recall, precision, and the F1 score. The proposed CPDCNN offers an overall accuracy of 96.60%, a recall rate of 0.9668, a precision of 0.9654, and an F1 score of 0.9582, which shows superiority over the existing state-of-the-art methods for knee tear detection.

Retraction Note to: Parallel deep convolutional neural network for content based medical image retrieval

Retraction Note to: Parallel deep convolutional neural network for content based medical image retrieval

VPipe: A Virtualized Acceleration System for Achieving Efficient and Scalable Pipeline Parallel DNN Training

The increasing computational complexity of DNNs achieved unprecedented successes in various areas such as machine vision and natural language processing (NLP), e.g., the recent advanced Transformer has billions of parameters. However, as large-scale DNNs significantly exceed GPU's physical memory limit, they cannot be trained by conventional methods such as data parallelism. Pipeline parallelism that partitions a large DNN into small subnets and trains them on different GPUs is a plausible solution. Unfortunately, the layer partitioning and memory management in existing pipeline parallel systems are fixed during training, making them easily impeded by out-of-memory errors and the GPU under-utilization. These drawbacks amplify when performing neural architecture search (NAS) such as the evolved Transformer, where different network architectures of Transformer needed to be trained repeatedly. vPipe is the first system that transparently provides dynamic layer partitioning and memory management for pipeline parallelism. vPipe has two unique contributions, including (1) an online algorithm for searching a near-optimal layer partitioning and memory management plan, and (2) a live layer migration protocol for re-balancing the layer distribution across a training pipeline. vPipe improved the training throughput of two notable baselines (Pipedream and GPipe) by 61.4-463.4 percent and 24.8-291.3 percent on various large DNNs and training settings.

Multi-Year ENSO Forecasts Using Parallel Convolutional Neural Networks With Heterogeneous Architecture

The El Niño-Southern Oscillation (ENSO) is one of the main drivers of the interannual climate variability of Earth and can cause a wide range of climate anomalies, so multi year ENSO forecasts are a paramount scientific issue. However, most existing works rely on the conventional iterative mechanism and, thus, fail to provide reliable long-term predictions due to error accumulation. Although methods based on deep learning (DL) apply the parallel modeling scheme for different lead times instead of a single iteration model, they leverage the same DL model for prediction, which can not fully mine the variability of different lead times, resulting in a decrease of prediction accuracy. To solve this problem, we propose a novel parallel deep convolutional neural network (CNN) with a heterogeneous architecture. In this study, by adaptively selecting network architectures for different lead times, we realize variability modeling of different tasks (lead times) and thereby improve the reliability of long-term predictions. Furthermore, we propose a relationship between different prediction lead times and neural network architecture from a unique perspective, namely, the receptive field originally proposed in computer vision. According to the spatio-temporal correlated area and sampling scale of lead times, the size of the convolution kernel and the mesh size of sampling are adjusted as the lead time increases. The Coupled Model Intercomparison Project phase 5 (CMIP5) from 1861 to 2004 and the Simple Ocean Data Assimilation (SODA) from 1871 to 1973 were used for model training, and the GODAS from 1982 to 2017 were used for testing the forecast skill of the model. Experimental results demonstrate that the proposed method outperforms the other well-known methods, especially for long-term predictions.

Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

Deep learning and radar make it feasible to automatically recognize human activities in various lighting conditions, even occlusion case, which significantly promotes the application of activity recognition in the fields of security surveillance, health care, and so on. In this paper, an approach for human activity recognition (HAR) using deep learning is proposed based on stepped frequency continues wave (SFCW) radar. Specifically, SFCW radar is utilized to generate two types of characteristic representation domains, namely multiple frequencies of spectrograms in time–frequency domain and range maps in range domain. On the one hand, spectrograms and range maps provide different types of features. On the other hand, multi-frequency spectrograms furnish same type of features while with different scattering properties and frequency resolutions. Then a specific deep learning network including multiple parallel deep convolutional neural networks (DCNNs) and a sparse autoencoder is designed to extract and fuse these features associated with human activities from the multi-frequency spectrograms and rang map. In particular, each DCNN is aimed at extracting the detailed micro-Doppler features from a spectrogram, while sparse autoencoder learns prime range distribution features by compressing each range map to reduce complexity and improve robustness. Experimental results verify that the proposed deep learning scheme achieves 96.42% recognition accuracy about six types of activities by incorporating three frequencies of spectrograms and range map, and surpasses two existed methods depending on single-frequency spectrogram and combination of single-frequency spectrogram and range map.

PdlADMM: An ADMM-based framework for parallel deep learning training with efficiency

Alternating Direction Methods of Multipliers (ADMM) has been proven to be a useful alternative to the popular gradient-based optimizers and successfully applied to train the DNN model. Whereas existing ADMM-based approaches generally do not achieve a good trade-off between the rapid convergence and fast training and do not support parallel DNN training with multiple GPUs as well. These drawbacks seriously hinder them from effectively training DNN models with modern GPU computing platforms which are always equipped with multiple GPUs. In this paper, we propose pdlADMM that can effectively train DNN in a data-parallel manner. The key insight of pdlADMM lies in that it explores efficient solutions for each sub-problem by comprehensively considering three main factors including computational complexity, convergence, and suitability to parallel computing. With more number of GPUs, pdlADMM remains rapid convergence and the computational complexity on each GPU tends to decline. Extensive experiments demonstrate the effectiveness of our proposal. Compared to the other two state state-of-the-art ADMM-based approaches, pdlADMM converges significantly faster, obtains better accuracy, and achieves very competitive training speed at the same time.

RETRACTED ARTICLE: Parallel deep convolutional neural network for content based medical image retrieval

DICOM images which helps in diagnosis and prognosis would be critical component in health care systems. Speedy recovery of past historic DICOM images based on the given query image is becoming a critical requirement for the Laboratories and Doctors for quick inference and accurate analogy of the patient conditions. In existing, It is also identified that there is a presence of imbalanced data set which degrade the retrieval accuracy of the model which may reduce by using extract the different kinds of features. The DCNN classifiers are trained by datasets whose data distributions of individual classes are not even or similar, they have always suffered from imbalanced classification performance against classes. Through DCNN can be used to minimize the gaps in terms of accuracy and retrieval but still efficiency parallelization would be essential for faster training and retrieval time. Time complexity is always been a major issue in DCNN, to overcome the above complexity the parallelization of model or data dimension need to be adapted. In this paper, parallel deep convolutional neural network (PDCNN) model is proposed by hyper parameter optimimzation for CBMIR system. The proposed model incorporating the low level content features, high level semantic features and compact features along with DCNN features to tackle the imbalanced dataset problem and reducing the DCNN training time for DICOM images. The high-level and compact features are extracted to resolve the imbalanced dataset problem by using the following algorithms: (a) local binary pattern (LBP), (b) histogram of oriented gradients (HOG) and (c) radon. The data parallelism was adopted in the proposed DCNN model to reduce the network training time by execution of DCNN layers across multiple CPU cores on a single PC. The implementation results for the proposed model in terms of Precision, Recall and F measure values are 87%, 87% and 92% respectively.

Prediction of Enzyme Function Based on Three Parallel Deep CNN and Amino Acid Mutation.

During the past decade, due to the number of proteins in PDB database being increased gradually, traditional methods cannot better understand the function of newly discovered enzymes in chemical reactions. Computational models and protein feature representation for predicting enzymatic function are more important. Most of existing methods for predicting enzymatic function have used protein geometric structure or protein sequence alone. In this paper, the functions of enzymes are predicted from many-sided biological information including sequence information and structure information. Firstly, we extract the mutation information from amino acids sequence by the position scoring matrix and express structure information with amino acids distance and angle. Then, we use histogram to show the extracted sequence and structural features respectively. Meanwhile, we establish a network model of three parallel Deep Convolutional Neural Networks (DCNN) to learn three features of enzyme for function prediction simultaneously, and the outputs are fused through two different architectures. Finally, The proposed model was investigated on a large dataset of 43,843 enzymes from the PDB and achieved 92.34% correct classification when sequence information is considered, demonstrating an improvement compared with the previous result.

Low voltage AC series arc fault detection method based on parallel deep convolutional neural network

The complexity and concealment of series arc fault threaten the safety of household power supply system. Detection of series arc faults using threshold and characteristics extracted from current or voltage may affected by load type and connection mode. Based on AlexNet, a new parallel deep convolutional neural network detection method is proposed in this paper. The series arc fault and normal operation current signal data of three types of load totally 7200 sets were collected respectively. Using the collected data, a training data set and four test data sets were constructed to train and test the proposed convolutional neural network. The experimental results show that the detection accuracy of parallel AlexNet is higher than that of AlexNet, with a maximum of 16.75 percentage points. The stability of parallel AlexNet is about twice as high as that of AlexNet, and the maximum is close to four times.