Deep Convolutional Neural Network Model Research Articles

Transferring deep convolutional neural network models for generalization mapping of autumn crops

Transferring deep convolutional neural network models for generalization mapping of autumn crops

Transfer Learning and Hybrid Deep Convolutional Neural Networks Models for Autism Spectrum Disorder Classification From EEG Signals

Transfer Learning and Hybrid Deep Convolutional Neural Networks Models for Autism Spectrum Disorder Classification From EEG Signals

Dominating Alzheimer's disease diagnosis with deep learning on sMRI and DTI-MD.

Alzheimer's disease (AD) is a progressive and irreversible neurodegenerative disorder that has become one of the major health concerns for the elderly. Computer-aided AD diagnosis can assist doctors in quickly and accurately determining patients' severity and affected regions. In this paper, we propose a method called MADNet for computer-aided AD diagnosis using multimodal datasets. The method selects ResNet-10 as the backbone network, with dual-branch parallel extraction of discriminative features for AD classification. It incorporates long-range dependencies modeling using attention scores in the decision-making layer and fuses the features based on their importance across modalities. To validate the effectiveness of our proposed multimodal classification method, we construct a multimodal dataset based on the publicly available ADNI dataset and a collected XWNI dataset, which includes examples of AD, Mild Cognitive Impairment (MCI), and Cognitively Normal (CN). On this dataset, we conduct binary classification experiments of AD vs. CN and MCI vs. CN, and demonstrate that our proposed method outperforms other traditional single-modal deep learning models. Furthermore, this conclusion also confirms the necessity of using multimodal sMRI and DTI data for computer-aided AD diagnosis, as these two modalities complement and convey information to each other. We visualize the feature maps extracted by MADNet using Grad-CAM, generating heatmaps that guide doctors' attention to important regions in patients' sMRI, which play a crucial role in the development of AD, establishing trust between human experts and machine learning models. We propose a simple yet effective multimodal deep convolutional neural network model MADNet that outperforms traditional deep learning methods that use a single-modality dataset for AD diagnosis.

Intelligent Machinery Fault Diagnosis With Event-Based Camera

Event-based cameras are the emerging bio-inspired technology in vision sensing. Different from the traditional standard cameras, the event-based cameras asynchronously record the brightness change per pixel, and have the great merits of high temporal resolution, low energy consumption, high dynamic range, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">etc</i> . While the event-based cameras have been initially exploited in several common vision-based tasks in the recent years, the investigation on machine condition monitoring problem is quite limited. This paper offers the first attempt in the current literature on exploring the contactless event vision data for machine fault diagnosis. A vibration event representation is proposed to transform the event records into typical data samples, and a deep convolutional neural network model is used for processing the event information. To enhance the model robustness against environmental noisy vision events, an event data augmentation method is proposed to introduce variations of the event patterns. A deep representation clustering method is further proposed to improve the pattern recognition performance with respect to different machine health conditions. Experiments on the event vision-based rotating machine fault diagnosis problem are carried out. It is extensively validated that high fault diagnosis accuracies can be obtained using the vision data from the event-based cameras, which are competitive with the popular accelerometer data. Considering the properties of flexibility, portability and data recognizability, the event-based cameras thus provide a promising new tool for contactless machine health condition monitoring and fault diagnosis.

Tomato Leaf Disease Detection Using CNN

One of the structure of economic development is farming involved with niche skills and technologies. More than 25% of total tomato crop production is lost annually because of various plant diseases. The farmers who grow tomatoes face huge economic losses every year, often due to various diseases that can infect tomatoes. The work carried aims to identify these diseases early and accurately, so can be informed well before the damage, which can minimize the waste. Convolution neural network (CNN) techniques, subset of Machine Learning (ML) are demonstrated to be very efficient for image datasets of various complex problems. Specifically, the work is focused to build and train a customized CNN model to classify the leaf images. Here we are using advanced deep learning architectures AlexNet and VGGNet-16 to recognize various diseases from image datasets collected from agricultural sources to develop models. Comparative analysis has resulted an improvement in accuracy by using deep learning architectures and CNN models. These models are responsible for helping to solve distribution problems, checking whether a leaf is healthy or not and, if so, what disease it is. Disease detected early and treated quickly minimizes the cost and improves the quality and quantity of the crop produced.

FPGA based Flexible Implementation of Light Weight Inference on Deep Convolutional Neural Networks

Standard Convolution (StdConv) is the main technique used in the state of the art Deep Convolutional Neural Networks (DCNNs). Fewer computations are achieved if Depthwise Separable Convolution technique (SepConv) is used as an alternative. A crucial issue in many applications like smart cameras and autonomous vehicles where low latency is essential stems from deploying a lightweight and low cost inference models. An acceptable accuracy should be kept with tolerable computations and memory access load. A flexible architecture for different DCNN convolution types and models is proposed. The flexibility comes from the sharing of one memory access unit with different types of layers regardless of the selected kernel size, by multiplying each weight vector by local operators with variant aperture. Moreover, one depthwise computation unit can be used for both standard and pointwise layers. The learnable parameters are quantized to 8-bits fixed point representation and that gives very limited reduction of accuracy and a considerable reduction of the Field-Programmable Gate Array (FPGA) resources. To reduce processing time, inter layer parallel computations are performed. The experiment is conducted by using grey scale ORL database with shallow Convolutional Neural Network (CNN) and the colored Canadian Institute for Advanced Research 10 classes (CIFAR-10) database with DCNN, and a comparable accuracies of 93% and 85.7% are achieved respectively using very low cost of Spartan 3E and moderate cost of zynq FPGA platforms

A comparative study on time series data-based artificial intelligence approaches for classifying cattle feeding behavior

&lt;span&gt;Cattle feeding behavior analysis is crucial for optimizing livestock management practices and ensuring animal well-being. This study presents a comparative analysis of three models: two machine learning algorithms including random forest and support vector machine (SVM), in addition to a deep learning convolutional neural networks (CNN) model, for classifying cattle feeding behaviors (eating, ruminating, and other) using time series data generated from a 3-axis accelerometer. The results of this study highlight the performance of these methods in accurately categorizing cattle feeding behaviors and demonstrate the importance of precise and efficient livestock monitoring and contributing to the improvement of animal well-being and enhancing the overall effectiveness of livestock operations.&lt;/span&gt;

Deep learning-based network anomaly detection and classification in an imbalanced cloud environment

With the advancements in computer networking, communication between end-to-end systems has increased drastically. However, security issues have also been raised. Thus, detecting anomalies from a complex cloud environment is still challenging. Therefore, the present article proposes the deep Convolutional Neural Network (CNN) model for detecting and classifying near-real-time network intrusions from an imbalanced cloud environment. The random forest model is also offered and implemented to select the best suitable features as input to the CNN model. The experiments were carried out on CSE-CIC-IDS2018 datasets. The results show that the proposed CNN model achieved 97.07% testing accuracy with a 2.93% error rate. The performance of the proposed model was also measured using precision, recall, and f1-score with 98.11, 96.93, and 97.52%. The results are more accurate, precise, promising, and able to detect network anomalies with the highest accuracy and can be successfully used in real-time Industry 4.0 systems.

Deep convolutional neural network (CNN) model optimization techniques—Review for medical imaging

The field of artificial intelligence (AI) and machine learning (ML) has been expanding and is explored by researchers in various fields. In medical diagnosis, for instance, the field of AI/ML is being explored because if medical diagnostic devices are built and designed with a backend of AI/ML, then the benefits would be unprecedented. Automated diagnostic tools would result in reduced health care costs, diagnosis without human intervention, overcoming human errors, and providing adequate and affordable medical care to a wider portion of the population with portions of the actual cost. One domain where AI/ML can make an immediate impact is medical imaging diagnosis (MID), namely the classification of medical images, where researchers have applied optimization techniques aiming to improve image classification accuracy. In this paper, we provide the research community with a comprehensive review of the most relevant studies to date on the use of deep CNN architecture optimization techniques for MID. As a case study, the application of these techniques to COVID-19 medical images were made. The impacts of the related variables, including datasets and AI/ML techniques, were investigated in detail. Additionally, the significant shortcomings and challenges of the techniques were touched upon. We concluded our work by affirming that the application of AI/ML techniques for MID will continue for many years to come, and the performance of the AI/ML classification techniques will continue to increase.

Leveraging Neutrosophic TOPSIS with Artificial Intelligence-Driven Tropical Cyclone Intensity Estimation for Weather Prediction

Tropical cyclones (TCs) are powerful, low-pressure weather systems attributed to heavy rainfall and strong winds, and have often resulted in extensive damage to coastal regions. TC intensity prediction, an essential aspect of meteorological forecasting, includes evaluating the strength of the storm to facilitate disaster preparedness and alleviate possible risks. Classical approaches for the prediction of TC intensity rely on different oceanic and atmospheric parameters, but the incorporation of artificial intelligence (AI) approaches, especially those leveraging image data, provides positive breakthroughs in efficiency and accuracy. By harnessing AI techniques like deep learning architectures and convolutional neural networks (CNNs), meteorologists could analyze radar data, satellite imagery, and other visual inputs to distinguish complicated patterns indicative of intensity changes and TC development. This combination of weather science and AI-driven image analysis enables more timely and precise predictions and improves our understanding of TC dynamics, eventually fortifying protection against the impacts of formidable storms. This article introduces Neutrosophic TOPSIS with Artificial Intelligence Driven Tropical Cyclone Intensity Estimation (NTOPSIS-TCIE) technique for Weather Prediction. The presented NTOPSIS-TCIE technique determines the intensities of the TC which in turn helps to forecast weather. In the NTOPSIS-TCIE technique, median filtering (MF) approach is used to remove the noise in the images. In addition, the features are extracted using deep convolutional neural network (CNN) model. To enhance the performance of the CNN model, Harris Hawks Optimization (HHO) algorithm is applied. Finally, the NTOPSIS model is employed for the prediction of TC intensities. The performance of the NTOPSIS-TCIE technique can be studied using TC image dataset and the results signify its promising results over other models

Network anomaly detection and performance evaluation of Convolutional Neural Networks on UNSW-NB15 dataset

The present study uses the benchmark UNSW-NB15 datasets to detect network anomalies using the proposed deep learning-based Convolutional Neural Network (CNN) model. Several studies have already worked on network anomaly detection with some limits. However, earlier research has used old datasets and obtained limited accuracy. In addition, previous studies could not detect the latest malicious activities. Therefore, in the present study, firstly, we implemented the machine learning-based random forest method to diminish the dimensionality of the dataset and pick the most notable features. In this case, the random forest method has set only fifteen features from the forty-one and reduced the complexity of the data. Subsequently, the CNN model was trained and tested on the UNSW-NB15 dataset with 99.00% testing accuracy. The performance of the offered CNN model was assessed using precision, recall and the f1-values. The present study’s outcome can be used in real-time malicious activity detection.

DIAGNOSIS OF KIDNEY CYST, TUMOR AND STONE FROM CT SCAN IMAGESUSING FEATURE FUSION HYPERGRAPH CONVOLUTIONAL NEURAL NETWORK (F2HCN2)

The development of a computational tool to diagnose kidney disorders in their early stages is necessary due to the rise in chronic kidney diseases (CKDs) and the global shortage of nephrologists. The three common renal disorders covered in this study are kidney stones, cysts, and tumors. Early diagnosis of these diseases from the computed tomography (CT) images is a challenging task. Yet, present graph convolutional neural network (GCNN) approaches have the issue of overdependence on the adjacency matrix. Moreover, compared to deep convolutional neural network (CNN) models, a single modal feature results in low accuracy and robustness. In this paper, we proposed the feature fusion hypergraph CNN (F&lt;sup&gt;2&lt;/sup&gt;HCN&lt;sup&gt;2&lt;/sup&gt;) to accurately diagnose kidney diseases in the early stage based on CT scan images. The discriminative features of the images are extracted using DarkNet19 and residual features are extracted using ResNet50. The extracted features are classified using feature fusion hypergraph CNN. The proposed model is trained with 12,446 CT whole urogram and abdomen images. The hypergraph representation learning is performed to train the network with the fused features. Deep learning metrics including accuracy, F1 score (F1), recall, positive predictive value (PPV), receiver operating characteristic curve, and area under curve (AUC) are used to validate the proposed model. It outperforms compared to other state-of-the-art algorithms with accuracy of 99.71&amp;#37;. The proposed F&lt;sup&gt;2&lt;/sup&gt;HCN&lt;sup&gt;2&lt;/sup&gt; is a robust computer-aided tool for the early diagnosis of kidney diseases. It will assist the radiologist for better prognosis for kidney related abnormalities.

A Combination Model of Shifting Joint Angle Changes With 3D-Deep Convolutional Neural Network to Recognize Human Activity.

Research in the field of human activity recognition is very interesting due to its potential for various applications such as in the field of medical rehabilitation. The need to advance its development has become increasingly necessary to enable efficient detection and response to a wide range of movements. Current recognition methods rely on calculating changes in joint distance to classify activity patterns. Therefore, a different approach is required to identify the direction of movement to distinguish activities exhibiting similar joint distance changes but differing motion directions, such as sitting and standing. The research conducted in this study focused on determining the direction of movement using an innovative joint angle shift approach. By analyzing the joint angle shift value between specific joints and reference points in the sequence of activity frames, the research enabled the detection of variations in activity direction. The joint angle shift method was combined with a Deep Convolutional Neural Network (DCNN) model to classify 3D datasets encompassing spatial-temporal information from RGB-D video image data. Model performance was evaluated using the confusion matrix. The results show that the model successfully classified nine activities in the Florence 3D Actions dataset, including sitting and standing, obtaining an accuracy of (96.72 ± 0.83)%. In addition, to evaluate its robustness, this model was tested on the UTKinect Action3D dataset, obtaining an accuracy of 97.44%, proving that state-of-the-art performance has been achieved.

Coronary heart disease classification using deep learning approach with feature selection for improved accuracy.

Coronary heart disease (CHD) is one of the deadliest diseases and a risk prediction model for cardiovascular conditions is needed. Due to the huge number of features that lead to heart problems, it is often difficult for an expert to evaluate these huge features into account. So, there is a need of appropriate feature selection for the given CHD dataset. For early CHD detection, deep learning modes (DL) show promising results in the existing studies. This study aimed to develop a deep convolution neural network (CNN) model for classification with a selected number of efficient features using the LASSO (least absolute shrinkage and selection operator) technique. Also, aims to compare the model with similar studies and analyze the performance of the proposed model using accuracy measures. The CHD dataset of NHANES (National Health and Nutritional Examination Survey) was examined with 49 features using LASSO technique. This research work is an attempt to apply an improved CNN model for the classification of the CHD dataset with huge features CNN model with feature extractor consists of a fully connected layer with two convolution 1D layers, and classifier part consists of two fully connected layers with SoftMax function was trained on this dataset. Metrics like accuracy recall, specificity, and ROC were used for the evaluation of the proposed model. The feature selection was performed by applying the LASSO model. The proposed CNN model achieved 99.36% accuracy, while previous studies model achieved over 80 to 92% accuracy. The application of the proposed CNN with the LASSO model for the classification of CHD can speed up the diagnosis of CHD and appears to be effective in predicting cardiovascular disease based on risk features.

Attention Feature Fusion Network via Knowledge Propagation for Automated Respiratory Sound Classification.

Goal: In light of the COVID-19 pandemic, the early diagnosis of respiratory diseases has become increasingly crucial. Traditional diagnostic methods such as computed tomography (CT) and magnetic resonance imaging (MRI), while accurate, often face accessibility challenges. Lung auscultation, a simpler alternative, is subjective and highly dependent on the clinician's expertise. The pandemic has further exacerbated these challenges by restricting face-to-face consultations. This study aims to overcome these limitations by developing an automated respiratory sound classification system using deep learning, facilitating remote and accurate diagnoses. Methods: We developed a deep convolutional neural network (CNN) model that utilizes spectrographic representations of respiratory sounds within an image classification framework. Our model is enhanced with attention feature fusion of low-to-high-level information based on a knowledge propagation mechanism to increase classification effectiveness. This novel approach was evaluated using the ICBHI benchmark dataset and a larger, self-collected Pediatric dataset comprising outpatient children aged 1 to 6 years. Results: The proposed CNN model with knowledge propagation demonstrated superior performance compared to existing state-of-the-art models. Specifically, our model showed higher sensitivity in detecting abnormalities in the Pediatric dataset, indicating its potential for improving the accuracy of respiratory disease diagnosis. Conclusions: The integration of a knowledge propagation mechanism into a CNN model marks a significant advancement in the field of automated diagnosis of respiratory disease. This study paves the way for more accessible and precise healthcare solutions, which is especially crucial in pandemic scenarios.

Advancing soil erosion prediction in Wadi Sahel-Soummam watershed Algeria: A comparative analysis of deep neural networks (DNN) and convolutional neural networks (CNN) models integrated with GIS

This study employs adaptive deep learning (utilizing DNN and CNN approaches) to accurately predict soil erosion, a crucial aspect of sustainable soil resource management. The goal is to develop fuzzy logic models for erosion forecasting in a large watershed with limited in-puts, comparing them to predictions from the Revised Universal Soil Loss Equation (RUSLE). Integration of GIS enables analysis of satellite data, providing crucial details like land use, slope, rainfall distribution, and flow direction. This synergistic approach enhances erosion prediction capabilities and yields spatial erosion distributions. Producing precise erosion risk maps within GIS is crucial for prioritizing high-risk areas and implementing effective conservation methods in the Wadi Sahel watershed, Algeria. The assessment in the Oued Sahel-Soummam watershed involved overlaying five RUSLE factor maps using Arc GIS spatial analysis, resulting in an aver-age annual soil loss of 4.22 tons per hectare. The DNN and CNN models were integrated with GIS for detailed calculation of annual average soil loss (tons per hectare per year) and mapping erosion risk areas in Wadi Sahel-Soummam watershed. Using the CNN model, estimated annual soil loss in Sahel-Soummam wadi was about 4.00 tons per hectare per year, while the DNN model estimated around 4.13 tons per hectare per year. This study employed two deep learning models for erosion prediction, with the DNN model featuring six hidden layers performing no-tably better than the compared CNN model.

Deep CNN and LSTM Approaches for Efficient Workload Prediction in Cloud Environment

In the dynamic landscape of cloud computing, efficient resource allocation and workload prediction are paramount for optimizing infrastructure utilization, cost management, and overall service quality. As a result, an efficient resource management approach involves pooling available resources among many customers in a manner that considers their energy usage, SLAs, and forecasting accuracy all at once. Based on the performance analysis results, it is determined that integrating these techniques would result in the most efficient and adaptable cloud data centre. We proposed a Model based on Deep Convolutional Neural Networks (DCNN) and Long Short-Term Memory (LSTM) for handling SLAs in the cloud from both the perspective of consumers and service providers. Subsequently, we delve into the methodology of applying deep CNN and LSTM models to the problem of workload prediction in cloud environments. This methodology encompasses data preprocessing, model architecture, training parameters, and the choice of performance metrics. To anticipate CPU consumption from time series data and detect SLA violations, we suggested a DCNN-LSTM model. The accuracy prediction, energy usage, CPU use, and Service Level Agreement monitoring are all a part of this model. The proposed method is effective in helping cloud providers cut down on service violations and associated fines. Regarding the composite metric of Energy SLA Violation, which assesses the combined aspects of energy use and adherence to Service Level Agreements (SLAs), DCNN-LSTM surpasses ARIMA-LSTM, CNN, LSTM, and ARIMA 6.8%, 10.88%, 16.6%, and 22.4%, respectively.

MRI Brain Tumor Analysis on Improved VGG-16 and Efficient NetB7 Models

In recent years, Magnetic Resonance Imaging (MRI) scanning has been the most rapidly developing field. Concerning the tumor’s size and specifics, diagnosing and classifying brain tumor’s is challenging and time-consuming for radiologists. The growth of abnormal cells in the brain is referred to as a brain tumor. A brain tumor is diagnosed in about 11,700 patients every year. It is estimated that 34% of males and 36% of females will survive five years after being diagnosed with malignant brain or different tumors. This study focuses on meningioma, pituitary, glioma and no tumors’, among the many brain tumors. Deep learning algorithms and machine learning methods were used to create an autonomous classification and segmentation system for brain tumors, significantly improving early detection. Using the Visual Geometry Group (16), parameters were set for training the model that detects brain tumors based on analysis of proposed literature solutions. Simple Convolution Neural Network (CNN) models such as VGG-16 and Efficient NetB7 perform well because they are among the highest-performing models. As a result of the study, quick, efficient, and precise decisions can be made using MRI to detect brain tumors. For this 7022 brain magnetic resonance images were used to train and test this model. According to experimental findings, the suggested differential deep CNN model could accurately categories MRI pictures of brain tumors, including aberrant and standard images, with a 98.19% accuracy rate.

Deblurring Videos Using Spatial-Temporal Contextual Transformer With Feature Propagation.

We present a simple and effective approach to explore both local spatial-temporal contexts and non-local temporal information for video deblurring. First, we develop an effective spatial-temporal contextual transformer to explore local spatial-temporal contexts from videos. As the features extracted by the spatial-temporal contextual transformer does not model the non-local temporal information of video well, we then develop a feature propagation method to aggregate useful features from the long-range frames so that both local spatial-temporal contexts and non-local temporal information can be better utilized for video deblurring. Finally, we formulate the spatial-temporal contextual transformer with the feature propagation into a unified deep convolutional neural network (CNN) and train it in an end-to-end manner. We show that using the spatial-temporal contextual transformer with the feature propagation is able to generate useful features and makes the deep CNN model more compact and effective for video deblurring. Extensive experimental results show that the proposed method performs favorably against state-of-the-art ones on the benchmark datasets in terms of accuracy and model parameters.

Fractal Dimension on CBCT Images and Modular Neural Networks to Identify Reduced Bone Mineral Density in Women

This paper provides two different methods to diagnose osteoporosis in women; the first method is the fractal analysis evaluated by CBCT at two bone locations (the mandible and the second cervical vertebrae) to see if there is any correlation between the two. At the same time, the second method is deep convolutional neural networks (DCNNs). One hundred eighty-eight patients' mandibular CBCT images were used, and DCNN models based on the ResNet-101 framework were employed. Dual X-ray absorptiometry of the hip and lumbar spine revealed that 139 of the 188 postmenopausal women tested had osteoporosis, whereas 49 had average bone mineral density. The second cervical vertebra and the mandible were selected as locations of interest for FD analysis on the CBCT images. Measurement accuracy, both within and between observers' agreements, and correlations between two data sets were all calculated. To evaluate osteoporosis, we used a segmented, three-phase approach. Stage 1 was devoted to the identification of mandibular bone slices. In Stage 2, the coordinates for the mandible's cross-sectional views were established, and Stage 3 calculated the thickness of the mandible bone, emphasizing osteoporotic variations. The average FD values within the interest area of the mandible were significantly lower in people with osteoporosis than in those with average bone mineral density. At the same time, the two groups had no significant difference in FD values at the second cervical vertebra. For the mandibular site, areas beneath the curve were 0.644 (P = 0.008), while the area under the curve for the vertebral site was 0.531 (P = 0.720). DCNN training in the first stage yielded an astounding 98.85% training accuracy, the second stage decreased L1 loss to a meager 1.02 pixels, and the bone thickness computation method used in the last stage had a mean squared error of 0. 8377. We concluded that FD was underutilized even though it distinguished between women with normal BMD and those with osteoporosis in the mandibular area. Additionally, even with small mandibular CBCT datasets, the results show the value of a modular transfer learning approach for osteoporosis detection.