Deep Learning Convolutional Neural Network Research Articles

Architectural Synergy: Investigating the Role of Artificial Neural Networks in Enabling Deep Learning

Convolutional Neural Networks (CNNs) have transformed the area of deep learning by demonstrating exceptional abilities in a range of tasks, including object identification, picture identification, and natural language processing. Yet, study and investigation into the complex interactions between various architectural elements inside CNNs known as architectural synergy remain continuing. This study addresses the consequences of architectural synergy for improving model performance, scalability, and reliability, and examines how it enables deep learning using CNNs. Using an extensive examination of extant literature and real-world implementation cases, we clarify the processes that underlie architectural synergy and underscore its capacity to enhance the capabilities of CNN-powered models. We hope to further our knowledge of the fundamental ideas behind the effectiveness of CNNs in deep learning by illuminating this important area of neural network architecture learning tasks.

Deep Learning based DWT- Bi-LSTM Classifier for Enhanced Cardiovascular Arrhythmia Classification

Nowadays heart diseases and their diagnosis have emerged as a prominent subject in health care systems, given that the heart performs a crucial role in the human body. Several computational techniques have been explored for the recognition and classification of cardiac diseases using Electrocardiogram (ECG) signals. Deep Learning (DL) is a present focus in healthcare solicitations, particularly in the classification of heartbeats in ECG signals. Many studies have utilized dissimilar DL models, including RNN (Recurrent Neural Networks), GRU (Gated Recurrent Unit), and CNN (Convolutional Neural Networks), to classify heartbeats using the MIT-BIH arrhythmia dataset. This article presents a methodical exploration of Bi-LSTM (Bi directional Long Short-Term Memory) based DL models for heartbeat classification using various quality metrics. Proposed variants include the Bi-LSTM model, demonstrating remarkable accuracy in classifying the heartbeats into five classes: Normal (N) beat, Supraventricular (S) beat, Ventricular contraction (V), Fusion beats (F), and Unclassifiable Beat (Q). The proposed technique outperforms present classifiers with Accuracy, Sensitivity, Specificity, and F1 score values of 98%, 96.9%, 97.4%, and 97.5% respectively. The simulations are conducted using MATLAB 2020a.

Galaxy mergers in UNIONS – I. A simulation-driven hybrid deep learning ensemble for pure galaxy merger classification

Abstract Merging and interactions can radically transform galaxies. However, identifying these events based solely on structure is challenging as the status of observed mergers is not easily accessible. Fortunately, cosmological simulations are now able to produce more realistic galaxy morphologies, allowing us to directly trace galaxy transformation throughout the merger sequence. To advance the potential of observational analysis closer to what is possible in simulations, we introduce a supervised deep learning Convolutional Neural Network (CNN) and Vision Transformer (ViT) hybrid framework, Mummi (MUlti Model Merger Identifier). Mummi is trained on realism-added synthetic data from IllustrisTNG100-1, and is comprised of a multi-step ensemble of models to identify mergers and non-mergers, and to subsequently classify the mergers as interacting pairs or post-mergers. To train this ensemble of models, we generate a large imaging dataset of 6.4 million images targeting UNIONS with RealSimCFIS. We show that Mummi offers a significant improvement over many previous machine learning classifiers, achieving 95% pure classifications even at Gyr long timescales when using a jury-based decision making process, mitigating class imbalance issues that arise when identifying real galaxy mergers from z = 0 to 0.3. Additionally, we can divide the identified mergers into pairs and post-mergers at 96% success rate. We drastically decrease the false positive rate in galaxy merger samples by 75%. By applying Mummi to the UNIONS DR5-SDSS DR7 overlap, we report a catalog of 13,448 high confidence galaxy merger candidates. Finally, we demonstrate that Mummi produces powerful representations solely using supervised learning, which can be used to bridge galaxy morphologies in simulations and observations.

Facial recognition for disease diagnosis using a deep learning convolutional neural network: a systematic review and meta-analysis.

With the rapid advancement of deep learning network technology, the application of facial recognition technology in the medical field has received increasing attention. This study aims to systematically review the literature of the past decade on facial recognition technology based on deep learning networks in the diagnosis of rare dysmorphic diseases and facial paralysis, among other conditions, to determine the effectiveness and applicability of this technology in disease identification. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines for literature search and retrieved relevant literature from multiple databases, including PubMed, on 31 December 2023. The search keywords included deep learning convolutional neural networks, facial recognition, and disease recognition. A total of 208 articles on facial recognition technology based on deep learning networks in disease diagnosis over the past 10years were screened, and 22 articles were selected for analysis. The meta-analysis was conducted using Stata 14.0 software. The study collected 22 articles with a total sample size of 57 539 cases, of which 43 301 were samples with various diseases. The meta-analysis results indicated that the accuracy of deep learning in facial recognition for disease diagnosis was 91.0% [95% CI (87.0%, 95.0%)]. The study results suggested that facial recognition technology based on deep learning networks has high accuracy in disease diagnosis, providing a reference for further development and application of this technology.

Cardiovascular disease risk factors prediction using deep learning convolutional neural networks

Heart disease remains a leading cause of mortality worldwide, prompting healthcare researchers to leverage analytical tools for comprehensive data analysis. This study focuses on exploring crucial parameters and employing deep learning (DL) techniques to enhance understanding and prediction of cardiovascular disease (CVD) risk factors. Utilizing SPSS and Weka tools, a cross-sectional and correlational design was employed to analyze extensive medical datasets. Binomial regression analysis revealed significant associations between age (𝑝 = 0.004) and body mass index (𝑝 = 0.002) with CVD development, highlighting their importance as risk factors. Leveraging Weka's DL algorithms, a predictive model was constructed to classify CVD causes. Particularly, convolutional neural networks (CNN) showcased remarkable accuracy, reaching 98.64%. The findings underscore the elevated risk of CVD among university students and employees in Saudi Arabia, emphasizing the need for heightened awareness and preventive measures, including dietary improvements and increased physical activity. This study underscores the importance of further research to enhance CVD risk perception among students and individuals in similar settings.

Single-cell hdWGCNA reveals metastatic protective macrophages and development of deep learning model in uveal melanoma

BackgroundAlthough there has been some progress in the treatment of primary uveal melanoma (UVM), distant metastasis remains the leading cause of death in patients. Monitoring, staging, and treatment of metastatic disease have not yet reached consensus. Although more than half of metastatic tumors (62%) are diagnosed within five years after primary tumor treatment, the remainder are only detected in the following 25 years. The mechanisms of UVM metastasis and its impact on prognosis are not yet fully understood.MethodsscRNA-seq data of UVM samples were obtained and processed, followed by cell type identification and characterization of macrophage subpopulations. High-dimensional weighted gene co-expression network analysis (HdWGCNA) was performed to identify key gene modules associated with metastatic protective macrophages (MPMφ) in primary samples, and functional analyses were conducted. Non-negative matrix factorization (NMF) clustering and immune cell infiltration analyses were performed using the MPMφ gene signatures. Machine learning models were developed using the identified metastatic protective macrophages related genes (MPMRGs) to distinguish primary from metastatic patients. A deep learning convolutional neural network (CNN) model was constructed based on MPMRGs and cell type associations. Lastly, a prognostic model was established using the MPMRGs and validated in independent cohorts.ResultsSingle-cell RNA-seq analysis revealed a unique immune microenvironment landscape in primary samples compared to metastatic samples, with an enrichment of macrophage cells. Using HdWGCNA, MPMφ and marker genes were identified. Functional analysis showed an enrichment of genes related to antigen processing progress and immune response. Machine learning and deep learning models based on key genes showed significant effectiveness in distinguishing between primary and metastatic patients. The prognostic model based on key genes demonstrated substantial predictive value for the survival of UVM patients.ConclusionOur study identified key macrophage subpopulations related to metastatic samples, which have a profound impact on shaping the tumor immune microenvironment. A prognostic model based on macrophage cell genes can be used to predict the prognosis of UVM patients.

Estimation of Leaf Water Content of a Fruit Tree by In Situ Vis-NIR Spectroscopy Using Multiple Machine Learning Methods in Southern Xinjiang, China

Water scarcity is one of the most significant environmental factors that inhibits photosynthesis and decreases the growth and productivity of plants. Using the deep learning convolutional neural network (CNN) model, this study evaluates the ability of spectroscopy to estimate leaf water content (LWC) in fruit trees. During midday, spectral data were acquired from leaf samples obtained from three distinct varieties of fruit trees, encompassing the spectral range spanning from 350 to 2500 nm. Then, for spectral preprocessing, the fractional order derivative (FOD) and continuous wavelet transform (CWT) algorithms were used to reduce the effects of scattering and noise on the collected spectra. Finally, the CNN model was developed to predict LWC in different fruit trees. The results showed that: (1) The spectra treated with CWT and FOD could improve the spectrum expression ability by improving the correlation between spectra and LWC. The correlation level of FOD treatment was higher than that of CWT treatment. (2) The CNN model was developed using FOD 1.2, and CWT 3 performed better than other traditional machine learning methods, such as RFR, SVR, and PLSR. (3) Further validation using additional samples demonstrated that the CNN model had good stability and quantitative prediction capability for the LWC of fruit trees (R2 > 0.95, root mean square error (RMSE) < 1.773%, and relative percentage difference (RPD) > 4.26). The results may provide an effective way to predict fruit LWC using a CNN-based model.

Comparison of data fusion strategies for automated prostate lesion detection using mpMRI correlated with whole mount histology

BackgroundIn this work, we compare input level, feature level and decision level data fusion techniques for automatic detection of clinically significant prostate lesions (csPCa).MethodsMultiple deep learning CNN architectures were developed using the Unet as the baseline. The CNNs use both multiparametric MRI images (T2W, ADC, and High b-value) and quantitative clinical data (prostate specific antigen (PSA), PSA density (PSAD), prostate gland volume & gross tumor volume (GTV)), and only mp-MRI images (n = 118), as input. In addition, co-registered ground truth data from whole mount histopathology images (n = 22) were used as a test set for evaluation.ResultsThe CNNs achieved for early/intermediate / late level fusion a precision of 0.41/0.51/0.61, recall value of 0.18/0.22/0.25, an average precision of 0.13 / 0.19 / 0.27, and F scores of 0.55/0.67/ 0.76. Dice Sorensen Coefficient (DSC) was used to evaluate the influence of combining mpMRI with parametric clinical data for the detection of csPCa. We compared the DSC between the predictions of CNN’s trained with mpMRI and parametric clinical and the CNN’s trained with only mpMRI images as input with the ground truth. We obtained a DSC of data 0.30/0.34/0.36 and 0.26/0.33/0.34 respectively. Additionally, we evaluated the influence of each mpMRI input channel for the task of csPCa detection and obtained a DSC of 0.14 / 0.25 / 0.28.ConclusionThe results show that the decision level fusion network performs better for the task of prostate lesion detection. Combining mpMRI data with quantitative clinical data does not show significant differences between these networks (p = 0.26/0.62/0.85). The results show that CNNs trained with all mpMRI data outperform CNNs with less input channels which is consistent with current clinical protocols where the same input is used for PI-RADS lesion scoring.Trial registrationThe trial was registered retrospectively at the German Register for Clinical Studies (DRKS) under proposal number Nr. 476/14 & 476/19.

Acoustic emission monitoring for damage diagnosis in composite laminates based on deep learning with attention mechanism

During service, advanced composite materials are susceptible to various forms of damage. Developing an efficient real-time Structural Health Monitoring (SHM) method is of profound significance to ensure the normal operational integrity of the advanced composite structures. This study introduces a novel end-to-end SHM approach employing acoustic emission (AE) techniques and a deep learning model with attention mechanisms (AM) to automate the damage diagnosis process and improve prediction accuracy. Low-velocity impact (LVI) experiments were carried out, recording AE signals for subsequent bi-spectrum analysis. Three dedicated AM-based Convolutional Neural Network (CNN) architectures were designed for damage diagnosis, utilizing 2D bi-spectrum contour images as input data. A comparison was made between the proposed CNN model, deep learning and traditional machine learning models in the context of damage diagnostics, revealing that CNN models combined AM exhibit significantly higher accuracy in pattern recognition tasks. The accuracies of the baseline CNN, self-attention-CNN, multi-head attention-CNN and convolutional block attention module-CNN are 98.22%, 98.50%, 98.85%, and 98.87%, respectively. The CNN embedded with AM demonstrated superior predictive performance and fewer instances of misclassification. Considering the specific demands of damage diagnostics in composite materials, the CBAM-CNN is better suited for composite material health monitoring tasks.

Deep learning classification of pediatric spinal radiographs for use in large scale imaging registries.

The purpose of this study is to develop and apply an algorithm that automatically classifies spine radiographs of pediatric scoliosis patients. Anterior-posterior (AP) and lateral spine radiographs were extracted from the institutional picture archive for patients with scoliosis. Overall, there were 7777 AP images and 5621 lateral images. Radiographs were manually classified into ten categories: two preoperative and three postoperative categories each for AP and lateral images. The images were split into training, validation, and testing sets (70:15:15 proportional split). A deep learning classifier using the EfficientNet B6 architecture was trained on the spine training set. Hyperparameters and model architecture were tuned against the performance of the models in the validation set. The trained classifiers had an overall accuracy on the test set of 1.00 on 1166 AP images and 1.00 on 843 lateral images. Precision ranged from 0.98 to 1.00 in the AP images, and from 0.91 to 1.00 on the lateral images. Lower performance was observed on classes with fewer than 100 images in the dataset. Final performance metrics were calculated on the assigned test set, including accuracy, precision, recall, and F1 score (the harmonic mean of precision and recall). A deep learning convolutional neural network classifier was trained to a high degree of accuracy to distinguish between 10 categories pre- and postoperative spine radiographs of patients with scoliosis. Observed performance was higher in more prevalent categories. These models represent an important step in developing an automatic system for data ingestion into large, labeled imaging registries.

Integrating the Capsule-like Smart Aggregate-Based EMI Technique with Deep Learning for Stress Assessment in Concrete.

This study presents a concrete stress monitoring method utilizing 1D CNN deep learning of raw electromechanical impedance (EMI) signals measured with a capsule-like smart aggregate (CSA) sensor. Firstly, the CSA-based EMI measurement technique is presented by depicting a prototype of the CSA sensor and a 2 degrees of freedom (2 DOFs) EMI model for the CSA sensor embedded in a concrete cylinder. Secondly, the 1D CNN deep regression model is designed to adapt raw EMI responses from the CSA sensor for estimating concrete stresses. Thirdly, a CSA-embedded cylindrical concrete structure is experimented with to acquire EMI responses under various compressive loading levels. Finally, the feasibility and robustness of the 1D CNN model are evaluated for noise-contaminated EMI data and untrained stress EMI cases.

Deep learning with convolution neural network detecting mesiodens on panoramic radiographs: comparing four models.

The aim of this study was to develop an optimal, simple, and lightweight deep learning convolutional neural network (CNN) model to detect the presence of mesiodens on panoramic radiographs. A total of 628 panoramic radiographs with and without mesiodens were used as training, validation, and test data. The training, validation, and test dataset were consisted of 218, 51, and 40 images with mesiodens and 203, 55, and 61 without mesiodens, respectively. Unclear panoramic radiographs for which the diagnosis could not be accurately determined and other modalities were required for the final diagnosis were retrospectively identified and employed as the training dataset. Four CNN models provided within software supporting the creation of neural network models for deep learning were modified and developed. The diagnostic performance of the CNNs was evaluated according to accuracy, precision, recall and F1 scores, receiver operating characteristics (ROC) curves, and area under the ROC curve (AUC). In addition, we used SHapley Additive exPlanations (SHAP) to attempt to visualize the image features that were important in the classifications of the model that exhibited the best diagnostic performance. A binary_connect_mnist_LeNet model exhibited the best performance of the four deep learning models. Our results suggest that a simple lightweight model is able to detect mesiodens. It is worth referring to AI-based diagnosis before an additional radiological examination when diagnosis of mesiodens cannot be made on unclear images. However, further revaluation by the specialist would be also necessary for careful consideration because children are more radiosensitive than adults.

Enhanced Estimation of Traffic Noise Levels Using Minute-Level Traffic Flow Data through Convolutional Neural Network

The advent of high-resolution minute-level traffic flow data from video surveillance on roads has opened up new opportunities for enhancing the estimation of traffic noise levels. In this study, we propose an innovative method that utilizes time series traffic flow data (TSTFD) to estimate traffic noise levels using a deep learning Convolutional Neural Network (CNN). Unlike traditional traffic flow data, TSTFD offer a unique structure and composition suitable for multidimensional data analysis. Our method was evaluated in a pilot study conducted in Foshan City, China, utilizing traffic flow information obtained from roadside video surveillance systems. Our results indicated that the CNN-based model surpassed traditional data-driven statistical models in estimating traffic noise levels, achieving a reduction in mean squared error (MSE) by 10.16%, mean absolute error (MAE) by 4.48%, and an improvement in the coefficient of determination (R²) by 1.73%. The model demonstrated robust generalization capabilities throughout the test period, exhibiting mean errors ranging from 0.790 to 1.007 dBA. However, the model’s applicability is constrained by the acoustic propagation environment, demonstrating effectiveness on roads with similar surroundings while showing limited applicability to those with different surroundings. Overall, this method is cost-effective and offers enhanced accuracy for the estimation of traffic noise level.

Evaluation of Denoising Performance of ResNet Deep Learning Model for Ultrasound Images Corresponding to Two Frequency Parameters.

Ultrasound imaging is widely used for accurate diagnosis due to its noninvasive nature and the absence of radiation exposure, which is achieved by controlling the scan frequency. In addition, Gaussian and speckle noises degrade image quality. To address this issue, filtering techniques are typically used in the spatial domain. Recently, deep learning models have been increasingly applied in the field of medical imaging. In this study, we evaluated the effectiveness of a convolutional neural network-based residual network (ResNet) deep learning model for noise reduction when Gaussian and speckle noises were present. We compared the results with those obtained from conventional filtering techniques. A dataset of 500 images was prepared, and Gaussian and speckle noises were added to create noisy input images. The dataset was divided into training, validation, and test sets in an 8:1:1 ratio. The ResNet deep learning model, comprising 16 residual blocks, was trained using optimized hyperparameters, including the learning rate, optimization function, and loss function. For quantitative analysis, we calculated the normalized noise power spectrum, peak signal-to-noise ratio, and root mean square error. Our findings showed that the ResNet deep learning model exhibited superior noise reduction performance to median, Wiener, and median-modified Wiener filter algorithms.

Identification and diagnosis of schizophrenia based on multichannel EEG and CNN deep learning model

This paper proposes a high-accuracy EEG-based schizophrenia (SZ) detection approach. Unlike comparable literature studies employing conventional machine learning algorithms, our method autonomously extracts the necessary features for network training from EEG recordings. The proposed model is a ten-layered CNN that contains a max pooling layer, a Global Average Pooling layer, four convolution layers, two dropout layers for overfitting prevention, and two fully connected layers. The efficiency of the suggested method was assessed using the ten-fold-cross validation technique and the EEG records of 14 healthy subjects and 14 SZ patients. The obtained mean accuracy score was 99.18 %. To confirm the high mean accuracy attained, we tested the model on unseen data with a near-perfect accuracy score (almost 100 %). In addition, the results we obtained outperform numerous other comparable works.

A Smart Visual Sensor for Smoke Detection Based on Deep Neural Networks.

The automatic detection of smoke by analyzing the video stream acquired by traditional surveillance cameras is becoming a more and more interesting problem for the scientific community thanks to the necessity to prevent fires at the very early stages. The adoption of a smart visual sensor, namely a computer vision algorithm running in real time, allows one to overcome the limitations of standard physical sensors. Nevertheless, this is a very challenging problem, due to the strong similarity of the smoke with other environmental elements like clouds, fog and dust. In addition to this challenge, data available for training deep neural networks is limited and not fully representative of real environments. Within this context, in this paper we propose a new method for smoke detection based on the combination of motion and appearance analysis with a modern convolutional neural network (CNN). Moreover, we propose a new dataset, called the MIVIA Smoke Detection Dataset (MIVIA-SDD), publicly available for research purposes; it consists of 129 videos covering about 28 h of recordings. The proposed hybrid method, trained and evaluated on the proposed dataset, demonstrated to be very effective by achieving a 94% smoke recognition rate and, at the same time, a substantially lower false positive rate if compared with fully deep learning-based approaches (14% vs. 100%). Therefore, the proposed combination of motion and appearance analysis with deep learning CNNs can be further investigated to improve the precision of fire detection approaches.

Diagnosis of Inflammations of the Paranasal Sinuses and Nasal Cavities in Deep Learning CNN Model

Diagnosis of Inflammations of the Paranasal Sinuses and Nasal Cavities in Deep Learning CNN Model

Development of an AI Model Utilizing Buildings’ Thermal Mass to Optimize Heating Energy and Indoor Temperature in a Historical Building Located in a Cold Climate

Historical buildings account for a significant portion of the energy use of today’s building stock, and there are usually limited energy saving measures that can be applied due to antiquarian and esthetic restrictions. The purpose of this case study is to evaluate the use of the building structure of a historical stone building as a heating battery, i.e., to periodically store thermal energy in the building’s structures without physically changing them. The stored heat is later utilized at times of, e.g., high heat demand, to reduce peaking as well as overall heat supply. With the help of Artificial Intelligence and Convolutional Neural Network Deep Learning Modelling, heat supply to the building is controlled by weather forecasting and a binary calendarization of occupancy for the optimization of energy use and power demand under sustained comfortable indoor temperatures. The study performed indicates substantial savings in total (by approximately 30%) and in peaking energy (by approximately 20% based on daily peak powers) in the studied building and suggests that the method can be applied to other, similar cases.

Trusted Operations of a Military Ground Robot in the Face of Man-in-the-Middle Cyberattacks Using Deep Learning Convolutional Neural Networks: Real-Time Experimental Outcomes

Trusted Operations of a Military Ground Robot in the Face of Man-in-the-Middle Cyberattacks Using Deep Learning Convolutional Neural Networks: Real-Time Experimental Outcomes

Elevating Performance Through AI-Driven Mock Interviews

Abstract: This paper proposes an innovative AI-based mock interview platform designed to enhance interview preparedness by assessing candidates across three key dimensions: emotions, confidence, and knowledge. Utilizing deep learning convolutional neural networks, the system analyzes facial expressions to gauge emotional responses, while speech recognition and natural language processing evaluate the candidate's confidence levels. Additionally, semantic analysis and keyword mapping assess the candidate's knowledge by comparing responses with relevant online resources. This comprehensive approach aims to reduce pre-interview anxiety, boost confidence, and refine interview skills, providing a more effective preparation tool compared to traditional methods.