Deep Learning Neural Network Research Articles

Advances in deep learning-based intelligent receivers

Abstract. This paper presents a summary of the progress of deep learning in the research of intelligent receivers and predicts its future development direction. First, the background of the era of intelligent receivers is introduced. With the rapid development of communication technology, intelligent receivers show insufficient flexibility, low accuracy, and spectral efficiency in detecting current radio signals. This is a challenge that is difficult to overcome in order to meet the needs of modern signal recovery. With the advent of artificial intelligence technologies such as deep learning, academia and industry began to explore the potential of applying these techniques to the field of communication receivers, leading to the emergence of intelligent receivers. This paper then introduces the fundamentals of wireless communication systems and deep learning neural networks, laying the groundwork for a deeper understanding of the subsequent content. Chapter 3 provides a comprehensive overview of the application of intelligent receivers in signal reception, including channel estimation, signal detection, modulation identification, demodulation and decoding. Finally, the paper considers the practical applications of intelligent receivers and speculates on their future development.

Digital rock modeling of deformed multi-scale media in deep hydrocarbon reservoirs based on in-situ stress-loading CT imaging and U-Net deep learning

Deep and ultra-deep hydrocarbon reservoirs are regarded as an attractive target for onshore oil and gas exploration and development in China. The multi-scale fractures are widely distributed in this type of reservoir. The fluid-solid coupling effect is strong, which can greatly affect the fluid flow, resulting in a low hydrocarbon recovery with the average value less than 15%. To explore the underlying flow dynamics during efficient development of deep and ultra-deep hydrocarbon reservoirs, it is of great importance to characterize the multi-scale fracture-pore structure evolution of rock affected by strong geo-stress field variation. To tackle the above issues, an actual rock drilled from a typical ultra-deep reservoir in Tarim Basin are selected to conduct in-situ stress-loading computed tomography (CT) scanning experiments, and the CT gray images of rock microstructure under different dynamic loading conditions are obtained. A fully convolutional neural network (U-Net) deep learning segmentation algorithm is introduced to accurately distinguish the rock skeleton, pore space and fractures in deep rock. The three-dimensional (3-D) digital rock model of deformed multi-scale fracture-porous media under different dynamic loading conditions is ultimately established to investigate the evolution of fracture morphology and deep rock microstructure as the in-situ stress is gradually loaded. It indicates that, the U-Net deep learning semantic segmentation algorithm can accurately segment CT images of deep rocks, and the established 3D digital rock model of fractured porous media can accurately represent the pore-throat distribution, fracture morphology, and pore-scale topological connectivity. At the initial stage of stress loading, there are few micro-fractures inside the rock, and the fracture connectivity is poor. Due to effect of rock compaction, the average pore throat radius increases while pore throat ratio, coordination number and tortuosity greatly decrease. As the effective stress is gradually loaded, the micro-fractures begin to propagate, leading to stronger heterogeneity of micro-fractures’ distribution and better topological connectivity, and both the coordination number and tortuosity gradually increase. After the deep rock is fractured, micro-fractures run through a fracture network and all the above topological parameters increase greatly.

Advancements in the use of AI in the diagnosis and management of inflammatory bowel disease

Inflammatory bowel disease (IBD) causes chronic inflammation of the colon and digestive tract, and it can be classified as Crohn’s disease (CD) and Ulcerative colitis (UC). IBD is more prevalent in Europe and North America, however, since the beginning of the 21st century it has been increasing in South America, Asia, and Africa, leading to its consideration as a worldwide problem. Optical colonoscopy is one of the crucial tests in diagnosing and assessing the progression and prognosis of IBD, as it allows a real-time optical visualization of the colonic wall and ileum and allows for the collection of tissue samples. The accuracy of colonoscopy procedures depends on the expertise and ability of the endoscopists. Therefore, algorithms based on Deep Learning (DL) and Convolutional Neural Networks (CNN) for colonoscopy images and videos are growing in popularity, especially for the detection and classification of colorectal polyps. The performance of this system is dependent on the quality and quantity of the data used for training. There are several datasets publicly available for endoscopy images and videos, but most of them are solely specialized in polyps. The use of DL algorithms to detect IBD is still in its inception, most studies are based on assessing the severity of UC. As artificial intelligence (AI) grows in popularity there is a growing interest in the use of these algorithms for diagnosing and classifying the IBDs and managing their progression. To tackle this, more annotated colonoscopy images and videos will be required for the training of new and more reliable AI algorithms. This article discusses the current challenges in the early detection of IBD, focusing on the available AI algorithms, and databases, and the challenges ahead to improve the detection rate.

A Machine Learning Approach for Modeling the Spatial-temporal Propagation Pattern of COVID-19

Abstract. Predicting infections resulting from epidemiological contagions involves a complex therapeutic endeavor that necessitates the consideration of various factors within the available data. COVID-19, a globally recognized highly dangerous disease, presents an opportunity to effectively understand its patterns and spatial determinants to combat similar ailments. This study seeks to employ a deep neural network and several machine learning techniques to identify the crucial factors influencing COVID-19 infections and to forecast the spatial-temporal spread of the disease. Previous research has focused on predicting outcomes using a limited set of 8 patient variables. To contribute to this field, the current investigation analyzes a dataset comprising 47,029 records of infected individuals who sought medical attention, with 18,433 of them testing positive for the coronavirus and requiring hospitalization. The study explores the prediction of illness and hospitalization needs based on geographical locations by utilizing machine learning algorithms such as Random Forest, Naive Bayes, Decision Tree, and K-Nearest Neighbor. The machine learning process commences with input data (comprising the 47,029 records), enabling the machine to identify patterns within the dataset and subsequently make informed decisions based on these patterns and insights gained. The primary objective is to allow the machine to autonomously learn and refine its predictions without human intervention. Applying these algorithms to COVID-19 data for spatial-temporal prediction revealed that the Random Forest algorithm achieved the highest accuracy of 0.84, while the Simple Bayes algorithm exhibited the lowest accuracy at 0.50. Additionally, the study compared random forest, decision tree, simple bayes, and K-nearest neighbor algorithms to predict the severity of road accidents.

A comparative study of different deep learning methods for time-series probabilistic residential load power forecasting

The widespread adoption of nonlinear power electronic devices in residential settings has significantly increased the stochasticity and uncertainty of power systems. The original load power data, characterized by numerous irregular, random, and probabilistic components, adversely impacts the predictive performance of deep learning techniques, particularly neural networks. To address this challenge, this paper proposes a time-series probabilistic load power prediction technique based on the mature neural network point prediction technique, i.e., decomposing the load power data into deterministic and stochastic components. The deterministic component is predicted using deep learning neural network technology, the stochastic component is fitted with Gaussian mixture distribution model and the parameters are fitted using great expectation algorithm, after which the stochastic component prediction data is obtained using the stochastic component generation method. Using a mature neural network point prediction technique, the study evaluates six different deep learning methods to forecast residential load power. By comparing the prediction errors of these methods, the optimal model is identified, leading to a substantial improvement in prediction accuracy.

Retraction Note: A novel vessel detection and classification algorithm using a deep learning neural network model with morphological processing (M-DLNN)

Retraction Note: A novel vessel detection and classification algorithm using a deep learning neural network model with morphological processing (M-DLNN)

Machine learning-aided hybrid technique for dynamics of rail transit stations classification: a case study

Accurate classification of rail transit stations is crucial for successful Transit-Oriented Development (TOD) and sustainable urban growth. This paper introduces a novel classification model integrating traditional methodologies with advanced machine learning algorithms. By employing mathematical models, clustering methods, and neural network techniques, the model enhances the precision of station classification, allowing for a refined evaluation of station attributes. A comprehensive case study on the Chengdu rail transit network validates the model’s efficacy, highlighting its value in optimizing TOD strategies and guiding decision-making processes for urban planners and policymakers. The study employs several regression models trained on existing data to generate accurate ridership forecasts, and data clustering using mathematical algorithms reveals distinct categories of stations. Evaluation metrics confirm the rationality and accuracy of the results. Additionally, a neural network achieving high accuracy on labeled data enhances the model’s predictive capabilities for unlabeled instances. The research demonstrates high accuracy, with the Mean Squared Error (MSE) for regression models (Multiple Linear Regression (MLR), Deep-Learning Neural Network (DNN), and K-Nearest Neighbor (KNN)) remaining below 0.012, while the neural networks used for station classification achieve 100% accuracy across seven time intervals and 98.15% accuracy for the eighth, ensuring reliable ridership forecasts and classification outcomes. Accuracy in rail transit station classification is critical, as it not only strengthens the model’s predictive capabilities but also ensures more reliable data-driven decisions for transit planning and development, allowing for more precise ridership forecasts and evidence-based strategies for optimizing TOD. This classification model provides stakeholders with valuable insights into the dynamics and features of rail transit stations, supporting sustainable urban development planning.

Multi-branch deep learning neural network prediction model for the development of angular biosensors based on sEMG.

Human gait motion intention recognition is very important for the lower extremity exoskeleton robot to accurately synchronize and respond to the user's natural motion. And motion intention recognition is generally performed through sEMG. Deep learning neural networks perform well in dealing with high-dimensional data and nonlinear relationships such as sEMG, but different deep learning neural networks have their own advantages in dealing with different types of data. Therefore, a multi-branch deep learning neural network, which enables different neural networks to process different feature items, could achieve more accurate and efficient motion intention recognition. The purpose of this study is to 1) Establish a multi-branch deep learning neural network model to achieve accurate gait recognition and effective estimation of joint angles. 2) Quantify the performance of the multi-branch deep learning neural network model in gait recognition and joint angle prediction using sEMG. This study involved the collection of sEMG and plantar pressure data during walking in human subjects. Firstly, the collected signals are filtered and denoised to ensure the quality and reliability of the data. Calculate the time domain features and the frequency domain features to capture the key information of gait. Then, using the sensitivity difference of different structural neural networks to different feature data, a multi-branch deep learning neural network model is developed, in which the extracted features are used as the input of the model. The output of the model includes gait cycle and joint angle, so as to realize the accurate recognition of human gait and the effective estimation of joint angle. The results show that the proposed method has high accuracy in identifying human gait and estimating joint angles. The multi-branch neural network model successfully integrates time-domain and frequency-domain features and provides reliable prediction of gait cycle and joint angle. The highest accuracy of gait recognition is 95.42%, the lowest is 90.11%, and the average is 92.16%. The average error of joint angle estimation is 3.19. This study designed a human walking gait recognition and joint angle prediction model to achieve accurate human lower limb motion intention recognition.The model can be integrated into the sEMG sensor to design a angular biosensors, which can predict the human joint angle in real time.

Three-dimensional convolutional neural network for leak detection and localization in smart water distribution systems

Smart meters such as advanced metering infrastructure (AMI) can significantly improve identifying realistic sized leaks in water distribution networks (WDNs). However, to date, detection/localization methods for AMI systems are extremely limited. In this study, to examine the benefits of using AMIs for leak detection within distribution network, a three-dimensional (3D) convolutional neural network (CNN) deep learning (DL) model is proposed that can account for temporally and spatially distributed information of pressures. The 3D CNN is tested for a real WDN in Austin using the realistic sized leaks (e.g., 3 L/s for 150-mm pipes) that are generated from hydraulic simulations. The model's performance is evaluated using detection probability, false alarm rate, and localization pipe distance metrics. In addition, the strength of using DL for leak identification is examined by comparing the CNN results with those from an optimization-based model. The 3D CNN performed better than the optimization model indicating that DL has advantages over conventional tools such as optimization methods. However, its adaptability may limit its use in some cases. Since DL can be significantly impacted by hydraulic simulation model, a way to handle modelling error must be determined. In addition, as network changes occur, retraining is required that may be time consuming and have difficulty with the number of failure conditions.

Bio-inspired feature selection for early diagnosis of Parkinson's disease through optimization of deep 3D nested learning.

Parkinson's disease (PD) is one of the most common neurodegenerative disorders that affect the quality of human life of millions of people throughout the world. The probability of getting affected by this disease increases with age, and it is common among the elderly population. Early detection can help in initiating medications at an earlier stage. It can significantly slow down the progression of this disease, assisting the patient to maintain a good quality of life for a more extended period. Magnetic resonance imaging (MRI)-based brain imaging is an area of active research that is used to diagnose PD disease early and to understand the key biomarkers. The prior research investigations using MRI data mainly focus on volume, structural, and morphological changes in the basal ganglia (BG) region for diagnosing PD. Recently, researchers have emphasized the significance of studying other areas of the human brain for a more comprehensive understanding of PD and also to analyze changes happening in brain tissue. Thus, to perform accurate diagnosis and treatment planning for early identification of PD, this work focuses on learning the onset of PD from images taken from whole-brain MRI using a novel 3D-convolutional neural network (3D-CNN) deep learning architecture. The conventional 3D-Resent deep learning model, after various hyper-parameter tuning and architectural changes, has achieved an accuracy of 90%. In this work, a novel 3D-CNN architecture was developed, and after several ablation studies, the model yielded results with an improved accuracy of 93.4%. Combining features from the 3D-CNN and 3D ResNet models using Canonical Correlation Analysis (CCA) resulted in 95% accuracy. For further enhancements of the model performance, feature fusion with optimization was employed, utilizing various optimization techniques. Whale optimization based on a biologically inspired approach was selected on the basis of a convergence diagram. The performance of this approach is compared to other methods and has given an accuracy of 97%. This work represents a critical advancement in improving PD diagnosis techniques and emphasizing the importance of deep nested 3D learning and bio-inspired feature selection.

Individual Buffalo Identification Through Muzzle Dermatoglyphics Images using Deep Learning Approaches

Animal identification is essential for routine farm operations, residue traceback, insurance, and ownership management. Owing to their uniqueness, incorrigible nature, tamperproof over time, environment-friendly, and pain-free, visual biometrics-based animal identification has recently gained momentum over traditional animal identification methods. Among visual biometrics-based cues, muzzle identification is a simple and relatively low-cost method. Therefore, to address the inherent significant limitations of conventional animal identification systems, we undertook this investigation to collect a database of digital images of muzzles that works as a benchmark, to apply deep learning frameworks to identify individual buffaloes from their muzzle images, and to compare their accuracy in terms of their identification capabilities. Muzzle images of 198 Surti buffaloes were subjected to transfer learning and fine-tuning processes in deep-learning neural networks. The performance was recorded for each pre-train model (ResNet50, InceptionV3, VGG16, AlexNet) with different hyperparameters of the epoch, batch size, and learning rate. A perusal of the data revealed that ResNet50 has the highest train accuracy (99.8%) and test accuracy (99.69%) among all four models used. AlexNet has the lowest train accuracy (90.8%) among the models. The findings concluded that all these four models could be applied to identify individual buffaloes; however, ResNet50 had the highest accuracy, and deep learning applications have great potential for individual buffalo identification and are promising tools for precision livestock farming

Advanced deep learning strategies for breast cancer image analysis

One of the leading causes of cancer-related deaths in women is breast cancer (BC). Patients' chances of survival are improved when this ailment is diagnosed in a timely manner and suitable treatments are prescribed. Thus, the chance of survival increases with the early detection of BC. Deep learning neural networks have garnered significant attention in recent years for use in BC screening, identification, and classification. Even though some encouraging results have surfaced, more improvement and confirmation are necessary. In this regard, the creation and comparison of many deep learning approaches for the early identification and classification of BC from mammography pictures is the main focus of our research. In this study, Innovative deep learning methods are developed by freezing the first 40 layers an dropping the 43 layers of MobileNetV2, InceptionV3 and DenseNet121 pretrained models, this approach was reinforced by using Data augmentation techniques. Our suggested methodologies are tested through simulations on MIAS dataset and SA private dataset. The three modified models achieved very high classification accuracies, specifically the modified DenseNet121 model, which reached 99,1% on MIAS dataset and 98,8% on SA dataset.

CNN for scalar-source distance estimation in grid-generated turbulence

Countermeasures against material leakage in industrial plants call for rapid leak source estimation techniques based on limited downstream information. Assuming that such scalar diffusion occurs in a turbulent environment, it is necessary to extract features from its instantaneous local concentration distribution and estimate the distance from the observation point to the source. To this end, we proposed an estimation method using a supervised deep learning of a convolutional neural network (CNN) and investigated its estimation performance in grid-generated turbulence fields. The grid turbulence fields were reproduced by direct numerical simulations at three Reynolds numbers (Re), and the CNN was trained using two-dimensional distribution images of passive scalars downstream. We confirmed that the images large enough to cover the spatial integral length scales of the scalar resulted in high estimation accuracy. For images with a coarsened brightness gradation, the small-scale feature of scalar fluctuations was lost, and the CNN attained a low estimation accuracy. By training the CNN with high-Re images, a high estimation accuracy was obtained, even for low-Re images. The opposite case (low-Re training and high-Re estimation) yielded low accuracy, where the CNN significantly underestimated the distance against the source. Based on these results, we discussed the generalizability and essential features to be learned by the CNN.

LungHist700: A dataset of histological images for deep learning in pulmonary pathology

Accurate detection and classification of lung malignancies are crucial for early diagnosis, treatment planning, and patient prognosis. Conventional histopathological analysis is time-consuming, limiting its clinical applicability. To address this, we present a dataset of 691 high-resolution (1200 × 1600 pixels) histopathological lung images, covering adenocarcinomas, squamous cell carcinomas, and normal tissues from 45 patients. These images are subdivided into three differentiation levels for both pathological types: well, moderately, and poorly differentiated, resulting in seven classes for classification. The dataset includes images at 20x and 40x magnification, reflecting real clinical diversity. We evaluated image classification using deep neural network and multiple instance learning approaches. Each method was used to classify images at 20x and 40x magnification into three superclasses. We achieved accuracies between 81% and 92%, depending on the method and resolution, demonstrating the dataset’s utility.

GREEN SYNTHESIS OF CaO–PdNps FROM LAWSONIA INERMIS LEAF EXTRACT: CATALYTIC REDUCTION OF METHYLENE BLUE WITH CENTRAL COMPOSITE DESIGN-ENHANCED NEURAL NETWORK PREDICTION

In this work, small-sized CaO–PdNps nanocomposites were synthesized using Lawsonia inermis leaf extract as a reducing and capping agent. X-ray diffraction (XRD), transmission electron microscopy (TEM), scanning electron microscopy (SEM), and Fourier transform infrared spectroscopy (FTIR) were used to analyze the produced nanocomposites. Based on these investigations, the average particle size was found to be [Formula: see text][Formula: see text]nm, with a predominantly spherical morphology. The resulting nanocomposite was then tested for catalytic activity in methylene blue reduction. The mass of the catalyst and dye, the concentration of sodium borohydride ([NaBH4]), and the reaction time were all optimized using a central composite design (CCD). A significant correlation between predicted and experimental degradation percentages was shown by the CCD model ([Formula: see text]), which also proved statistically significant (p-[Formula: see text]). Furthermore, a deep learning neural network (DLNN) was used to improve the prediction accuracy over and beyond CCD. There was a good connection between the two models as a result of using the CCD output as DLNN validation data. Mean absolute error (MAE), mean squared error (MSE), root mean squared error (RMSE), mean absolute percentage error (MAPE), and [Formula: see text] were used to assess the performance of the DLNN. The results ([Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text]) validated that all four metrics were successful in forecasting the percentage of degradation.

Inverse design of acoustic metamaterial-based vibration control for offshore wind turbines

This paper presents an inverse design approach for mechanical metamaterial turbines utilising a deep learning algorithm. We first establish a theoretical model for metamaterial turbines using the spectral element method to analyse their vibration characteristics. Subsequently, we predict the vibration transmission properties of these turbines and achieve on-demand inverse design of metastructures through a fully connected deep-learning neural network model. The efficacy of the forward prediction and reverse design network models is validated using an inverse neural network. Our results demonstrate a strong agreement between the predicted and target values, affirming the feasibility of employing deep learning for on-demand meta-turbine design. This intelligent design approach holds promise for expediting and enhancing the design of metamaterials tailored for low-frequency vibration isolation in engineering structures.

External validation of an artificial intelligence multi-label deep learning model capable of ankle fracture classification

BackgroundAdvances in medical imaging have made it possible to classify ankle fractures using Artificial Intelligence (AI). Recent studies have demonstrated good internal validity for machine learning algorithms using the AO/OTA 2018 classification. This study aimed to externally validate one such model for ankle fracture classification and ways to improve external validity.MethodsIn this retrospective observation study, we trained a deep-learning neural network (7,500 ankle studies) to classify traumatic malleolar fractures according to the AO/OTA classification. Our internal validation dataset (IVD) contained 409 studies collected from Danderyd Hospital in Stockholm, Sweden, between 2002 and 2016. The external validation dataset (EVD) contained 399 studies collected from Flinders Medical Centre, Adelaide, Australia, between 2016 and 2020. Our primary outcome measures were the area under the receiver operating characteristic (AUC) and the area under the precision-recall curve (AUPR) for fracture classification of AO/OTA malleolar (44) fractures. Secondary outcomes were performance on other fractures visible on ankle radiographs and inter-observer reliability of reviewers.ResultsCompared to the weighted mean AUC (wAUC) 0.86 (95%CI 0.82–0.89) for fracture detection in the EVD, the network attained wAUC 0.95 (95%CI 0.94–0.97) for the IVD. The area under the precision-recall curve (AUPR) was 0.93 vs. 0.96. The wAUC for individual outcomes (type 44A-C, group 44A1-C3, and subgroup 44A1.1-C3.3) was 0.82 for the EVD and 0.93 for the IVD. The weighted mean AUPR (wAUPR) was 0.59 vs 0.63. Throughout, the performance was superior to that of a random classifier for the EVD.ConclusionAlthough the two datasets had considerable differences, the model transferred well to the EVD and the alternative clinical scenario it represents. The direct clinical implications of this study are that algorithms developed elsewhere need local validation and that discrepancies can be rectified using targeted training. In a wider sense, we believe this opens up possibilities for building advanced treatment recommendations based on exact fracture types that are more objective than current clinical decisions, often influenced by who is present during rounds.

Comparison of DNA methylation based classification models for precision diagnostics of central nervous system tumors

As part of the advancement in therapeutic decision-making for brain tumor patients at St. Jude Children’s Research Hospital (SJCRH), we developed three robust classifiers, a deep learning neural network (NN), k-nearest neighbor (kNN), and random forest (RF), trained on a reference series DNA-methylation profiles to classify central nervous system (CNS) tumor types. The models’ performance was rigorously validated against 2054 samples from two independent cohorts. In addition to classic metrics of model performance, we compared the robustness of the three models to reduced tumor purity, a critical consideration in the clinical utility of such classifiers. Our findings revealed that the NN model exhibited the highest accuracy and maintained a balance between precision and recall. The NN model was the most resistant to drops in performance associated with a reduction in tumor purity, showing good performance until the purity fell below 50%. Through rigorous validation, our study emphasizes the potential of DNA-methylation-based deep learning methods to improve precision medicine for brain tumor classification in the clinical setting.

Enhancing temperature and torque prediction in permanent magnet synchronous motors using deep learning neural networks and BiLSTM RNNs

This study aims to develop an effective method for predicting the temperature and torque of Permanent Magnet Synchronous Motors (PMSMs) using deep learning techniques, which is crucial for optimizing motor performance and ensuring longevity, particularly in the automotive industry. Various Neural Network (NN) architectures, including a Recurrent Neural Network (RNN) with a Bidirectional Long Short-Term Memory (BiLSTM) unit, were employed to model the complex relationships between motor parameters, such as stator winding, current, torque, and permanent magnet temperature. The findings demonstrate that an NN with two hidden layers (64 and 32 neurons) achieved an R2 score of 0.99 for both torque and temperature prediction, while the BiLSTM network effectively modeled temporal dynamics, leading to high-fidelity rotor temperature predictions. This research provides a novel application of BiLSTM RNNs in accurately predicting PMSM temperatures, offering valuable insights for industries reliant on these motors. Integrating these models into motor control systems can enhance operational efficiency, reduce overheating risks, and extend motor lifespan, contributing to energy savings and environmental sustainability by lowering energy consumption and reducing waste.

A Method for Forecasting Geomagnetic Storms Based on Deep Learning Neural Networks Using Time Series of Matrix Observations of the Uragan Muon Hodoscope

A Method for Forecasting Geomagnetic Storms Based on Deep Learning Neural Networks Using Time Series of Matrix Observations of the Uragan Muon Hodoscope