Deep Artificial Neural Networks Research Articles

Utilizing artificial neural network system to predict the residual valve after endoscopic posterior urethral valve ablation.

To build, train, and assess the artificial neural network (ANN) system in estimatingthe residual valve rate after endoscopic valve ablationand compare the data obtained with conventional analysis. In a retrospective cross-sectional study between June 2010 and December 2020, 144 children with a history of posteriorurethral valve (PUV) who underwent endoscopic valve ablation were enrolled in the study. MATLAB software was used to design and train the network in a feed-forward backpropagationerror adjustment scheme. Preoperative and postoperative data from 101 patients (70%) (training set) were utilized to assessthe impact and relative significance of the necessity for repeatedablation. The validated suitably trained ANN was used to predict repeated ablation in the next 33 patients (22.9%) (test set)whose preoperative data were serially input into the system. To assess system accuracy in forecasting the requirement for repeatablation, projected values were compared to actual outcomes. The likelihood of predicting the residual valve was calculatedusing a three-layered backpropagating deep ANN using preoperativeand postoperative information. Of 144 operated cases, 33 (22.9%) had residual valvesand needs to repeated ablation. The ANN accuracy, sensitivity, and specificity for predicting the residual valve were 90.75%,92.73%, and 73.19%, respectively. Younger age at surgery, hyperechogenicity of the renal parenchyma, presence of vesicoureteralreflux (VUR), and grade of reflux before surgery were among the most significant characteristics that affectedpostoperative outcome variables, the need for repeated ablation, and were given the highest relative weight by the ANN system. Conclusions: The ANN is an integrated data-gathering tool for analyzing and finding relationships among variables as a complex non-linear statistical model. The results indicate that ANN is a valuable tool for outcome prediction of the residualvalveafter endoscopic valve ablation in patients with PUV.

A deep learning deviation-based scheme to defend against false data injection attacks in power distribution systems

Defending against false data injection attacks (FDIAs) in cyber-physical power systems is crucial. Detection in power distribution systems is complex due to load variations, uncertainties, and fewer meters. Defense strategies include model-driven and data-driven approaches, but model-based methods can trigger false alarms due to threshold setting issues. The current research proposes a novel data-driven method to address threshold setting issues in detecting and localizing FDIAs in power distribution systems. First, a dataset is created by recording estimated measurement values using an unscented Kalman filter and weighted least squares across various attack scenarios. These estimated measurements are then fed into a deep artificial neural network (ANN) for binary classification to detect attacks. The output, along with the estimated measurements, is used by another ANN to localize the corrupted meter zone. This deep learning-based approach improves threshold setting over the common chi-square method. Results show that the proposed deep learning method for FDIA detection and localization outperforms a recently proposed ensemble of shallow models. The area under the curve value increases by about 5% with lower training time. The approach is also effective against previously unseen attack strategies and different feeder topologies.

Forecasting of Turkey's Hazelnut Export Amounts According to Seasons with Dendritic Neuron Model Artificial Neural Network

It is seen that artificial neural networks have begun to be used extensively in the literature in solving the time series forecasting problem. In addition to artificial neural networks, classical forecasting methods can often be used to solve this problem. It is seen that classical forecasting methods give successful results for linear time series analysis. However, there is no linear relationship in many time series. Therefore, it can be thought that deep artificial neural networks, which contain more parameters but create more flexible non-linear model structures compared to classical time series forecasting methods, may enable the production of more successful forecasting methods. In this study, the problem of forecasting hazelnut export amounts according to seasons in Turkey with a dendritic neuron model artificial neural network is discussed. In this study, a training algorithm based on the particle swarm optimization algorithm is given for training the dendritic neuron model artificial neural network. The motivation of the study is to investigate Turkey's hazelnut export amounts according to seasons, using a dendritic neuron model artificial neural network. The performance of the proposed method has been compared with artificial neural networks used in the literature.

Machine learning optimization and 4E analysis of a CCHP system integrated into a greenhouse system for carbon dioxide capturing

The world has seen an increase in the popularity of renewable-based energy systems. However, because of their erratic nature, fossil fuels cannot be entirely phased out. Utilizing carbon dioxide in greenhouses close to power plants and generating extra power electricity from dissipated thermal energy are appealing strategies that cause us to have a cleaner environment and affordable power electricity. To exploit the high enthalpy of expelled gases from a gas turbine, the gas turbine was coupled with an absorption refrigerant cycle and an organic Rankine cycle and analyzed. Additionally, a deep artificial neural network method was used to calculate all functions in a rapid optimization and accurate 4E analysis. The carbon dioxide emissions subsided from 0.9183 to 0.41 kg CO2/s, and the optimization time improved 257 times after using the trained machine learning model. An adsorption technique for CO2 separation is used for the proposed system because of its lower energy consumption. The maximum exergy and energy efficiencies were attained at 36.71 % and 49.2 %, respectively, and parametric studies are being assessed. Due to increases in harvests and efficiencies, the net annual interest has increased by 21.8 %, up to 22.5 million dollars.

Digitalization of Analysis of a Concrete Block Layer Using Machine Learning as a Sustainable Approach

The concrete block pavement (CBP) system has a surface layer consisting of concrete block pavers and joint sand over a bedding sand layer. The non-homogeneous nature of the surface course of CBP, along with different laying patterns and shapes of block pavers, makes the analysis of CBP cumbersome. In this study, the surface course of CBP was modeled based on the slab action of the block pavers and joint sand, which are connected together in full contact. Four different laying patterns, including herringbone, stretcher, parquet, and square, were modeled using a finite element model. The elastic moduli of the block pavers varied from 2500 MPa to 45,000 MPa, with thicknesses ranging from 60 mm to 120 mm. As a result, modeling of CBP based on slab action can be considered a realistic strategy. In addition, a dataset was created based on quantitative inputs, e.g., elastic modulus and thickness of the block pavers, and qualitative input, i.e., block laying patterns. The approaches of machine learning adopted were support vector regression, Gaussian process regression, single-layer and deep artificial neural networks, and least squares boosting to implement prediction approach based on input and output. The analyses of statistical accuracy of all five machine learning methods showed high accuracy; however, the Gaussian process and deep artificial neural network methods resulted in the most accurate outputs and are recommended for further studies. Based on the machine learning models, digitalization is achieved through the development of simple, user-friendly software for electronic devices in order to perform a preliminary analysis of different laying patterns of CBP. Such a platform may result in less laboratory work and boosts the level of sustainability in concrete block pavement technology.

Predicting sessile droplet evaporation kinetics via cascaded deep networks and tree-based machine learning approach

The intricate nature of sessile droplet evaporation phenomenon makes detailed experimental studies time consuming and requires sophisticated apparatus; while complete numerical simulations are computationally expensive and also time-intensive. In this article, for the first time, we explore the applicability of machine learning (ML) approaches to predict the evaporation kinetics of generalized sessile droplets under various conditions. An in-house dataset, obtained using an experimentally well validated numerical model, is used to develop the ML models: deep artificial neural network (ANN) and decision tree algorithms such as Random Forest (RF) and extreme gradient boosting (XGB). The structures of these models are modified by cascading the output features according to the physics involved. This distinctive approach results in better prediction of the evaporation kinetics than basic ML models. The models are trained by a large set of input parameters for the target variables: viz. droplet evaporation rate, velocity scale, and temperature drop in the solid and liquid domain. Finally, the performance of these ML models is assessed by comparing their predictions with that of physics-based, experimentally validated, numerical models. Results show that the inclusion of additional features obtained using feature engineering significantly improve the prediction performance of ML algorithms, and consistently accurate predictions of droplet evaporation kinetics are obtained. Among various algorithms considered here, the ANN outperforms in term of various error matrices for most of the cases, followed by XGB, and RF models. Also, the highest mean average error (MAE) yield by the ML models for evaporation rate, velocity scale and temperature drop in liquid remains within ∼12.5%. In the case of temperature drop for the solid, the MAE is considerably higher due to large variability of the same target variable. Overall, the work clearly shows that ML algorithms can be used to obtain physically consistent predictions for sessile droplet evaporation parameters.

The development of machine learning approaches in two-dimensional NMR data interpretation for metabolomics applications

Metabolomics has been widely applied in human diseases and environmental science to study the systematic changes of metabolites over diverse types of stimuli. NMR-based metabolomics has been widely used, but the peak overlap problems in the one-dimensional (1D) NMR spectrum could limit the accuracy of quantitative analysis for metabolomics applications. Two-dimensional (2D) NMR has been applied to solve the 1D NMR overlap problem, but the data processing is still challenging. In this study, we built an automatic approach to process the 2D NMR data for quantitative applications using machine learning approaches. Partial least square discriminant analysis (PLS-DA), artificial neural network classification (ANN-DA), gradient boosted trees classification (XGBoost-DA), and artificial deep learning neural network classification (ANNDL-DA) were applied in combination with an automatic peak selection approach. Standard mixtures, sea anemone extracts, and mouse fecal samples were tested to demonstrate the approach. Our results showed that ANN-DA and ANNDL-DA have high accuracy in selecting 2D NMR peaks (around 90 %), which have a high potential application in 2D NMR-based metabolomics quantitively study, while PLS-DA and XGBoost-DA showed limitations in either data variation or overfitting. Our study built an automatic approach to applying 2D NMR data to routine quantitative analysis in metabolomics.

A hybrid deep recurrent artificial neural network with a simple exponential smoothing feedback mechanism

Both classical forecasting methods and machine learning approaches are used to solve forecasting problems. Deep artificial neural networks, one of the machine learning methods, are widely used today and give very good results. Recurrent neural networks, a type of deep neural network, are very important in forecasting problems. Simple recurrent artificial neural networks, which are the simplest deep recurrent neural networks, are often preferred in solving forecasting problems due to the small number of parameters they use. Simple exponential smoothing, one of the classical forecasting methods, attracts attention with its performance in solving forecasting problems. The motivation of the study is to create a new forecasting method by combining a classical and simple forecasting method with a deep recurrent artificial neural network in an architecture. In this, a new hybrid deep recurrent artificial neural network with a simple exponential smoothing feedback mechanism is proposed. The architecture of the proposed method is created as a combination of simple recurrent artificial neural networks and simple exponential smoothing methods. In the training of the proposed method, two training algorithms based on sine cosine optimization and particle swarm optimization algorithms are proposed. In these training algorithms, two different solution strategies such as restarting, and early stopping rule are used to avoid overfitting and local optimum problems. The performance of the proposed method is analysed using stock market datasets and compared with both different deep and shallow artificial neural networks and classical forecasting methods. As a result of the analyses, it is concluded that the proposed method is successful in one step ahead of forecasting performance.

Thy-DAMP: deep artificial neural network model for prediction of thyroid cancer mortality.

Despite the rising incidence of differentiated thyroid cancer (DTC), mortality rates have remained relatively low yet crucial for effective patient management. This study aims to develop a deep neural network capable of predicting mortality in patients with differentiated thyroid cancer. Leveraging data from the Surveillance, Epidemiology, and End Results (SEER) database, we developed Thy-DAMP (Deep Artificial Neural Network Model for Prediction of Thyroid Cancer) to forecast mortality in DTC patients. The dataset comprised demographic, histologic, and staging information. Following data normalization and feature encoding, the dataset was partitioned into training, testing, and validation subsets, with model hyperparameters fine-tuned via cross-validation. Among the 63,513 patients, the mean age was 48.22 years (SD = 14.8), with 77.32% being female. Papillary carcinoma emerged as the predominant subtype, representing 62.94% of cases. The majority presented with stage I disease (73.96%). Thy-DAMP demonstrated acceptable performance metrics on both the test and validation sets. Sensitivity was 83.24% (95% CI 76.95-88.40%), specificity was 93.53% (95% CI 93.01-94.02%), and accuracy stood at 93.33% (95% CI 92.82-93.83%). The model exhibited a positive predictive value of 19.76% (95% CI 18.20-21.42%) and a negative predictive value of 99.66% (95% CI 99.53-99.75%). Additionally, Thy-DAMP demonstrated a positive likelihood ratio of 12.86 (95% CI 11.62-14.23), a negative likelihood ratio of 0.18 (95% CI 0.13-0.25), and an area under the receiver operating characteristic curve (AUROC) of 0.95. The model was externally validated on a separate dataset with nearly identical performance. Thy-DAMP showcases considerable promise in accurately predicting mortality in DTC patients, leveraging limited set of patient data.

Deep learning of plausible bandgaps in dispersion curves of phononic crystals

Phononic crystals represent an interesting class of metamaterials that can be utilized to regulate or manipulate vibration, sound propagation, and thermal transport. Their useful features mainly arise from the bandgaps in their dispersion curves, preventing the passage of waves within specific frequency ranges. However, it is often costly and time-consuming to obtain the dispersion curves, and the reverse engineering of phononic crystals to have pre-defined bandgaps possesses even greater challenges. In this research, we address this issue by employing a deep artificial neural network to predict the bandgap ratio and the characteristics of plausible bandgaps, focusing on the localized resonance in columnar phononic crystals. We utilized two geometric parameters, i. e. the ratio of diameter and height of the cylindrical resonators relative to the lattice constant, achieving a determination coefficient of 0.9993 for predicting the characteristics of the bandgaps and 0.9827 for predicting the bandgap ratio. To verify the model and better understand its behavior, we introduce Shapley values. These values provide a comprehensive insight into how each geometric parameter influences the predicted bandgap ratios.

Optimising shapes of multiple pin fins in a microchannel using deep reinforcement learning and mesh deformation techniques

The utilisation of advanced pin fin designs in microchannels is useful for enhancing cooling efficiency. Advancements in machine learning and processing power have sparked interest in shape optimisation techniques. This research employs a novel framework that integrates Deep Artificial Neural Networks and Reinforcement Learning with a Computational Fluid Dynamics (CFD) solver to optimise multiple pin fin shapes within a microchannel. By incorporating Radial Basis Function interpolation and Proximal Policy Optimisation alongside FLUENT, acting as the CFD environment, the reinforcement learning agent adeptly explores the design space to enhance the thermohydraulic performance factor (TPF), aiming to maximise Nusselt number while minimising pressure loss. Unlike previous heat transfer optimisation studies, which typically required mesh regeneration at each step, the proposed framework could bypass the meshing step and alter the geometry directly by relying on the RBF interpolation technique to deform the mesh directly. Three distinct scenarios investigated in this study are uniform deformation of all pin fins, deformation of all pin fins arranged in two rows, and individual deformation of each pin fin. Extensive simulations, exceeding 90,000 different cases, demonstrate that although the optimisation process for the individual deformation of each pin fin requires more iterations compared to others, it surpasses them in terms of TPF improvement. Notably, significant improvements are achieved, such as a 49 % enhancement in Nusselt number and a 33 % reduction in pressure drop, culminating in an impressive 63 % increase in TPF compared to the initial geometry.

Using Machine Learning Algorithms to Develop a Predictive Model for Computing the Maximum Deflection of Horizontally Curved Steel I-Beams

Horizontally curved steel I-beams exhibit a complicated mechanical response as they experience a combination of bending, shear, and torsion, which varies based on the geometry of the beam at hand. The behaviour of these beams is therefore quite difficult to predict, as they can fail due to either flexure, shear, torsion, lateral torsional buckling, or a combination of these types of failure. This therefore necessitates the usage of complicated nonlinear analyses in order to accurately model their behaviour. Currently, little guidance is provided by international design standards in consideration of the serviceability limit states of horizontally curved steel I-beams. In this research, an experimentally validated dataset was created and was used to train numerous machine learning (ML) algorithms for predicting the midspan deflection at failure as well as the failure load of numerous horizontally curved steel I-beams. According to the experimental and numerical investigation, the deep artificial neural network model was found to be the most accurate when used to predict the validation dataset, where a mean absolute error of 6.4 mm (16.20%) was observed. This accuracy far surpassed that of Castigliano’s second theorem, where the mean absolute error was found to be equal to 49.84 mm (126%). The deep artificial neural network was also capable of estimating the failure load with a mean absolute error of 30.43 kN (22.42%). This predictive model, which is the first of its kind in the international literature, can be used by professional engineers for the design of curved steel I-beams since it is currently the most accurate model ever developed.

Improving deep neural network random initialization through neuronal rewiring

The deep learning literature is continuously updated with new architectures and training techniques. However, weight initialization is overlooked by most recent research, despite some intriguing findings regarding random weights. On the other hand, recent works have been approaching Network Science to understand the structure and dynamics of Artificial Neural Networks (ANNs) after training. Therefore, in this work, we analyze the centrality of neurons in randomly initialized networks. We show that a higher neuronal strength variance may decrease performance, while a lower neuronal strength variance usually improves it. A new method is then proposed to rewire neuronal connections according to a preferential attachment (PA) rule based on their strength, which significantly reduces the strength variance of layers initialized by common methods. In this sense, PA rewiring only reorganizes connections, while preserving the magnitude and distribution of the weights. We show through an extensive statistical analysis on image classification tasks that performance is improved in most cases, both during training and testing, when using both simple and complex architectures and learning schedules. Our results show that, aside from the magnitude, the organization of the weights is also relevant for better initialization of deep ANNs.

Osteoporotic vertebral compression fracture (OVCF) detection using artificial neural networks model based on the AO spine-DGOU osteoporotic fracture classification system

BackgroundOsteoporotic Vertebral Compression Fracture (OVCF) substantially reduces a person's health-related quality of life. Computer Tomography (CT) scan is currently the standard for diagnosis of OVCF. The aim of this paper was to evaluate the OVCF detection potential of artificial neural networks (ANN). MethodsModels of artificial intelligence based on deep learning hold promise for quickly and automatically identifying and visualizing OVCF. This study investigated the detection, classification, and grading of OVCF using deep artificial neural networks (ANN). Techniques: Annotation techniques were used to segregate the sagittal images of 1,050 OVCF CT pictures with symptomatic low back pain into 934 CT images for a training dataset (89%) and 116 CT images for a test dataset (11%). A radiologist tagged, cleaned, and annotated the training dataset. Disc deterioration was assessed in all lumbar discs using the AO Spine-DGOU Osteoporotic Fracture Classification System. The detection and grading of OVCF were trained using the deep learning ANN model. By putting an automatic model to the test for dataset grading, the outcomes of the ANN model training were confirmed. ResultsThe sagittal lumbar CT training dataset included 5,010 OVCF from OF1, 1942 from OF2, 522 from OF3, 336 from OF4, and none from OF5. With overall 96.04% accuracy, the deep ANN model was able to identify and categorize lumbar OVCF. ConclusionsThe ANN model offers a rapid and effective way to classify lumbar OVCF by automatically and consistently evaluating routine CT scans using AO Spine-DGOU osteoporotic fracture classification system.

A Deep Artificial Neural Network Model for Predicting the Mechanical Behavior of Triply Periodic Minimal Surfaces under Damage Loading

A Deep Artificial Neural Network Model for Predicting the Mechanical Behavior of Triply Periodic Minimal Surfaces under Damage Loading

Enhancing Solar Forecasting Accuracy with Sequential Deep Artificial Neural Network and Hybrid Random Forest and Gradient Boosting Models across Varied Terrains (Adv. Theory Simul. 7/2024)

Enhancing Solar Forecasting Accuracy with Sequential Deep Artificial Neural Network and Hybrid Random Forest and Gradient Boosting Models across Varied Terrains (Adv. Theory Simul. 7/2024)

A Baseline Electroencephalography Motor Imagery Brain-Computer Interface System Using Artificial Intelligence and Deep Learning

This paper presents a baseline or reference (single channel, single subject, single trial) electroencephalography (EEG) motor imagery (MI) brain computer interface (BCI) that harnesses deep learning artificial neural networks (ANNs) for brainwave signal classification. The EEG electrode or sensor is placed on the scalp within the frontal lobe of the right hemisphere of the brain and approximately above the motor cortex. Signal classification discriminates among three MI classes, namely, right first closed event, neutral event and left first closed event and the measured accuracy of the deep learning ANN was 83% which significantly outperforms chance classification. The effectiveness of the system is demonstrated by applying it to the navigation of a virtual environment, specifically, immersive 360-degree panoramas in equirectangular projection.

Алгоритм стеганографической защиты информации в видеофайлах на основе диффузионно-вероятностной модели с шумоподавлением

The results of a study are presented, the purpose of which was to develop a steganography algorithm for hiding text messages in video files. The algorithm is based on the use of a diffusion-probability model with noise reduction, which is implemented by a deep artificial neural network. The algorithm consists of two parts – for the parties sending and receiving the message. On the transmitting side, the following is carried out: synthesis of handwritten images of symbols (signatures) of the line of the hidden message, alignment of their frequency; applying direct diffusion to signatures, resulting in the generation of a noisy image that is deposited into a video stego container. At the receiving end, signatures are extracted from the video content, back diffusion is performed to obtain signatures of handwritten string characters, which are recognized using a convolutional neural network. The novelty of the research lies in the original developed algorithm for steganographic information protection in video files, as well as in a modified method of signature deposition based on the method of replacing the least significant bits. The method consists of bitwise embedding of bytes characterizing the pixel brightness level in the signature into the same blue brightness digits in a sequence of 8 frames of a video stego container. This method made it possible to significantly reduce the visible changes made to the video content when replacing not the least significant bits, but the middle significant bits in the stego container. This, in turn, provides greater resistance to compression attacks when transmitting information over the stegochannel. The practical significance of the research results lies in the developed software, with the help of which the algorithm for steganographic information protection in video files was tested, which showed high values of the peak signal-to-noise ratio and the index of structural similarity of images when embedding information in the middle bits of the bytes that set the brightness of the pixels of the stego container.

Representations and generalization in artificial and brain neural networks

Humans and animals excel at generalizing from limited data, a capability yet to be fully replicated in artificial intelligence. This perspective investigates generalization in biological and artificial deep neural networks (DNNs), in both in-distribution and out-of-distribution contexts. We introduce two hypotheses: First, the geometric properties of the neural manifolds associated with discrete cognitive entities, such as objects, words, and concepts, are powerful order parameters. They link the neural substrate to the generalization capabilities and provide a unified methodology bridging gaps between neuroscience, machine learning, and cognitive science. We overview recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and discuss theories connecting manifold dimension and radius to generalization capacity. Second, we suggest that the theory of learning in wide DNNs, especially in the thermodynamic limit, provides mechanistic insights into the learning processes generating desired neural representational geometries and generalization. This includes the role of weight norm regularization, network architecture, and hyper-parameters. We will explore recent advances in this theory and ongoing challenges. We also discuss the dynamics of learning and its relevance to the issue of representational drift in the brain.

Machine Learning Techniques in Enhanced Oil Recovery Screening Using Semisupervised Label Propagation

Summary Efficiently choosing the optimal enhanced oil recovery (EOR) technique is a critical requirement in reservoir engineering. Machine learning (ML) methods, with a well-established history of application, serve as a swift and dependable tool for EOR screening. In this paper, we aim to evaluate the effectiveness of various ML algorithms for EOR screening, utilizing a comprehensive database of nearly 1,000 EOR projects. This study delves into a comprehensive evaluation of regression and classification-based algorithms to develop a reliable screening system for EOR predictions and address challenges such as limited labeled data and missing values. Our analysis considered various EOR processes, including gas injection, chemical, and thermal EOR techniques. Various ML methods such as random forest (RF), K-nearest neighbors (KNN), support vector machine (SVM), shallow artificial neural networks (SANN), naive Bayes classifier (NBC), logistic regression (LR), and decision tree (DT) are applied, enabling both intermethod comparisons and evaluations against advanced methods, multiobjective deep artificial neural networks (MDANN), and multiobjective artificial neural networks (MANN). These advanced techniques provide the unique capability to concurrently address both regression and classification tasks. Considering that conventional methods can only be implemented on a single task, the RF, MANN, MDANN, and KNN algorithms demonstrated top-tier performance in our classification analysis. Regarding the regression task, KNN, RF, and MDANN displayed exceptional performance, signifying their prowess in predictive accuracy. However, MANN exhibited moderate performance in regression analysis. In addition, our study identified areas where certain algorithms, such as support vector regression (SVR), exhibited weaker performance, highlighting the importance of comprehensive model evaluation. This paper contributes novel insights into the application of ML techniques for EOR screening in the petroleum industry. By addressing challenges such as limited labeled data and missing values and by providing a thorough evaluation of various ML algorithms, our study offers valuable information for decision-makers in the oil and gas sector, aiding in the selection of suitable algorithms for EOR projects. In addition, the use of semisupervised label propagation and advanced techniques like KNN imputation adds to the existing body of literature, enhancing the state of knowledge in this domain.