Deep Artificial Neural Networks Research Articles

Robust neural networks using stochastic resonance neurons.

Various successful applications of deep artificial neural networks are effectively facilitated by the possibility to increase the number of layers and neurons in the network at the expense of the growing computational complexity. Increasing computational complexity to improve performance makes hardware implementation more difficult and directly affects both power consumption and the accumulation of signal processing latency, which are critical issues in many applications. Power consumption can be potentially reduced using analog neural networks, the performance of which, however, is limited by noise aggregation. Following the idea of physics-inspired machine learning, we propose here a type of neural network using stochastic resonances as a dynamic nonlinear node and demonstrate the possibility of considerably reducing the number of neurons required for a given prediction accuracy. We also observe that the performance of such neural networks is more robust against the impact of noise in the training data compared to conventional networks.

Quantifying uncertainty in neural network predictions of forced vibrations

AbstractThe prediction of forced vibrations in nonlinear systems is a common task in science and engineering, which can be tackled using various methodologies. A classical approach is based on solving differential (algebraic) equations derived from physical laws ('first principles'). Alternatively, Artificial Neural Networks (ANNs) may be applied, which rely on learning the dynamics of a system from given data. However, a fundamental limitation of ANNs is their lack of transparency, making it difficult to understand and trust the model's predictions. In this contribution, we follow a hybrid modelling approach combining a data‐based prediction using a stabilised Autoregressive Neural Network (s‐ARNN) with a priori knowledge from first principles. Moreover, aleatoric and epistemic uncertainty is quantified by a combination of mean‐variance estimation (MVE) and deep ensembles. Validating this approach for a classical Duffing oscillator suggests that the MVE ensemble is the most accurate and reliable method for prediction accuracy and uncertainty quantification. These findings underscore the significance of understanding uncertainties in deep ANNs and the potential of our method in improving the reliability of predictive nonlinear system modelling. We also demonstrate that including partially known dynamics can further increase accuracy, highlighting the importance of combining ANNs and physical laws.

Surrogate PHFGMC micromechanical models for multiscale analysis: AI-enhanced low-velocity impact analysis of hat-stiffened composite panels

The Parametric High-Fidelity Generalized Method of Cells (PHFGMC) has been established as an advanced micromechanical approach well-suited for analyzing the material nonlinearity and failure behavior of diverse periodic composite materials. In order to overcome the prohibitive computational cost of integrating micromechanical models into multiscale structural analyses as constitutive models, a proxy-surrogate modeling approach has been proposed by implementing a reduction modeling approach with deep Artificial Neural Networks (DNNs or ANNs). The PHFGMC-ANN approach has been employed to investigate the low velocity impact (LVI) analyses of hat-stiffened laminated composite panels under impact loading at various locations and energies with two different support conditions. Subsequent analysis of stiffened panels under compression loading has been conducted to understand the failure behavior of impacted panels. Further investigation was conducted into the separation between the skin and the stiffener, focusing on a single hat-stiffened coupon subjected to LVI. The analysis results have been compared against the experimental tests, and the comparison of interlaminar delamination has been used to demonstrate the efficacy of the new framework in integrating refined nonlinear micromechanical models within a multiscale analysis.

Multi-Timescale Energy Consumption Management in Smart Buildings Using Hybrid Deep Artificial Neural Networks

Demand side management is a critical issue in the energy sector. Recent events such as the global energy crisis, costs, the necessity to reduce greenhouse emissions, and extreme weather conditions have increased the need for energy efficiency. Thus, accurately predicting energy consumption is one of the key steps in addressing inefficiency in energy consumption and its optimization. In this regard, accurate predictions on a daily, hourly, and minute-by-minute basis would not only minimize wastage but would also help to save costs. In this article, we propose intelligent models using ensembles of convolutional neural network (CNN), long-short-term memory (LSTM), bi-directional LSTM and gated recurrent units (GRUs) neural network models for daily, hourly, and minute-by-minute predictions of energy consumptions in smart buildings. The proposed models outperform state-of-the-art deep neural network models for predicting minute-by-minute energy consumption, with a mean square error of 0.109. The evaluated hybrid models also capture more latent trends in the data than traditional single models. The results highlight the potential of using hybrid deep learning models for improved energy efficiency management in smart buildings.

A short trajectory is all you need: A transformer-based model for long-time dissipative quantum dynamics.

In this Communication, we demonstrate that a deep artificial neural network based on a transformer architecture with self-attention layers can predict the long-time population dynamics of a quantum system coupled to a dissipative environment provided that the short-time population dynamics of the system is known. The transformer neural network model developed in this work predicts the long-time dynamics of spin-boson model efficiently and very accurately across different regimes, from weak system-bath coupling to strong coupling non-Markovian regimes. Our model is more accurate than classical forecasting models, such as recurrent neural networks, and is comparable to the state-of-the-art models for simulating the dynamics of quantum dissipative systems based on kernel ridge regression.

Heart Sound Classification Using Harmonic and Percussive Spectral Features from Phonocardiograms with a Deep ANN Approach

Cardiovascular diseases (CVDs) are a leading cause of mortality worldwide, with a particularly high burden in India. Non-invasive methods like Phonocardiogram (PCG) analysis capture the acoustic activity of the heart. This holds significant potential for the early detection and diagnosis of heart conditions. However, the complexity and variability of PCG signals pose considerable challenges for accurate classification. Traditional methods of PCG signal analysis, including time-domain, frequency-domain, and time-frequency domain techniques, often fall short in capturing the intricate details necessary for reliable diagnosis. This study introduces an innovative approach that leverages harmonic–percussive source separation (HPSS) to extract distinct harmonic and percussive spectral features from PCG signals. These features are then utilized to train a deep feed-forward artificial neural network (ANN), classifying heart conditions as normal or abnormal. The methodology involves advanced digital signal processing techniques applied to PCG recordings from the PhysioNet 2016 dataset. The feature set comprises 164 attributes, including the Chroma STFT, Chroma CENS, Mel-frequency cepstral coefficients (MFCCs), and statistical features. These are refined using the ROC-AUC feature selection method to ensure optimal performance. The deep feed-forward ANN model was rigorously trained and validated on a balanced dataset. Techniques such as noise reduction and outlier detection were used to improve model training. The proposed model achieved a validation accuracy of 93.40% with sensitivity and specificity rates of 82.40% and 80.60%, respectively. These results underscore the effectiveness of harmonic-based features and the robustness of the ANN in heart sound classification. This research highlights the potential for deploying such models in non-invasive cardiac diagnostics, particularly in resource-constrained settings. It also lays the groundwork for future advancements in cardiac signal analysis.

Unified LSPG Model Reduction Framework and Assessment for Hypersonic Computational Fluid Dynamics

A unified least-squares Petrov–Galerkin (LSPG) framework for projection-based model order reduction featuring three different approximation manifolds [affine manifold, quadratic manifold, and nonlinear manifold built using a deep artificial neural network (ANN)] is presented. Its performance was assessed for a variable-speed version of the double-cone hypersonic benchmark problem. First, a high-dimensional viscous computational fluid dynamics model (HDM) was constructed, verified, and validated. The dimensionality of the HDM was then reduced using LSPG, each of the aforementioned approximation manifolds, and a global right reduced-order basis trained in the range 8≤M∞≤13. Each resulting global projection-based reduced-order model (PROM) was hyper-reduced and transformed into a hyper-reduced PROM (HPROM). The accuracy of each constructed HPROM was assessed for various quantities of interest and contrasted with that of snapshot interpolation. For this purpose, three different error measures were considered and discussed in the context of shock-dominated problems. Wall-clock and CPU time speedup factors are reported. Overall, it was shown that using a relatively small set of training data, all constructed LSPG HPROMs were nonlinearly stable, real-time capable, and highly predictive. The LSPG HPROM constructed using a nonlinear approximation manifold and an ANN was the most computationally efficient.

Predicting the risk of chronic kidney disease based on uric acid concentration in stones using biosensors integrated with a deep learning-based ANN system

Elevated levels of uric acid (UA) in the body may not only lead to the formation of stones but also increase the risk of developing chronic kidney disease (CKD). This study presents a biosensor for detecting UA concentration in stones and a deep learning-based artificial neural network (ANN) system for analyzing CKD risk. The biosensor is a screen-printed electrode (SPE) chip, whose surface was modified using oxygen plasma, enabling the detection of UA concentration via cyclic voltammetry. Experimental results show a good linear relationship between UA concentration and anodic peak current within the range of 0.15–5 mM. The surface modification method for this biosensor is simple and cost-effective. The ANN system took age and creatinine values as inputs, utilizing the Chronic_Kidney_Disease dataset and supplementary data from literatures for training. After detecting the UA concentration in stones using the biosensor, the result was converted into serum uric acid concentration, allowing the estimation of creatinine level, which was then used by the ANN to assess the risk of developing CKD. This system can assist urologists in determining whether patients should seek consultation with nephrologists for early diagnosis and treatment.

Terrain parameter identification for wheeled mobile robots using deep neural networks

This paper introduces an innovative approach to soil parameter estimation using deep artificial neural networks (ANN) tailored to estimate a wide range of soil types. Previous research in this field has relied on limited datasets and oversimplified assumptions in semi-empirical models. In this work, the sensitivity of stress distribution is analyzed with respect to soil parameters, revealing significant errors resulting from the simplifications in semi-empirical models and emphasizing the need for more accurate estimation methods with minimal reliance on simplifying assumptions. Training an ANN requires a comprehensive dataset, which has been a challenge due to limited access to diverse soil samples. Addressing this issue, the paper conducts simulations of a single wheel’s movement across a wide range of soil parameter combinations to generate the required dataset. These simulations are done using a dynamic motion-compliant semi-empirical model detailed in the article. Subsequently, the network’s hyperparameters are fine-tuned through a grid search, followed by extensive training of the optimized ANN model. In the end, the ANN demonstrates promising performance in accurately estimating soil parameters, validated through simulation results of both single wheel and Mars rover motion.

Utilizing artificial neural network system to predict the residual valve after endoscopic posterior urethral valve ablation.

To build, train, and assess the artificial neural network (ANN) system in estimatingthe residual valve rate after endoscopic valve ablationand compare the data obtained with conventional analysis. In a retrospective cross-sectional study between June 2010 and December 2020, 144 children with a history of posteriorurethral valve (PUV) who underwent endoscopic valve ablation were enrolled in the study. MATLAB software was used to design and train the network in a feed-forward backpropagationerror adjustment scheme. Preoperative and postoperative data from 101 patients (70%) (training set) were utilized to assessthe impact and relative significance of the necessity for repeatedablation. The validated suitably trained ANN was used to predict repeated ablation in the next 33 patients (22.9%) (test set)whose preoperative data were serially input into the system. To assess system accuracy in forecasting the requirement for repeatablation, projected values were compared to actual outcomes. The likelihood of predicting the residual valve was calculatedusing a three-layered backpropagating deep ANN using preoperativeand postoperative information. Of 144 operated cases, 33 (22.9%) had residual valvesand needs to repeated ablation. The ANN accuracy, sensitivity, and specificity for predicting the residual valve were 90.75%,92.73%, and 73.19%, respectively. Younger age at surgery, hyperechogenicity of the renal parenchyma, presence of vesicoureteralreflux (VUR), and grade of reflux before surgery were among the most significant characteristics that affectedpostoperative outcome variables, the need for repeated ablation, and were given the highest relative weight by the ANN system. Conclusions: The ANN is an integrated data-gathering tool for analyzing and finding relationships among variables as a complex non-linear statistical model. The results indicate that ANN is a valuable tool for outcome prediction of the residualvalveafter endoscopic valve ablation in patients with PUV.

A deep learning deviation-based scheme to defend against false data injection attacks in power distribution systems

Defending against false data injection attacks (FDIAs) in cyber-physical power systems is crucial. Detection in power distribution systems is complex due to load variations, uncertainties, and fewer meters. Defense strategies include model-driven and data-driven approaches, but model-based methods can trigger false alarms due to threshold setting issues. The current research proposes a novel data-driven method to address threshold setting issues in detecting and localizing FDIAs in power distribution systems. First, a dataset is created by recording estimated measurement values using an unscented Kalman filter and weighted least squares across various attack scenarios. These estimated measurements are then fed into a deep artificial neural network (ANN) for binary classification to detect attacks. The output, along with the estimated measurements, is used by another ANN to localize the corrupted meter zone. This deep learning-based approach improves threshold setting over the common chi-square method. Results show that the proposed deep learning method for FDIA detection and localization outperforms a recently proposed ensemble of shallow models. The area under the curve value increases by about 5% with lower training time. The approach is also effective against previously unseen attack strategies and different feeder topologies.

Forecasting of Turkey's Hazelnut Export Amounts According to Seasons with Dendritic Neuron Model Artificial Neural Network

It is seen that artificial neural networks have begun to be used extensively in the literature in solving the time series forecasting problem. In addition to artificial neural networks, classical forecasting methods can often be used to solve this problem. It is seen that classical forecasting methods give successful results for linear time series analysis. However, there is no linear relationship in many time series. Therefore, it can be thought that deep artificial neural networks, which contain more parameters but create more flexible non-linear model structures compared to classical time series forecasting methods, may enable the production of more successful forecasting methods. In this study, the problem of forecasting hazelnut export amounts according to seasons in Turkey with a dendritic neuron model artificial neural network is discussed. In this study, a training algorithm based on the particle swarm optimization algorithm is given for training the dendritic neuron model artificial neural network. The motivation of the study is to investigate Turkey's hazelnut export amounts according to seasons, using a dendritic neuron model artificial neural network. The performance of the proposed method has been compared with artificial neural networks used in the literature.

Machine learning optimization and 4E analysis of a CCHP system integrated into a greenhouse system for carbon dioxide capturing

The world has seen an increase in the popularity of renewable-based energy systems. However, because of their erratic nature, fossil fuels cannot be entirely phased out. Utilizing carbon dioxide in greenhouses close to power plants and generating extra power electricity from dissipated thermal energy are appealing strategies that cause us to have a cleaner environment and affordable power electricity. To exploit the high enthalpy of expelled gases from a gas turbine, the gas turbine was coupled with an absorption refrigerant cycle and an organic Rankine cycle and analyzed. Additionally, a deep artificial neural network method was used to calculate all functions in a rapid optimization and accurate 4E analysis. The carbon dioxide emissions subsided from 0.9183 to 0.41 kg CO2/s, and the optimization time improved 257 times after using the trained machine learning model. An adsorption technique for CO2 separation is used for the proposed system because of its lower energy consumption. The maximum exergy and energy efficiencies were attained at 36.71 % and 49.2 %, respectively, and parametric studies are being assessed. Due to increases in harvests and efficiencies, the net annual interest has increased by 21.8 %, up to 22.5 million dollars.

Digitalization of Analysis of a Concrete Block Layer Using Machine Learning as a Sustainable Approach

The concrete block pavement (CBP) system has a surface layer consisting of concrete block pavers and joint sand over a bedding sand layer. The non-homogeneous nature of the surface course of CBP, along with different laying patterns and shapes of block pavers, makes the analysis of CBP cumbersome. In this study, the surface course of CBP was modeled based on the slab action of the block pavers and joint sand, which are connected together in full contact. Four different laying patterns, including herringbone, stretcher, parquet, and square, were modeled using a finite element model. The elastic moduli of the block pavers varied from 2500 MPa to 45,000 MPa, with thicknesses ranging from 60 mm to 120 mm. As a result, modeling of CBP based on slab action can be considered a realistic strategy. In addition, a dataset was created based on quantitative inputs, e.g., elastic modulus and thickness of the block pavers, and qualitative input, i.e., block laying patterns. The approaches of machine learning adopted were support vector regression, Gaussian process regression, single-layer and deep artificial neural networks, and least squares boosting to implement prediction approach based on input and output. The analyses of statistical accuracy of all five machine learning methods showed high accuracy; however, the Gaussian process and deep artificial neural network methods resulted in the most accurate outputs and are recommended for further studies. Based on the machine learning models, digitalization is achieved through the development of simple, user-friendly software for electronic devices in order to perform a preliminary analysis of different laying patterns of CBP. Such a platform may result in less laboratory work and boosts the level of sustainability in concrete block pavement technology.

Predicting sessile droplet evaporation kinetics via cascaded deep networks and tree-based machine learning approach

The intricate nature of sessile droplet evaporation phenomenon makes detailed experimental studies time consuming and requires sophisticated apparatus; while complete numerical simulations are computationally expensive and also time-intensive. In this article, for the first time, we explore the applicability of machine learning (ML) approaches to predict the evaporation kinetics of generalized sessile droplets under various conditions. An in-house dataset, obtained using an experimentally well validated numerical model, is used to develop the ML models: deep artificial neural network (ANN) and decision tree algorithms such as Random Forest (RF) and extreme gradient boosting (XGB). The structures of these models are modified by cascading the output features according to the physics involved. This distinctive approach results in better prediction of the evaporation kinetics than basic ML models. The models are trained by a large set of input parameters for the target variables: viz. droplet evaporation rate, velocity scale, and temperature drop in the solid and liquid domain. Finally, the performance of these ML models is assessed by comparing their predictions with that of physics-based, experimentally validated, numerical models. Results show that the inclusion of additional features obtained using feature engineering significantly improve the prediction performance of ML algorithms, and consistently accurate predictions of droplet evaporation kinetics are obtained. Among various algorithms considered here, the ANN outperforms in term of various error matrices for most of the cases, followed by XGB, and RF models. Also, the highest mean average error (MAE) yield by the ML models for evaporation rate, velocity scale and temperature drop in liquid remains within ∼12.5%. In the case of temperature drop for the solid, the MAE is considerably higher due to large variability of the same target variable. Overall, the work clearly shows that ML algorithms can be used to obtain physically consistent predictions for sessile droplet evaporation parameters.

The development of machine learning approaches in two-dimensional NMR data interpretation for metabolomics applications

Metabolomics has been widely applied in human diseases and environmental science to study the systematic changes of metabolites over diverse types of stimuli. NMR-based metabolomics has been widely used, but the peak overlap problems in the one-dimensional (1D) NMR spectrum could limit the accuracy of quantitative analysis for metabolomics applications. Two-dimensional (2D) NMR has been applied to solve the 1D NMR overlap problem, but the data processing is still challenging. In this study, we built an automatic approach to process the 2D NMR data for quantitative applications using machine learning approaches. Partial least square discriminant analysis (PLS-DA), artificial neural network classification (ANN-DA), gradient boosted trees classification (XGBoost-DA), and artificial deep learning neural network classification (ANNDL-DA) were applied in combination with an automatic peak selection approach. Standard mixtures, sea anemone extracts, and mouse fecal samples were tested to demonstrate the approach. Our results showed that ANN-DA and ANNDL-DA have high accuracy in selecting 2D NMR peaks (around 90 %), which have a high potential application in 2D NMR-based metabolomics quantitively study, while PLS-DA and XGBoost-DA showed limitations in either data variation or overfitting. Our study built an automatic approach to applying 2D NMR data to routine quantitative analysis in metabolomics.

A hybrid deep recurrent artificial neural network with a simple exponential smoothing feedback mechanism

Both classical forecasting methods and machine learning approaches are used to solve forecasting problems. Deep artificial neural networks, one of the machine learning methods, are widely used today and give very good results. Recurrent neural networks, a type of deep neural network, are very important in forecasting problems. Simple recurrent artificial neural networks, which are the simplest deep recurrent neural networks, are often preferred in solving forecasting problems due to the small number of parameters they use. Simple exponential smoothing, one of the classical forecasting methods, attracts attention with its performance in solving forecasting problems. The motivation of the study is to create a new forecasting method by combining a classical and simple forecasting method with a deep recurrent artificial neural network in an architecture. In this, a new hybrid deep recurrent artificial neural network with a simple exponential smoothing feedback mechanism is proposed. The architecture of the proposed method is created as a combination of simple recurrent artificial neural networks and simple exponential smoothing methods. In the training of the proposed method, two training algorithms based on sine cosine optimization and particle swarm optimization algorithms are proposed. In these training algorithms, two different solution strategies such as restarting, and early stopping rule are used to avoid overfitting and local optimum problems. The performance of the proposed method is analysed using stock market datasets and compared with both different deep and shallow artificial neural networks and classical forecasting methods. As a result of the analyses, it is concluded that the proposed method is successful in one step ahead of forecasting performance.

Thy-DAMP: deep artificial neural network model for prediction of thyroid cancer mortality.

Despite the rising incidence of differentiated thyroid cancer (DTC), mortality rates have remained relatively low yet crucial for effective patient management. This study aims to develop a deep neural network capable of predicting mortality in patients with differentiated thyroid cancer. Leveraging data from the Surveillance, Epidemiology, and End Results (SEER) database, we developed Thy-DAMP (Deep Artificial Neural Network Model for Prediction of Thyroid Cancer) to forecast mortality in DTC patients. The dataset comprised demographic, histologic, and staging information. Following data normalization and feature encoding, the dataset was partitioned into training, testing, and validation subsets, with model hyperparameters fine-tuned via cross-validation. Among the 63,513 patients, the mean age was 48.22 years (SD = 14.8), with 77.32% being female. Papillary carcinoma emerged as the predominant subtype, representing 62.94% of cases. The majority presented with stage I disease (73.96%). Thy-DAMP demonstrated acceptable performance metrics on both the test and validation sets. Sensitivity was 83.24% (95% CI 76.95-88.40%), specificity was 93.53% (95% CI 93.01-94.02%), and accuracy stood at 93.33% (95% CI 92.82-93.83%). The model exhibited a positive predictive value of 19.76% (95% CI 18.20-21.42%) and a negative predictive value of 99.66% (95% CI 99.53-99.75%). Additionally, Thy-DAMP demonstrated a positive likelihood ratio of 12.86 (95% CI 11.62-14.23), a negative likelihood ratio of 0.18 (95% CI 0.13-0.25), and an area under the receiver operating characteristic curve (AUROC) of 0.95. The model was externally validated on a separate dataset with nearly identical performance. Thy-DAMP showcases considerable promise in accurately predicting mortality in DTC patients, leveraging limited set of patient data.

Deep learning of plausible bandgaps in dispersion curves of phononic crystals

Phononic crystals represent an interesting class of metamaterials that can be utilized to regulate or manipulate vibration, sound propagation, and thermal transport. Their useful features mainly arise from the bandgaps in their dispersion curves, preventing the passage of waves within specific frequency ranges. However, it is often costly and time-consuming to obtain the dispersion curves, and the reverse engineering of phononic crystals to have pre-defined bandgaps possesses even greater challenges. In this research, we address this issue by employing a deep artificial neural network to predict the bandgap ratio and the characteristics of plausible bandgaps, focusing on the localized resonance in columnar phononic crystals. We utilized two geometric parameters, i. e. the ratio of diameter and height of the cylindrical resonators relative to the lattice constant, achieving a determination coefficient of 0.9993 for predicting the characteristics of the bandgaps and 0.9827 for predicting the bandgap ratio. To verify the model and better understand its behavior, we introduce Shapley values. These values provide a comprehensive insight into how each geometric parameter influences the predicted bandgap ratios.

Optimising shapes of multiple pin fins in a microchannel using deep reinforcement learning and mesh deformation techniques

The utilisation of advanced pin fin designs in microchannels is useful for enhancing cooling efficiency. Advancements in machine learning and processing power have sparked interest in shape optimisation techniques. This research employs a novel framework that integrates Deep Artificial Neural Networks and Reinforcement Learning with a Computational Fluid Dynamics (CFD) solver to optimise multiple pin fin shapes within a microchannel. By incorporating Radial Basis Function interpolation and Proximal Policy Optimisation alongside FLUENT, acting as the CFD environment, the reinforcement learning agent adeptly explores the design space to enhance the thermohydraulic performance factor (TPF), aiming to maximise Nusselt number while minimising pressure loss. Unlike previous heat transfer optimisation studies, which typically required mesh regeneration at each step, the proposed framework could bypass the meshing step and alter the geometry directly by relying on the RBF interpolation technique to deform the mesh directly. Three distinct scenarios investigated in this study are uniform deformation of all pin fins, deformation of all pin fins arranged in two rows, and individual deformation of each pin fin. Extensive simulations, exceeding 90,000 different cases, demonstrate that although the optimisation process for the individual deformation of each pin fin requires more iterations compared to others, it surpasses them in terms of TPF improvement. Notably, significant improvements are achieved, such as a 49 % enhancement in Nusselt number and a 33 % reduction in pressure drop, culminating in an impressive 63 % increase in TPF compared to the initial geometry.