Deep Feedforward Neural Network Research Articles

Air quality index prediction with optimisation enabled deep learning model in IoT application

ABSTRACT The development of industrial and urban places caused air pollution, which has resulted in a variety of effects on individuals and the atmosphere over the years. The measurement of the air quality index (AQI) depends on various environmental situations, such as emissions, dispersions, and chemical reactions. This paper developed the Internet of Things (IoT)-based Deep Learning (DL) technique for predicting air quality. Initially, the IoT simulation is performed, where the nodes receive input data. The routing technique is used to identify the best route toward the Base station (BS). The proposed Tangent Two-Stage Algorithm (TTSA) is used in the routing mechanism. For AQI prediction, the time series data is transmitted to the BS. The Z-score normalisation is employed to neglect the unessential data. Furthermore, feature indicator extraction is employed to extract the relevant feature indicators. The Deep Feedforward Neural Network (DFNN) is used to predict air quality. Furthermore, the proposed Fractional Tangent Two-Stage Optimisation (FTTSA) is employed for the training process of DFNN. Moreover, metrics such as energy, time, and distance are used to evaluate the routing process, and superior results such as 0.979J, 0.025s and 0.196 m are obtained. Furthermore, the AQI is predicted by metrics like root mean square error (RMSE), R-squared (R2), mean square error (MSE), and mean absolute percentage error (MAPE), whereas the superior values such as 0.602, 0.598, 0.362, and 0.456 are attained.

Improved Early Detection of Tube Leaks Faults in Pulverised Coal-fired Boiler Using Deep Feed Forward Neural Network

Boiler tube leaks significantly reduce the operational availability of power units, yet their early detection and prediction have not been fully realised in the industry. This paper introduces a novel approach employing deep feedforward neural networks for early detection of boiler tube leaks in pulverised coal-fired boilers. Early detection enhances repair planning, minimising downtime and production losses. It also improves monitoring and control of boiler tube failures, thereby optimising power plant operations and revenue. Diverse deep neural network models were developed and rigorously tested by leveraging 9 years of operational data (2012–2020). Exhaustive hyper-parameter optimisation, involving seven parameters, substantially improved predictive accuracy. By achieving training and testing accuracies of 82.8% to 99.3%, the study assessed their ability to detect boiler tube leaks over the same 9-year span, providing insights into leak detection capabilities. The models notably predicted all 12 identified tube leak events, averaging a 14-day lead time before boiler shutdown. In addition to leak prediction, a leak detection matrix was devised to analyse residual behaviour and reduce the likelihood of false alarms. However, the models’ predictive performance was observed to be limited to the following year, with satisfactory results for 2021 only. Incorporating the 2021 data into retraining significantly improved the predictions for 2022. The study concludes that while the models demonstrate robust short-term prediction capabilities, they require continuous retraining to maintain accuracy and relevance. This ongoing refinement is essential for keeping the models up-to-date and reliable in predicting future boiler tube leaks.

DELM: Deep Ensemble Learning Model for Anomaly Detection in Malicious Network Traffic-based Adaptive Feature Aggregation and Network Optimization

With the rapid advancements in internet technology, the complexity and sophistication of network traffic attacks are increasing, making it challenging for traditional anomaly detection systems to analyze and detect malicious network attacks. The increasing advancedness of cyber threats calls for innovative approaches to identify malicious patterns within network traffic precisely. The primary issue lies in the fact that these approaches do not focus on the essential adaptive features of network traffic. We proposed an effective anomaly detection system for malicious network traffic attacks called the Deep Ensemble Learning Model (DELM). We leverage the structure of the Feedforward Deep Neural Network (FDNN), and Deep Belief Network (DBN), incorporating multiple hidden layers with non-linear activation functions. Integrating Adaptive Feature Aggregation (AFA) with the FDNN algorithm dynamically adjusts the feature aggregation process based on incoming traffic characteristics to improve adaptability. The Conditional Generative Network was employed to enhance DELM for generating data for minority classes. To improve the model’s accuracy, we applied batch normalization and data augmentation techniques for preprocessing, utilized n-gram, one-hot encoding, and feature aggregation methods for effective feature extraction. This study significantly contributes to network security by enhancing systems for detecting malicious network traffic. With its interpretability and adaptability, our proposed model shows promise in addressing the evolving cyber threat and fortifying critical network infrastructure. The experimental results demonstrate that our model performs with higher stability than the existing state-of-the-art detection approaches, as reflected by its higher accuracy, precision, recall, F1-score, and AUC-ROC.

Neural networks for ID gap orbit distortion compensation in PETRA III

Undulators are used in storage rings to produce extremely brilliant synchrotron radiation. In the ideal case, a perfectly tuned undulator always has a first and second field integrals equal to zero. But, in practice, field integral changes during gap movements can never be avoided for real-life devices. As they significantly impact the circulating electron beam, there is the need to routinely compensate such effects. Deep Neural Networks can be used to predict the distortion in the closed orbit induced by the undulator gap variations on the circulating electron beam. In this contribution several state-of-the-art deep learning algorithms were trained on measurements from PETRA III. The different architecture performances are then compared to identify the best model for the gap-induced distortion compensation. It is found that realistic data, with several gaps moving simultaneously to arbitrary gaps must be used to train the neural networks. The deep feed-forward neural network was found to be more effective than the recurrent and convolutional neural networks.

Performance of Deep Feed-Forward Neural Network Algorithm Based on Content-Based Filtering Approach

Background: Selecting a restaurant in a diverse city like Bandung can be challenging. This study leverages Twitter data and local restaurant information to develop an advanced recommendation system to improve decision-making. Objective: The system integrates content-based filtering (CBF) with deep feedforward neural network (DFF) classification to enhance the accuracy and relevance of restaurant recommendations. Methods: Data was sourced from Twitter and PergiKuliner, with restaurant-related tweets converted into rating values. The CBF combined Bag of Words (BoW) and cosine similarity, followed by DFF classification. SMOTE was applied during training to address data imbalance. Results: The initial evaluation of CBF showed a Mean Absolute Error (MAE) of 0.0614 and a Root Mean Square Error (RMSE) of 0.0934. The optimal DFF configuration in the first phase used two layers with 32/16 nodes, a dropout rate of 0.3, and a 20% test size. This setup achieved an accuracy of 81.08%, precision of 82.89%, recall of 76.93%, and f1-scores of 79.23%. In the second phase, the RMSprop optimizer improved accuracy to 81.30%, and tuning the learning rate to 0.0596 further increased accuracy to 89%, marking a 9.77% improvement. Conclusion: The research successfully developed a robust recommendation system, significantly improving restaurant recommendation accuracy in Bandung. The 9.77% accuracy increase highlights the importance of hyperparameter tuning. SMOTE also proved crucial in balancing the dataset, contributing to a well-rounded learning model. Future studies could explore additional contextual factors and experiment with recurrent or convolutional neural networks to enhance performance further.

Good results from sensor data: Performance of machine learning algorithms for regression problems in chemical sensors

Accurately predicting unseen data, instead of mere memorization of training examples, is a critical goal of machine learning. This generalization is particularly important in the field of chemical sensors, where the ability to accurately predict the chemical properties or concentration levels of unknown samples is crucial. The paper presents a comprehensive yet accessible introduction to various machine learning concepts, highlighting the importance of model interpretability and generalization in ensuring reliable and accurate results in this context. Nonlinear sensor array data are utilized to introduce key concepts (e.g., bias-variance tradeoff) and techniques (linear models, partial least squares regression, support vector machines, k-nearest neighbors, decision trees, ensemble methods, automated machine learning, symbolic regression, and artificial neural networks), providing a solid foundation to make informed decisions when selecting machine learning techniques for sensor-specific regression applications. The results clearly indicate a number of conclusions. First, overparameterized deep feedforward neural networks show great accuracy and generalization when trained on a sufficiently large dataset. Second, symbolic regression models proved to be more accurate than deep feedforward neural networks and classical machine learning techniques on smaller datasets. Third, the performance of various machine learning models was dataset-dependent, showing the importance of comparative studies to determine the most suitable approach. It is clear that the optimal model cannot be known a priori. This paper aims to provide a starting point for investigations on the performance of different machine learning techniques in chemical sensor applications.

Deep learning and optimization enabled multi-objective for task scheduling in cloud computing

ABSTRACT In cloud computing (CC), task scheduling allocates the task to best suitable resource for execution. This article proposes a model for task scheduling utilizing the multi-objective optimization and deep learning (DL) model. Initially, the multi-objective task scheduling is carried out by the incoming user utilizing the proposed hybrid fractional flamingo beetle optimization (FFBO) which is formed by integrating dung beetle optimization (DBO), flamingo search algorithm (FSA) and fractional calculus (FC). Here, the fitness function depends on reliability, cost, predicted energy, and makespan, the predicted energy is forecasted by a deep residual network (DRN). Thereafter, task scheduling is accomplished based on DL using the proposed deep feedforward neural network fused long short-term memory (DFNN-LSTM), which is the combination of DFNN and LSTM. Moreover, when scheduling the workflow, the task parameters and the virtual machine’s (VM) live parameters are taken into consideration. Task parameters are earliest finish time (EFT), earliest start time (EST), task length, task priority, and actual task running time, whereas VM parameters include memory utilization, bandwidth utilization, capacity, and central processing unit (CPU). The proposed model DFNN-LSTM+FFBO has achieved superior makespan, energy, and resource utilization of 0.188, 0.950J, and 0.238, respectively.

Predicting Color Schemes Based on Genetic Algorithm and Deep Feedforward Neural Network Model

Predicting Color Schemes Based on Genetic Algorithm and Deep Feedforward Neural Network Model

Rule Extraction in Trained Feedforward Deep Neural Networks

Explainability is a key aspect of machine learning, necessary for ensuring transparency and trust in decision-making processes. As machine learning models become more complex, the integration of neural and symbolic approaches has emerged as a promising solution to the explainability problem. One effective solution involves using search techniques to extract rules from trained deep neural networks by examining weight and bias values and calculating their correlation with outputs. This article proposes incorporating cosine similarity in this process to narrow down the search space and identify the critical path connecting inputs to final results. Additionally, the integration of first-order logic (FOL) is suggested to provide a more comprehensive and interpretable understanding of the decision-making process. By leveraging cosine similarity and FOL, an innovative algorithm capable of extracting and explaining rule patterns learned by a feedforward trained neural network was developed and tested in two use cases, demonstrating its effectiveness in providing insights into model behavior.

Achieving Occam's razor: Deep learning for optimal model reduction.

All fields of science depend on mathematical models. Occam's razor refers to the principle that good models should exclude parameters beyond those minimally required to describe the systems they represent. This is because redundancy can lead to incorrect estimates of model parameters from data, and thus inaccurate or ambiguous conclusions. Here, we show how deep learning can be powerfully leveraged to apply Occam's razor to model parameters. Our method, FixFit, uses a feedforward deep neural network with a bottleneck layer to characterize and predict the behavior of a given model from its input parameters. FixFit has three major benefits. First, it provides a metric to quantify the original model's degree of complexity. Second, it allows for the unique fitting of data. Third, it provides an unbiased way to discriminate between experimental hypotheses that add value versus those that do not. In three use cases, we demonstrate the broad applicability of this method across scientific domains. To validate the method using a known system, we apply FixFit to recover known composite parameters for the Kepler orbit model and a dynamic model of blood glucose regulation. In the latter, we demonstrate the ability to fit the latent parameters to real data. To illustrate how the method can be applied to less well-established fields, we use it to identify parameters for a multi-scale brain model and reduce the search space for viable candidate mechanisms.

Fault Type and Fault Location Detection in Transmission Lines with 6-Convolutional Layered CNN

In this article, we propose a data-driven method for short-circuit fault detection in transmission lines that exploits the capabilities of convolutional neural networks (CNNs). CNNs, a class of deep feedforward neural networks, can autonomously detect different features from data, eliminating the need for manual intervention. To mitigate the effects of noise and increase network robustness, we present a CNN architecture with six convolutional layers. The study uses a single busbar power system model developed with the PSCAD simulation program to evaluate the performance of the proposed method. The proposed CNN method is also compared with machine learning methods such as LSTM, SVM and ELM. Our results show a high success rate of 98.4% across all fault impedances, confirming the effectiveness of the proposed CNN methods in accurately detecting short-circuit faults based on current and voltage measurements.

On neural networks application in integral sliding mode control

In this paper, the combined use of deep neural networks (DNNs) and integral sliding mode (ISM) control is investigated, giving rise to novel DNN-ISM control strategies for nonlinear systems affine in the control law, characterized by fully unknown dynamics and subject to disturbances. In particular, two feed-forward DNNs are used to approximate the drift term and the control effectiveness matrix of the nonlinear systems model. The DNNs weights are tuned online relying on adaptation laws derived from Lyapunov’s stability analysis. The DNNs with their weights tuning mechanism are exploited as building blocks in the design of an ISM control scheme to solve trajectory tracking problems. The proposed framework is then extended to the case in which the system dynamics is subject to time-varying state constraints, producing a second scheme, still of DNN-ISM type. Both control schemes are analyzed, providing interesting theoretical guarantees. Possible solutions to reduce, in practical implementations, the chattering effect, as well as the initial estimation error of the DNNs are also discussed. The proposed DNNs-based control schemes are finally assessed in simulation relying on a realistic model of an anthropomorphic 7-axes robot manipulator.

An intelligent computational iBCMO-DNN algorithm for stochastic thermal buckling analysis of functionally graded porous microplates using modified strain gradient theory

The authors propose an intelligent computational method using the deep feedforward neural network integrated with an improved balance composite motion optimization algorithm to formulate a so-called iBCMO-DNN for solving the stochastic thermal buckling problems of functionally graded porous microplates. In the present approach, the deterministic behaviors of the functionally graded porous microplates were firstly analyzed by the combination of a unified higher-order shear deformation theory, modified strain gradient theory, and Ritz-type series solutions, and then stochastic responses under uncertainty of material properties are obtained by the iBCMO-DNN algorithm. The deep neural network with the long short-term memory model is used as a surrogate method to replace the time-consuming computational model, while the improved balance composite motion optimization is used to search for the optimal solutions. The obtained numerical results for various boundary conditions, uncertainty parameters, and three types of temperature distribution indicated that the proposed method achieves its accuracy and effectiveness in predicting stochastic thermal buckling of the functionally graded porous microplates. The improved balance composite motion optimization algorithm demonstrates a computational time ∼1.7 times faster than its predecessor. Furthermore, integrating a deep neural network into the improved algorithm reduces computational time to about 2/5 compared to the method without it.

PLIC-Net: A machine learning approach for 3D interface reconstruction in volume of fluid methods

The accurate reconstruction of immiscible fluid–fluid interfaces from the volume fraction field is a critical component of geometric Volume of Fluid methods. A common strategy is the Piecewise Linear Interface Calculation (PLIC), which fits a plane in each mixed-phase computational cell. However, recent work goes beyond PLIC by using two planes or even a paraboloid. To select such planes or paraboloids, complex optimization algorithms as well as carefully crafted heuristics are necessary. Yet, the potential exists for a well-trained machine learning model to efficiently provide broadly applicable solutions to the interface reconstruction problem at lower costs. In this work, the viability of a machine learning approach is demonstrated in the context of a single plane reconstruction. A feed-forward deep neural network is used to predict the normal vector of a PLIC plane given volume fraction and phasic barycenter data in a 3×3×3 stencil. The PLIC plane is then translated in its cell to ensure exact volume conservation. Our proposed neural network PLIC reconstruction (PLIC-Net) is equivariant to reflections about the Cartesian planes. Training data is analytically generated with O(106) randomized paraboloid surfaces, which allows for the sampling a broad range of interface shapes. PLIC-Net is tested in multiphase flow simulations where it is compared to standard LVIRA and ELVIRA reconstruction algorithms, and the impact of training data statistics on PLIC-Net’s performance is also explored. It is found that PLIC-Net greatly limits the formation of spurious planes and generates cleaner numerical break-up of the interface. Additionally, the computational cost of PLIC-Net is lower than that of LVIRA and ELVIRA. These results establish that machine learning is a viable approach to Volume of Fluid interface reconstruction and is superior to current reconstruction algorithms for some cases.

Explainable Deep Learning Model for Predicting Serious Adverse Events in Hospitalized Geriatric Patients Within 72 Hours.

The global aging population presents a significant challenge, with older adults experiencing declining physical and cognitive abilities and increased vulnerability to chronic diseases and adverse health outcomes. This study aims to develop an interpretable deep learning (DL) model to predict adverse events in geriatric patients within 72 hours of hospitalization. The study used retrospective data (2017-2020) from a major medical center in Taiwan. It included non-trauma geriatric patients who visited the emergency department and were admitted to the general ward. Data preprocessing involved collecting prognostic factors like vital signs, lab results, medical history, and clinical management. A deep feedforward neural network was developed, and performance was evaluated using accuracy, sensitivity, specificity, positive predictive value (PPV), and area under the receiver operating characteristic curve (AUC). Model interpretation utilized the Shapley Additive Explanation (SHAP) technique. The analysis included 127,268 patients, with 2.6% experiencing imminent intensive care unit transfer, respiratory failure, or death during hospitalization. The DL model achieved AUCs of 0.86 and 0.84 in the validation and test sets, respectively, outperforming the Sequential Organ Failure Assessment (SOFA) score. Sensitivity and specificity values ranged from 0.79 to 0.81. The SHAP technique provided insights into feature importance and interactions. The developed DL model demonstrated high accuracy in predicting serious adverse events in geriatric patients within 72 hours of hospitalization. It outperformed the SOFA score and provided valuable insights into the model's decision-making process.

Performance evaluation of deep learning approaches for fault diagnosis of rotational mechanical systems using vibration, sound, and acoustic emission signals

The present study emphasizes an optimized deep learning algorithm for gearbox fault detection using vibration, sound, and acoustic emission signals. Statistical and acoustic features are extracted from these signals, and various neural network algorithms are explored. The supervised deep feed forward neural network (DFFNN) demonstrates excellent performance with vibration signals but limited accuracy with sound and acoustic emission signals. To address this, unsupervised algorithms are optimized and compared with vibration-based classification. The findings show that unsupervised neural networks, particularly the auto-encoder and stacked auto-encoder architectures, achieve improved classification accuracy by leveraging the unique characteristics of acoustic emission signals. The unsupervised models also effectively overcome the vanishing gradient problem via regularization, enhancing their training efficiency. The stacked auto-encoder, with multiple layers of encoders and decoders, reduces computation time by 40% and memory consumption. These optimized algorithms hold promise for automated fault detection systems. The auto-encoder and stacked auto-encoder, utilizing vibration, sound, and acoustic emission signals, offer enhanced classification accuracy and can facilitate real-time monitoring of rotating mechanical systems. However, further optimization is needed to maximize their performance. In a nutshell, the supervised DFFNN excels in utilizing vibration signals for fault detection, while the unsupervised models exploit the distinctive characteristics of acoustic emission signals. Future research will focus on refining these algorithms to enhance their effectiveness. Implementing these optimized deep learning approaches can lead to autonomous fault detection systems, eliminating the need for continuous human supervision.

Electronic properties prediction enhancement of 36 ternary III-IB-VI alloys using a deep feed-forward neural network

The electronic properties of 36 ternary III-IB-VI monolayers have been investigated using a correlation study, in which a comprehensive study of the band structure of these alloys is provided. To this end, a combination of density functional theory calculations and a deep feed-forward neural network is conducted. Principal component analysis and data augmentation are also implemented as pre-processing. The former reduces the size of features to mitigate redundancy, and the latter can prevent overfitting. The deep feed-forward neural network is then used for the final prediction. During the study, bandgaps of the alloys are predicted indirectly and compared with directly obtained bandgaps. The results of the predictions show a maximum precision of 0.00089 for nMSE using the deep feed-forward neural network, which confirms strong dependencies between the electronic properties of the alloys.

Predicting the shear modulus and damping ratio of soils using machine learning models

In geotechnical engineering, the accurate estimation of fundamental soil properties, such as the shear modulus ratio (G/G max) and damping ratio (D), is crucial to design and analyze various structures subjected to dynamic loads. This study presents a comprehensive investigation on harnessing the power of machine learning techniques to precisely predict G/G max and D of granular soils. Using an extensive dataset gathered from cyclic triaxial and resonant column tests on diverse mixtures of sand and gravel, combined with previous research findings, a series of advanced machine learning algorithms including shallow neural networks, support vector regression, gradient boosting regression, and deep feed forward neural network (DFFNN) were developed. The proposed models elucidate various influential parameters, including the grading characteristics, void ratio, confining pressure, consolidation stress ratio, and specimen preparation techniques. The superiority of the DFFNN model in terms of accuracy and predictive performance was demonstrated through rigorous evaluation and comparison. This study contributes to a better understanding of soil behavior under dynamic conditions. It also provides a robust framework to employ machine learning in predicting G/G max and D of granular soils, thereby enhancing the efficiency and reliability of geotechnical designs and construction practices.

KFDAE: CircRNA-Disease Associations Prediction Based on Kernel Fusion and Deep Auto-Encoder.

CircRNA has been proved to play an important role in the diseases diagnosis and treatment. Considering that the wet-lab is time-consuming and expensive, computational methods are viable alternative in these years. However, the number of circRNA-disease associations (CDAs) that can be verified is relatively few, and some methods do not take full advantage of dependencies between attributes. To solve these problems, this paper proposes a novel method based on Kernel Fusion and Deep Auto-encoder (KFDAE) to predict the potential associations between circRNAs and diseases. Firstly, KFDAE uses a non-linear method to fuse the circRNA similarity kernels and disease similarity kernels. Then the vectors are connected to make the positive and negative sample sets, and these data are send to deep auto-encoder to reduce dimension and extract features. Finally, three-layer deep feedforward neural network is used to learn features and gain the prediction score. The experimental results show that compared with existing methods, KFDAE achieves the best performance. In addition, the results of case studies prove the effectiveness and practical significance of KFDAE, which means KFDAE is able to capture more comprehensive information and generate credible candidate for subsequent wet-lab.

Exploring Physics-Informed Neural Networks for the Generalized Nonlinear Sine-Gordon Equation

The nonlinear sine-Gordon equation is a prevalent feature in numerous scientific and engineering problems. In this paper, we propose a machine learning-based approach, physics-informed neural networks (PINNs), to investigate and explore the solution of the generalized non-linear sine-Gordon equation, encompassing Dirichlet and Neumann boundary conditions. To incorporate physical information for the sine-Gordon equation, a multiobjective loss function has been defined consisting of the residual of governing partial differential equation (PDE), initial conditions, and various boundary conditions. Using multiple densely connected independent artificial neural networks (ANNs), called feedforward deep neural networks designed to handle partial differential equations, PINNs have been trained through automatic differentiation to minimize a loss function that incorporates the given PDE that governs the physical laws of phenomena. To illustrate the effectiveness, validity, and practical implications of our proposed approach, two computational examples from the nonlinear sine-Gordon are presented. We have developed a PINN algorithm and implemented it using Python software. Various experiments were conducted to determine an optimal neural architecture. The network training was employed by using the current state-of-the-art optimization methods in machine learning known as Adam and L-BFGS-B minimization techniques. Additionally, the solutions from the proposed method are compared with the established analytical solutions found in the literature. The findings show that the proposed method is a computational machine learning approach that is accurate and efficient for solving nonlinear sine-Gordon equations with a variety of boundary conditions as well as any complex nonlinear physical problems across multiple disciplines.