Feed-forward Neural Network Model Research Articles

Deep Learning Ensemble for Flood Probability Analysis

Predicting flood events is complex due to uncertainties from limited gauge data, high data and computational demands of traditional physical models, and challenges in spatial and temporal scaling. This research innovatively uses only three remotely sensed and computed factors: rainfall, runoff and temperature. We also employ three deep learning models—Feedforward Neural Network (FNN), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM)—along with a deep neural network ensemble (DNNE) using synthetic data to predict future flood probabilities, utilizing the Savitzky–Golay filter for smoothing. Using a hydrometeorological dataset from 1993–2022 for the Nile River basin, six flood predictors were derived. The FNN and LSTM models exhibited high accuracy and stable loss, indicating minimal overfitting, while the CNN showed slight overfitting. Performance metrics revealed that FNN achieved 99.63% accuracy and 0.999886 ROC AUC, CNN had 95.42% accuracy and 0.893218 ROC AUC, and LSTM excelled with 99.82% accuracy and 0.999967 ROC AUC. The DNNE outperformed individual models in reliability and consistency. Runoff and rainfall were the most influential predictors, while temperature had minimal impact.

Transfer learning for probabilistic localization of hidden cracks in concrete structures

AbstractThe utility of discriminative supervised learning models built using multiple training-data sources is investigated for hidden crack localization in concrete. Feed-forward neural network (FFNN) is chosen as the model architecture, and transfer learning is used to assimilate the information obtained from different sources (computational physics simulations and laboratory experiments). The labeled training data consists of values of a damage index and the known locations of hidden cracks. The classification models need to learn how the presence of damage (hidden cracks) affects the damage index at different sensors for different test conditions. To this end, diagnostic FFNN models are built by sequentially adding and training new hidden layers to assimilate labeled information from computer models (different model geometries, test conditions, crack lengths, crack locations) and laboratory experiments on a plain cement slab. These transfer learning-based models are then used to localize damage in concrete specimens that reflect real-world conditions (i.e., specimens with steel reinforcement and randomly distributed aggregate). The actual damage state in these specimens is determined by extracting cores and performing petrographic studies on the extracted cores. The damage probability estimated by transfer learning-based models is compared with the petrographic damage rating index (DRI) to identify the most suitable approach to train the diagnostic models. The transfer learning-based diagnostic methodology shows promise and could be used in various structural health monitoring applications, where sufficient labeled data are typically not available from a single data source.

An Enhanced Edge Computing Technique for Detection of Voltage Fluctuation in Grid-tied Renewable Energy

Renewable energy sources (RES) such as solar photovoltaic and wind are becoming the most attractive power generation options in many nations. Even while high penetration seems likely, power quality anomalies such as voltage fluctuation, harmonics, and frequency fluctuation associated with RES hinder seamless integration. The variability and unpredictability of these sources create the most oddities. In grid-tied renewable energy, monitoring power quality efficiently is crucial. Power grid monitoring solutions in related literature use sensor-based cloud and edge computing techniques. The existing systems struggle with excessive latency when delivering large amounts of generated data to the cloud. To fill this gap, a new approach for the detection and localization of voltage fluctuation is proposed in this study. The approach integrated three techniques namely; feed-forward neural network (FFNN), Stockwell transform, and anomaly-aware edge computing to detect and locate voltage fluctuation in a GtRE. Using MATLAB/Simulink, virtual emulation of a modified IEEE 33 Bus and a GtRE representing a section of Ado Ekiti (in Nigeria) low-voltage distribution grid are carried out for data generation and system evaluation. Feature extraction was carried out in a Python IDE using Stockwell transform. The voltage fluctuation events are detected and localized based on the extracted features using the trained FFNN model deployed and evaluated within three microcontroller-based computing devices. The proposed approach integrated anomaly-aware with edge computing to send only voltage data that are considered abnormal to a dedicated data center for visualization and storage. Performance evaluation of the proposed technique on the simulated GtRE demonstrates a significant decrease of 98% and 90% in latency when compared to cloud computing and conventional edge computing respectively. Comparison of the proposed approach to two closely related solutions in literature also demonstrates a 50% and 92.5 % reduction in latency. The contribution of the study is the reduced latency and minimal bandwidth utilization achieved by the implementation of the developed technique.

Overview of Artificial Neural Network-Based Solution for Ordinary and Partial Differential Equations by Feed Forward Method Using Python

In the modern era, models such as Neural Networks are increasingly used to deal with ordinary and partial differential equations (ODEs-PDEs) because of their related complexity. The main aim of the paper is to focus on Artificial Neural Networks as a method to solve these equations since it is possible to approximate complex nonlinear relationships with great accuracy and adaptively reduce computational costs, thus enhancing the accuracy of solutions within highly dimensional and nonlinear problems. The traditional methods are not applicable to partial differential equations due to the problem of nonlinearity and high dimensionality. However, the Feed Forward method implemented in Python is quite a powerful alternative that can easily approximate complicated functions and very well have a hold on nonlinear equations. This research paper contains differential equations as nonlinear and linear equations and Feed Forward neural network models for both solutions of ODEs and PDEs. Python enables the integration of AI-based algorithms for the enhancement of more accurate and efficient results. This work compares the existing solutions based on ANNs and further highlights the possible advantages of using ANNs in ordinary as well as complex differential equations problems. The results indicate that ANNs have immense potential for further developing computational algorithms applicable to solving ODEs and PDEs, particularly in real-time applications that demand high precision and adaptability.

Data-driven Smart Farming to Grade and Classify Tomatoes using CNN and FFNN for Agricultural Innovation

Identifying images poses a challenge in computer vision, but the use of deep learning methods has greatly enhanced the performance of image classification systems. In this research, Convolutional Neural Networks (CNN) and Feed Forward Neural Networks (FFNN) have been utilized for image classification. CNN is extremely effective in picture classification, which extracts relevant information from images using convolutional and pooling layers to minimize the dimensionality of the derived features, while FFNN algorithm is a classic neural network with fully linked layers. It can be used to further process the features extracted by CNN. The study makes use of CNN and FFNN models to train a huge dataset of tomato images to categorize them based on their type, ripeness, and damage status. CNN is found to be more effective in the case of tomato classification as compared to FFNN algorithm in all the use cases. The accuracy for classification of an image (tomato or not) using CNN is 95.83%, type classification using CNN is 81.52%, whereas using FFNN is 66.30%; ripeness grading for CNN is 92.86%, whereas for FFNN it is 57.14%; and damage status grading is 92.86% using CNN and 67.86% using FFNN. Therefore, it can be concluded that quality processing of tomatoes can be improved using CNN.

Estimating Models and Evaluating their Efficiency under Multicollinearity in Multiple Linear Regression: A Comparative Study

Multicollinearity between independent variables occurs in multiple linear regression analysis characterized by high correlations, which complicates discerning individual variable effect, impacting model accuracy, stability, and interpretation of relationships. The research aims to diagnose the multicollinearity problem between explanatory variables in the linear regression model and identifying the variables causing this problem based on the variance inflation factor (VIF), then estimation and evaluate the performance of three alternative methods, which are Ridge regression, principal components analysis, and Feedforward Neural Networks (FFNN) models with one and two hidden layers and application of the models to compressive strength data for high-performance concrete. The results showed that Ridge regression and PCA effectively addressed multicollinearity problem, but the single hidden layer model FFNN showed superior predictive accuracy in estimating the compressive strength of high-performance concrete when comparing RMSE, MAE, and R2 values.

Optimizing the Learning Rate Hyperparameter for Hybrid BiLSTM-FFNN Model in a Tourism Recommendation System

Indonesia, with its abundant natural resources, is rich in captivating tourist attractions. Tourism, a vital economic sector, can be significantly influenced by digitalization through social media. However, the overwhelming amount of information available can confuse tourists when selecting suitable destinations. This research aims to develop a tourism recommendation system employing content-based filtering (CBF) and hybrid Bidirectional Long Short-Term Memory Feed-Forward Neural Network (BiLSTM-FFNN) model to assist tourists in making informed choices. The dataset comprises 9,504 rating matrices obtained from tweet data and reputable web sources. In various experiments, the hybrid BiLSTM-FFNN model demonstrated superior performance, achieving an accuracy of 93.36% following optimization with the Stochastic Gradient Descent (SGD) algorithm at a learning rate of about 0.193. The accuracy, after applying Synthetic Minority Over-sampling Technique (SMOTE) and fine-tuning the learning rate hyperparameter, showed a 14.3% improvement over the baseline model. This research contributes by developing a recommendation system method that integrates CBF and hybrid deep learning with high accuracy and provides a detailed analysis of optimization techniques and hyperparameter tuning.

Deep Learning-Driven Analysis of a Six-Bar Mechanism for Personalized Gait Rehabilitation

Abstract Recent advances in robotics and artificial intelligence have highlighted the potential for the integration of computational intelligence in enhancing the functionality and adaptability of robotic systems, particularly in rehabilitation. Designing robotic exoskeletons for the lower limb rehabilitation of post-stroke patients requires frequent adjustments to accommodate individual differences in leg anatomy. This complex engineering challenge necessitates a deep understanding of human physiology, robotics, and optimization to develop adaptive robotic systems and also to swiftly quantify the required adjustments and implement them for each patient. The conventional approaches, which mostly rely on heuristics and manual tuning, often struggle to achieve optimal results. This paper presents a novel method that integrates a genetic algorithm (GA) with a deep learning approach to generate a gait trajectory of the ankle joint from a six-bar linkage mechanism of fixed dimensions. Later, using the same approach, the inverse kinematics solution for this mechanism is also devised whereby, the set of the link dimensions of the six-bar linkage mechanism is obtained for the given gait trajectory of an individual to achieve customization. We simulated the kinematic behavior of the six-bar linkage mechanism within defined mechanical constraints and utilized the generated data for training a feedforward neural network and long short-term memory models. The proposed model, when trained, can produce accurate lengths for the desired gait trajectories in the sagittal plane and vice-versa which further validates our proposed approach for inverse kinematics solution.

Industrial data-driven machine learning soft sensing for optimal operation of etching tools

Smart Manufacturing, or Industry 4.0, has gained significant attention in recent decades with the integration of Internet of Things (IoT) and Information Technologies (IT). As modern production methods continue to increase in complexity, there is a greater need to consider what variables can be physically measured. This advancement necessitates the use of physical sensors to comprehensively and directly gather measurable data on industrial processes; specifically, these sensors gather data that can be recontextualized into new process information. For example, artificial intelligence (AI) machine learning-based soft sensors can increase operational productivity and machine tool performance while still ensuring that critical product specifications are met. One industry that has a high volume of labor-intensive, time-consuming, and expensive processes is the semiconductor industry. AI machine learning methods can meet these challenges by taking in operational data and extracting process-specific information needed to meet the high product specifications of the industry. However, a key challenge is the availability of high quality data that covers the full operating range, including the day-to-day variance. This paper examines the applicability of soft sensing methods to the operational data of five industrial etching machines. Data is collected from readily accessible and cost-effective physical sensors installed on the tools that manage and control the operating conditions of the tool. The operational data are then used in an intelligent data aggregation approach that increases the scope and robustness for soft sensors in general by creating larger training datasets comprised of high value data with greater operational ranges and process variation. The generalized soft sensor can then be fine-tuned and validated for a particular machine. In this paper, we test the effects of data aggregation for high performing Feedforward Neural Network (FNN) models that are constructed in two ways: first as a classifier to estimate product PASS/FAIL outcomes and second as a regressor to quantitatively estimate oxide thickness. For PASS/FAIL classification, a data aggregation method is developed to enhance model predictive performance with larger training datasets. A statistical analysis method involving point-biserial correlation and the Mean Absolute Error (MAE) difference score is introduced to select the optimal candidate datasets for aggregation, further improving the effectiveness of data aggregation. For large datasets with high quality data that enable model training for more complex tasks, regression models that predict the oxide thickness of the product are also developed. Two types of models with different loss functions are tested to compare the effects of the Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE) loss functions on model performance. Both the classification and regression models can be applied in industrial settings as they provide additional information regarding the process outcome. Individually, these models can reduce the number of metrology steps in semiconductor factories, and when developed further, can empower the development of advanced process control strategies.

Machine Learning and Deep Learning Models for Predicting Noncovalent Inhibitors of AmpC β-Lactamase.

Continuous use of antibiotics leads to the ability of bacteria to adapt by developing complex antibiotic resistance (AR) mechanisms. The synthesis of β-lactamases is a widely observed AR mechanism. The class C β-lactamase (AmpC) causes significant resistance toward β-lactam antibiotics, and new treatments are urgently needed. Noncovalent inhibitors have been developed against a broad spectrum of β-lactamases. In this study, we developed robust and accurate models for predicting noncovalent inhibitors of AmpC using large compound data sets and machine/deep learning modeling. We created support vector machine (SVM), random forest (RF), and feed-forward neural network (FFNN) classification models. The cross-validation (CV) accuracies varied between 80 and 82%, as combined models reached an accuracy of 83%. We analyzed the physicochemical characteristics of the noncovalent inhibitors and predicted the binding modes for some of them. Such models are helpful for identifying new noncovalent inhibitors in order to establish novel solutions against the growing resistance to standard β-lactam inhibitors. The best RF, SVM, and FFNN models for predicting noncovalent inhibitors of AmpC β-lactamase are available in the GitHub repository, https://github.com/UPCmctr/ML-DL-AmpC-B-lactamase.

Impact of wearable device data and multi-scale entropy analysis on improving hospital readmission prediction.

Unplanned readmissions following a hospitalization remain common despite significant efforts to curtail these. Wearable devices may offer help identify patients at high risk for an unplanned readmission. We conducted a multi-center retrospective cohort study using data from the All of Us data repository. We included subjects with wearable data and developed a baseline Feedforward Neural Network (FNN) model and a Long Short-Term Memory (LSTM) time-series deep learning model to predict daily, unplanned rehospitalizations up to 90 days from discharge. In addition to demographic and laboratory data from subjects, post-discharge data input features include wearable data and multiscale entropy features based on intraday wearable time series. The most significant features in the LSTM model were determined by permutation feature importance testing. In sum, 612 patients met inclusion criteria. The complete LSTM model had a higher area under the receiver operating characteristic curve than the FNN model (0.83 vs 0.795). The 5 most important input features included variables from multiscale entropy (steps) and number of active steps per day. Data available from wearable devices can improve ability to predict readmissions. Prior work has focused on predictors available up to discharge or on additional data abstracted from wearable devices. Our results from 35 institutions highlight how multiscale entropy can improve readmission prediction and may impact future work in this domain. Wearable data and multiscale entropy can improve prediction of a deep-learning model to predict unplanned 90-day readmissions. Prospective studies are needed to validate these findings.

Efficient Solutions to Time-Fractional Telegraph Equations with Chebyshev Neural Networks

Abstract The objective of this study is to employ artificial neural networks (ANNs) as a new method for solving time fractional telegraph equations (TFTEs), which are typically tackled using the Caputo fractional derivative in scientific investigations. The utilisation of Chebyshev polynomials in place of a hidden layer serves to improve computing performance and expand the range of input patterns. A feed-forward neural network (NN) model is utilised, using Adam optimisation, to further refine network parameters and minimise mistakes. Two problems are selected, and the proposed method is employed to obtain their numerical solutions in order to showcase the effectiveness of the methodology. The efficiency of the proposed method is demonstrated by a comparison with the analytical solutions.

A statistical modelling approach to feedforward neural network model selection

Feedforward neural networks (FNNs) can be viewed as non-linear regression models, where covariates enter the model through a combination of weighted summations and non-linear functions. Although these models have some similarities to the approaches used within statistical modelling, the majority of neural network research has been conducted outside of the field of statistics. This has resulted in a lack of statistically based methodology, and, in particular, there has been little emphasis on model parsimony. Determining the input layer structure is analogous to variable selection, while the structure for the hidden layer relates to model complexity. In practice, neural network model selection is often carried out by comparing models using out-of-sample performance. However, in contrast, the construction of an associated likelihood function opens the door to information-criteria-based variable and architecture selection. A novel model selection method, which performs both input- and hidden-node selection, is proposed using the Bayesian information criterion (BIC) for FNNs. The choice of BIC over out-of-sample performance as the model selection objective function leads to an increased probability of recovering the true model, while parsimoniously achieving favourable out-of-sample performance. Simulation studies are used to evaluate and justify the proposed method, and applications on real data are investigated.

Monitoring and Predicting Air Quality with IoT Devices

The growing concern about air quality and its influence on human health has prompted the development of sophisticated monitoring and forecast systems. This article gives a thorough investigation into forecasting the air quality index (AQI) with an Internet of Things (IoT) device that analyzes temperature, humidity, PM10, and PM2.5 levels. The dataset used for this analysis comprises 5869 data points across six critical parameters essential for accurate air quality prediction. The data from these sensors is sent to the ThingSpeak cloud platform for storage and preliminary analysis. The system forecasts AQI using a TensorFlow-based regression model, delivering real-time insights. The combination of IoT technology and machine learning improves the accuracy and responsiveness of air quality monitoring systems, making it a useful tool for environmental management and public health protection. This work presents comparatively the effectiveness of feedforward neural network models trained with the ‘adam’ and ‘RMSprop’ optimizers over different epochs, as well as the machine learning algorithm random forest with varying numbers of estimators to forecast AQI. The models were trained using both types of regression analysis: linear regression and random forest regression. The findings show that the model achieves a high degree of accuracy, with the predictions closely aligning with the actual AQI values, thus having the potential to significantly reduce the negative health impact associated with poor air quality, protecting public health and alerting users when pollution levels are higher than allowed. Specifically, the random forest model with 100 estimators delivers the best overall performance for both AQI 10 and AQI 2.5, achieving the lowest Mean Absolute Error (MAE) of 0.2785 for AQI 10 and 0.2483 for AQI 2.5. This integration of IoT technology and advanced predictive analysis addresses the significant worldwide issue of air pollution by identifying the pollution hotspots and allowing decision-makers for quick reactions, and the development of effective strategies to reduce pollution sources.

Hyperspectral Imaging and Machine Learning: A Promising Tool for the Early Detection of Tetranychus urticae Koch Infestation in Cotton

Monitoring Tetranychus urticae Koch in cotton crops is challenging due to the vast crop areas and clustered mite attacks, hindering early infestation detection. Hyperspectral imaging offers a solution to such a challenge by capturing detailed spectral information for more accurate pest detection. This study evaluated machine learning models for classifying T. urticae infestation levels in cotton using proximal hyperspectral remote sensing. Leaf reflection data were collected over 21 days, covering various infestation levels: no infestation (0 mites/leaf), low (1–10), medium (11–30), and high (>30). Data were preprocessed, and spectral bands were selected to train six machine learning models, including Random Forest (RF), Principal Component Analysis–Linear Discriminant Analysis (PCA-LDA), Feedforward Neural Network (FNN), Support Vector Machine (SVM), k-Nearest Neighbor (kNN), and Partial Least Squares (PLS). Our analysis identified 31 out of 281 wavelengths in the near-infrared (NIR) region (817–941 nm) that achieved accuracies between 80% and 100% across 21 assessment days using Random Forest and Feedforward Neural Network models to distinguish infestation levels. The PCA loadings highlighted 907.69 nm as the most significant wavelength for differentiating levels of two-spotted mite infestation. These findings are significant for developing novel monitoring methodologies for T. urticae in cotton, offering insights for early detection, potential cost savings in cotton production, and the validation of the spectral signature of T. urticae damage, thus enabling more efficient monitoring methods.

Optimization of renewable energy hybrid power plant in the state of Karnataka, India

This paper proposes a methodology on reliability and cost minimization of the electricity produced by Renewable energy sources. The approach is to use Hybrid Renewable Energy Systems, which involves integrating various renewable sources. The objective of this study is to address these challenges by HRES method in the state of Karnataka. In Karnataka, there has been large scale curtailment of renewable energy generation due to the various structural issues. This study would give a solution to reliability and cost related issues faced by this state. For the study a Mathematical Model has been developed using Feed Forward Neural Network (FNN) and Chemical Reaction Optimization algorithms. Chemical reaction Optimization model is chosen to remove biases of the Artificial Neural Network Model of FNN. This model will be used to simulate interaction between Hybrid Renewable Energy Systems. The result of the study will provide a power system with high & optimized renewable energy output in the state of Karnataka, India which will be reliable and cost effective.

Predicting Market Capitalization of Large Global Companies using Ordinary Least Square, Feedforward and Bayesian Neural Network Models

Predicting Market Capitalization of Large Global Companies using Ordinary Least Square, Feedforward and Bayesian Neural Network Models

Artificial Intelligence Using FFNN Models for Computing Soil Complex Permittivity and Diesel Pollution Content

Soil pollution caused by hydrocarbons, such as diesel, poses significant risks to both human health and the ecosystem. The evaluation of soil pollution and various soil engineering applications often relies on the analysis of complex permittivity, encompassing parameters such as dielectric constant and dielectric loss. Various computational models, including theoretical physics-based models, mixture theory models, statistical empirical models, and artificial neural network (ANN) models, have been explored for computing soil complex permittivity and predicting water and pollutant content. Theoretical models require detailed data that is often unavailable, and thus have limited applicability. Mixture models tend to underestimate soil characteristics due to inaccuracies in permittivity estimation of soil phases. While empirical models are widely used, their applicability is restricted to specific soil types, datasets, and locations. ANN models offer promising predictions, accommodating nonlinear phenomena and allowing for missing information and variables. In this study, capacitive electromagnetic electrode sensors were utilized to determine the complex permittivity of soil contaminated with varying levels of diesel at different moisture levels. Theoretical mixture, empirical, and Feed Forward Neural Network (FFNN) models were employed to compute the permittivity of polluted soil based on its phases and to predict the level of diesel pollution. A comparison of these modeling approaches revealed that the FFNN model exhibited the best performance. The ANN model demonstrated superior performance metrics, including a high correlation coefficient and lower mean square error. Specifically, the correlation coefficients for the FFNN model were 0.9942 for training samples, 0.9967 for validation samples, and 0.9977 for test samples. Additionally, the ANN model yielded the lowest mean square error compared to the other three models. Doi: 10.28991/CEJ-2024-010-09-018 Full Text: PDF

Research on weathering recognition model of ancient glass based on three-layer neural network

In this paper, a three-layer neural network-based weathering recognition model for ancient glass is proposed to cope with the limitations of traditional weathering analysis methods in dealing with complexity and large-scale data. By constructing a multi-layer feed-forward neural network model containing input, hidden and output layers, this paper achieves high-precision classification of the weathering state of ancient glass. The experimental results show that the accuracy of the model reaches 98.0% and 100% on the training set and test set, respectively, which proves the effectiveness and efficiency of the method in the recognition of ancient glass weathering. This study demonstrates the potential of neural network application in cultural relics conservation and provides a theoretical basis for further optimisation of cultural relics identification techniques.

Artificial Neural Network-driven Optimization of Fe3O4 Nanoparticles/PVDF Macrospheres in Fenton-like System for Methylene Blue Degradation

Efficient degradation of industrial dyes remains a critical challenge in environmental engineering. This study introduces a novel Fe3O4 nanoparticles/PVDF macrospheres in a Fenton-like system, optimized using an Artificial Neural Network (ANN) for the degradation of Methylene Blue (MB). A feedforward backpropagation neural network model to optimize and predict the performance of this advanced oxidation process under various operational conditions. The model was trained, validated, and tested with robust datasets, demonstrating high predictive accuracy and generalization capability. The Mean Square Error (MSE) and Root Mean Square Error (RMSE) during testing were 0.0200 and 0.1414, respectively, indicating precise predictions. The coefficient of determination (R²) and correlation coefficient (R) were exceptionally high at 0.9744 and 0.9871, affirming the model's ability to capture the underlying dynamics of the degradation process effectively. The ANN-driven approach not only enhanced the efficiency of the MB degradation process but also provided significant insights into the scalability and applicability of the Fe3O4/PVDF system for practical water treatment solutions. This study underscores the potential of integrating advanced machine learning techniques with chemical engineering processes to achieve sustainable and efficient environmental management solutions, particularly for the treatment of recalcitrant wastewater contaminants.