Feedforward Network Research Articles

Utility of Certain AI Models in Climate-Induced Disasters

To address the current challenge of climate change at the local and global levels, this article discusses a few important water resources engineering topics, such as estimating the energy dissipation of flowing waters over hilly areas through the provision of regulated stepped channels, predicting the removal of silt deposition in the irrigation canal, and predicting groundwater level. Artificial intelligence (AI) in water resource engineering is now one of the most active study topics. As a result, multiple AI tools such as Random Forest (RF), Random Tree (RT), M5P (M5 model trees), M5Rules, Feed-Forward Neural Networks (FFNNs), Gradient Boosting Machine (GBM), Adaptive Boosting (AdaBoost), and Support Vector Machines kernel-based model (SVM-Pearson VII Universal Kernel, Radial Basis Function) are tested in the present study using various combinations of datasets. However, in various circumstances, including predicting energy dissipation of stepped channels and silt deposition in rivers, AI techniques outperformed the traditional approach in the literature. Out of all the models, the GBM model performed better than other AI tools in both the field of energy dissipation of stepped channels with a coefficient of determination (R2) of 0.998, root mean square error (RMSE) of 0.00182, and mean absolute error (MAE) of 0.0016 and sediment trapping efficiency of vortex tube ejector with an R2 of 0.997, RMSE of 0.769, and MAE of 0.531 during testing. On the other hand, the AI technique could not adequately understand the diversity in groundwater level datasets using field data from various stations. According to the current study, the AI tool works well in some fields of water resource engineering, but it has difficulty in other domains in capturing the diversity of datasets.

Network anomaly detection using Deep Autoencoder and parallel Artificial Bee Colony algorithm-trained neural network

Cyberattacks are increasingly becoming more complex, which makes intrusion detection extremely difficult. Several intrusion detection approaches have been developed in the literature and utilized to tackle computer security intrusions. Implementing machine learning and deep learning models for network intrusion detection has been a topic of active research in cybersecurity. In this study, artificial neural networks (ANNs), a type of machine learning algorithm, are employed to determine optimal network weight sets during the training phase. Conventional training algorithms, such as back-propagation, may encounter challenges in optimization due to being entrapped within local minima during the iterative optimization process; global search strategies can be slow at locating global minima, and they may suffer from a low detection rate. In the ANN training, the Artificial Bee Colony (ABC) algorithm enables the avoidance of local minimum solutions by conducting a high-performance search in the solution space but it needs some modifications. To address these challenges, this work suggests a Deep Autoencoder (DAE)-based, vectorized, and parallelized ABC algorithm for training feed-forward artificial neural networks, which is tested on the UNSW-NB15 and NF-UNSW-NB15-v2 datasets. Our experimental results demonstrate that the proposed DAE-based parallel ABC-ANN outperforms existing metaheuristics, showing notable improvements in network intrusion detection. The experimental results reveal a notable improvement in network intrusion detection through this proposed approach, exhibiting an increase in detection rate (DR) by 0.76 to 0.81 and a reduction in false alarm rate (FAR) by 0.016 to 0.005 compared to the ANN-BP algorithm on the UNSW-NB15 dataset. Furthermore, there is a reduction in FAR by 0.006 to 0.0003 compared to the ANN-BP algorithm on the NF-UNSW-NB15-v2 dataset. These findings underscore the effectiveness of our proposed approach in enhancing network security against network intrusions.

Indirect measurement of bridge surface roughness using vibration responses of a two-axle moving vehicle based on physics-constrained generative adversarial network

This study addresses the challenge of indirectly measuring bridge surface roughness through the vibration responses of a moving vehicle, which is crucial for pavement maintenance and bridge safety assessment. A physics-constrained generative adversarial network (PC-GAN) was proposed for the probabilistic estimation of surface roughness. The method consists of two steps: initially, a GAN informed by physics-based knowledge extracts combined information of bridge vibration deflection and surface roughness from vehicle accelerations. Subsequently, a feed-forward network isolates the bridge surface roughness from the combined data. Numerical examples validate the PC-GAN method, demonstrating sustained high accuracy under challenging conditions, including ISO 8608 level C road roughness, vehicle speeds up to 8 m s-1, 10 % deviation in vehicle parameters, 10 % environmental noise, and 10 % vehicle damping ratio. Laboratory tests further confirmed the method's efficacy, with the successful detection of artificial barriers on the bridge surface and a mean relative error of 3.33 % in height estimation. The PC-GAN method is demonstrated to be a robust tool for estimating bridge surface roughness under various numerical and laboratory conditions. These findings provide valuable insights for the rapid inspection of bridge pavement conditions using vibration responses from moving test vehicles.

Sequential Recommendation Model Based on Deep Residual Recurrent Neural Network

In order to solve the problem that the sequence recommendation model based on RNN (Recurrent Neural Network) is prone to gradient vanishing or exploding when processing long sequences, which leads to the instability of the recommendation model training process, residual connection, layer normalization and feedforward neural network modules are introduced on the basis of the traditional gated recurrent unit (GRU), and a sequence recommendation model DeepGRU based on deep residual recurrent neural network is proposed. It is verified on 3 public datasets. The experimental results show that the DeepGRU has obvious advantages over the most advanced sequence recommendation methods (the average recommendation accuracy is improved 8.68%). The ablation experiment verifies the effectiveness of the introduced residual connection and other modules under the DeepGRU framework. In addition, the DeepGRU effectively alleviates the problem of unstable training process when processing long sequences.

Machine learning model‐based optimal tracking control of nonlinear affine systems with safety constraints

AbstractThis work focuses on the development of a machine learning (ML) model‐based framework for safe optimal tracking control of a class of nonlinear control‐affine systems to ensure simultaneous closed‐loop stability and safety. Specifically, a novel multilayer feedforward neural network (FNN) with a control‐affine architecture is designed to model nonlinear dynamic systems. Subsequently, a model‐based reinforcement learning (RL) framework is presented, utilizing a novel cost function with Control Lyapunov‐Barrier Function (CLBF) properties, to learn both the control policy and the optimal value function for an infinite‐horizon optimal tracking control problem for nonlinear systems with safety constraints. The efficacy of the proposed methodology is demonstrated through simulations of a one‐link robot manipulator and a chemical process example.

A Wasserstein perspective of Vanilla GANs

The empirical success of Generative Adversarial Networks (GANs) caused an increasing interest in theoretical research. The statistical literature is mainly focused on Wasserstein GANs and generalizations thereof, which especially allow for good dimension reduction properties. Statistical results for Vanilla GANs, the original optimization problem, are still rather limited and require assumptions such as smooth activation functions and equal dimensions of the latent space and the ambient space. To bridge this gap, we draw a connection from Vanilla GANs to the Wasserstein distance. By doing so, existing results for Wasserstein GANs can be extended to Vanilla GANs. In particular, we obtain an oracle inequality for Vanilla GANs in Wasserstein distance. The assumptions of this oracle inequality are designed to be satisfied by network architectures commonly used in practice, such as feedforward ReLU networks. By providing a quantitative result for the approximation of a Lipschitz function by a feedforward ReLU network with bounded Hölder norm, we conclude a rate of convergence for Vanilla GANs as well as Wasserstein GANs as estimators of the unknown probability distribution.

Applicability domain of a calibration model based on neural networks and infrared spectroscopy

Artificial neural networks are used as calibration models in routine analytical determinations that involve spectroscopic data. To ensure that the model will generate reliable predictions for new samples, the applicability domain must be well defined. This article describes a strategy for establishing the limits of the applicability domain when the calibration model is a feed-forward neural network. The applicability domain was defined by two limits: 1) the 0.99 quantile of the squared Mahalanobis distance calculated from the network activations of the training set and 2) the 0.99 quantile of the reconstruction error of the training spectra using either an autoencoder network or a decoder network. A new sample with a squared Mahalanobis distance and/or spectral residuals beyond these limits is said to be outside the applicability domain, and the prediction is questionable. The approach was illustrated by predicting the density of diesel fuel samples from mid-infrared spectra and the fat content in meat from near-infrared spectra. The methodology could correctly detect anomalous spectra in prediction using either the autoencoder or the decoder.

Prediction of one-dimensional acoustic field with axial temperature gradient using neural networks

The study of sound propagation inside the ducts finds extensive application in aerospace, automobiles, speech, and biomedical sectors. This paper presents a neural network-based formulation to estimate the acoustic field inside a uniform duct with temperature gradient along the axial direction. The governing differential equation is derived from the momentum, energy, and state equations. The acoustic field is approximated with a feedforward neural network, and the problem is converted to an unconstrained optimization problem using the trial solution method. The training process is performed using the L-BFGS optimizer and the sine activation function. The acoustic field inside the duct is predicted up to 2000 Hz for linear and exponential temperature profiles. The predicted results are in good agreement with those obtained from the traditional Range-Kutta solver with a maximum relative error of 0.1%. Furthermore, the classical Helmholtz equation without the temperature gradient is solved using the developed neural network formulation. Using these results, a comparative study is conducted to understand the effect of the temperature gradient on the acoustic field inside the duct.

Validation of large language models for detecting pathologic complete response in breast cancer using population-based pathology reports

AimsThe primary goal of this study is to evaluate the capabilities of Large Language Models (LLMs) in understanding and processing complex medical documentation. We chose to focus on the identification of pathologic complete response (pCR) in narrative pathology reports. This approach aims to contribute to the advancement of comprehensive reporting, health research, and public health surveillance, thereby enhancing patient care and breast cancer management strategies.MethodsThe study utilized two analytical pipelines, developed with open-source LLMs within the healthcare system’s computing environment. First, we extracted embeddings from pathology reports using 15 different transformer-based models and then employed logistic regression on these embeddings to classify the presence or absence of pCR. Secondly, we fine-tuned the Generative Pre-trained Transformer-2 (GPT-2) model by attaching a simple feed-forward neural network (FFNN) layer to improve the detection performance of pCR from pathology reports.ResultsIn a cohort of 351 female breast cancer patients who underwent neoadjuvant chemotherapy (NAC) and subsequent surgery between 2010 and 2017 in Calgary, the optimized method displayed a sensitivity of 95.3% (95%CI: 84.0–100.0%), a positive predictive value of 90.9% (95%CI: 76.5–100.0%), and an F1 score of 93.0% (95%CI: 83.7–100.0%). The results, achieved through diverse LLM integration, surpassed traditional machine learning models, underscoring the potential of LLMs in clinical pathology information extraction.ConclusionsThe study successfully demonstrates the efficacy of LLMs in interpreting and processing digital pathology data, particularly for determining pCR in breast cancer patients post-NAC. The superior performance of LLM-based pipelines over traditional models highlights their significant potential in extracting and analyzing key clinical data from narrative reports. While promising, these findings highlight the need for future external validation to confirm the reliability and broader applicability of these methods.

Artificial Neural Networks to Predict Metabolic Syndrome without Invasive Methods in Adolescents.

Background/Objectives: The prevalence of metabolic syndrome (MetS) is increasing worldwide, and an increasing number of cases are diagnosed in younger age groups. This study aimed to propose predictive models based on demographic, anthropometric, and non-invasive clinical variables to predict MetS in adolescents. Methods: A total of 2064 adolescents aged 18-19 from São Luís-Maranhão, Brazil were enrolled. Demographic, anthropometric, and clinical variables were considered, and three criteria for diagnosing MetS were employed: Cook et al., De Ferranti et al. and the International Diabetes Federation (IDF). A feed-forward artificial neural network (ANN) was trained to predict MetS. Accuracy, sensitivity, and specificity were calculated to assess the ANN's performance. The ROC curve was constructed, and the area under the curve was analyzed to assess the discriminatory power of the networks. Results: The prevalence of MetS in adolescents ranged from 5.7% to 12.3%. The ANN that used the Cook et al. criterion performed best in predicting MetS. ANN 5, which included age, sex, waist circumference, weight, and systolic and diastolic blood pressure, showed the best performance and discriminatory power (sensitivity, 89.8%; accuracy, 86.8%). ANN 3 considered the same variables, except for weight, and exhibited good sensitivity (89.0%) and accuracy (87.0%). Conclusions: Using non-invasive measures allows for predicting MetS in adolescents, thereby guiding the flow of care in primary healthcare and optimizing the management of public resources.

Mechanism and Data-Driven Fusion SOC Estimation

An accurate assessment of the state of charge (SOC) of electric vehicle batteries is critical for implementing frequency regulation and peak shaving. This study proposes mechanism- and data-driven SOC fusion calculation methods. First, a second-order Thevenin battery model is developed to obtain the physical parameters of the battery. Second, data from the Thevenin battery model and data from four standard cycling conditions in the electric vehicle industry are added to the dataset of the feed-forward neural network data-driven model to construct the test and training sets of the data-driven model. Finally, the error of the mechanism and data-driven fusion modeling method is quantitatively analyzed by comparing the estimation error of the method for the battery SOC at different temperatures with the accuracy of the data-driven SOC estimation method. The simulation results show that the root mean square error, the mean age absolute error, and the maximum error of mechanism and data-driven method for the estimation error of battery SOC are lower than those of the data-driven method by 0.9%, 0.65%, and 1.3%, respectively. The results show that the mechanism and data-driven fusion SOC estimation method has better generalization performance and higher SOC estimation accuracy.

Artificial Neural Networks for Classification Tasks: A Systematic Literature Review

Artificial neural networks (ANNs) have become indispensable tools for solving classification tasks across various domains. This systematic literature review explores the landscape of ANN utilization in classification, addressing three key research questions: the types of architectures employed, their accuracy, and the data utilized. The review encompasses 30 studies published between 2019 and 2024, revealing Convolutional Neural Networks (CNNs) as the predominant architecture in image-related tasks, followed by Multilayer Perceptron (MLP) architectures for general classification tasks. Feed Forward Neural Networks (FFNN) exhibited the highest average accuracy with a 97.12%, with specific studies achieving exceptional results across diverse classification tasks. Moreover, the review identifies digitized images as a commonly utilized data source, reflecting the broad applicability of ANNs in tasks such as medical diagnosis and remote sensing. The findings underscore the importance of machine learning approaches, highlight the robustness of ANNs in achieving high accuracy, and suggest avenues for future research to enhance interpretability, efficiency, and generalization capabilities, as well as address challenges related to data quality.

Comparison and integration of physical and interpretable AI-driven models for rainfall-runoff simulation

Precise streamflow forecasting in river systems is crucial for water resources management and flood risk assessment. The Tagus Headwaters River Basin (THRB) in Spain is a key hydrological hub, providing regulated flow for agricultural, urban, and energy sectors, and facilitating water transfer to the Segura River Basin, a key semi-arid region. Given the basin's near-total allocation of water resources, accurate streamflow simulations are essential to optimize the socio-economic distribution and ensure sustainable management across both interconnected basins. This study conducts a comprehensive evaluation of the Soil and Water Assessment Tool (SWAT+), support vector regression (SVR), feed forward neural network (FFNN), and long short-term memory (LSTM) models in simulating the rainfall-runoff process at four gauging stations within the THRB. An ensemble machine learning technique is then applied to assess improvements in streamflow estimation. Results revealed that AI-based models significantly surpassed the SWAT+ model in performance. Furthermore, the application of an ensemble technique enhanced the precision of rainfall-runoff modeling by 18 to 26% during the calibration period and 4.1 to 9.2% during the validation period, compared to individual AI-based models. Additionally, the SWAT+ model's precision improved by 44 to 74% and 40 to 55% for the respective periods. The use of Shapley Additive Explanations (SHAP) methodology allowed the results of the ensemble with machine learning to be more interpretable by explaining how each model contributes to the prediction. This research offers significant contributions to hydrological modeling, highlighting the importance of ensemble techniques in elevating predictive accuracy for various river basins.

Ứng dụng phương pháp học máy dự báo mức độ căng thẳng tài chính của doanh nghiệp niêm yết tại Việt Nam

This study utilizes the traditional statistical method and machine learning methods to forecast the level of financial distress of listed companies in Vietnam. The data used in the study was collected from 646 listed companies on the HOSE and HNX exchanges during the period from 2012 to 2022. The study experiments with the dataset on 6 machine learning models: Multiple Regression Model (MRL), LASSO, Generalized Additive Model (GAM), Random Forests (RF), Gradient Boosting Regression Trees (GBRT), and a single layer, feed forward neural network (NN), as well as the traditional statistical method. The research results demonstrate a congruence between the outcomes of the traditional method and modern machine learning models. Among these, the model with the highest accuracy is Random Forest (RF) with an accuracy rate of 98.8%. The variables that most influence the financial distress status of companies are TANG, ROA, LTD, NPM. Based on these results, our study proposes recommendations to support informed and effective financial decisions for stakeholders (companies, regulatory agencies, shareholders, creditors, investors).

Hybrid Approach of Multiclassification for Lung Cancer Types Identification

As a widespread health concern, lung cancer requires sophisticated detection techniques for a precise classification. This research proposes an integrated framework for improving how well lung cancer is detected by combining processing methods for images and machine learning. The dataset, which includes normal cases, large-cell carcinoma, squamous cell carcinoma, and adenocarcinoma, allows for a detailed investigation of various forms of lung cancer. K-means clustering is the initial stage of the approach of segmenting complex images. Utilizing the VGG16 model, features are extracted that are important for classification. Several classification models, such as SVM, Logistic Regression, and Feedforward Neural Network, are trained using the combined features from clustering and VGG16. This research goes beyond the dichotomy of benign and malignant cases, investigating in more detail the subtypes of large-cell carcinoma, squamous cell carcinoma, and adenocarcinoma. Including different classes of lung cancer increases the granularity of the classification process, improving diagnostic accuracy and clinical insights. The results highlight how well the suggested strategy works, which is a major advancement in the accurate identification of distinct kinds of lung cancer.

DeepMarkerNet: Leveraging supervision from the Duchenne Marker for spontaneous smile recognition

Distinguishing between spontaneous and posed smiles from videos poses a significant challenge in pattern classification literature. Researchers have developed feature-based and deep learning-based solutions for this problem. To this end, deep learning outperforms feature-based methods. However, certain aspects of feature-based methods could improve deep learning methods. For example, previous research has shown that Duchenne Marker (or D-Marker) features from the face play a vital role in spontaneous smiles, which can be useful to improve deep learning performances. In this study, we propose a deep learning solution that leverages D-Marker features to improve performance further. Our multi-task learning framework, named DeepMarkerNet, integrates a transformer network with the utilization of facial D-Markers for accurate smile classification. Unlike past methods, our approach simultaneously predicts the class of the smile and associated facial D-Markers using two different feed-forward neural networks, thus creating a symbiotic relationship that enriches the learning process. The novelty of our approach lies in incorporating supervisory signals from the pre-calculated D-Markers (instead of as input in previous works), harmonizing the loss functions through a weighted average. In this way, our training utilizes the benefits of D-Markers, but the inference does not require computing the D-Marker. We validate our model’s effectiveness on four well-known smile datasets: UvA-NEMO, BBC, MMI facial expression, and SPOS datasets, and achieve state-of-the-art results.

Research on turbulence model parameters for the prediction of waterjet propulsion pump hydrodynamic performance

Abstract The hydrodynamic performance of waterjet propulsion pumps has an important impact on improving the speed of waterjet propulsion ships. High-precision prediction of the hydrodynamic performance of water-jet propulsion pumps based on CFD is of great significance to improving the pump design quality. Based on the Reynolds-averaged Naiver-Stokes equation, this paper conducts a grid-independence verification on a water jet propulsion pump based on hydrodynamic test data, uses the Latin hypercube sampling method to generate SST k-ω turbulence model parameter space calculation samples, performs steady-state calculation on all samples, analyzes the sensitivity of changes in the parameters of the turbulence model to the prediction of the external characteristics of the waterjet propulsion pump, and finally obtains the influence of changes in the parameters of the turbulence model on the prediction of the external characteristics of the pump. Among them, the sensitivities of the three parameters β *, β1 and a1 are higher. The results show that the prediction accuracy of the external characteristics of the water jet propulsion pump can be effectively improved by correcting these three highly sensitive parameters, and the correction values of the three key parameters are obtained. The prediction error of the external characteristics of the water jet propulsion pump is reduced from 4.3% to less than 2.1%. A feedforward neural network is used to establish a mapping model between the three turbulence model parameters and the head coefficient and power coefficient of the waterjet propulsion pump in order to achieve rapid prediction. The genetic algorithm is applied to optimize the weights, thresholds and other coefficients in the neural network, thereby improving the prediction accuracy. Based on the optimized neural network, the genetic algorithm is used to optimize the three turbulence model parameters, and the optimized coefficients are obtained. The optimized turbulence model parameters are put into the hydrodynamic model for RANS simulation to verify the degree of optimization of the model coefficients by the genetic algorithm.

Identification and Quantification of Common Adulterants in Extra Virgin Olive Oil Using Microwave Dielectric Spectroscopy Aided by Feedforward Neural Networks

Identification and Quantification of Common Adulterants in Extra Virgin Olive Oil Using Microwave Dielectric Spectroscopy Aided by Feedforward Neural Networks

Industrial data-driven machine learning soft sensing for optimal operation of etching tools

Smart Manufacturing, or Industry 4.0, has gained significant attention in recent decades with the integration of Internet of Things (IoT) and Information Technologies (IT). As modern production methods continue to increase in complexity, there is a greater need to consider what variables can be physically measured. This advancement necessitates the use of physical sensors to comprehensively and directly gather measurable data on industrial processes; specifically, these sensors gather data that can be recontextualized into new process information. For example, artificial intelligence (AI) machine learning-based soft sensors can increase operational productivity and machine tool performance while still ensuring that critical product specifications are met. One industry that has a high volume of labor-intensive, time-consuming, and expensive processes is the semiconductor industry. AI machine learning methods can meet these challenges by taking in operational data and extracting process-specific information needed to meet the high product specifications of the industry. However, a key challenge is the availability of high quality data that covers the full operating range, including the day-to-day variance. This paper examines the applicability of soft sensing methods to the operational data of five industrial etching machines. Data is collected from readily accessible and cost-effective physical sensors installed on the tools that manage and control the operating conditions of the tool. The operational data are then used in an intelligent data aggregation approach that increases the scope and robustness for soft sensors in general by creating larger training datasets comprised of high value data with greater operational ranges and process variation. The generalized soft sensor can then be fine-tuned and validated for a particular machine. In this paper, we test the effects of data aggregation for high performing Feedforward Neural Network (FNN) models that are constructed in two ways: first as a classifier to estimate product PASS/FAIL outcomes and second as a regressor to quantitatively estimate oxide thickness. For PASS/FAIL classification, a data aggregation method is developed to enhance model predictive performance with larger training datasets. A statistical analysis method involving point-biserial correlation and the Mean Absolute Error (MAE) difference score is introduced to select the optimal candidate datasets for aggregation, further improving the effectiveness of data aggregation. For large datasets with high quality data that enable model training for more complex tasks, regression models that predict the oxide thickness of the product are also developed. Two types of models with different loss functions are tested to compare the effects of the Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE) loss functions on model performance. Both the classification and regression models can be applied in industrial settings as they provide additional information regarding the process outcome. Individually, these models can reduce the number of metrology steps in semiconductor factories, and when developed further, can empower the development of advanced process control strategies.

Remote sensing image classification based on non-linear enhanced attention mechanism

Abstract In recent years, Transformer technology has gradually shown promising applications in the field of computer vision, becoming a research hotspot. However, traditional vision Transformers suffer from significant computational burdens. Although previous studies have attempted to alleviate this issue by reducing the computational load of attention mechanisms, their methods still require improvement. Additionally, past research often overlooked the non-linear representation of values within attention mechanisms, posing another challenge to address. Therefore, this paper proposes a novel attention mechanism aimed at reducing the computational burden of traditional attention mechanisms while enhancing their ability to express non-linearities in values. Furthermore, conventional feedforward neural networks typically output results directly after passing through an activation function, which may limit the model’s representational capacity due to the simplistic structure leading to singular feature information. To address this issue, this study introduces a more robust feature fusion convolutional feedforward network. Through these methods, the aim is to enhance the accuracy of Transformers in the field of remote-sensing image classification.