Overfitting Prevention in Accident Prediction Models: Bayesian Regularization of Artificial Neural Networks

Estimating groundwater quality parameters through conventional methods is time-consuming through laboratory measurements for megacities. There is a need to develop models that can help decision-makers make policies for sustainable groundwater reserves. The current study compared the efficiency of multivariate linear regressions (MLR) and artificial neural network (ANN) models in the prediction of groundwater parameters for total dissolved solids (TDS) for three sub-divisions in Lahore, Pakistan. The data for this study were collected every quarter of a year for six years. ANN was applied to investigate the feasibility of feedforward, backpropagation neural networks with three training functions T-BR (Bayesian regularization backpropagation), T-LM (Levenberg–Marquardt backpropagation), and T-SCG (scaled conjugate backpropagation). Two activation functions were used to analyze the performance of algorithmic training functions, i.e., Logsig and Tanh. Input parameters of pH, electrical conductivity (EC), calcium (Ca2+), magnesium (Mg2+), chloride (Cl−), and sulfate (SO42−) was used to predict TDS as an output parameter. The computed values of TDS by ANN and MLR were in close agreement with their respective measured values. Comparative analysis of ANN and MLR showed that TDS root means square error (RMSE) for city sub-division and Pearson’s coefficient of correlation (r) for ANN and MLR were 2.9% and 0.981 and 4.5% and 0.978, respectively. Similarly, for the Farrukhabad sub-division, RMSE and r for ANN were 4.9% and 0.952, while RMSE and r for MLR were 5.5% and 0.941, respectively. For the Shahadra sub-division, RMSE was 10.8%, r was 0.869 for ANN, RMSE was 11.3%, and r was 0.860 for MLR. The results exhibited that the ANN model showed less error in results than MLR. Therefore, ANN can be employed successfully as a groundwater quality prediction tool for TDS assessment.

Use of Accident Prediction Models in Road Safety Management – An International Inquiry

A flood is an extremely dangerous disaster that can wipe away an entire city, coastline, and rural area. The flood can cause wide destrotion to property and life that has the supreme corrosive force and can be highly damaging. In order to decrease the damages caused by the flood, an Artificial Neural Network (ANN) model has been established to predict flood in Sungai Isap, Kuantan, Pahang, Malaysia. This model is able to initiate the same brain thinking process and avoid the influence of the predict judgment. In this paper, presentation and comparison that using Bayesian Regularization (BR) back-propagation, Levenberg-Marquardt (LM) back-propagation and Gradient Descent (GD) back-propagation algorithms will be organized and carry out the result flood prediction. The predicted result of the Bayesian Regularization indicates a satisfactory performance. The conclusions also indicate that Bayesian Regularization is more versatile than Levenberg-Marquart and Gradient Descent with that can be backup or a practical tool for flood prediction. Temperature, precipitation, dew point, humidity, sea level pressure, visibility, wind, and river level data collected from January 2013 until May 2015 in the city of Sungai Isap, Kuantan is used for training, validation, and testing of the network model. The comparison is shown on the basis of mean square error (MSE) and regression (R). The prediction by training function Bayesian Regularization back-propagation found to be more suitable to predict flood model.

This paper presents a comparative analysis of Levenberg-Marquardt (LM) and Bayesian Regularization (BR) backpropagation algorithms in development of different Artificial Neural Networks (ANNs) to estimate the output power of a Photovoltaic (PV) module. The proposed ANNs undergo training, validation and testing phases on 10000+ combinations of data including the real-time measurements of irradiance level (W/m2) and PV output power (W) as well as the calculations of the Sun’s position in the sky and the PV module surface temperature (°C). The overall performance of the LM and the BR algorithms are analyzed during the development phases of the ANNs, and also the results of implementation of each ANN in different time intervals with different input types are compared. The comparative study presents the trade-offs of utilizing LM and BR algorithms in order to develop the best ANN architecture for PV output power estimation.

In this study, three artificial neural network (ANN) models are developed with back propagation (BP) optimization algorithms to predict various lightning damage modes in carbon/epoxy laminates. The proposed ANN models use three input variables associated with lightning waveform parameters (i.e., the peak current amplitude, rising time, and decaying time) to predict fiber damage, matrix damage, and through-thickness damage in the composites. The data used for training and testing the networks was actual lightning damage data collected from peer-reviewed published literature. Various BP training algorithms and network architecture configurations (i.e., data splitting, the number of neurons in a hidden layer, and the number of hidden layers) have been tested to improve the performance of the neural networks. Among the various BP algorithms considered, the Bayesian regularization back propagation (BRBP) showed the overall best performance in lightning damage prediction. When using the BRBP algorithm, as expected, the greater the fraction of the collected data that is allocated to the training dataset, the better the network is trained. In addition, the optimal ANN architecture was found to have a single hidden layer with 20 neurons. The ANN models proposed in this work may prove useful in preliminary assessments of lightning damage and reduce the number of expensive experimental lightning tests.

The groundwater level is required to keep within the permissible limit for sustainable groundwater development in any area. In the present study, an Artificial Neural Network (ANN) model has been developed for groundwater development with respect to state variables of a groundwater system, i.e., a maximum depth to water table for agricultural purposes. The zonal cropping areas are considered as inputs to the ANN model. The methodology has been illustrated in the Yamuna-Hindon Inter basin, India. The ANN model is performed for two different training algorithms like (i) Levenberg–Marquardt (LM) and (ii) Bayesian regularization (BR) and their performance was compared with the backpropagation (BP) algorithm. The prediction accuracy of both algorithms was tested using performance indices viz. mean square error (MSE), root mean square error (RMSE), and correlation coefficient (R2). The performance of both the ANN training algorithms in predicting maximum depth to water table over the study area was found to be almost similarly good. However, the performance of the LM algorithm was found slightly superior to that of the BR algorithm as well as the BP algorithm.KeywordsArtificial Neural Network (ANN)Feedforward Multilayer Neural Network (FNN)Levenberg–Marquardt algorithm (LM)Bayesian regularization algorithm (BR)Groundwater modeling

The search for better climate change adaptation techniques for addressing environmental and economic issues due to changing climate is of paramount interest in the current era. One of the many ways Pacific Island regions and its people get affected is by dry spells and drought events from extreme climates. A drought is simply a prolonged shortage of water supply in an area. The impact of drought varies both temporally and spatially that can be catastrophic for such regions with lack of resources and facilities to mitigate the drought impacts. Therefore, forecasting drought events using predictive models that have practical implications for understanding drought hydrology and water resources management can allow enough time to take appropriate adaption measures. This study investigates the feasibility of the Artificial Neural Network (ANN) algorithms for prediction of a drought index: Standardized Precipitation-Evapotranspiration Index (SPEI). The purpose of the study was to develop an ANN model to predict the index in two selected regions in Queensland, Australia. The first region, is named as the grassland and the second as the temperate region. The monthly gridded meteorological variables (precipitation, maximum and minimum temperature) that acted as input parameters in ANN model were obtained from Australian Water Availability Project (AWAP) for 1915–2013 period. The potential evapotranspiration (PET), calculated using thornthwaite method, was also an input variable, while SPEI was the predictand for the ANN model. The input data were divided into training (80%), validation (10%) and testing (10%) sets. To determine the optimum ANN model, the Levenberg-Marquardt and Broyden-Fletcher-Goldfarb-Shanno quasi-Newton backpropagation algorithms were used for training the ANN network and the tangent sigmoid, logarithmic sigmoid and linear activation algorithms were used for hidden transfer and output functions. The best architecture of input-hidden neuron-output neurons was 4-28-1 and 4-27-1 for grassland and temperate region, respectively. For evaluation and selection of the optimum ANN model, the statistical metrics: Coefficient of Determination (R 2 ), Willmott’s Index of Agreement (d), Nash-Sutcliffe Coefficient of Efficiency (E), Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE) were employed. The R 2 , d, E, RMSE and MAE for optimum ANN models were 0.9839, 0.9909, 0.9838, 0.1338, 0.0882 and 0.9886, 0.9935, 0.9874, 0.1198, 0.0814 for grassland and temperate region, respectively. When prediction errors were analysed, a value of 0.0025 to 0.8224 was obtained for the grassland region, and a value of 0.0113 to 0.6667 was obtained for the temperate region, indicating that the ANN model exhibit a good skill in predicting the monthly SPEI. Based on the evaluation and statistical analysis of the predicted SPEI and its errors in the test period, we conclude that the ANN model can be used as a useful data-driven tool for forecasting drought events. Broadly, the ANN model can be applied for prediction of other climate related variables, and therefore can play a vital role in the development of climate change adaptation and mitigation plans in developed and developing nations, and most importantly, in the Pacific Island Nations where drought events have a detrimental impact on economic development.

The main meteorological parameters which influencing the rainfall can be distilled from the MODIS satellite cloud imagery and the artificial neural network (ANN) model constructed by these meteorological parameters and can be applied on distributed rainfall estimation. Because it is difficult to decide the structure of back propagation neural network (BPNN) and to solve the problem of local convergence, an appropriate training and modeling method of ANN such as the real code genetic algorithm (RGA) is vital to the accuracy of rainfall estimation. The data of the simulation tests show that the Mean Relative Error (MRE) of BPA model is 23.6%, while the MRE of RGA model is 20.7%, Compared with the ANN trained by BPA, the estimation error of the ANN trained by RGA is cut down by 2.9%, and the Root Mean Squared Error (RMSE) is cut down by 2.5% at the same time, hence, the results prove that the ANN model trained using RGA will significantly outperform the back propagation algorithm (BPA) trained ANN model and improve the precision of rainfall estimation. Keywords: remote sensing; EOS/MODIS; artificial neural network (ANN); back propagation algorithm (BPA); genetic algorithm (GA); distributed rainfall estimation 1. INTRODUCTION Rainfall precipitation is an important but highly variable atmospheric parameter, and in a large river basin, different area has different weather condition, conventional methods of retrieved meteorological parameters are pretty difficult to satisfy the hydrological need. While the technology of remote sensing can obtain the distributed meteorological parameters in each unit area of the river basin, therefore, remote sensing is more effective and convenient than conventional methods in relevant surveys and studies. Moreover, the existing rainfall station network cannot provide the temporal and spatial coverage which are necessary for sufficient monitoring, so their application for accurate precipitation estimation with good temporal and spatial coverage is hampered by the existing technical limitation problems. Compared with the existing rainfall station network, the satellite measurements have the advantage of providing spatially and temporally homogeneous observations over a large area, such as GMS, TM, AVHRR and MODIS satellite images. In these satellite sensors, the moderate resolution imaging spectroradiometer (MODIS) has the wide spectral range and spatial coverage of 36 spectral bands sampling the electromagnetic spectrum from 0.4 to 14 um with a spatial resolution ranging from 250 to 1,000 meters

In the present study, first lactation test day and monthly milk records of 301 Murrah buffaloes were used for prediction of first lactation 305-day milk yield (FL305DMY) using artificial neural network (ANN) and was compared with multiple linear regression (MLR). Models were evaluated on the basis of coefficient of determination and root mean square error (RMSE). Two different input sets (Input set-1 and Input set-2) were used in the study. In input set-1, four test day milk yields (6th, 36th, 66th and 96th day of lactation) along with age at first calving (AFC) and peak yield (PY) were taken together and in input set-2, four monthly milk yields record (1st, 2nd, 3rd and 4th month yield) along with AFC and PY were taken together. The ANN was trained using back propagation (BP) algorithm which is also known as Bayesian regularization (BR). ANN achieved highest accuracy of 82% with lowest RMSE value of 16.46% for input set-1 while MLRs accuracy was 80.53% with RMSE value of 17.48%. Higher accuracy and lower RMSE value for ANN clearly showed its better performance than MLR model. Hence, ANN could be alternatively used as a tool for prediction of FL305DMY in Murrah buffaloes using input set-1 with more than 80% accuracy. So, 96th day test day yield (TD4) can be used for prediction of FL305DMY and as a trait for early genetic evaluation of sires.

Accident prediction models are invaluable tools that have many applications in road safety analysis. However, there are certain statistical issues related to accident modeling that either deserve further attention or have not been dealt with adequately in the road safety literature. This paper discusses and illustrates how to deal with two statistical issues related to modeling accidents using Poisson and negative binomial regression. The first issue is that of model building or deciding which explanatory variables to include in an accident prediction model. The study differentiates between applications for which it is advisable to avoid model over-fitting and other applications for which it is desirable to fit the model to the data as closely as possible. It then suggests procedures for developing parsimonious models, i.e., models that are not over-fitted, and best-fit models. The second issue discussed in the paper is that of outlier analysis. The study suggests a procedure for the identification and exclusion of extremely influential outliers from the development of Poisson and negative binomial regression models. The procedures suggested for model building and conducting outlier analysis are more straightforward to apply in the case of Poisson regression models because of an added complexity presented by the shape parameter of the negative binomial distribution. The paper, therefore, presents flowcharts detailing the application of the procedures when modeling is carried out using negative binomial regression. The described procedures are then applied in the development of negative binomial accident prediction models for the urban arterials of the cities of Vancouver and Richmond located in the province of British Columbia, Canada. Key words: accident prediction models, overfitting, parsimony, outlier analysis, Poisson regression, negative binomial regression.

This research project aims to investigate the efficacy of artificial neural networks (ANN) in mapping dry flue gas desulphurization (DFGD). Bayesian regularization (BR) and Levenberg–Marquardt (LM) training algorithms were used for DFGD modelling. The input layer feed data contained diatomite to Ca(OH)2 ratio, hydration time, hydration temperature, sulphation temperature, and inlet gas concentration, while the output layer metadata were sorbent conversion and sulphation responses. The hyperbolic tangent (tansig), sigmoid (logsig), and linear (purelin) activation functions were compared to ascertain the best network learning model. The number of hidden layer cells also varied between 7 and 10, given the existence of multiple output feed data. BR and LM performance evaluation was based on coefficient of determination (R2), root mean square error (RMSE), and mean square error (MSE) mathematical analysis. BR was a superlative training tool compared to LM, with lower RMSE and MSE values. The goodness of fit data for both techniques was close to unity, clarifying that ANN using BR and LM tools can be used to predict DGFD outcome. The shrinking core model was used to analyze the desulphurization reaction and concluding the chemical reaction was the reaction controlling step.

Development of a computational predictive model for the nonlinear in-plane compressive response of sandwich panels with bio-foam

This paper offers a comprehensive investigation into the forward and inverse kinematics of a wrist rehabilitation robot, utilizing the Denavit-Hartenberg method for forward kinematics (FK) and a geometric approach, as well as artificial neural networks (ANN) and adaptive Neuro-Fuzzy inference systems (ANFIS) for inverse kinematics (IK) analysis. While the geometric method entails precise parameter measurements and faces uncertainties, ANN and ANFIS are explored as potential remedies to enhance accuracy and robustness. Evaluating 11 different training functions sourced from existing literature, our study conducts a thorough assessment of their performance within an ANN network. We aim to pinpoint the most suitable training function for achieving optimal IK solutions in the context of a wrist rehabilitation robotic. Additionally, the ANFIS model, trained using Fuzzy C-Means (FCM), sets itself apart from Grid Partitioning (GP) and Subtractive Clustering (SC). Among the ANN training functions, Bayesian regularization with 5 hidden layers emerges as the most effective, yielding low root mean square error (RMSE) values of 0.003, 0.004, and 0.007 degrees for pronation/supination (P/S), abduction/adduction (AB/AD), and flexion/extension (F/E), respectively. Conversely, ANFIS, trained with FCM, demonstrates satisfactory yet less precise results, with RMSE values of 0.191, 0.082, and 0.165 degrees for P/S, AB/AD, and F/E, respectively. Despite its adequacy, ANFIS trails behind ANN, showcasing RMSE reductions of 98.4%, 95.1%, and 95.7% for P/S, AB/AD, and F/E angles, respectively. This study contributes to leveraging ANN and ANFIS for IK analysis in wrist rehabilitation robotics, highlighting the efficacy of ANN, particularly when employing Bayesian regularization, to enhance accuracy.

Sight distance (stopping, passing, and decision) is a key element in highway geometric design. Existing models for evaluating sight distance on vertical alignments are applicable only to simple, isolated elements such as a crest vertical curve, a sag vertical curve, and a reverse vertical curve (a sag curve following a crest curve, or vice versa). This paper presents an analytical methodology for evaluating sight distance on complex vertical alignments that involve any combination of vertical alignment elements. The methodology can be used for evaluating passing sight distance on two-lane highways, and stopping sight distance and decision sight distance on all highways. Sight distance controlled by the headlight beam can also be evaluated. The locations of sight-hidden dips, which may develop when a sag vertical curve follows a crest vertical curve with or without a common tangent, can be determined. Also, sight distances obstructed by overpasses are evaluated. A profile of the available sight distance can be established and used to evaluate sight distance deficiency and the effect of alignment improvements. A software was developed and can be used for determining the available sight distance accurately. The software may replace the current field and graphical practice for establishing the no-passing zones and evaluating stopping and decision sight distances on complex vertical alignments. Key words: sight distance, vertical alignment, highway, passing zones.

Phenotyping is an important step for successful animal selective breeding. Computer vision systems, such as digital image analysis, paired with machine learning (e.g., artificial neural networks, ANN), have the potential to be used in precision aquaculture and genetic improvement programs. Digital image analyses are suitable for determining morphometric traits (e.g., length, height, and width) and traits difficult to measure using traditional techniques, such as body areas, while reducing animal handling. Furthermore, images can provide explanatory variables for posterior prediction of growth, carcass, and fillet traits through ANN models. Therefore, we aimed to (i) develop a fast and straightforward method to measure the length, height, and body areas of Nile tilapia using digital image analysis, (ii) estimate genetic parameters for these traits, and (iii) apply image traits in machine learning algorithms to predict body weight (BW), carcass weight (CW), fillet weight (FW), and fillet yield (FY). The fish used in the study belonged to the 10th and 11th generation of the Nile tilapia breeding program (TILAMAX strain) of the Universidade Estadual de Maringá. In the first study, 656 fish (366 days old at harvest, BW of 414 ± 98 g) were photographed and subjected to image analysis to measure the trunk area (TA), head area (HA), caudal fin area (CFA), and fillet area (FA). Heritability estimates (h 2 ) for BW, TA, HA, CFA, and FA were 0.25, 0.23, 0.26, 0.21, and 0.25, respectively. Genetic correlations between the traits were positive and high, ranging from 0.70 to 0.98. We highlight the genetic correlation between BW and TA (r G = 0.98) and FA (r G = 0.97). Given the observed results, it can be concluded that selecting for body areas obtained by digital image analysis can lead to indirect genetic gains in weight and other areas. However, genetic correlations of these body areas with fillet weight and fillet yield were unknown. For the second study, 1,161 fish (427 days old at harvest, BW of 1,093 ± 346 g) were photographed. Body lengths (3 sections), heights (5 sections), TA, HA, FA, and total area (TOT) were measured from the coordinate values (x and y values in the center of each pixel) of 20 pre-set landmarks on the surface of fish images using the free R software. The proposed method allowed to measure 12 traits in 46 s. The h 2 for lengths and heights were moderate to high, ranging from 0.22 to 0.37. The h 2 values for TA, HA, FA, and TOT were 0.26, 0.35, 0.25, and 0.27, respectively. Positive and moderate to high genetic correlations were observed between morphometric traits and BW (0.66 to 0.98), FW (0.50 to 0.91), and CW (0.77 to 0.98). We highlight the genetic correlation of TA with BW (r G = 0.98), FW (r G = 0.91), and CW (r G = 0.96). The TA/TOT ratio showed a positive and moderate genetic correlation (0.54) with FY. We investigated five supervised machine learning methods for predicting BW, CW, FW, and FY using image traits: multiple linear regression, feed-forward artificial neural network, deep learning, Bayesian regularization for feed-forward neural networks, and random forests. To verify the effectiveness of prediction methods, we used a 10-fold cross-validation procedure with 5 replicates, and the folds were randomly split to provide the training (n = 1045) and validation (n = 116) datasets. Pearson’s correlation coefficient (r), mean absolute error (MAE), and root mean square error (RMSE) between predicted and observed values were calculated. In general, the Bayesian regularization model showed better performance and accuracy in predicting BW (r = 0.99, MAE = 39.54, RMSE = 54.70), CW (r = 0.98, MAE = 27.82, RMSE = 40.03), and FW (r = 0.96, MAE = 23.26, RMSE = 33.42). For FY prediction, all evaluated models had low performance and accuracy (r = 0.29, MAE = 1.55, RMSE = 2.24). The findings demonstrate that digital image analysis is a promising tool for measuring morphometric traits in Nile tilapia, given its non-invasive nature, fast operation, and low cost. Additionally, it was found that body areas can be used as selection criteria, particularly in future studies on body shape changes, with positively correlated responses to FW and positive, albeit lower, correlations with FY. Finally, the Bayesian regularization for the feed-forward neural network method showed the best performance in predicting BW, CW, and FW in Nile tilapia from image traits as predictor variables. Keywords: Morphometric traits. Computer vision. Genetic parameters. Irregular polygons. Artificial neural networks.