Bayesian Regularization Method Research Articles

Efficient Compound Activity Prediction Model Based on Bayesian Regularization Method for Candidate Drugs

In breast cancer diagnoses, antagonist compounds are used to resist <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula> Subtype of Estrogen Receptor (ER <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha$ </tex-math></inline-formula> ) bioactivity. However, those compounds are difficult to be obtained in the process of drug screening. In this paper, an efficient bioactivity prediction model is proposed with consideration of Absorption, Distribution, Metabolism, Excretion and Toxicity (ADMET) properties as an auxiliary prediction to achieve the minimization of the damage caused to human body whereas maximizing the efficacy of estrogen activity inhibitors. In the proposed prediction model, Pearson correlation analysis and Bayesian regularization algorithm under neural network are specifically used to train and analyze the data of bioactivity. To calculate the loss of neural network, cross-entropy and supervise learning are applied. The results show that the accuracy of our proposed compound prediction model can reach up to 92.2%, 94.3%, and 90.2% for Caco2, CYP3A4, and hERG, respectively. Moreover, the Area Under the Curve (AUC) value of our proposed model is 0.9752, which is 13-14% higher than that of SVM method. In addition, the Matthews Correlation Coefficient (MCC) can reach up to 0.93 ± 0.03, 37-40% higher than that of SVM method. Due to correlation analysis and valid data screening, our model can shorten the prediction time by 15%. Due to its advantages in accuracy and speed of prediction, this research can contribute to the upgrading of diagnosis and treatment of breast cancer.

3-D Source Location by Neural Network for FBG Acoustic Emission Sensors

Fiber Bragg grating acoustic emission sensors have been used in many applications. In this paper, based on four fiber Bragg grating acoustic emission sensors, an acoustic emission location experiment is carried out on the surface of a cylindrical polymer-bonded explosive specimen. Due to the difference in the strain sensitivity of fiber Bragg grating in different directions, the traditional time-difference location method is not fit for fiber Bragg grating acoustic emission sensors. A 4-layer dense neural network for fiber Bragg grating acoustic emission sensors, together with a Bayesian regularization method is used to calculate the coordinates of the acoustic emission wave source. The experimental results show that the neural network location method is suitable for fiber Bragg grating acoustic emission sensors.

Optimal Energy Control of Battery Hybrid System for Marine Vessels by Applying Neural Network Based on Equivalent Consumption Minimization Strategy

This paper introduces an optimal energy control method whose rule-based control employs the equivalent consumption minimization strategy as the design standard to support a neural network technique. Using the proposed control method, the output command values for each power source based on the load of the ship and the state of charge of the battery satisfy the target of energy optimization. Based on the rules, the load of the ship and the state of charge of the battery were the input in the neural network, and the outputs of two generators were recorded as the output values of the neural network. To optimize the weights of the neural network and reduce the error between the predicted values and results, the Bayesian regularization method was employed, and a single hidden layer with 20 nodes, 2 input layers, and 2 output layers were considered. For the hidden layer, the tansigmoid function was applied, and for the activation functions of the output layers, linear functions were adopted considering the correlation between the input and output data used for training the neural network. The propulsion motor was fitted with a speed controller to ensure a stable speed, and a torque load was applied on the propulsion motor. To verify the accuracy of the neural network learning, a generator–battery hybrid system simulation was conducted using MATLAB Simulink, and the neural network learned values were compared with the generator output command values obtained based on the load of the ship and the battery state of charge. Additionally, it was confirmed that the generator command values were consistent with the neural network learned values, and the stability of the system was maintained by controlling the speed, voltage, and current control of the propulsion motor under various loads of the ship and different battery charge statuses.

Best Model and Performance of Bayesian Regularization Method for Data Prediction

The backpropagation algorithm has many training and activation functions that can be used to influence or maximize prediction results, all of which have their respective advantages and disadvantages. The purpose of this paper is to analyze one of the training functions of the backpropagation algorithm which can be used as a reference for use in data prediction problems in the form of models and best performance. The training function is the Bayesian Regularization method. This method is able to train the network by optimizing the Levenberg-Marquardt by updating the bias and weights. The research dataset used to analyze the data in this paper is Formal Education Participation in Indonesia 2015-2020 which consists of the School Participation Rate, the Gross Enrollment Rate, and the Pure Enrollment Rate. The 2015-2016 dataset is used as training data with a 2017 target, while the 2018-2019 dataset is the test data with a 2020 target. The models used are 2-10-1, 2-15-1, and 2-20-1. Based on the analysis and calculation process, the results of the 2-15-1 model are the best with an epoch of 217 iterations and an MSE of 0.00002945, this is because the epoch is not too large and has the smallest MSE compared to the other 2 models.

A partially confirmatory approach to scale development with the Bayesian Lasso.

The exploratory and confirmatory approaches of factor analysis lie on two ends of a continuum of substantive input for scale development. Recent advancements in Bayesian regularization methods enable more flexibility in covering a wide range of the substantive continuum. Based on the Bayesian Lasso (least absolute shrinkage and selection operator) methods for the regression model and covariance matrix, this research proposes a partially confirmatory approach to address the loading and residual structures at the same time. With at least one specified loading per item, a one-step procedure can be applied to figure out both structures simultaneously. With a few specified loadings per factor, a two-step procedure is preferred to capture the model configuration correctly. In both cases, the Bayesian hierarchical formulation is implemented using Markov Chain Monte Carlo estimation with different Lasso or regular priors. Both simulated and real data sets were analyzed to evaluate the validity, robustness, and practical usefulness of the proposed approach across different situations. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

NEURAL NETWORK FORECASTING OF ENERGY CONSUMPTION OF A METALLURGICAL ENTERPRISE

The subject of the research is the methods of constructing and training neural networks as a nonlinear modeling apparatus for solving the problem of predicting the energy consumption of metallurgical enterprises. The purpose of this work is to develop a model for forecasting the consumption of the power system of a metallurgical enterprise and its experimental testing on the data available for research of PJSC "Dneprospetsstal". The following tasks have been solved: analysis of the time series of power consumption; building a model with the help of which data on electricity consumption for a historical period is processed; building the most accurate forecast of the actual amount of electricity for the day ahead; assessment of the forecast quality. Methods used: time series analysis, neural network modeling, short-term forecasting of energy consumption in the metallurgical industry. The results obtained: to develop a model for predicting the energy consumption of a metallurgical enterprise based on artificial neural networks, the MATLAB complex with the Neural Network Toolbox was chosen. When conducting experiments, based on the available statistical data of a metallurgical enterprise, a selection of architectures and algorithms for learning neural networks was carried out. The best results were shown by the feedforward and backpropagation network, architecture with nonlinear autoregressive and learning algorithms: Levenberg-Marquard nonlinear optimization, Bayesian Regularization method and conjugate gradient method. Another approach, deep learning, is also considered, namely the neural network with long short-term memory LSTM and the adam learning algorithm. Such a deep neural network allows you to process large amounts of input information in a short time and build dependencies with uninformative input information. The LSTM network turned out to be the most effective among the considered neural networks, for which the indicator of the maximum prediction error had the minimum value. Conclusions: analysis of forecasting results using the developed models showed that the chosen approach with experimentally selected architectures and learning algorithms meets the necessary requirements for forecast accuracy when developing a forecasting model based on artificial neural networks. The use of models will allow automating high-precision operational hourly forecasting of energy consumption in market conditions.&#x0D; Keywords: energy consumption; forecasting; artificial neural network; time series.

Suggestion for a new deterministic model coupled with machine learning techniques for landslide susceptibility mapping

Deterministic models have been widely applied in landslide risk assessment (LRA), but they have limitations in obtaining various geotechnical and hydraulic properties. The objective of this study is to suggest a new deterministic method based on machine learning (ML) algorithms. Eight crucial variables of LRA are selected with reference to expert opinions, and the output value is set to the safety factor derived by Mohr–Coulomb failure theory in infinite slope. Linear regression and a neural network based on ML are applied to find the best model between independent and dependent variables. To increase the reliability of linear regression and the neural network, the results of back propagation, including gradient descent, Levenberg–Marquardt (LM), and Bayesian regularization (BR) methods, are compared. An 1800-item dataset is constructed through measured data and artificial data by using a geostatistical technique, which can provide the information of an unknown area based on measured data. The results of linear regression and the neural network show that the special LM and BR back propagation methods demonstrate a high determination of coefficient. The important variables are also investigated though random forest (RF) to overcome the number of various input variables. Only four variables—shear strength, soil thickness, elastic modulus, and fine content—demonstrate a high reliability for LRA. The results show that it is possible to perform LRA with ML, and four variables are enough when it is difficult to obtain various variables.

Walking Speed Estimation From a Wearable Insole Pressure System Embedded With an Accelerometer Using Bayesian Neural Network

Abstract In this study, we introduced a machine learning method for estimating human walking speed using plantar pressure and acceleration data. A pressure-derivative method using pretest feature selection was proposed to extract speed-related features from plantar pressure sensors. The maximum, minimum, and standard deviation of acceleration data were also selected as neural network inputs. To improve the generalization ability of the neural network, Bayesian regularization method was adopted. Experiments were conducted under seven different walking speeds to validate the performance of the proposed method. The results show that a strong linear correlation (R = 0.995) exists between the estimated and actual walking speed. The average error of the proposed method is 0.003 ± 0.043 m/s (mean ± root-mean-square error), which is better than previous works. It is suggested that including the speed-related information of both stance and swing phase would give a new insight for achieving a high accuracy of walking speed estimation.

Spatiotemporal Bayesian Regularization for Cardiac Strain Imaging: Simulation and In Vivo Results.

Cardiac strain imaging (CSI) plays a critical role in the detection of myocardial motion abnormalities. Displacement estimation is an important processing step to ensure the accuracy and precision of derived strain tensors. In this paper, we propose and implement Spatiotemporal Bayesian regularization (STBR) algorithms for two-dimensional (2-D) normalized cross-correlation (NCC) based multi-level block matching along with incorporation into a Lagrangian cardiac strain estimation framework. Assuming smooth temporal variation over a short span of time, the proposed STBR algorithm performs displacement estimation using at least four consecutive ultrasound radio-frequency (RF) frames by iteratively regularizing 2-D NCC matrices using information from a local spatiotemporal neighborhood in a Bayesian sense. Two STBR schemes are proposed to construct Bayesian likelihood functions termed as Spatial then Temporal Bayesian (STBR-1) and simultaneous Spatiotemporal Bayesian (STBR-2). Radial and longitudinal strain estimated from a finite-element-analysis (FEA) model of realistic canine myocardial deformation were utilized to quantify strain bias, normalized strain error and total temporal relative error (TTR). Statistical analysis with one-way analysis of variance (ANOVA) showed that all Bayesian regularization methods significantly outperform NCC with lower bias and errors (p < 0.001). However, there was no significant difference among Bayesian methods. For example, mean longitudinal TTR for NCC, SBR, STBR-1 and STBR-2 were 25.41%, 9.27%, 10.38% and 10.13% respectively An in vivo feasibility study using RF data from ten healthy mice hearts were used to compare the elastographic signal-to-noise ratio (SNRe) calculated using stochastic analysis. STBR-2 had the highest expected SNRe both for radial and longitudinal strain. The mean expected SNRe values for accumulated radial strain for NCC, SBR, STBR-1 and STBR-2 were 5.03, 9.43, 9.42 and 10.58, respectively. Overall results suggest that STBR improves CSI in vivo.

Comprehensive Evaluation Model of Maritime Industrial Economic Activities Based on an AHP and BP Neural Network

Yang, M., Chen, J., and Zhou, Y., 2020. Comprehensive evaluation model of maritime industrial economic activities based on an AHP and BP neural network. In: Liu, X. and Zhao, L. (eds.), Today's Modern Coastal Society: Technical and Sociological Aspects of Coastal Research. Journal of Coastal Research, Special Issue No. 111, pp. 178–182. Coconut Creek (Florida), ISSN 0749-0208.Based on a comprehensive evaluation of maritime industrial economic activities that are difficult to quantify and with which subjective factors interfere strongly, this paper uses the analytic hierarchy process and BP neural network to build a comprehensive evaluation model. Taking a marine enterprise as an example, the first-level and second-level indicators of the economic activities of the enterprise are adopted. The model determines the input weights of the first-level indicators according to the expert scoring method, multiplies the 27 second-level indicators under the first-level indicators by the weights of the first-level indicators as the input indicators of the model, and establishes training based on a Bayesian regularization and gradient descent method. The three-layer BP neural network application model has a better chance of improving the problem that subjective factors dominate and lead to lower evaluation accuracy of industrial economic activities. Taking a comprehensive evaluation experiment of industrial economic activities of a plant in 2020 as an example, the experimental results prove that the evaluation results of the model have guiding significance for the analysis and evaluation of industrial economic activities.

Optimization of an explosive waste incinerator via an artificial neural network surrogate model

In this study, the operating conditions for a novel method of explosive waste disposal have been optimized to minimize the formation of NOx emissions. Previous methods of disposal include rotary kilns but have disadvantages, especially associated with the formation of NOx. To address this, development of a fluidized bed reactor as an incinerator has been proposed. However, as the number of input variables to consider increases, optimization of the design through computational fluid dynamics (CFD) simulations alone becomes challenging, because of the significant computational time required. Thus, we proposed to build a surrogate model of CFD output using an artificial neural network (ANN) that can be used to locate the optimal operating conditions. To perform the optimization on five variables, finite ranges of these variables were selected and sampled using a Latin Hypercube Method, giving a total of 300 random input samples. CFD simulations were performed using each of these conditions, and the ANN surrogate model was designed to match the results. Due to its higher accuracy, the surrogate model from the Bayesian Regularization method was selected, which had an error rate of around 1% when compared to the CFD results. Using this surrogate model, a 34% reduction in NOx emissions from the reactor was achievable. Overall, the fluidized bed reactors show promise with regard to the reduction of NOx emissions from explosive waste disposal, and surrogate modeling with ANNs represents a useful method for reducing the CFD computation time used for multivariable optimization of such a system.

ANN based method for improving gold price forecasting accuracy through modified gradient descent methods

Forecast of prices of financial assets including gold is of considerable importance for planning the economy. For centuries, people have been holding gold for many important reasons such as smoothening inflation fluctuations, protection from an economic crisis, sound investment etc.. Forecasting of gold prices is therefore an ever important exercise undertaken both by individuals and groups. Various local, global, political, psychological and economic factors make such a forecast a complex problem. Data analysts have been increasingly applying Artificial Intelligence (AI) techniques to make such forecasts. In the present work an inter comparison of gold price forecasting in Indian market is first done by employing a few classical Artificial Neural Network (ANN) techniques, namely Gradient Descent Method (GDM), Resilient Backpropagation method (RP), Scaled Conjugate Gradient method (SCG), Levenberg-Marquardt method (LM), Bayesian Regularization method (BR), One Step Secant method (OSS) and BFGS Quasi Newton method (BFG). Improvement in forecasting accuracy is achieved by proposing and developing a few modified GDM algorithms that incorporate different optimization functions by replacing the standard quadratic error function of classical GDM. Various optimization functions investigated in the present work are Mean median error function (MMD), Cauchy error function (CCY), Minkowski error function (MKW), Log cosh error function (LCH) and Negative logarithmic likelihood function (NLG). Modified algorithms incorporating these optimization functions are referred to here by GDM_MMD, GDM_CCY, GDM_KWK, GDM_LCH and GDM_NLG respectively. Gold price forecasting is then done by employing these algorithms and the results are analysed. The results of our study suggest that the forecasting efficiency improves considerably on applying the modified methods proposed by us.

Design of Neural Network With Levenberg-Marquardt and Bayesian Regularization Backpropagation for Solving Pantograph Delay Differential Equations

In this paper, novel computing paradigm by exploiting the strength of feed-forward artificial neural networks (ANNs) with Levenberg-Marquardt Method (LMM), and Bayesian Regularization Method (BRM) based backpropagation is presented to find the solutions of initial value problems (IVBs) of linear/nonlinear pantograph delay differential equations (LP/NP-DDEs). The dataset for training, testing and validation is created with reference to known standard solutions of LP/NP-DDEs. ANNs are implemented using the said dataset for approximate modeling of the system on mean squared error based merit functions, while learning of the adjustable parameters is conducted with efficacy of LMM (ANN-LMM) and BRMs (ANN-BRM). The performance of the designed algorithms ANN-LMM and ANN-BRM on IVPs of first, second and third order NP-FDEs are verified by attaining a good agreement with the available solutions having accuracy in the range from 10−5 to 10−8 and are further endorsed through error histograms and regression measures.

An Extension of Unified Bayesian Tikhonov Regularization Method and Application to Image Restoration

An Extension of Unified Bayesian Tikhonov Regularization Method and Application to Image Restoration

Информационная обработка данных динамики уровня речных вод при мониторинге устойчивости геотехнической системы

Introduction. During the operation of urban and rural geotechnical systems, the mechanical indicators of the stability of the soil base are significantly influenced by the regime and dynamics of the physicochemical properties of nearby water bodies, caused by the development of karst-suffusion processes. The results of information analysis of data on the dynamics of the river runoff level and water salinity make it possible to increase the accuracy of predictive estimates of the loss of stability of the geotechnical system due to the development of karst-suffusion processes. Goals and objectives. The aim of the work is to improve the safety of operation of geotechnical systems and increase the efficiency of modeling and forecasting systems for geodynamics by developing an algorithm for assessing changes in the risk of developing suffusion processes based on an analysis of the dynamics of the level of groundwater and surface waters. Methods. The paper analyzes the data on the number of karst sinkholes depending on the dynamics of the river water level, obtained on the basis of statistical sources and reports on regime observations, as well as a result of field research. In the course of data processing, a spline interpolation method was used, an algorithm and a neural network for predicting a failure using the Bayesian regularization method based on a network training function was developed, which updates the weight and bias of the value in accordance with the Levenberg-Marquardt optimization. Results and its discussion. Based on the results of practical use, the effectiveness of the developed algorithm was confirmed in identifying the dynamics of the formation of dips during information processing of data on changes in the water level in the Oka River and data on the appearance of new dips in the period from 2012 to 2019. Conclusion. The results obtained in the work make it possible to judge the presence of the expediency of using the developed algorithm for assessing the occurrence of karst sinkholes when monitoring the stability of geotechnical systems and assessing the safety of their operation in general. Prospects for further work are associated with expanding the set of training data and adapting the structure of the neural network to the individual characteristics of the territories by introducing additional geological, hydrological and climatic parameters into the processing. Key words: geotechnical system, stability, karst sinkhole, bifurcation parameters, information processing of data.

Analysis of small-angle scattering data using model fitting and Bayesian regularization

The structure of macromolecules can be studied by small-angle scattering (SAS), but as this is an ill-posed problem, prior knowledge about the sample must be included in the analysis. Regularization methods are used for this purpose, as already implemented in indirect Fourier transformation and bead-modeling-based analysis of SAS data, but not yet in the analysis of SAS data with analytical form factors. To fill this gap, a Bayesian regularization method was implemented, where the prior information was quantified as probability distributions for the model parameters and included via a functional S. The quantity Q = χ2 + αS was then minimized and the value of the regularization parameter α determined by probability maximization. The method was tested on small-angle X-ray scattering data from a sample of nanodiscs and a sample of micelles. The parameters refined with the Bayesian regularization method were closer to the prior values as compared with conventional χ2 minimization. Moreover, the errors on the refined parameters were generally smaller, owing to the inclusion of prior information. The Bayesian method stabilized the refined values of the fitted model upon addition of noise and can thus be used to retrieve information from data with low signal-to-noise ratio without risk of overfitting. Finally, the method provides a measure for the information content in data, N g, which represents the effective number of retrievable parameters, taking into account the imposed prior knowledge as well as the noise level in data.

A comparison of sparse Bayesian regularization methods on computed tomography reconstruction

Design of regularization term is an important part of solution of an ill-posed linear inverse problem. Another important issue is selection of tuning parameters of the regularization term. We address this problem using Bayesian approach which treats tuning parameters as unknowns and estimates them from the data. Specifically, we study a regularization model known as Automatic Relevance Determination (ARD) and several methods of its solution. The first approach is the conventional Variational Bayes method using the symmetrical factorization of the posterior of the vector of unknowns and the vector of tuning parameters. The second approach is based on the idea of marginalization over the vector of unknowns or the vector of tuning parameters, while the complementary vector is estimated using maximum likelihood. The resulting algorithm is thus an optimization task with non-convex objective function, which is solved using standard gradient methods.The proposed algorithms are tested on real tomographic X-ray data and the comparison with conventional regularization techniques (Tikhonov and Lasso) is performed. The algorithm using marginalization over the tuning parameter is found to be closest to the ground truth with acceptable computational cost.MATLAB® implementation of the reconstruction algorithms is freely available for download.

Mixture Semisupervised Bayesian Principal Component Regression for Soft Sensor Modeling

In this paper, a mixture semisupervised Bayesian principal component regression-based soft sensor modeling method for nonlinear industrial process with multiple operating modes is presented. In many chemistry processes, part of output data samples may be unavailable due to the difficulties in measurement or recording. The semisupervised method is introduced to efficiently deal with the unlabeled data set. Moreover, the Bayesian regularization method is proposed to determine the unknown dimensionality of latent variables space in each submode by introducing three different formulations of two hyperparameters to construct the Gaussian prior distributions over the loading and regression matrices. The formulation of this method is derived in expectation maximization algorithm scheme, and the formulas to update unknown parameters are derived. The effectiveness of the proposed method is verified through a numerical example, the Tennessee Eastman benchmark process, and the comparisons with the existing method.

Utilization of the LMS Algorithm to Filter the Predicted Course by Means of Neural Networks for Monitoring the Occupancy of Rooms in an Intelligent Administrative Building

For monitoring the occupancy of individual rooms in an intelligent administrative building (IAB), a wide variety of sensors can be used, by means of which the presence of a person in the monitored area can be determined. For a larger administrative-type building, installation of additional sensors means considerable investment costs. As the present-day standard, temperature or also humidity sensors are installed in individual IAB rooms. The paper describes the proposed method for the determination of occupancy of the monitored area by means of prediction of the course of CO2(ppm) from the measured values of humidity rH(%), indoor temperature Ti(°C) and outdoor temperature To(°C), using the gradient algorithm of back-propagation of error for adaptation of the multilayer feedforward Artificial Neural Network (ANN) in the IAB areas with utilization of the Bayesian regularization method (BRM) to obtain information on the occupancy of individual rooms. The LMS algorithm was used to filter the predicted course in order to determine the occupancy of the monitored areas more precisely. The advantage of the proposed method is the utilization of common operating sensors to obtain information on the state of operational-technical functions in the IAB for the purpose of optimum control of the operational-technical functions of the IAB on the basis of predictable needs of persons using the IAB areas.

A Nonlinear Autoregressive Scheme for Time Series Prediction via Artificial Neural Networks

This article is devoted to a time series prediction scheme involving the nonlinear autoregressive algorithm and its applications. The scheme is implemented by means of an artificial neural network containing a hidden layer. As a training algorithm we use scaled conjugate gradient (SCG) method and the Bayesian regularization (BReg) method. The first method is applied to time series without noise, while the second one can also be applied for noisy datasets. We apply the suggested scheme for prediction of time series arising in oil and gas pricing using 50 and 100 past values. Results of numerical simulations are presented and discussed.