Conjugate Gradient Descent Algorithm Research Articles

Examination of the effect of ANN and NLPCA technique on prediction performance in patients with breast tumors

Breast cancer is among the most prevalent cancer kinds worldwide. The aim of this study is to examine the effect of combining Artificial Neural Networks and Nonlinear Principal Component Analysis techniques on prediction performance in patients with breast tumors. In the application, a network containing 5 layers, including the input, the coding, the bottleneck, the decoding and the output, was used for the 30 variable data set of 569 breast tumor patients. The training algorithm of choice was the Conjugate Gradient Descent (CGD) algorithm. In this study, artificial neural networks (ANN) and nonlinear principal component analysis were coupled. NLPCA was first applied to dimension reduction in artificial neural networks. Using both the original data set and the decreased size, artificial neural networks were used in the second stage to develop prediction models. By contrasting the performance of these two prediction models with one another, the outcomes were understood. 96.37% of the variation was explained by the two fundamental components that were found using NLPCA. The prediction models developed for the original data set and the dimension-reduced data set have R2 values of 91% and 87%, respectively. The advantages of the NLPCA and ANN combination for breast tumor patients are demonstrated by this study. It is believed that utilizing principal components as inputs can cut down on complexity and extraneous information.

Comparison of different supervised learning algorithms for position analysis of the slider-crank mechanism

Slider-crank mechanisms have a wide range of applications in machine design, primarily for converting rotary motion into reciprocating motion, as seen in internal combustion engines, pumps, compressors, and human-powered vehicles. Due to their widespread usage, analytical and graphical solutions for the position analysis of slider-crank mechanisms are introduced in various textbooks and lecture notes. Recently, artificial neural network algorithms have been applied to various design applications across different research fields, and mechanism design is no exception. However, investigations into the application of neural network algorithms specifically to slider-crank mechanisms are quite limited. In this study, the analytical position analysis of the slider-crank mechanism is presented, and datasets are obtained first. Subsequently, the application of following three different supervised learning algorithms to the position analysis of the slider-crank (R-RRT) mechanism is investigated using analytical solution datasets: the Levenberg-Marquardt Backpropagation (LM) algorithm, Bayesian Regularization (BR) algorithm, and Scaled Conjugate Gradient Descent (SCG) algorithm. These three algorithms were chosen due to their distinct characteristics. The main objectives of this study are to highlight the differences among various neural network algorithms' applications and to understand the applicability and suitability of different neural network algorithms for kinematic position analysis in mechanisms. The results indicate that the Bayesian regularization algorithm yields the best results, while the Levenberg-Marquardt Backpropagation algorithm exhibits the best performance.

Financial Early Warning Model for Listed Companies Based on the Smart Sensor Data Network

This paper uses intelligent sensors to build a data network, collects information about listed enterprises in all aspects, performs frequency statistics and semantic analysis based on the financial domain lexicon on information related to listed enterprises, and introduces big data variables into a nonlinear support vector machine early warning model of enterprises combined with financial indicators. This paper introduces online reviews as big data indicators based on financial early warning theories and methods. Suitable financial indicators and big data indicators are selected for the financial early warning model to filter the available indicators. The prediction results of only financial indicators are compared with the prediction results of incorporating big data indicators. Quantify the comment information related to enterprises on the platform through sentiment classification and statistics on the number of comment information posted. The cost-sensitive support vector machine is used as the base classifier of the improved AdaBoost algorithm to build a dynamic imbalance warning model. To address the problems of large complexity and computation of unbalanced big data classification, high reliance on a priori knowledge, and classification performance to be improved, the classification and detection method of the intelligent sensor data network is proposed. By introducing migration learning, the problem of knowledge acquisition and training efficiency of high-dimensional complex data feature extraction in a big data environment is effectively solved, and the network performance is optimized by a conjugate gradient descent algorithm. Through simulation experiments, the prediction accuracy of the model for positive class samples is significantly improved after the nonequilibrium processing, the recall rate of the network model reaches 84.15%, and the prediction accuracy of the network model under different time steps reaches more than 90%. The experiment proves that the model can send out financial alert information more quickly and efficiently and accurately when a financial crisis may occur, relative to the traditional financial forecasting methods.

Modeling Average Rainfall in Nigeria With Artificial Neural Network (ANN) Models and Seasonal Autoregressive Integrated Moving Average (SARIMA) Models

Rainfall prediction is one of the most essential and challenging operational obligations undertaken by meteorological services globally. In this article we conduct a comparative study between the ANN models and the traditional SARIMA models to show the most suitable model for predicting rainfall in Nigeria. Average monthly rainfall data in Nigerian for the period Jan. 1991 to Dec.2020 were considered. The ACF and PACF plots clearly identifies the SARIMA (1,0,2)x(1,1,2)12 as an appropriate model for predicting average monthly rainfall. The performance of the trained Neural Network (NN) analysis clearly favours Levenberg-Marquardt (LM) over the Scaled Conjugate Gradient Descent (SCGD) algorithms and Bayesian Regularization (BR) method with Average Absolute Error 0.000525056. The forecasting performance metric using the RSME and MAE, showed that Neural Network trained by Levenberg-Marquadrt algorithm gives better predicted values of Nigerian rainfall than the SARIMA (1,0,2)x(1,1,2)12.

An online conjugate gradient algorithm for large-scale data analysis in machine learning

In recent years, the amount of available data is growing exponentially, and large-scale data is becoming ubiquitous. Machine learning is a key to deriving insight from this deluge of data. In this paper, we focus on the large-scale data analysis, especially classification data, and propose an online conjugate gradient (CG) descent algorithm. Our algorithm draws from a recent improved Fletcher-Reeves (IFR) CG method proposed in Jiang and Jian[13] as well as a recent approach to reduce variance for stochastic gradient descent from Johnson and Zhang [15]. In theory, we prove that the proposed online algorithm achieves a linear convergence rate under strong Wolfe line search when the objective function is smooth and strongly convex. Comparison results on several benchmark classification datasets demonstrate that our approach is promising in solving large-scale machine learning problems, viewed from the points of area under curve (AUC) value and convergence behavior.

Overcoming information reduced data and experimentally uncertain parameters in ptychography with regularized optimization.

The overdetermination of the mathematical problem underlying ptychography is reduced by a host of experimentally more desirable settings. Furthermore, reconstruction of the sample-induced phase shift is typically limited by uncertainty in the experimental parameters and finite sample thicknesses. Presented is a conjugate gradient descent algorithm, regularized optimization for ptychography (ROP), that recovers the partially known experimental parameters along with the phase shift, improves resolution by incorporating the multislice formalism to treat finite sample thicknesses, and includes regularization in the optimization process, thus achieving reliable results from noisy data with severely reduced and underdetermined information.

Prediction of Draft Force of a Chisel Cultivator Using Artificial Neural Networks and Its Comparison with Regression Model

In this study, artificial neural networks (ANNs) were used to predict the draft force of a rigid tine chisel cultivator. The factorial experiment based on the randomized complete block design (RCBD) was used to obtain the required data and to determine the factors affecting the draft force. The draft force of the chisel cultivator was measured using a three-point hitch dynamometer and data were collected using a DT800 datalogger. A recurrent back-propagation multilayer network was selected to predict the draft force of the cultivator. The gradient descent algorithm with momentum, Levenberg–Marquardt algorithm, and scaled conjugate gradient descent algorithm were used for network training. The tangent sigmoid transfer function was the activation functions in the layers. The draft force was predicted based on the tillage depth, soil moisture content, soil cone index, and forward speed. The results showed that the developed ANNs with two hidden layers (24 and 26 neurons in the first and second layers, respectively) with the use of the scaled conjugate gradient descent algorithm outperformed the networks developed with other algorithms. The average simulation accuracy and the correlation coefficient for the prediction of draft force of a chisel cultivator were 99.83% and 0.9445, respectively. The linear regression model had a much lower accuracy and correlation coefficient for predicting the draft force compared to the ANNs.

Quantitative X-ray fluorescence imaging of gold nanoparticles using joint L1 and total variation regularized reconstruction.

This work proposed a joint L1 and total variation (TV) regularized reconstruction method for X-ray fluorescence tomography (XFT), and investigated the performance of this method in quantitative imaging of gold nanoparticles (GNPs). We developed a dual-modality XFT/CT imaging system which consisted of a benchtop X-ray source, a translation/rotation stage, a silicon drift detector for X-ray fluorescence (XRF) detection, and a flat panel detector for transmission X-ray detection. A pencil-beam collimator was 3D printed with steel and employed in sample excitation. The sensitivity of the XFT imaging system was determined by imaging water phantoms with multiple inserts containing GNP solutions of various concentrations (0.02-0.16 wt.%). A joint L1 and total variation (TV) regularized algorithm was developed for XFT reconstruction, where the L1 regularization was used to reduce image artifacts and the TV regularization was used to preserve the shape of targets. Nonlinear conjugate gradient (NCG) descent algorithm with backtracking line search was adopted to solve the reconstruction problem. We compared the L1 + TV regularization method with filtered back projection (FBP), maximum likelihood expectation maximization (ML-EM), L1 regularization, and TV regularization methods. Contrast-to-noise ratio (CNR), Dice similarity coefficient (DSC) and localization error (LE) metrics were used to compare the performance of different methods. The CT and XFT imaging doses were also measured using EBT2 radiochromic films. The 3D printed pencil-beam collimator shaped an excitation beam with a 2 mm full width at half maximum at the imaging isocenter. Based on the phantom imaging experiments, the joint L1 and TV regularization method performed better than FBP, ML-EM, L1 regularization and TV regularization methods, with higher localization accuracy (offset <0.6 mm), CNR and DSC values. Compared with CT, XFT with L1 + TV regularized reconstruction demonstrated higher sensitivity in GNP imaging, and could detect GNP at a concentration of 0.02 wt.% or lower. Moreover, there existed a significant linear correlation (R2>0.99) between the reconstructed and true GNP concentration. The estimated XFT imaging dose is about 41.22 cGy under current setting. The joint L1 + TV regularized reconstruction algorithm performed better in noise suppression and shape preservation. Using the L1 + TV regularized reconstruction, the XFT system is able to localize GNP targets with submillimeter accuracy and quantify GNP distribution at a concentration of 0.02 wt.% or lower.

Computed Tomography Medical Image Compression using Conjugate Gradient

Image compression which is a subset of data compression plays a crucial task in medical field. The medical images like CT, MRI, PET scan and X-Ray imagery which is a huge data, should be compressed to facilitate storage capacity without losing its details to diagnose the patient correctly. Now a days artificial neural network is being widely researched in the field of image processing. This paper examines the performance of a feed forward artificial neural network with learning algorithm as conjugate gradient. Various update parameters are considered in conjugate gradient methodology. This work performs a comparison between Conjugate gradient technique and Gradient Descent algorithm. MSE and PSNR are used as quality metrics. The investigation is carried on CT scan of lower abdomen medical image.

A novel radio map construction method with reduced human efforts for Wi-Fi localisation system

Fingerprint based Wi-Fi localisation system often takes a lot of human efforts to measure the received signal strength (RSS) of dense grid in an indoor environment. In this paper, we propose a novel fingerprint database construction method with reduced human effort to obtain optimal length of RSS time series and grid division. We verify the chaotic characteristics of RSS time series, and use phase space reconstruction algorithm to calculate the optimal length of RSS data to be collected at each reference point. Then Gaussian process regression (GPR) for fingerprinting based indoor localisation is used to construct the database with information of limited reference points. The hyper-parameters of GPR is calculated by conjugate gradient descent algorithm. The performance of the proposed radio map construction framework is validated in real indoor environment, and with using Bayesian positioning method, the localisation error mean can be 1.5 m while the construction time of radio map is greatly reduced with ensuring accuracy.

Student’s Academic Performance Prediction Using Factor Analysis Based Neural Network

This study focused on the statistical technique using the neural network, hybrid models and factor analysis on constructing the new factors affecting students learning styles of the survey done among university students in predicting academic performance. The data were collected using survey questionnaires and students’ academic records. The methodologies used were descriptive statistics, factor analysis, neural network and hybrid models technique using the following Learning algorithms; Levenberg-Marquardt (LM), Bayesian Regularization (BR), BFGS Quasi-Newton (BFG), Scaled Conjugate Gradient (SCG), Gradient Descent (GD) in artificial neural network model while for the second Hybrid model only the best two algorithms where use; Levenberg-Marquardt (LM), Bayesian Regularization (BR). The results showed ten new factors were successfully constructed using factor analysis and the proposed hybrid models show that though it took longer time and number of epochs to train the hybrid models by Bayesian Regularization Algorithms, and it gives more accurate predictions than both the Levenberg-Marquadrt, Scaled Conjugate Gradient, Gradient Descent and BFGS Quasi-Newton (BFG) Algorithms. In a nutshell, the finding indicates that Bayesian Regularization is the best learning algorithms in both Neural Network and Hybrid models for predicting students’ academic performance.

Artificial Neural Network Study to Predict the Amount of Carried Weight by Rail Transportation System

Keretapi Tanah Melayu Berhad (KTMB) is the main rail operator in Peninsular Malaysia. KTMB provides cargo services which are safe, efficient and trustworthy. KTMB also has services that are connected to the port and inland port in Peninsular Malaysia. However, they remove suffered three major derailments in 2017. On November 23, a cargo train had an accident when 12 cargo trains traveling southward slipped between National Bank Station and Kuala Lumpur Station due to heavy weight and oversized loads carried by the cargo train. This study is conducted to predict the amount of carried weight of cargo by KTMB using Artificial Neural Network model. Datasets used in this study was taken from Department of Statistics Malaysia Official Portal from year 2001 to 2016. There are three algorithms chosen in this study which are Conjugate Gradient Descent (CGD), Quasi-Newton (QN) and Lavenberg-Marquardt (LM) algorithm. The best algorithm is selected to predict the amount of carried weight by comparing the value of error measures of the three algorithms which are Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE). Therefore, CGD is the best algorithm that produces smallest error of RMSE and MAPE. By using CGD algorithm, the results show the forecast value of carried weight for five years ahead which is from year 2017 until 2021 is decrease.

Conjugate gradient descent learned ANN for Indian summer monsoon rainfall and efficiency assessment through Shannon-Fano coding

Work reported in the present paper demonstrates a neurocomputing based predictive model for the average rainfall in India during the season of summer monsoon. Backpropagation method with Conjugate Gradient Descent algorithm has been implemented to develop the neurocomputing model. After three runs of the model, it is found that a high prediction yield is available. Finally, Shannon-Fano coding has been implemented and the coding efficiency has been measured by dividing the error percentage of prediction into various classes. The efficiency of Conjugate Gradient Descent algorithm for multilayer ANN has been finally established through Shannon-Fano coding.

Prediction of PM2.5 using an ensemble of artificial neural networks and regression models

Inhaling particulate matter such as PM2.5 can have a hazardous impact on the human health. In order to predict the PM2.5 concentration, Artificial Neural Networks trained with conjugate gradient descent such as Multi Layer Perceptron (MLP), cascade forward neural network, Elman neural network, Radial Basis Function (RBF) neural network and Non-linear Autoregressive model with exogenous input (NARX) along with regression models such as Multiple Linear Regression (MLR) consisting of batch gradient descent, stochastic gradient descent, mini-batch gradient descent and conjugate gradient descent algorithms and Support Vector Regression (SVR) were implemented. In these models, the concentration of PM2.5 was the dependent variable and the data related to concentrations of PM2.5, SO2, O3 and meteorological data including average Maximum Temperature (MAX T), daily wind speed (WS) for the years 2010–2016 in Houston and New York were the independent variables. For the final forecast, several ensemble models of individual neural network predictors and individual regression predictors have been presented.

Quantitative characterization of turbidity by radiative transfer based reflectance imaging.

A new and noncontact approach of multispectral reflectance imaging has been developed to inversely determine the absorption coefficient of μ a , the scattering coefficient of μs and the anisotropy factor g of a turbid target from one measured reflectance image. The incident beam was profiled with a diffuse reflectance standard for deriving both measured and calculated reflectance images. A GPU implemented Monte Carlo code was developed to determine the parameters with a conjugate gradient descent algorithm and the existence of unique solutions was shown. We noninvasively determined embedded region thickness in heterogeneous targets and estimated in vivo optical parameters of nevi from 4 patients between 500 and 950nm for melanoma diagnosis to demonstrate the potentials of quantitative reflectance imaging.

Guarantees of riemannian optimization for low rank matrix completion

We establish the exact recovery guarantees for a class of Riemannian optimization methods based on the embedded manifold of low rank matrices for matrix completion. Assume \begin{document}$ m $\end{document} entries of an \begin{document}$ n\times n $\end{document} rank \begin{document}$ r $\end{document} matrix are sampled independently and uniformly with replacement. We first show that with high probability the Riemannian gradient descent and conjugate gradient descent algorithms initialized by one step hard thresholding are guaranteed to converge linearly to the measured matrix provided \begin{document}$ \begin{align*} m\geq C_\kappa n^{1.5}r\log^{1.5}(n), \end{align*} $\end{document} where \begin{document}$ C_\kappa $\end{document} is a numerical constant depending on the condition number of the measured matrix. Then the sampling complexity is further improved to \begin{document}$ \begin{align*} m\geq C_\kappa nr^2\log^{2}(n) \end{align*} $\end{document} via the resampled Riemannian gradient descent initialization. The analysis of the new initialization procedure relies on an asymmetric restricted isometry property of the sampling operator and the curvature of the low rank matrix manifold. Numerical simulation shows that the algorithms are able to recover a low rank matrix from nearly the minimum number of measurements.

Cosmic Microwave Background Mapmaking with a Messenger Field

Abstract We apply a messenger field method to solve the linear minimum-variance mapmaking equation in the context of Cosmic Microwave Background (CMB) observations. In simulations, the method produces sky maps that converge significantly faster than those from a conjugate gradient descent algorithm with a diagonal preconditioner, even though the computational cost per iteration is similar. The messenger method recovers large scales in the map better than conjugate gradient descent, and yields a lower overall χ 2. In the single, pencil beam approximation, each iteration of the messenger mapmaking procedure produces an unbiased map, and the iterations become more optimal as they proceed. A variant of the method can handle differential data or perform deconvolution mapmaking. The messenger method requires no preconditioner, but a high-quality solution needs a cooling parameter to control the convergence. We study the convergence properties of this new method and discuss how the algorithm is feasible for the large data sets of current and future CMB experiments.

Forecasting Air Quality Index Using an Ensemble of Artificial Neural Networks and Regression Models

Abstract Air is the most essential constituent for the sustenance of life on earth. The air we inhale has a tremendous impact on our health and well-being. Hence, it is always advisable to monitor the quality of air in our environment. To forecast the air quality index (AQI), artificial neural networks (ANNs) trained with conjugate gradient descent (CGD), such as multilayer perceptron (MLP), cascade forward neural network, Elman neural network, radial basis function (RBF) neural network, and nonlinear autoregressive model with exogenous input (NARX) along with regression models such as multiple linear regression (MLR) consisting of batch gradient descent (BGD), stochastic gradient descent (SGD), mini-BGD (MBGD) and CGD algorithms, and support vector regression (SVR), are implemented. In these models, the AQI is the dependent variable and the concentrations of NO2, CO, O3, PM2.5, SO2, and PM10 for the years 2010–2016 in Houston and Los Angeles are the independent variables. For the final forecast, several ensemble models of individual neural network predictors and individual regression predictors are presented. This proposed approach performs with the highest efficiency in terms of forecasting air quality index.

Privacy-Preserving Distributed Linear Regression on High-Dimensional Data

Abstract We propose privacy-preserving protocols for computing linear regression models, in the setting where the training dataset is vertically distributed among several parties. Our main contribution is a hybrid multi-party computation protocol that combines Yao’s garbled circuits with tailored protocols for computing inner products. Like many machine learning tasks, building a linear regression model involves solving a system of linear equations. We conduct a comprehensive evaluation and comparison of different techniques for securely performing this task, including a new Conjugate Gradient Descent (CGD) algorithm. This algorithm is suitable for secure computation because it uses an efficient fixed-point representation of real numbers while maintaining accuracy and convergence rates comparable to what can be obtained with a classical solution using floating point numbers. Our technique improves on Nikolaenko et al.’s method for privacy-preserving ridge regression (S&amp;P 2013), and can be used as a building block in other analyses. We implement a complete system and demonstrate that our approach is highly scalable, solving data analysis problems with one million records and one hundred features in less than one hour of total running time.

Direct diffusion tensor estimation using a model-based method with spatial and parametric constraints.

To develop a new model-based method with spatial and parametric constraints (MB-SPC) aimed at accelerating diffusion tensor imaging (DTI) by directly estimating the diffusion tensor from highly undersampled k-space data. The MB-SPC method effectively incorporates the prior information on the joint sparsity of different diffusion-weighted images using an L1-L2 norm and the smoothness of the diffusion tensor using a total variation seminorm. The undersampled k-space datasets were obtained from fully sampled DTI datasets of a simulated phantom and an ex-vivo experimental rat heart with acceleration factors ranging from 2 to 4. The diffusion tensor was directly reconstructed by solving a minimization problem with a nonlinear conjugate gradient descent algorithm. The reconstruction performance was quantitatively assessed using the normalized root mean square error (nRMSE) of the DTI indices. The MB-SPC method achieves acceptable DTI measures at an acceleration factor up to 4. Experimental results demonstrate that the proposed method can estimate the diffusion tensor more accurately than most existing methods operating at higher net acceleration factors. The proposed method can significantly reduce artifact, particularly at higher acceleration factors or lower SNRs. This method can easily be adapted to MR relaxometry parameter mapping and is thus useful in the characterization of biological tissue such as nerves, muscle, and heart tissue.