Types Of Learning Algorithms Research Articles

Application of power-law committee machine to combine five machine learning algorithms for enhanced oil recovery screening

One of the main challenges in screening of enhanced oil recovery (EOR) techniques is the class imbalance problem, where the number of different EOR techniques is not equal. This problem hinders the generalization of the data-driven methods used to predict suitable EOR techniques for candidate reservoirs. The main purpose of this paper is to propose a novel approach to overcome the above challenge by taking advantage of the Power-Law Committee Machine (PLCM) technique optimized by Particle Swam Optimization (PSO) to combine the output of five cutting-edge machine learning methods with different types of learning algorithms. The PLCM method has not been used in previous studies for EOR screening. The machine learning models include the Artificial Neural Network (ANN), CatBoost, Random Forest (RF), K-Nearest Neighbors (KNN), and Support Vector Machine (SVM). The CatBoost is used for the first time in this work for screening of EOR methods. The role of the PSO is to find the optimal values for the coefficients and exponents of the power-law model. In this study, a bigger dataset than those in previous studies, including 2563 successful worldwide EOR experiences, was gathered. A bigger dataset improves the generalization of the data-driven methods and prevents overfitting. The hyperparameters of the individual machine-learning models were tuned using the fivefold cross-validation technique. The results showed that all the individual methods could predict the suitable EOR method for unseen cases with an average score of 0.868. Among the machine learning models, the KNN and SVM had the highest scores with a value of 0.894 and 0.892, respectively. Nonetheless, after combining the output of the models using the PLCM method, the score of the predictions improved to 0.963, which was a substantial increase. Finally, a feature importance analysis was conducted to find out the most influential parameters on the output. The novelty of this work is having shown the ability of the PLCM technique to construct an accurate model to overcome the class-imbalance issue in EOR screening by utilizing different types of data-driven models. According to feature importance analysis, oil gravity and formation porosity were recognized as the most influential parameters on EOR screening.

Adversarial Transferability in Embedded Sensor Systems: An Activity Recognition Perspective

Machine learning algorithms are increasingly used for inference and decision-making in embedded systems. Data from sensors are used to train machine learning models for various smart functions of embedded and cyber-physical systems ranging from applications in healthcare, autonomous vehicles, and national security. However, recent studies have shown that machine learning models can be fooled by adding adversarial noise to their inputs. The perturbed inputs are called adversarial examples. Furthermore, adversarial examples designed to fool one machine learning system are also often effective against another system. This property of adversarial examples is called adversarial transferability and has not been explored in wearable systems to date. In this work, we take the first stride in studying adversarial transferability in wearable sensor systems from four viewpoints: (1) transferability between machine learning models; (2) transferability across users/subjects of the embedded system; (3) transferability across sensor body locations; and (4) transferability across datasets used for model training. We present a set of carefully designed experiments to investigate these transferability scenarios. We also propose a threat model describing the interactions of an adversary with the source and target sensor systems in different transferability settings. In most cases, we found high untargeted transferability, whereas targeted transferability success scores varied from 0% to 80%. The transferability of adversarial examples depends on many factors such as the inclusion of data from all subjects, sensor body position, number of samples in the dataset, type of learning algorithm, and the distribution of source and target system dataset. The transferability of adversarial examples decreased sharply when the data distribution of the source and target system became more distinct. We also provide guidelines and suggestions for the community for designing robust sensor systems. Code and dataset used in our analysis is publicly available here. 1

Fake Customer Review Detection System

Online purchasing is rising bit by bit since each service or product is easily accessible. Sellers are obtaining more reaction to one’s corporation factors. Several people generally frustrated kinds of persons misdirect others by sharing false comments to encourage or damage the image of any specific goods or services according to wish. Such people are known as perception spammers and the false reviews they give are considered as fake comments. Although customer reviews could be beneficial, naïve confidence in such comments is unsafe for either the buyers or sellers. Many consumers read research before making any online purchase. Moreover, the comments could be misleading for additional benefit or profit, so any buying decision relied on web comments should be taken carefully. Our work is mainly directed to SA at the document level, more specifically, on movie reviews dataset. Machine learning techniques and SA methods are expected to have a major positive effect, especially for the detection processes of fake reviews in restaurant reviews, e-commerce, social commerce environments, and other domains. In machine learning-based techniques, algorithms such as SVM, NB, and NLP are applied for the classification purposes SVM is a type of learning algorithm that represents supervised machine learning approaches, and it is an excellent successful prediction approach. The SVM is also a robust classification approach. The main goal of our study is to classify restaurant reviews as a real review or fake review using SA algorithms with supervised learning techniques. Keywords: — Supervised Machine Learning Techniques, Support Vector Machine, Natural Language Processing and Naıve Bayes.

Short-Term Solar PV Power Generation Day-Ahead Forecasting Using Artificial Neural Network: Assessment and Validation

Solar photovoltaics (PV) is considered an auspicious key to dealing with energy catastrophes and ecological contamination. This type of renewable energy is based on climatic conditions to produce electrical power. In this article, a multilayer feedforward neural network (MLFFNN) is implemented to predict and forecast the output power for a solar PV power station. The MLFFNN is designed using the module temperature and the solar radiation as the two main only inputs, whereas the expected power is its output. Data of approximately one week (6-days) are obtained from a real PV power station in Egypt. The data of the first five days are used to train the MLFFNN. The training of the designed MLFFNN is executed using two types of learning algorithms: Levenberg-Marquardt (LM) and error backpropagation (EBP). The data of the sixth day, which are not used for the training, are used to check the efficiency and the generalization capability of the trained MLFFNN by both algorithms. The results provide evidence that the trained MLFFNN is running very well and efficiently to predict the power correctly. The results obtained from the trained MLFFNN by LM (MLFFNN-LM) are compared with the corresponding ones obtained by the MLFFNN trained by EBP (MLFFNN-EBP). From this comparison, the MLFFNN-LM has slightly lower performance in the training stage and slightly better performance in the stage of effectiveness investigation compared with the MLFFNN-EBP. Finally, a comparison with other previously published approaches is presented. Indeed, predicting the power correctly using the artificial NN is useful to avoid the fall of the power that maybe happen at any time.

Structural Health Monitoring for impact localisation via machine learning

The focus of this manuscript is Machine learning applied to the Structural Health Monitoring. Two different algorithms have been used to detect impacts on an aluminium plate, one based on the polynomial regression, the other one based on a shallow neural network. Both are supervised algorithms in which some data impacts are used for training, while a complementary subset of data is used for test. The two methods have been preliminarily optimized in terms of training and testing performance and, subsequently, compared in terms of accuracy. By using K-Fold cross validation procedure, with different combinations of training/test sets, the performances of the polynomial models with different degrees were evaluated calculating the Mean Radial Error. For the shallow neural network, three type of learning algorithms were compared: Levenberg-Marquardt, Bayesian Regularization, Scaled Conjugate Gradient. The study confirmed the effectiveness of Machine learning applied to the detection of impacts.

Double Deep Reinforcement Learning-Based Energy Management for a Parallel Hybrid Electric Vehicle With Engine Start–Stop Strategy

Committed to optimizing the fuel economy of hybrid electric vehicles (HEVs), improving the working conditions of the engine, and promoting research on deep reinforcement learning (DRL) in the field of energy management strategies (EMSs), this article first proposed a DRL-based EMS combined with a rule-based engine start–stop strategy. Moreover, considering that both the engine and the transmission are controlled components, this article developed a novel double DRL (DDRL)-based EMS, which uses a deep Q-network (DQN) to learning the gear-shifting strategy and uses a deep deterministic policy gradient (DDPG) to control the engine throttle opening, and the DDRL-based EMS realizes the multiobjective synchronization control by different types of learning algorithms. After off-line training, the simulation result of the online test shows that the fuel consumption gaps of the proposed DRL- and DDRL-based EMSs are −0.55% and 2.33% compared to that of the deterministic dynamic programming (DDP)-based EMS by overcoming some inherent flaws of DDP, respectively. The computational efficiency has been significantly improved, and the average output time per action is 0.91 ms. Therefore, the control strategy that combines learning- and rule-based controls and the multiobjective control strategies both have the potential to ensure optimization and real-time efficiency.

Combination of Reduction Detection Using TOPSIS for Gene Expression Data Analysis

In high-dimensional data analysis, Feature Selection (FS) is one of the most fundamental issues in machine learning and requires the attention of researchers. These datasets are characterized by huge space due to a high number of features, out of which only a few are significant for analysis. Thus, significant feature extraction is crucial. There are various techniques available for feature selection; among them, the filter techniques are significant in this community, as they can be used with any type of learning algorithm and drastically lower the running time of optimization algorithms and improve the performance of the model. Furthermore, the application of a filter approach depends on the characteristics of the dataset as well as on the machine learning model. Thus, to avoid these issues in this research, a combination of feature reduction (CFR) is considered designing a pipeline of filter approaches for high-dimensional microarray data classification. Considering four filter approaches, sixteen combinations of pipelines are generated. The feature subset is reduced in different levels, and ultimately, the significant feature set is evaluated. The pipelined filter techniques are Correlation-Based Feature Selection (CBFS), Chi-Square Test (CST), Information Gain (InG), and Relief Feature Selection (RFS), and the classification techniques are Decision Tree (DT), Logistic Regression (LR), Random Forest (RF), and k-Nearest Neighbor (k-NN). The performance of CFR depends highly on the datasets as well as on the classifiers. Thereafter, the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method is used for ranking all reduction combinations and evaluating the superior filter combination among all.

A Study on the Application and the Advancement of Deep Neural Network Algorithm

Deep neural network is a new type of learning algorithm, which has both global and local aspects and performs well in pattern recognition and computational speed. In recent years, deep neural network algorithm has been widely used in scientific research and real life, but its complexity, parallelism and other characteristics lead it to be a very challenging and innovative research area. This study briefly introduces the basic principles and theoretical knowledge of deep neural network algorithms, and mainly discusses their applications and Advancement of feature extraction in the field.

Digital education as a method of reducing pedagogical goals and objectives

The article focuses on a comparative characteristic of predigital and digital education. The study methodology deploys the differentiation of philosophical and scientific paradigms. Pre-digital education is associated with the classical paradigm. Accordingly, the parameters of pre-digital education include the notions of finite and true knowledge, a singular type of learning algorithms, a balanced learning process, and the deterministic influence of educational programs. The indicators of digital education are identified within the framework of the non-classical and post-non-classical paradigms. The non-classical form of digital education is characterized by the presence of alternative educational programs, the polysemantic object of study, infinite knowledge, and pragmatic orientation. This form of education accentuates the subjective and personal context, the opportunities for plural explanations, and the technologies for studying the singular, the local, and the fragmental. Digital education contributes to modeling infinite discourses and intersubjective practices. All educational innovations within digital education appear as a way of ontological and gnoseological construction. Digital education enhances the role of interdisciplinary education. The competencies of “cross-cutting” and extensive understanding of a complex subject appear. Students should be focused on the results of teaching in various disciplines. A consequence of the expanding learning opportunities is the necessary reduction of pedagogical goals and objectives.

Prediction of Critical Flashover Voltage of High Voltage Insulators Leveraging Bootstrap Neural Network

Understanding the flashover performance of the outdoor high voltage insulator has been in the interest of many researchers recently. Various studies have been performed to investigate the critical flashover voltage of outdoor high voltage insulators analytically and in the laboratory. However, laboratory experiments are expensive and time-consuming. On the other hand, mathematical models are based on certain assumptions which compromise on the accuracy of results. This paper presents an intelligent system based on Artificial Neural Networks (ANN) to predict the critical flashover voltage of High-Temperature Vulcanized (HTV) silicone rubber in polluted and humid conditions. Various types of learning algorithms are used, such as Gradient Descent (GD), Levenberg-Marquardt (LM), Conjugate Gradient (CG), Quasi-Newton (QN), Resilient Backpropagation (RBP), and Bayesian Regularization Backpropagation (BRBP) to train the ANN. The number of neurons in the hidden layers along with the learning rate was varied to understand the effect of these parameters on the performance of ANN. The proposed ANN was trained using experimental data obtained from extensive experimentation in the laboratory under controlled environmental conditions. The proposed model demonstrates promising results and can be used to monitor outdoor high voltage insulators. It was observed from obtained results that changing of the number of neurons, learning rates, and learning algorithms of ANN significantly change the performance of the proposed algorithm.

Robust adaptive repetitive control for a class of nonlinear periodically time-varying systems

This paper presents a robust adaptive repetitive control (RARC) method for a class of periodically time-varying nonlinear systems with aperiodic uncertainties. A σ modification is introduced in the learning algorithm of RARC, in order to guarantee robustness of the system undertaken. The closed-loop type learning algorithm is examined and it is shown that the realisability cannot be assured when the σ modification is applied. To avoid the causality contradiction, an open-loop type learning algorithm with switching σ modification is proposed to guarantee robustness and achieve the asymptotic convergence of the tracking error, when the disturbances disappear. Extension to the RARC for robotic manipulators is given and the numerical simulation is carried out to verify the effectiveness of the learning control scheme.

Study on the Influence of Measuring AE Sensor Type on the Effectiveness of OLTC Defect Classification.

The principal objective of this study is to improve the diagnostics of power transformers, which are the key element of supplying electricity to consumers. On Load Tap Changer (OLTC), which is the object of research, the results of which are presented in this article, is one of the most important elements of these devices. The applied diagnostic method is the acoustic emission (AE) method, which has the main advantage over others, that it is considered as a non-destructive testing method. At present, there are many measuring devices and sensors used in the AE method, there are also some international standards, according to which, measurements should be performed. In the presented work, AE signals were measured in laboratory conditions with various OLTC defects being simulated. Five types of sensors were used for the measurement. The recorded signals were analyzed in the time and frequency domain and using discrete wavelet transformation. Based on the results obtained, sets of indicators were determined, which were used as features for an autonomous classification of the type of defect. Several types of learning algorithms from the group of supervised machine learning were considered in the research. The performance of individual classifiers was determined by several quality evaluation measures. As a result of the analyses, the type and characteristics of the most optimal algorithm to be used in the process of classification of the OLTC fault type were indicated, depending on the type of sensor with which AE signals were recorded.

Predicting radioactive waste glass dissolution with machine learning

The vitrification of high-level nuclear waste within borosilicate glass and its disposition within a multi-barrier repository deep underground is accepted as the best form of disposal. Here, the ability of machine learning to predict both static and dynamic glass leaching behavior is analysed using large-scale unstructured multi-source data, covering a diverse range of experimental conditions and glass compositions. Machine learning can accurately predict leaching behavior, predict missing data, and time forecast. Accuracy depends upon the type of learning algorithm, model input variables, and diversity or size of the underlying dataset. For static leaching, the bagged random forest method predicts well, even when either pH or glass composition are neglected as input variables, additionally showing potential in predicting independent glass dissolution data. For dynamic leaching, accuracy improves if replacing final pH with a species dissolution rate as an input variable, although results show no preferred output species (Si, Na, or Al).

Identification of non-linear stochastic systems using a new Hammerstein-Wiener neural network: a simulation study through a non-linear hydraulic process

Hammerstein-Wiener models have been proved to be suitable in modelling a class of typical non-linear dynamic systems. This paper aims at developing a Hammerstein-Wiener Neural Network (HWNN) which formulates a Hammerstein-Wiener mathematical model, in order to identify a nonlinear dynamic system operating in a stochastic environment. A central aspect is that a general situation has been considered including non-invertible non-linearity output and correlation of stochastic disturbances after the dynamic linear block. Different from the existing parameter identification methods, the model is developed to handle two types of learning algorithms that can directly obtain the parameters of the unknown time-varying nonlinear system. Firstly, all neural network weights in HWNN are adapted using a Back Propagation-based Gradient algorithm (BPG). The second method, namely Recursive Least Square Back Propagation based Gradient (RLSBPG) is derived from the BPG algorithm to achieve the parametric estimation of the Hammerstein scheme where the remaining parameters are estimated by the least-squares approach based on fuzzy technique to ameliorate the estimation quality. The convergence analysis of the algorithms is presented, and their performances are tested through a simulation study of a nonlinear hydraulic process.

Identification of non-linear stochastic systems using a new Hammerstein-Wiener neural network: a simulation study through a non-linear hydraulic process

Hammerstein-Wiener models have been proved to be suitable in modelling a class of typical non-linear dynamic systems. This paper aims at developing a Hammerstein-Wiener Neural Network (HWNN) which formulates a Hammerstein-Wiener mathematical model, in order to identify a nonlinear dynamic system operating in a stochastic environment. A central aspect is that a general situation has been considered including non-invertible non-linearity output and correlation of stochastic disturbances after the dynamic linear block. Different from the existing parameter identification methods, the model is developed to handle two types of learning algorithms that can directly obtain the parameters of the unknown time-varying nonlinear system. Firstly, all neural network weights in HWNN are adapted using a Back Propagation-based Gradient algorithm (BPG). The second method, namely Recursive Least Square Back Propagation based Gradient (RLSBPG) is derived from the BPG algorithm to achieve the parametric estimation of the Hammerstein scheme where the remaining parameters are estimated by the least-squares approach based on fuzzy technique to ameliorate the estimation quality. The convergence analysis of the algorithms is presented, and their performances are tested through a simulation study of a nonlinear hydraulic process.

On artificial neural networks approach with new cost functions

In this manuscript, the artificial neural networks approach involving generalized sigmoid function as a cost function, and three-layered feed-forward architecture is considered as an iterative scheme for solving linear fractional order ordinary differential equations. The supervised back-propagation type learning algorithm based on the gradient descent method, is able to approximate this a problem on a given arbitrary interval to any desired degree of accuracy. To be more precise, some test problems are also given with the comparison to the simulation and numerical results given by another usual method.

Performance Comparison of Neural Network Training Algorithms for Modeling Customer Churn Prediction

Predicting customer churn has become the priority of every telecommunication service provider as the market is becoming more saturated and competitive. This paper presents a comparison of neural network learning algorithms for customer churn prediction. The data set used to train and test the neural network algorithms was provided by one of the leading telecommunication company in Malaysia. The Multilayer Perceptron (MLP) networks are trained using nine (9) types of learning algorithms, which are Levenberg Marquardt backpropagation (trainlm), BFGS Quasi-Newton backpropagation (trainbfg), Conjugate Gradient backpropagation with Fletcher-Reeves Updates (traincgf), Conjugate Gradient backpropagation with Polak-Ribiere Updates (traincgp), Conjugate Gradient backpropagation with Powell-Beale Restarts (traincgb), Scaled Conjugate Gradient backpropagation (trainscg), One Step Secant backpropagation (trainoss), Bayesian Regularization backpropagation (trainbr), and Resilient backpropagation (trainrp). The performance of the Neural Network is measured based on the prediction accuracy of the learning and testing phases. LM learning algorithm is found to be the optimum model of a neural network model consisting of fourteen input units, one hidden node and one output node. The best result of the experiment indicated that this model is able to produce the performance accuracy of 94.82%.

A comparative study of the feed forward back propagation (FFBP) and layer recurrent (LR) neural network model for forecasting ground level ozone concentration

Ozone (O3) in the troposphere is considered as a secondary air pollutant and has an adverse impact on human health and climatic condition. In many countries including India, O3 is listed as one of the criteria pollutants. Thus, a proper forecasting technique of ozone concentration is necessary for protecting the human health. The concentration of ozone in the troposphere depends on the meteorological condition and precursor’s levels. Hence, it is essential to consider these dependent factors in the development of prediction model. The study aims to develop an ozone forecasting model using artificial neural network (ANN). Three-year air pollution and meteorological data (1 January 2009 to 31 December 2011) of Kolkata City was used for model development. Two types of learning algorithms [feed forward back propagation (FFBP) and layer recurrent (LR)] were used for training the ANN model. Four meteorological factors (relative humidity, temperature, wind speed, and wind direction) along with the NO2 concentration and previous day’s ozone concentration were used as input parameters in the model for predicting the ozone concentration. The number of neurons in the hidden layers of a neural network model was optimized for both the algorithms. The number of input combinations was also optimized using forward search algorithm. The model performances were tested using four statistical indices [percentage of root mean square error (RMSE), coefficient of determination (R 2), fractional bias (FB), index of agreement (IOA)] for evaluating the ANN models.

Застосування інтелектуальної діагностичної інфраструктури для управління кібербезпекою. Частина 2. Підтримка життєвого циклу системи кіберзахисту

How use of structural features in the construction of hybrid models allows supporting of models adjustment and adaptation to problem-subject environment was considered in the article. The following features were attributed to structural ones: the type of learning algorithm; kind of activation function; the number of layers of the neural network; type of neurons; way of spreading information in neural networks; method of evaluating and interpreting the results of the neural network; the format of fuzzy inference rules; fuzzification and defuzzification method; way to implement the operations of fuzzy implication and logical operations NOT, AND, OR; kind of used genetic operators and the target functions, etc. We propose to use a neural network approach as a basis for the decision of difficulty tasks using decision-support systems. Its effectiveness can be enhanced by: prior training or adjustment of individual neural modules for solvable problem; incorporation of knowledge about the peculiarities of the domain in the hierarchical (multilayer) neural networks structure; application of basic types of hybrid models in which neural network communicates with other information technologies.

Reaching optimized parameter set: protein secondary structure prediction using neural network

We propose an optimized parameter set for protein secondary structure prediction using three layer feed forward back propagation neural network. The methodology uses four parameters viz. encoding scheme, window size, number of neurons in the hidden layer and type of learning algorithm. The input layer of the network consists of neurons changing from 3 to 19, corresponding to different window sizes. The hidden layer chooses a natural number from 1 to 20 as the number of neurons. The output layer consists of three neurons, each corresponding to known secondary structural classes viz. alpha helix, beta strands and coils respectively. It also uses eight different learning algorithms and nine encoding schemes. Exhaustive experiments were performed using non-homologues dataset. The experimental results were compared using performance measures like Q3, sensitivity, specificity, Mathew correlation coefficient and accuracy. The paper also discusses the process of obtaining a stabilized cluster of 2530 records from a collection of 11340 records. The graphs of these stabilized clusters of records with respect to accuracy are concave, convergence is monotonic increasing and rate of convergence is uniform. The paper gives BLOSUM62 as the encoding scheme, 19 as the window size, 19 as the number of neurons in the hidden layer and One- Step Secant as the learning algorithm with the highest accuracy of 78%. These parameter values are proposed as the optimized parameter set for the three layer feed forward back propagation neural network for the protein secondary structure predictionv