Multiplicative Neuron Model Research Articles

The Forecasting of TheNumber Tourists Arriving in Turkey with Intuitionistic Fuzzy Regression Functions Approach

The impacts of the tourism sector on countries are felt in various areas such as economy, cultural heritage and social development. Tourism contributes significantly to a country's foreign exchange earnings and positively affects the trade network. Tourists' spending boosts local economies and increases employment. These effects are particularly important for Turkey. Tourist visits can be used as a tool for regional promotion. Therefore, tourism demand forecasting is necessary to make the best use of these positive effects on Turkey's economic development and to plan tourism activities. Artificial neural network methods and fuzzy systems for time series forecasting problem are frequently used analysis methods in recent years. In this study, the time series of the total number of tourists visiting Turkey on a monthly basis is analyzed with the intuitionistic fuzzy regression functions approach, which is a generalization of the fuzzy regression functions approach. The analysis performance of the intuitionistic fuzzy regression functions approach is evaluated using fuzzy regression functions approach, multilayer perceptron artificial neural network and multiplicative neuron model artificial neural networks. As a result of the analysis, it is concluded that the intuitionistic fuzzy regression approach produces better forecasting results than both some artificial neural network models and the fuzzy regression functions approach. Since this is the first time that the intuitionistic fuzzy regression functions approach has been used in forecasting the number of tourists, the study aims to contribute to the literature and to help tourism industry employees to be more efficient and successful by providing them with the opportunity to make better future planning.

A new single multiplicative neuron model artificial neural network based on black hole optimization algorithm: forecasting the amounts of clean water given to metropolis

A new single multiplicative neuron model artificial neural network based on black hole optimization algorithm: forecasting the amounts of clean water given to metropolis

Evaluating single multiplicative neuron models in physics-informed neural networks for differential equations

Machine learning is a prominent and highly effective field of study, renowned for its ability to yield favorable outcomes in estimation and classification tasks. Within this domain, artificial neural networks (ANNs) have emerged as one of the most powerful methodologies. Physics-informed neural networks (PINNs) have proven particularly adept at solving physics problems formulated as differential equations, incorporating boundary and initial conditions into the ANN’s loss function. However, a critical challenge in ANNs lies in determining the optimal architecture, encompassing the selection of the appropriate number of neurons and layers. Traditionally, the Single Multiplicative Neuron Model (SMNM) has been explored as a solution to this issue, utilizing a single neuron with a multiplication function in the hidden layer to enhance computational efficiency. This study initially aimed to apply the SMNM within the PINNs framework, targeting the differential equation y′-y=0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$y'-y=0$$\\end{document} with boundary conditions y(0)=1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$y(0) = 1$$\\end{document} and y(1)=e\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$y(1) = e$$\\end{document}. Upon implementation, however, it was discovered that while the conventional SMNM approach was theorized to offer significant advantages, multiplicative aggregate function led to a failure in convergence. Consequently, we introduced a “mimic single multiplicative neuron model (mimic-SMNM)” employing an architecture with a single neuron, designed to simulate the SMNM’s conceptual advantages while ensuring convergence and computational efficiency. Comparative analysis revealed that the real-PINNs accurately solved the equation, the true SMNM failed to converge, and the mimic model was highlighted for its architectural simplicity and computational feasibility, directly implying it is faster and more efficient than real PINNs for the solution of simple differential equations. Furthermore, our findings demonstrated that our proposed mimic-SMNM model achieves a five-times increase in computational speed compared to real PINNs after 30,000 epochs.

Water Quality Assessment with Artificial Neural Network Models: Performance Comparison Between SMN, MLP and PS-ANN Methodologies

AbstractIdentifying and measuring potential sources of pollution is essential for water management and pollution control. Using a range of artificial intelligence models to analyze water quality (WQ) is one of the most effective techniques for estimating water quality index (WQI). In this context, machine learning–based models are introduced to predict the WQ factors of Southeastern Black Sea Basin. The data comprising monthly samples of different WQ factors were collected for 12 months at eight locations of the Türkiye region in Southeastern Black Sea. The traditional evaluation with WQI of surface water was calculated as average (i.e. good WQ). Single multiplicative neuron (SMN) model, multilayer perceptron (MLP) and pi-sigma artificial neural networks (PS-ANNs) were used to predict WQI, and the accuracy of the proposed algorithms were compared. SMN model and PS-ANNs were used for WQ prediction modeling for the first time in the literature. According to the results obtained from the proposed ANN models, it was found to provide a highly reliable modeling approach that allows capturing the nonlinear structure of complex time series and thus to generate more accurate predictions. The results of the analyses demonstrate the applicability of the proposed pi-sigma model instead of using other computational methods to predict WQ both in particular and other surface water resources in general.

A new genetic algorithm method based on statistical-based replacement for the training of multiplicative neuron model artificial neural networks

In this study, we propose a new genetic algorithm that uses a statistical-based chromosome replacement strategy determined by the empirical distribution of the objective function values. The proposed genetic algorithm is further used in the training process of a multiplicative neuron model artificial neural network. The objective function value for the genetic algorithm is the root mean square error of the multiplicative neuron model artificial neural network prediction. This combination of methods is proposed for a particular type of problems, that is, time-series prediction. We use different subsets of three stock exchange time series to test the performance of the proposed method and compare it against similar approaches, and the results prove that the proposed genetic algorithm for the multiplicative neuron model of the artificial neural network works better than many other artificial intelligence optimization methods. The ranks of the proposed method are 1.78 for the Nikkei data sets, 1.55 for the S&P500 data sets and 1.22 for the DOW JONES data sets for data corresponding to different years, according to the root mean square error, respectively. Moreover, the overall mean rank is 1.50 for the proposed method. Also, the proposed method obtains the best performance overall as well as the best performance for all the individual tests. The results certify that our method is robust and efficient for the task investigated.

A Robust Learning Algorithm Based on Particle Swarm Optimization for Pi-Sigma Artificial Neural Networks.

Artificial neural networks (ANNs) have been frequently used in forecasting problems in recent years. One of the most popular types of ANNs in these days is Pi-Sigma artificial neural networks (PS-ANNs). PS-ANNs have a high order ANN structure and they use both multiplicative and additive neuron models in their architecture. PS-ANNs produce superior forecasting performance because of their high order structure. PS-ANNs are affected negatively by an outlier or outliers in a data set because of having a multiplicative neuron model in their architecture. In this study, a new robust learning algorithm based on particle swarm optimization and Huber's loss function for PS-ANNs is proposed. To evaluate the performance of the proposed method, Dow Jones stock exchange and Australian beer consumption data sets are analyzed and the obtained results are compared with many ANNs types proposed in the literature. Besides, the performance of the proposed method in outlier cases is also investigated by injecting outliers into these data sets. It is seen that the proposed learning algorithm has a satisfying performance both the data have an outlier or outliers' case and original case.

A new nonlinear causality test based on single multiplicative neuron model artificial neural network: a case study for Turkey’s macroeconomic indicators

A new nonlinear causality test based on single multiplicative neuron model artificial neural network: a case study for Turkey’s macroeconomic indicators

An Evolutionary Field Theorem: Evolutionary Field Optimization in Training of Power-Weighted Multiplicative Neurons for Nitrogen Oxides-Sensitive Electronic Nose Applications.

Neuroevolutionary machine learning is an emerging topic in the evolutionary computation field and enables practical modeling solutions for data-driven engineering applications. Contributions of this study to the neuroevolutionary machine learning area are twofold: firstly, this study presents an evolutionary field theorem of search agents and suggests an algorithm for Evolutionary Field Optimization with Geometric Strategies (EFO-GS) on the basis of the evolutionary field theorem. The proposed EFO-GS algorithm benefits from a field-adapted differential crossover mechanism, a field-aware metamutation process to improve the evolutionary search quality. Secondly, the multiplicative neuron model is modified to develop Power-Weighted Multiplicative (PWM) neural models. The modified PWM neuron model involves the power-weighted multiplicative units similar to dendritic branches of biological neurons, and this neuron model can better represent polynomial nonlinearity and they can operate in the real-valued neuron mode, complex-valued neuron mode, and the mixed-mode. In this study, the EFO-GS algorithm is used for the training of the PWM neuron models to perform an efficient neuroevolutionary computation. Authors implement the proposed PWM neural processing with the EFO-GS in an electronic nose application to accurately estimate Nitrogen Oxides (NOx) pollutant concentrations from low-cost multi-sensor array measurements and demonstrate improvements in estimation performance.

Single Neuron for Solving XOR like Nonlinear Problems.

XOR is a special nonlinear problem in artificial intelligence (AI) that resembles multiple real-world nonlinear data distributions. A multiplicative neuron model can solve these problems. However, the multiplicative model has the indigenous problem of backpropagation for densely distributed XOR problems and higher dimensional parity problems. To overcome this issue, we have proposed an enhanced translated multiplicative single neuron model. It can provide desired tessellation surface. We have considered an adaptable scaling factor associated with each input in our proposed model. It helps in achieving optimal scaling factor value for higher dimensional input. The efficacy of the proposed model has been tested by randomly increasing input dimensions for XOR-type data distribution. The proposed model has crisply classified even higher dimensional input in their respective class. Also, the computational complexity is the same as that of the previous multiplicative neuron model. It has shown more than an 80% reduction in absolute loss as compared to the previous neuron model in similar experimental conditions. Therefore, it can be considered as a generalized artificial model (single neuron) with the capability of solving XOR-like real problems.

A new automatic forecasting method based on a new input significancy test of a single multiplicative neuron model artificial neural network

ABSTRACT The model adequacy and input significance tests have not been proposed as features for the specification of a single multiplicative neuron model artificial neural networks in the literature. Moreover, there is no systematic approach based on hypothesis tests for using single multiplicative neuron model artificial neural networks for forecasting purposes like classical time series forecasting methods. In this study, new methods are proposed to solve these problems. The performance of the proposed test procedures is investigated in a simulation study. According to simulation results, the proposed tests have very good performance. Moreover, the test procedures are illustrated by using two real-world examples. The second contribution of the paper is that an automatic forecasting method is proposed based on input significance and model adequacy tests and the particle swarm optimization-based learning algorithm. The proposed automatic forecasting method is applied to M4 competition hourly data sets, and it is the best pure machine learning method among others in the competition. The proposed automatic forecasting method is more accurate than all benchmarks, such as MLP, RNN, and ETS, which were proposed by the competition organizers.

Time-series interval prediction under uncertainty using modified double multiplicative neuron network

This paper presents a hybrid intelligent approach for constructing prediction intervals (PIs) of terrain profiles over time under uncertainty. It utilizes the double multiplicative neuron (DMN) model and the modified particle swarm optimization (MPSO) algorithm to calculate the upper and lower bounds of unknown elevations ahead on terrain profiles based on the vehicles’ track. MPSO withholds particles generating the positive PIs in the training epochs, in order to prevent the occurrence of unreasonable upside-down PIs that are brought by conventional methods. MPSO adjusts the parameters of the DMN model iteratively by minimizing the value of the proposed cost function. The fitness function aims to enhance DMN’s capability of forecasting terrain trends by integrating a trend indicator with PIs coverage probability and interval widths. This study utilizes the terrain profiles of 3 arc-seconds resolution to verify the effectiveness of the proposed MPSO-DMNT approach for one-step and multi-step PIs estimation. Experimental results demonstrate that the proposed approach (1) overcomes the limitations of the conventional PIs indicators; (2) improves the prediction accuracy for terrain trends by 18.8% in the training data and 15.4% in the testing data, and reduces the computational burden by 31.6% in the training data and 8% in the testing data over the lower upper bound estimation (LUBE) method; (3) achieves comparative coverage probability and interval widths to LUBE using a low-complexity single-layered network. The proposed hybrid approach can be used as an auxiliary decision-making tool for terrain avoidance and terrain following in flight.

Data-driven time series prediction based on multiplicative neuron model artificial neuron network

This paper develops a hybrid approach combining the neural network and the nonlinear filtering to model and predict terrain profiles for both air and ground vehicles. To simplify the neural network structures and reduce the number of synaptic weights and biases, the multiplicative neuron model (MNM) is utilized to describe the relationship between the unknown elevation ahead and the last few height values on the terrain profile. This paper adopts the gradient descent algorithm (GDA) to train the MNM terrain model and stores the MNM parameters into a nonlinear state-space model. The state vector in the state-space model (i.e., parameters of MNM) evolve agilely once absorbing new observations and measurement of elevation values by the Bootstrap Particle Filter (BPF) algorithm. Data-driven predictions on terrain profiles can be achieved through the updated MNM model. This study utilizes two types of terrain profiles to verify the effectiveness of the proposed MNM–BPF approach. Experimental results on two public datasets indicate that the proposed approach not only overcomes the limitations of conventional terrain models that cannot dynamically tune model parameters according to the newly input information, but also provides a simple but effective single-layered network for modeling terrain profiles. The well-trained MNM–BPF model can achieve the lowest root mean square errors (RMSE) (i.e., 17.3211 on the NS profile, 19.0366 on the EW profile) and average error (AE) (i.e., 1.5852 on the NS profile, 0.14885 on the EW profile) in the low-resolution dataset. The lowest RMSE (i.e., 0.16549 on the left profile, 0.29926 on the right profile) and mean absolute error (MAE) (i.e., 0.13467 on the left profile, 0.23933 on the right profile) results are obtained in the high-resolution dataset. Overall, the developed model is superior to the state-of-the-art models in at least four of the six performance metrics and reduces RMSE by 40.8%, 17.2%, 13.1%, and 6.8% on average on the four testing terrain profiles, respectively. The developed approach can be used as a decision tool for the accurate prediction of terrain profiles with different resolutions.

Online short-term load forecasting methods using hybrids of single multiplicative neuron model, particle swarm optimization variants and nonlinear filters

Short-term load (STL) forecasting plays a significant role in modern power system management. Improving the accuracy of STL forecasting is helpful for power enterprises to design reasonable operation planning, thus improving the economic and social benefits of the system. Hybrid methods consisted of single multiplicative neuron (SMN) model, various particle swarm optimization (PSO) algorithms and nonlinear filters are developed in this study. For this purpose, a SMN model based nonlinear state–space model is established using the weights and biases of SMN model and the output of SMN model to present the state vector and the measurement equation at first. Then PSO variants are used to optimize the weights and biases of SMN model with known STL training data. Finally, the nonlinear filters are employed to perform dynamic state estimation for STL forecasting by taking the optimal weights and biases of SMN model during training phase as the initialized value, and the STL forecasting results are represented by the predicted measurement value of nonlinear filters. The effectiveness of the suggested approaches is tested by using the STL datasets from Australian energy market operator, and the experimental results have demonstrated their attractiveness of the proposed methods by compared with other methods.

Threshold single multiplicative neuron artificial neural networks for non-linear time series forecasting

Single multiplicative neuron artificial neural networks have different importance than many other artificial neural networks because they do not have complex architecture problem, too many parameters and they need more computation time to use. In single multiplicative neuron artificial neural network, it is assumed that there is a one data generation process for time series. Many time series need an assumption that they have two data generation process or more. Based on this idea, the threshold model structure can be employed in a single multiplicative neuron model artificial neural network for taking into considering data generation processes problem. In this study, a new artificial neural network type is proposed and it is called a threshold single multiplicative neuron artificial neural network. It is assumed that time series have two data generation processes according to the architecture of single multiplicative neuron artificial neural network. Training algorithms are proposed based on harmony search algorithm and particle swarm optimization for threshold single multiplicative neuron artificial neural network. The proposed method is tested by various time series data sets and compared with well-known forecasting methods by considering different error measures. Finally, the performance of the proposed method is evaluated by a simulation study.

A hybrid algorithm based on artificial bat and backpropagation algorithms for multiplicative neuron model artificial neural networks

In the literature, the multiplicative neuron model artificial neural networks are trained by gradient-based or some artificial intelligence optimization algorithms. It is well known that the hybrid algorithms give successful results than classical algorithms in the literature and the use of hybrid systems increase day by day. From this point of view, different from other studies contribute to multiplicative neuron model artificial neural networks, the properties of an artificial intelligence optimization technique, artificial bat algorithm, and a gradient-based algorithm, backpropagation learning algorithm, is used together firstly by using the proposed method in this study. Thus, both a derivative and a heuristic algorithm were used together firstly for multiplicative neuron model artificial neural networks. The proposed method is applied to three well-known different real-world time series data. The performance of the proposed method is both compared with gradient-based optimization algorithms, some artificial optimization algorithms used for the training of artificial neural networks and some popular analyze methods. The analysis results show that the proposed hybrid method has superior performance than other methods.

Effect of Dispersion on Classification of Different Datasets Using Multiplicative Neuron Model

The classification of nonlinear problems, single Multiplicative Neuron model is used. Several different databases like Fisher’s Iris database, wiconsin Breast Cancer, Mammographic Mass and Pima Indian have selected for study. The measure of dispersion has been found of all datasets by measuring standard deviation and range of dataset. The classification results of all problems compared and comparative analysis has been made of all datasets, considering several elements of Neural Network such as number of epochs, cost function, MSE and misclassification rate. After comparing various performance parameters mainly misclassification rate with measures of dispersion, it is found that misclassification rate of multiplicative neuron model depends only patterns of datasets. Seeing the results of study, it can be said that classification rate does not depend on the measure of dispersion (Standard deviation and Range) of datasets.

MULTIPLICATIVE NEURON MODEL BASED ON SINE COSINE ALGORITHM FOR TIME SERIES PREDICTION

Time series prediction is a method to predict the system behavior in the future based on current given data. Neural Networks (NNs) approach is a well-known technique that is useful for time series prediction. In the literature many NN models such Multilayer Perceptron (MLP), Pi-Sigma NN (PSNN), Recurrent NN etc. are proposed for solving time series prediction. In this paper, we use Multiplicative Neuron Model (MNM) to predict time series. For training this model, we propose use newly developed evolutionary optimization algorithm called Sine Cosine algorithm (SCA), and this algorithm has not been used as far as we know in training the MNM. The proposed SCA-MNM model is employed for the most known time series problems. In this paper, the application of the SCA-MNM on time prediction is illustrated using two mostly used datasets Mackey-Glass time series dataset, Box-Jenkins gas furnace dataset. To investigate the effect of the proposed SCA-MNM model, comparisons were made with some of the results given in the literature.

Prediction of TAIEX based on hybrid fuzzy time series model with single optimization process

All fuzzy time series approaches proposed in the literature consider three steps constituting the solution process as separate processes. Thus, model error is the sum of the errors that may occur in each step. In this regard, synchronous evaluation of the steps constituting the analysis process will produce a single model error and will lead to a reduction in the model error. Within the scope of this study, we proposed an approach which evaluates the steps constituting fuzzy time series analysis in one process synchronously to forecast the Taiwan Stock Exchange Capitalization Weighted Stock Index. In the proposed approach, defuzzification step is eliminated by using real values of time series as target values in the identification of fuzzy relations step. In this respect, determination of fuzzy cluster centres in fuzzification and the training of artificial neural network with single multiplicative neuron which is used the identification of fuzzy relation are carried out in a single optimization process with particle swarm optimization. This work also covers comprehensive literature review and summary info of related methodologies including fuzzy time series, particle swarm optimization, single multiplicative neuron model and fuzzy C-means clustering. The proposed method is applied to twelve different time series and a total of twenty-four measurements are done and superior forecasting performance of the method is proven.

Single Multiplicative Neuron Model Artificial Neural Network with Autoregressive Coefficient for Time Series Modelling

Single multiplicative neuron model and multilayer perceptron have been commonly used for time series prediction problem. Having a simple structure and features of easily applicable differentiates the single multiplicative neuron model from the multilayer perception. While, multilayer perceptron just as many other artificial neural networks are data-based methods, single multiplicative neuron model has a model structure due to it is composed of a single neuron. Multilayer perceptron can highly compliance with data by changing its architecture, though single multiplicative neuron model, in this respect, is insufficient. In this study, to overcome this problem of single multiplicative neuron model, a new model that its weights and biases are obtained by way of autoregressive equations is proposed. Since the time indexes are considered to determine weights and biases from the autoregressive models, the proposed neural network can be evaluated as a data-based model. To show the performance and capability of the proposed method, various implementations have been executed over some well-known data sets. And the obtained results demonstrate that data-based proposed method has outstanding forecasting performance.

A combined robust fuzzy time series method for prediction of time series

In case of outlier(s) it is inevitable that the performance of the fuzzy time series prediction methods is influenced adversely. Therefore, current prediction methods will not be able to provide satisfactory accuracy rates for defuzzified outputs (predictions) when the data has outlier(s). In this study, not only to be able to sort out this problem but also to be able to improve the forecasting accuracy, we propose a combined robust approach for fuzzy time series by assessing how the prediction performance of the methods will be affected from the outlier(s). In the proposed model, different from the current models, both crisp values and membership values are used as inputs and also real time series observations are taken as outputs. The proposed model therefore does not require defuzzification transaction and uses single multiplicative neuron model to determine the fuzzy relations and a robust fitness function in its training process. While performing the training process of this model by particle swarm optimization within a combined single optimization process, using crisps values and membership values together provides successful results by getting further information. The various implementations are illustrated to show that the proposed model could obtain more accurate and robust results in forecasting.an example is illustrated to show that the proposed method could obtain more accurate and robust results in forecasting.an example is illustrated to show that the proposed method could obtain more accurate and robust results in forecasting.an example is illustrated to show that the proposed method could obtain more accurate and robust results in forecasting.