Activation Function In Output Layer Research Articles

Development of a coffee wilt disease identification model using deep learning

Coffee, commonly known as Coffee Arabica, is Ethiopia's most significant beverage. Coffee production in Ethiopia is affected by a number of abiotic and biotic factors, the most prominent of which are diseases caused by a range of etiologic agents, especially fungus. A multitude of diseases harm the stems, leaves, fruits, and roots of the crop, lowering its yield and marketability. Ethiopia was the first place where the Fusarium wilt disease on Coffee Arabica was discovered. In this study, a deep learning method for the automatic detection of coffee wilt disease is proposed. The investigation was carried out in three stages. First, we collected images of healthy and diseased coffee. Secondly, we developed a convolutional neural network (CNN) that can tell the difference between healthy images and images of infected/diseased coffee leaves. Finally, we tuned the dataset to train and test the developed model. For the purpose of experimentation, we collected 4000 images of healthy and infected coffee leaves. 80% of the obtained dataset was utilized to train the model, while the remaining 20% is randomly picked to test the CNN model. The suggested model efficiently grouped the input image with a mean training accuracy of 98.1% and a mean test accuracy of 97.9% as a result of experimentation using a learning rate of 0.0001, a Sigmoid output layer activation function, 100 epochs, and an 8:2 training and testing dataset ratio.

DETECTION OF ATTACKS ON A COMPUTER NETWORK BASED ON THE USE OF NEURAL NETWORKS COMPLEX

Purpose. The article is aimed at the development of a methodology for detecting attacks on a computer network. To achieve this goal the following tasks were solved: to develop a methodology for detecting attacks on a computer network based on an ensemble of neural networks using normalized data from the open KDD Cup 99 database; when performing machine training to identify the optimal parameters of the neural network which will provide a sufficiently high level of reliability of detection of intrusions into the computer network. Methodology. As an architectural solution of the attack detection module, a two-level network system is proposed, based on an ensemble of five neural networks of the multilayer perceptron type. The first neural network to determine the category of attack class (DoS, R2L, U2R, Probe) or the fact that there was no attack; other neural networks – to detect the type of attack, if any (each of these four neural networks corresponds to one class of attack and is able to identify types that belong only to this class). Findings. The created software model was used to study the parameters of the neural network configuration 41–1–132–5, which determines the category of the attack class on the computer network. It is determined that the optimal training speed is 0.001. The ADAM algorithm proved to be the best for optimization. The ReLU function is the most suitable activation function for the hidden layer, and the hyperbolic tangent function – for the output layer activation function. Accuracy in test and validation samples was 92.86 % and 91.03 %, respectively. Originality. The developed software model, which uses the Python 3.5 programming language, the integrated development environment PyCharm 2016.3 and the Tensorflow 1.2 framework, makes it possible to detect all types of attacks of DoS, U2R, R2L, Probe classes. Practical value. Graphical dependencies of accuracy of neural networks at various parameters are received: speed of training; activation function; optimization algorithm. The optimal parameters of neural networks have been determined, which will ensure a sufficiently high level of reliability of intrusion detection into a computer network.

Activation functions selection for BP neural network model of ground surface roughness

Roughness prediction of ground surfaces is critical in understanding and optimizing the grinding process. However, it is hitherto difficult to predict accurately the ground surface roughness by theoretical and empirical models due to the complexity of grinding process. BP neural network (BPNN), which can be used to establish the relationship between processing parameters and surface roughness, avoids the difficulty of revealing the complex physical mechanism and thus has unique potential in automatic optimization of grinding process in industrial practice. Activation function is one of the most important factors affecting the efficiency and accuracy of BPNN. Nevertheless, it is often selected arbitrarily or at most by trials or tuning. This paper proposes an activation function selection approach in which virtual data generated from the approximate physical model are employed to evaluate the performance of the BPNN in practice application. The results show that with tansig as the activation function of hidden layer and purelin as the activation function of output layer, the BPNN model can obtain the highest learning efficiency. Moreover, when the activation function of hidden layer is sigmoid, whose shape factor is 1–3, and the output layer activation function is purelin, the model can predict more precisely. Finally, the proposed approach is validated by comparing the performance of BPNN obtained from the virtual data and the experimental data. Obtained results showed that the proposed approach is a simple and effective way to determine the activation function of BPNN.

Artificial neural network models predicting the leaf area index: a case study in pure even-aged Crimean pine forests from Turkey

BackgroundLeaf Area Index (LAI) is an important parameter used in monitoring and modeling of forest ecosystems. The aim of this study was to evaluate performance of the artificial neural network (ANN) models to predict the LAI by comparing the regression analysis models as the classical method in these pure and even-aged Crimean pine forest stands.MethodsOne hundred eight temporary sample plots were collected from Crimean pine forest stands to estimate stand parameters. Each sample plot was imaged with hemispherical photographs to detect the LAI. The partial correlation analysis was used to assess the relationships between the stand LAI values and stand parameters, and the multivariate linear regression analysis was used to predict the LAI from stand parameters. Different artificial neural network models comprising different number of neuron and transfer functions were trained and used to predict the LAI of forest stands.ResultsThe correlation coefficients between LAI and stand parameters (stand number of trees, basal area, the quadratic mean diameter, stand density and stand age) were significant at the level of 0.01. The stand age, number of trees, site index, and basal area were independent parameters in the most successful regression model predicted LAI values using stand parameters (R2adj. = 0.5431). As corresponding method to predict the interactions between the stand LAI values and stand parameters, the neural network architecture based on the RBF 4–19-1 with Gaussian activation function in hidden layer and the identity activation function in output layer performed better in predicting LAI (SSE (12.1040), MSE (0.1223), RMSE (0.3497), AIC (0.1040), BIC (− 77.7310) and R2 (0.6392)) compared to the other studied techniques.ConclusionThe ANN outperformed the multivariate regression techniques in predicting LAI from stand parameters. The ANN models, developed in this study, may aid in making forest management planning in study forest stands.

Using artificial neural network for predicting and controlling the effluent chemical oxygen demand in wastewater treatment plant

PurposeModeling Wastewater Treatment Plant (WWTP) constitutes an important tool for controlling the operation of the process and for predicting its performance with substantial influent fluctuations. The purpose of this paper is to apply an artificial neural network (ANN) approach with a feed-forward back-propagation in order to predict the ten-month performance of Touggourt WWTP in terms of effluent Chemical Oxygen Demand (CODeff).Design/methodology/approachThe influent variables such as (pHinf), temperature (TEinf), suspended solid (SSinf), Kjeldahl Nitrogen (KNinf), biochemical oxygen demand (BODinf) and chemical oxygen demand (CODinf) were used as input variables of neural networks. To determine the appropriate architecture of the neural network models, several steps of training were conducted, namely the validation and testing of the models by varying the number of neurons and activation functions in the hidden layer, the activation function in output layer as well as the learning algorithms.FindingsThe better results were achieved with an architecture network [6-50-1], hyperbolic tangent sigmoid activation functions at hidden layer, linear activation functions at output layer and a Levenberg – Marquardt method as learning algorithm. The results showed that the ANN model could predict the experimental results with high correlation coefficient 0.89, 0.96 and 0.87 during learning, validation and testing phases, respectively. The overall results indicated that the ANN modeling approach can provide an effective tool for simulating, controlling and predicting the performance of WWTP.Originality/valueThis work is the first of its kind in this region due to the remarkable development in terms of population and agricultural activity in the region, which drove to the increase of water pollutants, so it is necessary to use the modern technologies to modeling and controlling of WWTP.

Joint detection, channel estimation, and interference cancellation in downlink MC-CDMA communication systems using complex-valued multilayer neural networks

Data detection in the presence of interference is one of the main challenges in multicarrier code division multiple access (MC-CDMA) communication systems. In this paper, a new detection technique for downlink MC-CDMA systems is proposed. This technique uses complex-valued multilayer neural networks at the receiver side. With the new definition for desired responses (±(1+J) instead of ±1, where $$ J = \sqrt {{ - 1}} $$ ), the convergence rate is increased (in the training process) regarding to saturation of imaginary output and the performance is increased because of increasing Euclidean distance of output neuron inputs in two states of desired outputs (with factor of $$ \sqrt {2} $$ ). The performance of the proposed method is improved further by considering two various saturation coefficients (in the activation function of output layer) in the training and test processes. Since the last performance improving lead to low convergence rate, this effect is compensated by correcting the coefficient of training rate in the output layer. Simulation results confirm the high convergence rate, low computational complexity, and also good performance of the proposed method in wide range of SNRs.

Improving back propagation with a new error function

In a recent study, a modified form of the error-of-performance function for back-propagation (BP) learning in feedforward neural networks (NN) was shown to improve the learning speed. In ( ∂ E ∂ω ) the derivative term f j 1 = (1 − f j)·f rmj of the output layer activation function f j was removed and therefore the updating of the coupling strengths via the BP formula was less suppressed. Nevertheless, the linear dependence on Δ j = t j − z j remained where t j, z j are the target and actual values of the output unit j. In this letter, we show how this unwanted Δ j dependence can also be removed by defining a new error function. A linear combination of the recently studied “cross-entropy cost function,” reminicent of the sum over the quadratic errors, and our new error function, reminicent of the sum over the absolute value of the errors, significantly improves BP learning.