Layer Activation Function Research Articles

Neural network based control of PH in a laboratory-scale plant

We analyse the automation of a laboratory-scale pH system. We develop a neural controller based on an internal model control strategy. Feedforward backpropagation neural networks with only one hidden layer and sigmoidal activation functions are used. The training of such networks is based on input-output data of the plant. The results obtained operating the controlled system under different perturbation patterns (changes in set point and in the input water flow rate) show very stable and relatively fast responses within the whole operating range of the system, when compared with classical PID control.

Improving back propagation with a new error function

In a recent study, a modified form of the error-of-performance function for back-propagation (BP) learning in feedforward neural networks (NN) was shown to improve the learning speed. In ( ∂ E ∂ω ) the derivative term f j 1 = (1 − f j)·f rmj of the output layer activation function f j was removed and therefore the updating of the coupling strengths via the BP formula was less suppressed. Nevertheless, the linear dependence on Δ j = t j − z j remained where t j, z j are the target and actual values of the output unit j. In this letter, we show how this unwanted Δ j dependence can also be removed by defining a new error function. A linear combination of the recently studied “cross-entropy cost function,” reminicent of the sum over the quadratic errors, and our new error function, reminicent of the sum over the absolute value of the errors, significantly improves BP learning.

Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks

We give conditions ensuring that multilayer feedforward networks with as few as a single hidden layer and an appropriately smooth hidden layer activation function are capable of arbitrarily accurate approximation to an arbitrary function and its derivatives. In fact, these networks can approximate functions that are not differentiable in the classical sense, but possess only a generalized derivative, as is the case for certain piecewise differentiable functions. The conditions imposed on the hidden layer activation function are relatively mild; the conditions imposed on the domain of the function to be approximated have practical implications. Our approximation results provide a previously missing theoretical justification for the use of multilayer feedforward networks in applications requiring simultaneous approximation of a function and its derivatives.