Data Mining and Knowledge Discovery, Neural Networks in

Abstract

Activation function The activation or transfer function transforms the weighted inputs of a neuron into an output signal. Activation functions often have a “squashing” effect. Common activation functions used in neural networks are threshold, linear, sigmoid, hyperbolic, and Gaussian. Artificial neural network An artificial neural network is a system composed of many simple, but highly interconnected processing nodes (named neurons) which operate in parallel and collectively. It resembles biological nervous systems in two basic functions: (1) Experiential knowledge is acquired through a learning process and can be retrieved again later. (2) The knowledge is stored in the strength (weights) of the connections between the neurons. Artificial neuron An artificial neuron receives a number of inputs, which may be either external inputs to the neural network or outputs of other neurons. Each input connection is assigned a weight, similar to the synaptic efficacy of a biological neuron. The weighted sum of inputs is compared against an activation level (threshold) to determine the activation value of the neuron. Feedback networks In feedback or recurrent networks, signals may flow in both directions. Feedback networks are dynamic such that they have a state that is changing continuously until it reaches an equilibrium point. Learning rule The learning rule describes the way a neural network is trained, i.e., how its free parameters undergo changes to fit the network to the training data. Feedforward network Feedforward neural networks are organized in one or more layers of processing units. In a feedforward neural network, the signal is allowed to flow one way only, i.e., from inputs to outputs. There are no feedback loops, i.e., the outputs of a layer do not affect its inputs.