Fundamentals of Neural Networks

Abstract

Neural networks (NNs), the parallel distributed processing and connectionist models which we referred to as ANN systems, represent some of the most active research areas in artificial intelligence (AI) and cognitive science today. The main concepts of ANNs are related to human brain. The capabilities of the human brain have always fascinated scientists and led them to investigate its inner workings. Over the past 50 years, a number of models have been developed in an attempt to replicate the brain’s various functions. At the same time, the development of computers was taking a totally different direction. As a result, today’s computer architectures, operating systems, and programming have very little in common with information processing as performed by the brain. Currently, we are experiencing a reevaluation of the brain’s abilities, and models of information processing in the brain have been translated into algorithms and made widely available. The basic building block of these brain models (i.e., ANNs) is an information processing unit that models neuron. An artificial neuron of this kind performs only rather simple mathematical operations. The artificial neuron in performing more complex operation is derived solely from the way in which large numbers of neurons may be connected to form a network. Since the various neural models replicate different abilities of the brain, they can be utilized to solve different types of problems, such as the storage and retrieval of information, the modeling of functional relationships and the representation of large amounts of data. Thus, different kinds of neurons, as functions of the brain model, have been proposed and studied. Promising results for a number of problems such as pattern recognition, category formation, speech production, and addressable memory and optimization (specially in control theory) have been reported [1]–[7]. To model neurons, the major function used in the ANN is a sigmoid-type function (SF).