Modular Neural Network Research Articles

Hybrid intelligent diagnosis approach based on soft computing from signal and image knowledge representations for a biomedical application

Fault diagnosis is a complex and fuzzy cognitive process, and soft computing methods as neural networks and fuzzy logic, have shown great potential in the development of decision support systems. Dealing with expert (human) knowledge consideration, Computer-Aided Diagnosis (CAD) dilemma is one of the most interesting, but also one of the most difficult problems. Among difficulties contributing to challenging nature of this problem, one can mention the need of fine classification and decision-making. In this paper, a brief survey on fault diagnosis systems is given first. Then, from a fault diagnosis system analysis of the classification and decision-making problem, a global diagnosis synopsis is deduced. Afterwards, a hybrid intelligent diagnosis approach, based on soft computing implying modular neural networks for classification and fuzzy logic for decision-making, is suggested from signal and image representations. The suggested approach is developed in biomedicine for a CAD, from Auditory Brainstem Response test, and the prototype design and experimental results are presented. In fact, a double classification is exploited in a primary fuzzy diagnosis, to ensure a satisfactory reliability. Then, this reliability is reinforced using a confidence parameter with the primary diagnosis result, exploited in a final fuzzy diagnosis giving the appropriate diagnosis with a confidence index. Indeed, experimental results demonstrate the efficiency and reliability of CAD for three classes: two auditory pathologies Retro-cochlear Class (RC) and Endo-cochlear Class (EC), and Normal auditory Class (NC). The generalization rate of NC is clearly higher for primary fuzzy diagnosis and final fuzzy diagnosis than that of the two classifications. The obtained rates for RC and EC are higher than obtained by image classification but quite similar than those obtained by signal classification. An important contribution of the final fuzzy diagnosis is the fact that a confidence index is associated with each fault diagnosis. Finally, a discussion is given with regard to the reliability and large application field of the suggested approach.

Determination of Weights for Multiobjective Decision Making or Machine Learning

Decision-making processes in complex systems generally require the mechanisms to make the tradeoff among contradicting design criteria. When multiple objectives are involved in decision making or machine learning, a crucial step is to determine the weights of individual objectives to the system-level performance. Determining the weights of multiobjectives is an evaluation process, and it has been often treated as an optimization problem. However, our preliminary investigation has shown that existing methodologies in dealing with the weights of multiobjectives have some obvious limitations in the sense that the determination of weights is tackled as a single optimization problem, a result based on such an optimization is incomprehensive, and it can even be unreliable when the information about multiple objectives is incomplete such as an incompleteness caused by poor data. The constraints of weights are also discussed. Variable weights are natural in decision-making processes. Therefore, we are motivated to develop a systematic methodology in determining variable weights of multiobjectives. The roles of weights in an original multiobjective decision-making or machine-learning problem are analyzed, and the weights are determined with the aid of a modular neural network. The inconsistency issue of weights is particularly discussed.

New method for prediction and solving the problem of drilling fluid loss using modular neural network and particle swarm optimization algorithm

Loss circulation is a common problem in drilling industry that causes high expenditure on drilling companies. Nowadays minimizing of loss circulation is a main goal and preference for drilling engineers. Artificial intelligence (Al) is a new method of solving engineering problems that has the ability to consider all effective parameters simultaneously. Moreover, it has generalization and the ability to learn directly from field data. In this paper, two models were designed using Al and data of 38 wells located in Maroun oil field. Both models were developed by modular neural network, to predict loss circulation in quality and quantity. Then, the particle swarm optimization algorithm was used to minimize loss circulation. The accuracy of two models in predicting loss circulation quantitatively and qualitatively is 0.94 and 0.98 %, respectively.

Modular Neural Networks architecture optimization with a new nature inspired method using a fuzzy combination of Particle Swarm Optimization and Genetic Algorithms

We describe in this paper a new hybrid approach for optimization combining Particle Swarm Optimization (PSO) and Genetic Algorithms (GAs) using Fuzzy Logic to integrate the results. The new optimization method combines the advantages of PSO and GA to provide an improved FPSO+FGA hybrid method. Fuzzy Logic is used to combine the results of the PSO and GA in the best way possible. Also Fuzzy Logic is used to adjust parameters in FPSO and FGA. The proposed optimization method was tested with a set of benchmark mathematical functions and then with a more complex problem of neural network architecture optimization. The results of the proposed hybrid optimization method are shown to outperform other methods for these problems.

Deinterlacing algorithm using gradient-regularized modular neural networks

An intrafield deinterlacing algorithm based on gradient-regularized modular neural networks is proposed. The proposed method defines six gradient regularization terms for every missing pixel. Different modular neural networks are selectively used according to the gradient of the pixel to be interpolated. With the statistics of the six gradient regularization terms, a more robust output is generated by modular neural networks. When compared with existing deinterlacing algorithms, the proposed algorithm improves the peak signal-to-noise-ratio while achieving better subjective quality.

Using augmenting modular neural networks to evolve neuro-controllers for a team of underwater vehicles

The paper presents a new generative neuro-evolutionary method called augmenting modular neural networks (AMNN). As the name of the method implies, its purpose is to construct neural networks with a modular architecture. In addition to the modularity itself, neural networks evolving according to AMNN are also characterized by gradually expanding architecture. In the beginning of the evolutionary process, all networks consist of only output modules (or a single module). After some time, if the architecture of all networks is insufficient to effectively perform a task, all of them are augmented by one hidden module. In the following generations, further hidden modules are also added and this procedure is continued until some stopping criterion is satisfied. To test performance of AMNN, the method was used to evolve neuro-controllers for a team of underwater vehicles whose common goal was to capture other vehicle behaving by a deterministic strategy (predator---prey problem). The experiments were carried out in simulation, whereas their results were used to compare AMNN with neuro-evolutionary methods designed for building monolithic neural networks.

A Forest Health Assessment Method Based on ABC-MNN

To better support forest sustainable management, this paper explores the technical framework for forest health assessment, and then focuses on how to better execute the evaluation part of this technical framework. Modular neural networks (MNN) have been shown to be more efficient for classification problems than the conventional monolithic artificial neural network. Therefore, this was used in the evaluation part of this technical framework. Unfortunately, the conventional back propagation (BP) algorithm which is commonly used to train each single artificial neural network (ANN) in MNN has slow convergence speed and the local minimum. To solve this issue, the artificial bee colony (ABC) algorithm is introduced. By combining the robust global searching ability of ABC algorithm with the strong nonlinear mapping and learning ability of BP algorithm, the hybrid ABC-BP algorithm was adopted as the learning algorithm of each single ANN in MNN. In summary, this paper presents a new model of MNN based on the hybrid ABC-BP algorithm (ABC-MNN, for short) to be used for forest health assessment. Its effectiveness is supported and illustrated in the accompanying experimental results.

Prediction of Depth of Sedation from Biological Signals Using Continuous Restricted Boltzmann Machine

Proper anesthesia is very important for patients to get through surgery without pain and then avoid some other problems. By monitoring the depth of sedation for a patient, it could help a clinician to provide a suitable amount of anesthetic and other clinical treatment. In hospital, a patient is usually monitored by different types of biological systems. To predict the depth of sedation from biological signals is able to ease patient monitoring services. In this study, continuous restricted Boltzmann machines based neural network is proposed to predict the depth of sedation. The biological signals including heart rate, blood pressure, peripheral capillary oxygen saturation, and body weight are selected as analytic features. To improve the accuracy, the signals related to the state of anesthesia including fractional anesthetic concentration, end-tidal carbon dioxide, fraction inspiration carbon dioxide, and minimum alveolar concentration are also adopted in this study. Using minimizing contrastive divergence, a continuous restricted Boltzmann machine is trained and then used to predict the depth of sedation. The experimental results showed that the proposed approach outperforms feed-forward neural network and modular neural network. Besides, it would be able to ease patient monitoring services by using biological systems and promote healthcare quality.

An Assessment of a Proposed Hybrid Neural Network for Daily Flow Prediction in Arid Climate

Rainfall-runoff simulation in hydrology using artificial intelligence presents the nonlinear relationships using neural networks. In this study, a hybrid network presented as a feedforward modular neural network (FF-MNN) has been developed to predict the daily rainfall-runoff of the Roodan watershed at the southern part of Iran. This FF-MNN has three layers—input, hidden, and output. The hidden layer has two types of neural expert or module. Hydrometeorological data of the catchment were collected for 21 years. Heuristic method was used to develop the MNN for exploring daily flow generalization. Two training algorithms, namely, backpropagation with momentum and Levenberg-Marquardt, were used. Sigmoid and linear transfer functions were employed to explore the network’s optimum behavior. Cross-validation and predictive uncertainty assessments were carried out to protect overtiring and overparameterization, respectively. Results showed that the FF-MNN could satisfactorily predict stream flow during testing period. The Nash-Sutcliff coefficient, coefficient of determination, and root mean square error obtained using MNN during training and test periods were 0.85, 0.85, and 39.4 and 0.57, 0.58, and 32.2, respectively. The predictive uncertainties for both periods were 0.39 and 0.44, respectively. Generally, the study showed that the FF-MNN can give promising prediction for rainfall-runoff relations.

A modular neural network scheme applied to fault diagnosis in electric power systems.

This work proposes a new method for fault diagnosis in electric power systems based on neural modules. With this method the diagnosis is performed by assigning a neural module for each type of component comprising the electric power system, whether it is a transmission line, bus or transformer. The neural modules for buses and transformers comprise two diagnostic levels which take into consideration the logic states of switches and relays, both internal and back-up, with the exception of the neural module for transmission lines which also has a third diagnostic level which takes into account the oscillograms of fault voltages and currents as well as the frequency spectrums of these oscillograms, in order to verify if the transmission line had in fact been subjected to a fault. One important advantage of the diagnostic system proposed is that its implementation does not require the use of a network configurator for the system; it does not depend on the size of the power network nor does it require retraining of the neural modules if the power network increases in size, making its application possible to only one component, a specific area, or the whole context of the power system.

Investigation of Malting Process Using Artificial Neural Network

Malting is a complex biotechnological process that includes steeping; germination and drying of cereal grains under controlled conditions of temperature and humidity. In this research malting process parameters were predict by modular neural network with different activation function included, logsig-logsig, tanh-tanh, logsigtanh, logsig-identity and tanh-identity. Steeping time (x1) and germination time (x2) were used as input parameters and hot water extract (y1), malting yield (y2) and enzyme activity (β-Gluconase) (y3) were selected as output parameters. The results showed that using perceptron neural network with tanh-identity activation function had the best result among all of activation functions to predict effective parameters of malting process. As well, this network was able to predict hot water extract, malting yield and enzyme activity (β - Gluconase) with R2 value of 1, 0.984 and 0.995, respectively.

Use of Neural Networks in Prediction and Simulation of Steel Surface Roughness

Researches on machined surface roughness prediction in the face milling process of steel are presented in the paper. The data for modelling by the application of neural networks have been collected by the central composite design of experiment. Input variables are the parameters of machining (number of revolutions – cutting speed, feed and depth of cut) and the way of cooling, while the machined surface roughness is output variable. In the modelling process the algorithms Back-Propagation Neural Network, Modular Neural Network and Radial Basis Function Neural Network have been used. Various architectures of neural networks have been investigated on a data sample and they have generated the prediction results which are at the RMS (Root Mean Square) error level of 5.24 % in the learning phase (8.53 % in the validation phase) for the Radial Basis Function Neural Network, 6.02 % in the learning phase (8.87 % in the validation phase) for the Modular Neural Network and for the Back-Propagation Neural Network 6.46 % in the learning phase (7.75 % in the validation phase).

A Novel Topology in Modular Ann Approach for Multi-Modal Concept Identification and Image Retrieval

Concept identification and automated image annotation helps to overcome the non-trivial issue of image retrieval. It is accomplished by decomposing the image at object level and extracting concepts from objects. Several applications like biometrics, geographical information systems, and automatic target recognition possess highly diffused feature vectors. Hence, a conventional Artificial Neural Network (ANN) fails to deliver truthful results. In this work, we have developed a modified back propagation algorithm and have proposed a novel topology in Modular Artificial Neural Network (MANN) to rectify the problem faced in concept identification. In this topology we have trained each module of ANN for one object in one verse all fashion and highly ranked output is taken as classified output. Our argument is supported by combined application of afore-mentioned algorithm and novel topology using MATLAB simulations on two different datasets.

Aircraft recognition using modular extreme learning machine

In this paper, a novel recognition scheme is proposed for identifying the aircrafts of different types based on multiple modular neural network classifiers. Three moment invariants including Hu moments, Zernike moments and Wavelet moments are extracted from the characteristics exhibited by aircrafts and used as the input variables of each modular neural network respectively. Each modular neural network consists of multiple single-hidden layer feedforward networks which are trained using the extreme learning machine and different clustering data subsets. A clustering and selection method is used to get the classification rate of each modular neural network and then based on their weighted sum the final classification output is obtained. The proposed recognition scheme is finally evaluated by recognizing six different types of aircraft models and the simulation results show the superiority of the proposed method compared with the single ELM classifier and other classification algorithms.

Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure

A new model of a modular neural network (MNN) using a granular approach and its optimization with hierarchical genetic algorithms is proposed in this paper. This model can be used in different areas of application, such as human recognition and time series prediction. In this paper, the proposed model is tested for human recognition based on the ear biometric measure. A benchmark database of the ear biometric measure is used to illustrate the advantages of the proposed model over existing approaches in the literature. The proposed method consists in the optimization of the design parameters of a modular neural network, such as number of modules, percentage of data for the training phase, goal error, learning algorithm, number of hidden layers and their respective number of neurons. This method also finds out the amount of and the specific data that can be used for the training phase based on the complexity of the problem.

Network Modularity: Back to the Future in Motor Control

Optogenetic analysis has revealed the existence of multiple rhythm-generating neural networks that drive leg motoneuron pools in the lumbar spinal cord of rodents. These findings extend the concept of a modular neural network organization for locomotion from invertebrates and lower vertebrates to mammals.

Neural networks-based modeling applied to a process of heavy metals removal from wastewaters

This article approaches the problem of environment pollution with heavy metals from disposal of industrial wastewaters, namely removal of these metals by means of biosorbents, particularly with Romanian peat (from Poiana Stampei). The study is carried out by simulation using feed-forward and modular neural networks with one or two hidden layers, pursuing the influence of certain operating parameters (metal nature, sorbent dose, pH, temperature, initial concentration of metal ion, contact time) on the amount of metal ions retained on the unit mass of sorbent. In neural network modeling, a consistent data set was used, including five metals: lead, mercury, cadmium, nickel and cobalt, the quantification of the metal nature being done by its electronegativity. Even if based on successive trials, the method of designing neural models was systematically conducted, recording and comparing the errors obtained with different types of neural networks, having various numbers of hidden layers and neurons, number of training epochs, or using various learning methods. The errors with values under 5% make clear the efficiency of the applied method.

단위 신경망과 특징벡터 차원 축소 기반의 음악 분위기 자동판별

본 논문에서는 개인화된 분위기 분류 모델 대신에 대중의 분위기 분류 모델을 제안한다. 분위기 판별 성능을 개선하기 위해 두 가지 접근 방법을 선택하였는데, 그 첫 번째가 표준편차에 기초한 특징축소이다. 이는 음악의 특징을 추출하기 위해 사용하는 MIRtoolbox에서 추출되는 391개의 특징들을 모두 사용할 경우의 성능 저하 문제를 해결하기 위한 방법이다. 실험결과, 본 논문에서 제안한 특징축소 방법이 기존의 차원 축소 방법인 R-Square와 PCA보다 성능이 좋음을 확인할 수 있었다. 그리고 특징축소 방법만으로는 성능 개선에 한계가 있어 두 번째 개선방법으로 단위 신경망을 사용하여 추가의 성능 개선을 시도하였다. 실험결과 이 역시 유효한 성능 개선이 이루어짐을 확인할 수 있었다. This paper focuses on building a generalized mood classification model with many mood classes instead of a personalized one with few mood classes. Two methods are adopted to improve the performance of mood classification. The one of them is feature reduction based on standard deviation of feature values, which is designed to solve the problem of lowered performance when all 391 features provided by MIR toolbox used to extract features of music. The experiments show that the feature reduction methods suggested in this paper have better performance than that of the conventional dimension reduction methods, R-Square and PCA. As performance improvement by feature reduction only is subject to limit, modular neural network is used as another method to improve the performance. The experiments show that the method also improves performance effectively.

Swarm Optimized Modular Neural Network Based Diagnostic System for Breast Cancer Diagnosis

Artificial Neural Networks have long been considered a simple yet powerful and elegant paradigm for solving problems related to Pattern Recognition, Machine Learning and Knowledge Discovery. However, performance of traditional, monolithic neural network based systems, suffers when faced with complex problems which involve a large number of decision variables or dimensions.Also, performance of any such system depends on the architecture of the neural network involved. The architecture usually remains sub-optimal as human expertise is generally used to design the optimal architecture. In this paper, we describe how the twin paradigms of modularity and swarm intelligence based optimization could be successfully used to overcome these concerns. Here, instead of using a single monolithic expert, we use a modular neural network where several independent neural network experts ind ividually work upon the inputs and give their outputs which is then integrated using an Integrator (here, a Fuzzy C -Means Integrator). Also, swarm intelligence has been used to determine the connections in each individual expert for achieving an optimizedarchitecture for each expert. This approach has been used for the diagnosis of breast cancer disease. Experimental results showthat the proposed approach gives a better diagnostic ability than those of other traditional methods used.

Developmental model of face and object recognition using modular neural network

Developmental model of face and object recognition using modular neural network