Modular Neural Network Research Articles

Design of a hierarchy modular neural network and its application in multimodal emotion recognition

Achievement of the fusion for different modalities is a critical issue for multimodal emotion recognition. Feature-level fusion methods cannot deal with missing or corrupted data, while decision-level fusion methods may lose the correlation information between different modalities. To solve the above problems, a hierarchy modular neural network (HMNN) is proposed and is applied for multimodal emotion recognition. First, an HMNN is constructed to mimic the hierarchy modular architecture as demonstrated in the human brain. Each module contains several submodules dealing with features from different modalities. Connections are built between submodules within the same module and between corresponding submodules from different modules. Then, a learning algorithm based on Hebbian learning is used to train the connection weights in HMNN, which simulates the learning mechanism of the human brain. HMNN recognizes the label based on the activity level of each module and adopts the winner-take-all strategy. Finally, the proposed HMNN is applied on a public dataset for multimodal emotion recognition. Experimental results show that the proposed HMNN improves the recognition results, when compared with other decision-fusion methods, including support vector machine, as well as neural networks such as back-propagation and radial basis function neural networks. Furthermore, the inter-submodule connections in one module realizes information integration from different modalities and improves the performance of HMNN. Besides, the experiments suggest the effectiveness of HMNN on dealing with missing/corrupted data.

Estimation of CNC Grinding Process Parameters Using Different Neural Networks

Continuation of research on solving the problem of estimation of CNC grinding process parameters of multi-layer ceramics is presented in the paper. Heuristic analysis of the process was used to define the attributes of influence on the grinding process and the research model was set. For the problem of prediction - estimation of the grinding process parameters the following networks were used in experimental work: Modular Neural Network (MNN), Radial Basis Function Neural Network (RBFNN), General Regression Neural Network (GRNN) and Self-Organizing Map Neural Network (SOMNN). The experimental work, based on real data from the technological process was performed for the purpose of training and testing various architectures and algorithms of neural networks. In the architectures design process different rules of learning and transfer functions and other attributes were used. RMS error was used as a criterion for value evaluation and comparison of the realised neural networks and was compared with previous results obtained by Back-Propagation Neural Network (BPNN). In the validation phase the best results were obtained by Back-Propagation Neural Network (RMSE 12, 43 %), Radial Basis Function Neural Network (RMSE 13, 24 %, ), Self-Organizing Map Neural Network (RMSE 13, 38 %) and Modular Neural Network (RMSE 14, 45 %). General Regression Neural Network (RMSE 21, 78 %) gave the worst results.

Optimization of type-1, interval type-2 and general type-2 fuzzy inference systems using a hierarchical genetic algorithm for modular granular neural networks

In this paper, a new method for fuzzy inference system optimization is presented. The proposed method performs type-1, interval type-2 and general type-2 fuzzy inference systems design using a hierarchical genetic algorithm as optimization method. This method is an improvement of a fuzzy system optimization approach presented in previous works where only the optimization of type-1 and interval type-2 fuzzy inference systems was performed using a human recognition application. The human recognition is performed using three biometric measures namely iris, ear, and voice, where the main idea is to perform the combination of responses in the modular neural networks using an optimized fuzzy inference system to improve the final results without and with noisy conditions. The main contribution of the proposed method is an optimization able to select type of fuzzy logic, granulation of each fuzzy variable and fuzzy rules selection to design optimal fuzzy inference systems applied in combining modular neural networks responses. The results obtained show the effectiveness of the proposed method to design structures of fuzzy inference systems. Statistical comparisons are performed with previous results, where better results can be observed using the proposed method. The design of optimal structures of fuzzy inference systems include among other parameters; type of fuzzy logic (type-1, interval and general type-2 fuzzy logic), type of inference model (Mamdani model or Sugeno model), and consequents of the fuzzy if–then rules.

A classifier of matrix modular neural network to simplify complex classification tasks

This paper proposes the matrix modular neural network (MMNN), which is a modular neural network and adopts a novel task decomposition technique to solve complex problems, such as the large training sets and the category asymmetric training sets. A complex problem can be decomposed into many easier problems, each of which is dealt in two subspaces and can be solved by a single neural network module. All of these modules form a neural network matrix, which produces an output matrix that leads to an integration machine so that finally a classification decision result can be efficiently made. This paper’s theoretic analyses and experiments show that the MMNN can reduce the learning time and improve the generalization capability and the classification accuracy of neural networks.

Modular and deep QoE/QoS mapping for multimedia services over satellite networks

SummarySatellite networks are promising alternatives to deal with the increasing traffic volume and provide universal access for end users. Quality of experience (QoE) as an aggregate of user perception and network quality will be a critical success factor for multimedia service delivery over satellite networks. Quality of experience prediction based on quality of service (QoS) has received much attention recently, but it has been little investigated in satellite networks. In this paper, on the basis of the modular neural network and deep belief networks (DBNs), we design a QoE/QoS mapping method for multimedia services over satellite networks to translate QoS parameters into user QoE. The task of QoE/QoS mapping for multimedia services is decomposed into several different subtasks by using traffic classification. Heterogeneous DBNs are used to learn the mapping relationships between QoE and QoS for different types of services (ie, subtask) simultaneously. An integrative approach based on relative distance is exploited to select subneural network(s) to predict resulting QoE collaboratively. To determine the weight parameters of the QoE/QoS correlation model, the dataset composed of QoS parameters and subjective opinion scores is built on the basis of satellite network simulator and subjective test. Finally, the effectiveness of our QoE/QoS mapping method is validated, and the relationship between adoption rating and opinion scores is analyzed.

An Efficient Fabric Defect Prediction Based on Modular Neural Network Classifier with Alternative Hard C-Means Clustering

In India, textile industry has been mainly focused because it increased the economy day by day. But, it has some problem in the field of quality control. At present, it is mainly solved visually through skilled workers. Though, due to the human errors and eye fatigue, the system reliability has been restricted. So, in this research has been focused automatic fabric defect detection scheme. Here, Modular Neural Network (MNN) is proposed for fabric defect detection and classification with low cost and high accurate rate via using image processing schemes in the woven fabrics. At first, the images are collected from the machine and then preprocessed by using Enhanced Directional Switching Median Filter (EDWF) to reduce the impulse and stationary noise. To attain high accurate prediction, the preprocessed image has been segmented by using Alternative Hard C-Means (AHCM) cluster. After clustering, the images are converted to binary image. Then, the first order features has been extracted from the image. The extracted features are given as input to MNN, which classifies the fabric defects. In MNN, the weight factors are calculated by using back propagation algorithm and generate the output. The simulation results show that the proposed MNN attained high accuracy rate of 96.7% when compared to existing Artificial Neural Network (ANN) than Support Vector Machine with Genetic Algorithm (SVM-GA) classification algorithms.

A Direct Position-Determination Approach for Multiple Sources Based on Neural Network Computation.

The most widely used localization technology is the two-step method that localizes transmitters by measuring one or more specified positioning parameters. Direct position determination (DPD) is a promising technique that directly localizes transmitters from sensor outputs and can offer superior localization performance. However, existing DPD algorithms such as maximum likelihood (ML)-based and multiple signal classification (MUSIC)-based estimations are computationally expensive, making it difficult to satisfy real-time demands. To solve this problem, we propose the use of a modular neural network for multiple-source DPD. In this method, the area of interest is divided into multiple sub-areas. Multilayer perceptron (MLP) neural networks are employed to detect the presence of a source in a sub-area and filter sources in other sub-areas, and radial basis function (RBF) neural networks are utilized for position estimation. Simulation results show that a number of appropriately trained neural networks can be successfully used for DPD. The performance of the proposed MLP-MLP-RBF method is comparable to the performance of the conventional MUSIC-based DPD algorithm for various signal-to-noise ratios and signal power ratios. Furthermore, the MLP-MLP-RBF network is less computationally intensive than the classical DPD algorithm and is therefore an attractive choice for real-time applications.

Modular-Echo State Network Applied on Forecasting of Photovoltaic Power Generation

According to issues of the power system interconnection caused by uncertainty and intermittent of photo-voltaic (PV) power generation, a model based on modular-echo state network (M-ESN) is given to forecast power generation. Firstly, it is established forecast sub-models by modular neural networks (MNN) considering different seasons. Then, dividing each sub-model is based on similar days from history data of photovoltaic power generation with the average temperature as samples to train the model and generated power forecasting by the echo state network (ESN). Finally, integration output forecast numbers are gained. The results show that, compared with the ESN forecast model and BP forecast model, this prediction model has faster forecast speed, higher accuracy and better stability.

A hybrid model based on modular neural networks and fuzzy systems for classification of blood pressure and hypertension risk diagnosis

In this paper, a hybrid model using modular neural networks and fuzzy logic was designed to provide the hypertension risk diagnosis of a person. This model considers age, risk factors and behavior of the blood pressure in a period of 24 h, using as a basis the Framingham Heart Study.Records of blood pressure are collected with the ambulatory blood pressure monitoring (ABPM), a device which takes readings for a period of time of 24 h. A modular neural network was designed, with three modules, of which the first and second modules correspond to the systolic and diastolic pressures and the last one to the heart rate. Each module is trained with the data obtained by the ABPM of different patients, this in order that the neural network learns the different behaviors that the blood pressure may have. Also, different architectures and learning methods are considered to obtain the best possible architecture. In addition, two fuzzy inference systems (FISs) for classification purpose are proposed, the first one for the heart rate level and the second one for the night profile of the patient. These were tested with different types of membership functions and then selecting the FIS that obtained the best results. Furthermore, a third FIS as a blood pressure classifier is also used.The different proposed methodologies were tested, in the case of the modular neural network to find the architecture that produces better results and in the fuzzy inference systems to find which membership functions were the ideal ones for the case study, in this way obtaining overall good results. For the case of the modular neural network, the learning accuracy in the first module is 98%, in the second module is 97.62% and the third module is 97.83% respectively. For the night profile, the fuzzy system is compared to a traditional system of production rules, and it is noted that the first one gives all correct outputs and the second one just gives 53% of the outputs, this is due to the uncertainty handling that fuzzy systems can provide, which the traditional system cannot because its rules are very strict.Hybrid intelligent systems for the solution of this kind of complex problems have excellent performance, due to the good learning in each module of the neural network and the classification uncertainty that is well managed by the fuzzy systems, obtaining with this a hybrid combination for achieving good results.

Methodological Research for Modular Neural Networks Based on “an Expert With Other Capabilities”

This article contains a new subnet training method for modular neural networks, proposed with the inspiration of the principle of “an expert with other capabilities”. The key point of this method is that a subnet learns the neighbor data sets while fulfilling its main task: learning the objective data set. Additionally, a relative distance measure is proposed to replace the absolute distance measure used in the classical subnet learning method and its advantage in the general case is theoretically discussed. Both methodology and empirical study of this new method are presented. Two types of experiments respectively related with the approximation problem and the prediction problem in nonlinear dynamic systems are designed to verify the effectiveness of the proposed method. Compared with the classical subnet learning method, the average testing error of the proposed method is dramatically decreased and more stable. The superiority of the relative distance measure is also corroborated.

Cloud manufacturing service evaluation based on modular neural network

In order to solve the evaluation problem of cloud manufacturing service and then minimise the time of choosing the best service, a comprehensive evaluation index system of cloud manufacturing services is constructed in the light of the characteristics of cloud manufacturing service. The service attributes, manufacturing attributes and network attributes are taken into account from two views of the online characteristics and the offline characteristics of cloud manufacturing service in the evaluation index system. And on this basis, a quantitative evaluation model based on modular neural network (MNN) is proposed. Firstly, the clustering algorithm subtractive-k-means is introduced and used to divide the task into several subtasks based on the idea of modularisation. Then, the subtasks are processed through the trained BP network. Finally, the self-adaptive genetic algorithm is used to optimise the integration weights of each subtask processing module. Combining with the cloud manufacturing evaluation index system, this modular neural network model can be used to effectively evaluate the cloud manufacturing service and help customers quickly select the best cloud manufacturing service.

A Hybrid Approach for Modular Neural Network Design Using Intercriteria Analysis and Intuitionistic Fuzzy Logic

Intercriteria analysis (ICA) is a new method, which is based on the concepts of index matrices and intuitionistic fuzzy sets, aiming at detection of possible correlations between pairs of criteria, expressed as coefficients of the positive and negative consonance between each pair of criteria. Here, the proposed method is applied to study the behavior of one type of neural networks, the modular neural networks (MNN), that combine several simple neural models for simplifying a solution to a complex problem. They are a tool that can be used for object recognition and identification. Usually the inputs of the MNN can be fed with independent data. However, there are certain limits when we may use MNN, and the number of the neurons is one of the major parameters during the implementation of the MNN. On the other hand, a high number of neurons can slow down the learning process, which is not desired. In this paper, we propose a method for removing part of the inputs and, hence, the neurons, which in addition leads to a decrease of the error between the desired goal value and the real value obtained on the output of the MNN. In the research work reported here the authors have applied the ICA method to the data from real datasets with measurements of crude oil probes, glass, and iris plant. The method can also be used to assess the independence of data with good results.

Design of Dynamic Modular Neural Network Based on Adaptive Particle Swarm Optimization Algorithm

To solve the problem that subnetwork output cannot be optimally integrated in a modular neural network (MNN), this paper proposes an adaptive particle swarm optimization algorithm for dynamic MNN (APSO-DMNN). First, the method identifies the distribution of samples and updates the training parameters based on data potential. Second, the MNN activates the corresponding subnetworks according to the input data. Calculate the weights based on an APSO algorithm, which can dynamically optimize the contribution of the output. Then, the inertia weights in the APSO algorithm are adjusted by a nonlinear function in order to avoid being trapped into local optimal values. Finally, the proposed APSO-DMNN can be obtained based on the optimal integration and dynamic adjustment. Comparisons with other algorithms indicate that the proposed method is more effective in modeling and predicting.

A Hybrid Intelligent Model for Enhancing Healthcare Services on Cloud Environment

Cloud computing plays a major role in addressing the challenges of healthcare services such as diagnosis of diseases, telemedicine, maximize utilization of medical resources, etc. Early detection of chronic kidney disease on cloud environment is a big challenge that is facing healthcare providers. This paper concentrates on the using of intelligent techniques such as Decision Tree, Clustering, Linear Regression, Modular Neural Network, and Back Propagation Neural Network to address this challenge. In this paper, the researchers propose a hybrid intelligent model based on cloud computing for early revealing of chronic kidney disease. Two intelligent techniques were used: linear regression and neural network. Linear regression was used to define crucial factors that have an impact on chronic kidney disease. The proposed model for early revealing of chronic kidney disease was built using Neural Network. The accuracy of proposed model is 97.8%. This model outperforms on the other models existed in the previous works in terms of the accuracy and precision, recall and F1 score.

Cloud manufacturing service evaluation based on modular neural network

Cloud manufacturing service evaluation based on modular neural network

Time Series Analysis Based on Modular Architectures of Neural Networks

Paper presents a modular approach for time series analysis area. We consider the most important characteristics of modular architectures of neural networks and their advantages under traditional monolithic neural networks. The main idea of this paper is take answer - why modular neural networks have so high performance in many tasks. Also we present few examples of modular approaches which can be applied for time series analysis problem.

Algorithms of hierarchical mixture of opinions of experts in problems of synthesis of information management systems city development

In the present article we will consider a class of associative machines with dynamic structure where the entrance signal exerts direct impact on the mechanism of association of output signals of experts. At the same time we are interested in such group of expert decisions at which separate expert responses unite not linearly through hierarchically organized lock networks. Hierarchical mixture of opinions of experts, along with simple mixture are examples of modular networks: neural network of a module if the calculations executed by it can be distributed on several subsystems processing different entrance signals and not crossed in the work. Output signals of these subsystems unite the integrative module which exit does not possess feedback with subsystems. In fact, the integrative module makes the decision as output signals of subsystems are grouped in the general output signal of system, and identifies what examples are samples for training of concrete modules. The most general definition of modular neural network: any set of algorithms of data processing, including algorithms of the artificial neural networks grouped for the solution of some uniform task. Automatically determine the class of associative machines with dynamic structure where the entrance signal exerts direct impact on the mechanism of association of output signals of experts, at the same time group of expert decisions at which separate expert responses unite not linearly through hierarchically organized lock networks is considered.

From Classification to Regression Multitasking QSAR Modeling Using a Novel Modular Neural Network: Simultaneous Prediction of Anticonvulsant Activity and Neurotoxicity of Succinimides.

Succinimides, which contain a pharmacophore responsible for anticonvulsant activity, are frequently used antiepileptic drugs and the synthesis of their new derivatives with improved efficacy and tolerability presents an important task. Nowadays, multitarget/tasking methodologies focused on quantitative-structure activity relationships (mt-QSAR/mtk-QSAR) have an important role in the rational design of drugs since they enable simultaneous prediction of several standard measures of biological activities at diverse experimental conditions and against different biological targets. Relating to this very topic, the mt-QSAR/mtk-QSAR methodology can give only binary classification models, and as such, in this study a regression mtk-QSAR (rmtk-QSAR) model based on a novel modular neural network (MNN) has been proposed. The MNN uses standard classification mtk-QSAR models as input modules, while the regression is performed by the output module. The rmtk-QSAR model has been successfully developed for the simultaneous prediction of anticonvulsant activity and neurotoxicity of succinimides, with a satisfactory accuracy in testing (R2 = 0.87). Thus, the proposed mtk-QSAR regression method can be regarded as a viable alternative to the standard QSAR methodology.

The Assimilation of Multi-Type Information for Seasonal Precipitation Forecasting Using Modular Neural Network

The rainfall occurrences are triggered by different types of climate sources not restricted to past precipitation values but may include climate indices such as El Nino/Southern Oscillation, Indian Ocean Dipole, and Madden Julian Oscillation. In this paper, we investigated the effectiveness of assimilating two sources of inputs for heavy precipitation forecasting using modular neural network. The assimilated input was obtained by merging two input variable sources (climate indices and precipitation records) according to their individual weighting factor determined by correlation test. To simulate the hydrologic response using merged product, a modular neural network model was developed. The modular concept was implemented by separating the precipitation events based on seasonal monsoon and trained the subset of seasonal data using modular neural network. Four subsets of monthly precipitation data were sampled to evaluate modular neural network model at 1-month lead-time with single precipitation neural network model and multiple linear regression as benchmark models. The results show that the merging method can effectively assimilate information from two sources of inputs to improve the accuracy of heavy precipitation forecasting.

Artificial neural network based screening of cervical cancer using a hierarchical modular neural network architecture (HMNNA) and novel benchmark uterine cervix cancer database

The work reported in this paper presents a novel hierarchical modular neural network architecture (HMNNA) for automated screening of cervical cancer. HMNNA consists of three neural networks trained specifically on different areas of problem space under consideration, and the trained networks are then arranged in a tree structure forming hierarchical modular neural network architecture. The three specialized neural networks are trained by Levenberg–Maarquardt neural network algorithm. As compared to the standard back propagation algorithm, Levenberg–Maarquardt is fast and stable for convergence with only one drawback, i.e., storage requirement for estimated Hessian Matrix. For training and testing of HMNNA, a huge primary database is created which contains 8091 cervical cell images pertaining to 200 clinical cases collected from two health care institutions of northern India. The raw cases of cervical cancer in the form of Pap smear slides were photographed under a multi-headed digital microscope. Individual cells were manually cropped off from these slide images which were then passed through a feature extraction module for morphological profiling. Each cell was calibrated on the basis of 40 features from both cytoplasm and nucleus. After profiling, these cells were vigilantly assigned cell classes as per the latest 2001-Bethesda system of cervical cancer cell classification, by trained cytotechnicians and histopathologists. HMNNA is also trained and tested on the Herlev Benchmark dataset created by the Denmark University, which consists of 1417 cervical cancer cells. Both the datasets have seven classes of diagnosis, i.e., superficial squamous, intermediate squamous, columnar, mild dysplasia, moderate dysplasia, severe dysplasia, and carcinoma in situ, corresponding to the level of abnormality in cervical cells. These datasets are available in public domain at http://digitalpapsmeardb.in/ and http://mde-lab.aegean.gr/index.php/downloads. The screening potential of the HMNNA is compared with 25 well-known machine learning algorithms available in MatlabR2016 (Machine learning and statistics toolbox 10.2) and monolithic neural network algorithms available in Matlab neural network pattern recognition toolbox. The HMNNA outperformed in all the 25 algorithms for both the datasets. For the Novel Benchmark database, it produced a classification accuracy of 95.32% with an F-value of 0.949310 and classification accuracy of 88.41% with an F-value of 0.89145 for the Herlev dataset. The screening potential of HMNNA was also evaluated and compared with the other diagnostic systems available in the recently published literature and was found to be performing much better than the counterparts on multiple parameters of performance evaluation.