Modular Neural Network Architecture Research Articles

Computer Vision Based on a Modular Neural Network for Automatic Assessment of Physical Therapy Rehabilitation Activities.

Physical rehabilitation techniques during the treatment of clinical pathology are one of the most challenging areas for the medical structure, patients, and families. In large and continental countries, remote monitoring of this treatment is essential. However, equipment and medical follow-up during exercises still have high costs. With the improvement of computer vision and machine learning techniques, some computational, less expensive alternatives have been proposed in the literature. However, monitoring patients during physical rehabilitation exercises with the help of artificial intelligence by a health professional, especially from the capture of visual signals, is still a challenge and poorly explored in the scientific-technological literature. This work aims to propose a new methodology based on computer vision and machine learning for remote tracking of the body joints of patients during physiotherapy rehabilitation exercises. As a new contribution, this work presents a modular neural network architecture composed of two modules: one for detecting physical exercises and another for measuring how much is correct. Another contribution is a strategy for expanding databases, considering that generic databases for this type of exercise are rare on the internet. The results showed that both modules obtained more than 90% of accuracy in recognition and their respective validation.

Neural network approximations for Calabi-Yau metrics

Ricci flat metrics for Calabi-Yau threefolds are not known analytically. In this work, we employ techniques from machine learning to deduce numerical flat metrics for K3, the Fermat quintic, and the Dwork quintic. This investigation employs a simple, modular neural network architecture that is capable of approximating Ricci flat Kähler metrics for Calabi-Yau manifolds of dimensions two and three. We show that measures that assess the Ricci flatness and consistency of the metric decrease after training. This improvement is corroborated by the performance of the trained network on an independent validation set. Finally, we demonstrate the consistency of the learnt metric by showing that it is invariant under the discrete symmetries it is expected to possess.

Measurement results for compensation of impairments of LTE stimulated direct‐conversion transmitter through augmented MoE

This work presents a neural network-based one-step solution for modeling predistorter and mitigating the nonlinear distortion of PA along with impairments, I/Q imbalance, and DC offset of the direct conversion transmitter for wide-band signals. In this work, a modified form of Mixture of experts (MoE), augmented MoE is used as a predistorter to linearize direct conversion transmitter. MoE model encompasses a family of modular neural network architectures having several expert networks connected to a single gating network, and it follows the Divide-and-Conquer principle. The direct-conversion transmitter with class AB-PA was stimulated with a wide-band three carriers Long-Term Evolution (LTE) with different bandwidth signals. The presented method's measurement results show that the proposed model has good linearization performance in the presence of transmitter impairments.

A Committee Machine Neural Network for Dynamic and its Inverse Modeling of Distortions and Impairments in Wireless Transmitters

The tradeoff between bandwidth efficiency and power efficiency with nonlinear distortion introduced by high power amplifier (HPA) in a modern communication system, at a high bit rate has become a prominent problem. Digital predistortion (DPD), the most cost-effective and yet lesser complex method is a widely used linearization method. In this method, the dynamic nonlinear characteristics of the PA are used to obtain an inverse transfer function to abolish the nonlinearity. The development of a good DPD technique depends on its ability to model an accurate PA behavior and its inverse behavior. Recently, several neural network-based predistortion linearizers had been proposed, and these linearizers are effectively proving their worth, but these methods report inadequate generalization performance in the presence of system imperfections like DC offset, IQ imbalance, and both. The present paper proposes a Mixture of Experts (MOE) based committee machine neural network for modeling the PA/transmitter system transfer function and its inverse transfer function under different system conditions. MOE models encompass a family of modular neural network architectures having several experts network and a single gating network, and targeting to divide a complex problem into simple subtasks. An algorithm based on a maximum likelihood criterion is considered for the MOE network training. The results obtained with the proposed model show good generalization performance with nonlinear HPA and with memory effect in the presence of system imperfections.

A Novel Approach for Vehicle Type Classification and Speed Prediction Using Deep Learning

In Vehicles automation system, Classification and speed detection has become an important research challenge in road safety and intelligent transportation system. Many systems like pattern recognition, image processing and machine learning technologies have overcome numerous hindrances to accomplish this goal. In this paper, we demonstrate a speed detection system and vehicle type classification founded on deep learning technique. Moreover, we built up Modular Neural Network (MNN) architecture, advancement algorithm and its parameters are acquired by training dataset. This integrated part of a system will enhance to finding in automation detection and traffic flow management system.

Blood Pressure Classification Using the Method of the Modular Neural Networks.

In this paper, we present a new model based on modular neural networks (MNN) to classify a patient's blood pressure level (systolic and diastolic pressure and pulse). Tests are performed with the Levenberg-Marquardt (trainlm) and scaled conjugate gradient backpropagation (traincsg) training methods. The modular neural network architecture is formed by three modules. In the first module we consider the diastolic pressure data; in the second module we use details of the systolic pressure; in the third module, pulse data is used and the response integration is performed with the average method. The goal is to design the best MNN architecture for achieving an accurate classification. The results of the model show that MNN presents an excellent classification for blood pressure. The contribution of this work is related to helping the cardiologist in providing a good diagnosis and patient treatment and allows the analysis of the behavior of blood pressure in relation to the corresponding diagnosis, in order to prevent heart disease.

Artificial neural network based screening of cervical cancer using a hierarchical modular neural network architecture (HMNNA) and novel benchmark uterine cervix cancer database

The work reported in this paper presents a novel hierarchical modular neural network architecture (HMNNA) for automated screening of cervical cancer. HMNNA consists of three neural networks trained specifically on different areas of problem space under consideration, and the trained networks are then arranged in a tree structure forming hierarchical modular neural network architecture. The three specialized neural networks are trained by Levenberg–Maarquardt neural network algorithm. As compared to the standard back propagation algorithm, Levenberg–Maarquardt is fast and stable for convergence with only one drawback, i.e., storage requirement for estimated Hessian Matrix. For training and testing of HMNNA, a huge primary database is created which contains 8091 cervical cell images pertaining to 200 clinical cases collected from two health care institutions of northern India. The raw cases of cervical cancer in the form of Pap smear slides were photographed under a multi-headed digital microscope. Individual cells were manually cropped off from these slide images which were then passed through a feature extraction module for morphological profiling. Each cell was calibrated on the basis of 40 features from both cytoplasm and nucleus. After profiling, these cells were vigilantly assigned cell classes as per the latest 2001-Bethesda system of cervical cancer cell classification, by trained cytotechnicians and histopathologists. HMNNA is also trained and tested on the Herlev Benchmark dataset created by the Denmark University, which consists of 1417 cervical cancer cells. Both the datasets have seven classes of diagnosis, i.e., superficial squamous, intermediate squamous, columnar, mild dysplasia, moderate dysplasia, severe dysplasia, and carcinoma in situ, corresponding to the level of abnormality in cervical cells. These datasets are available in public domain at http://digitalpapsmeardb.in/ and http://mde-lab.aegean.gr/index.php/downloads. The screening potential of the HMNNA is compared with 25 well-known machine learning algorithms available in MatlabR2016 (Machine learning and statistics toolbox 10.2) and monolithic neural network algorithms available in Matlab neural network pattern recognition toolbox. The HMNNA outperformed in all the 25 algorithms for both the datasets. For the Novel Benchmark database, it produced a classification accuracy of 95.32% with an F-value of 0.949310 and classification accuracy of 88.41% with an F-value of 0.89145 for the Herlev dataset. The screening potential of HMNNA was also evaluated and compared with the other diagnostic systems available in the recently published literature and was found to be performing much better than the counterparts on multiple parameters of performance evaluation.

OS 26-01 CLASSIFICATION OF ARTERIAL HYPERTENSION USING A COMPUTATIONAL MODEL BASED ON ARTIFICIAL MODULAR NEURAL NETWORKS

Objective: The development of an artificial modular neural network (MNN) method for diagnosing and classification of arterial Hypertension based on the level of the blood pressure (BP) of a patient is presented. The main goal is to diagnose the degree of hypertension based on the BP values using MNN applying response integration via a gating network approach. Design and Method: This study was performed with 28 patients to classify the BP levels, based on the European Society of Hypertension (ESH) and the European Society of Cardiology (ESC) Guidelines of Hypertension. We collected patient data with the Ambulatory blood pressure monitoring (ABPM), which is a tool that can help diagnose hypertension. The main goal is to model the 24-hr ABPM patterns in patients with the MNN and classify the BP of the patient using the Levenberg-Marquardt algorithm. The proposed computational method using Artificial MNN consists on designing the best system architecture of the MNN based on previous data. Results: Based on ABPM data of the 28 patients, we implemented a computational system based on three different MNN architectures. The first one achieved a classification rate of 93.3%, the second one of 91.7% and the third one of 89.7% respectively, and with this we can note that excellent classifications results are obtained. The best architecture of the MNN for achieving these results is the following: 25 neurons in the first layer and 30 neurons for the second layer for each of the modules, and the Target Error for learning is 0.002 and 500 epochs are used during learning. Conclusions: We can conclude that with the proposed method using a MNN and the European Guidelines of Hypertension, good results are obtained for diagnosis and classification of hypertension. We can note that MNNs have proven to be a reliable and accurate technique when compared to conventional classification methods.

Modular Neural Networks architecture optimization with a new nature inspired method using a fuzzy combination of Particle Swarm Optimization and Genetic Algorithms

We describe in this paper a new hybrid approach for optimization combining Particle Swarm Optimization (PSO) and Genetic Algorithms (GAs) using Fuzzy Logic to integrate the results. The new optimization method combines the advantages of PSO and GA to provide an improved FPSO+FGA hybrid method. Fuzzy Logic is used to combine the results of the PSO and GA in the best way possible. Also Fuzzy Logic is used to adjust parameters in FPSO and FGA. The proposed optimization method was tested with a set of benchmark mathematical functions and then with a more complex problem of neural network architecture optimization. The results of the proposed hybrid optimization method are shown to outperform other methods for these problems.

RETRACTED: A modular neural network architecture with concept

This paper focuses on the powerful concept of modularity. It is descried how this concept is deployed in natural neural networks on an architectural as well as on a functional level. Furthermore, different approaches for modular neural networks are discussed. By means of these methods, a two-layer modular neural system is introduced. The basic building blocks of the architecture are multilayer perceptions (MP) with the backpropagation (BP) algorithm. This modular network is designed to combine two different approaches of generalization known from connectionist and logical neural networks; this enhances the generalization abilities of the network. Experiments described in this paper show that the architecture is especially useful in solving problems with a large number of input attributes.

Genetic Optimization of Neural Networks for Person Recognition Based on the Iris

This paper describes the application of modular neural network architectures for person recognition using the human iris image as a biometric measure. The iris database was obtained from the Institute of Automation of the Academy of Sciences China (CASIA). We show simulation results with the modular neural network approach, its optimization using genetic algorithms, and the integration with different methods, such as: the gating network method, type-1 fuzzy integration and optimized fuzzy integration using genetic algorithms. Simulation results show a good identification rate using fuzzy integrators and the best structure found by the genetic algorithm.

Genetic Optimization of Neural Networks for Person Recognition based on the Iris

This paper describes the application of modular neural network architectures for person recognition using the human iris image as a biometric measure. The iris database was obtained from the Institute of Automation of the Academy of Sciences China (CASIA). We show simulation results with the modular neural network approach, its optimization using genetic algorithms, and the integration with different methods, such as: the gating network method, type-1 fuzzy integration and optimized fuzzy integration using genetic algorithms. Simulation results show a good identification rate using fuzzy integrators and the best structure found by the genetic algorithm.

Asymptotic Convergence Properties of the EM Algorithm for Mixture of Experts

Mixture of experts (ME) is a modular neural network architecture for supervised classification. The double-loop expectation-maximization (EM) algorithm has been developed for learning the parameters of the ME architecture, and the iteratively reweighted least squares (IRLS) algorithm and the Newton-Raphson algorithm are two popular schemes for learning the parameters in the inner loop or gating network. In this letter, we investigate asymptotic convergence properties of the EM algorithm for ME using either the IRLS or Newton-Raphson approach. With the help of an overlap measure for the ME model, we obtain an upper bound of the asymptotic convergence rate of the EM algorithm in each case. Moreover, we find that for the Newton approach as a specific Newton-Raphson approach to learning the parameters in the inner loop, the upper bound of asymptotic convergence rate of the EM algorithm locally around the true solution Θ* is [Formula: see text], where ϵ>0 is an arbitrarily small number, o(x) means that it is a higher-order infinitesimal as x → 0, and e(Θ*) is a measure of the average overlap of the ME model. That is, as the average overlap of the true ME model with large sample tends to zero, the EM algorithm with the Newton approach to learning the parameters in the inner loop tends to be asymptotically superlinear. Finally, we substantiate our theoretical results by simulation experiments.

Adaptive sensor drift counteraction by a modular neural network

The response properties of sensors such as electronic noses vary in time due to internal or environmental factors. Recalibration is often costly or technically infeasible, which is why algorithms aimed at addressing the sensor drift problem at the data processing level have been developed. These falls in two categories: The pre-processing approaches, such as component correction [1], try to extract the direction and amount of drift in the training data and remove the drift component during operation. Adaptive algorithms, such as the self-organizing map [2], try to counteract the drift during runtime by adjusting the network to the incoming data.We have previously suggested a modular neural network architecture as a model of cortical layer 4 [3]. Here we show how it quite well can handle the sensor drift problem in chemosensor data. It creates a distributed and redundant code suitable for a noisy and drifting environment. A feature extraction layer governed by competitive learning allows for network adaptation during runtime. In addition, training data can be utilized to create a prediction of the underlying drift to further improve the network performance. Hence, we attempt to combine the two aforementioned methodological categories into one network model.The capabilities of the proposed network are demonstrated on surrogate data as well as real-world data collected from an electronic nose.

Differentiation of Two Subtypes of Adult Hydrocephalus by Mixture of Experts

This paper illustrates the use of mixture of experts (ME) network structure to guide model selection for diagnosis of two subtypes of adult hydrocephalus (normal-pressure hydrocephalus-NPH and aqueductal stenosis-AS). The ME is a modular neural network architecture for supervised learning. Expectation-Maximization (EM) algorithm was used for training the ME so that the learning process is decoupled in a manner that fits well with the modular structure. To improve classification accuracy, the outputs of expert networks were combined by a gating network simultaneously trained in order to stochastically select the expert that is performing the best at solving the problem. The classifiers were trained on the defining features of NPH and AS (velocity and flux). Three types of records (normal, NPH and AS) were classified with the accuracy of 95.83% by the ME network structure. The ME network structure achieved accuracy rates which were higher than that of the stand-alone neural network models.

Predicting the dynamic properties of glyben using a modular neural network (MNN)

Glyben is an artificial soil comprised of bentonite and glycerin that is used in scaled physical model testing. The dynamic properties of glyben are strongly influenced by the percent glycerin by mass in the artificial soil, temperature, time after mixing, shear strain amplitude, excitation frequency, and confining stress. This paper describes the development and testing of a modular neural network (MNN) that is suitable for predicting the dynamic properties of glyben. The MNN architecture comprises an input layer, two expert modules (neural networks) linked by a gating network, and an output layer. The MNN is trained using 124 datasets obtained from the literature and tested as part of the current study to evaluate its accuracy. It is shown that the developed MNN can adequately predict the dynamic properties of glyben.

Fingerprint recognition using the fuzzy Sugeno integral for response integration in modular neural networks

We describe in this paper a new approach for pattern recognition using modular neural networks with a fuzzy logic method for response integration. We proposed a new architecture for modular neural networks for achieving pattern recognition in the particular case of human fingerprints. Also, the method for achieving response integration is based on the fuzzy Sugeno integral. Response integration is required to combine the outputs of all the modules in the modular network. We have applied the new approach for fingerprint recognition with a real database of fingerprints from students of our institution.

Implementing wavelet transform/mixture of experts network for analysis of electrocardiogram beats

Abstract: Mixture of experts (ME) is a modular neural network architecture for supervised learning. This paper illustrates the use of the ME network structure to guide model selection for classification of electrocardiogram (ECG) beats. The expectation maximization algorithm is used for training the ME so that the learning process is decoupled in a manner that fits well with the modular structure. The ECG signals were decomposed into time–frequency representations using discrete wavelet transforms and statistical features were calculated to depict their distribution. The ME network structure was implemented for ECG beats classification using the statistical features as inputs. To improve classification accuracy, the outputs of expert networks were combined by a gating network simultaneously trained in order to stochastically select the expert that is performing the best at solving the problem. Five types of ECG beats (normal beat, congestive heart failure beat, ventricular tachyarrhythmia beat, atrial fibrillation beat, partial epilepsy beat) obtained from the Physiobank database were classified with an accuracy of 96.89% by the ME network structure. The ME network structure achieved accuracy rates which were higher than those of the stand‐alone neural network models.

Modular Neural Network Architecture for Precise Condition Monitoring

Unplanned production shutdown due to equipment failure is the source of the highest cost in the manufacturing and process industries. Traditional fault detection methods are able to monitor the process and detect deterioration of the equipment after their degradation and malfunction occurs. This paper presents an intelligent technique based on a neural network (NN) that monitors the health of the equipment and forecasts faults by detecting any onset of failures. In this approach, an adaptive modular NN architecture that is capable of monitoring the health of industrial machines is introduced. This technique is applied to a subsystem of a machining center. The high accuracy of the technique is verified by extensive tests, resulting in over 99% precision.

A modular neural network applied to image transformation and mental images

This paper presents an original modular neural network architecture whose modules are multilayer perceptrons. The modules’ inputs are external inputs or hidden layers of other modules, thereby allowing them to be connected in a general manner. Based on this flexible architecture, networks with high numbers of inputs and outputs can be elaborated and properly trained. A suitable application is image transformation, i.e. the transformation of many input pixels into as many output pixels. Some architectural variations are presented; first the localization over a fraction of the network of a specific transformation’s training, and then the merging of two input images into a single output image. As a case study, we use the modular network to model mental images and their transformations (mental rotation, mental assemblage). It should eventually prove to be a valuable tool for image processing applications, such as super-resolution, or for the elaboration of complex cognitive systems.