Modular Neural Network Research Articles

Achieving synchronization and chimera state of modular neural networks by using dynamic learning to adjust electromagnetic induction

Achieving synchronization and chimera state of modular neural networks by using dynamic learning to adjust electromagnetic induction

Emotion Recognition Through Analysis of Speech – A Review

The feature extraction is very important for emotion recognition through speech. There are several approaches when dealing with emotion recognition. In this paper, we present different feature extraction approaches as well as different models used to differentiate between a neutral speech versus an emotional speech sample. This research is instrumental for the digitization and preservation of cultural heritage, as it allows us to capture and analyze the emotional nuances in historical audio recordings, ensuring their accurate representation for future generations. We have selected two works consisting of a total of four different methods for emotion recognition. In the first paper by Jacob (2017), we look at Decision tree and Logistic Regression. Decision tree attains an 84.45% accuracy on the test class whereas logistic regression is able to achieve an accuracy of 66.85% after stepwise regression. These methods contribute to the digital archiving of cultural heritage by providing robust tools for analyzing and preserving the emotional content of spoken artifacts. In another paper by Bhatti et all. (2004), sequential forward selection (SFS) was used to create subsets from the given features and relevance of the subsets of features. General regression neural network was used to evaluate the accuracy which was found to be 80.69%. As a complementary purpose, modular neural network was performed with an accuracy of 83.31% with the same dataset. These techniques enhance our ability to maintain the integrity and emotional depth of cultural heritage recordings in digital archives.

Population bursts in a modular neural network as a mechanism for synchronized activity in KNDy neurons.

The pulsatile activity of gonadotropin-releasing hormone neurons (GnRH neurons) is a key factor in the regulation of reproductive hormones. This pulsatility is orchestrated by a network of neurons that release the neurotransmitters kisspeptin, neurokinin B, and dynorphin (KNDy neurons), and produce episodic bursts of activity driving the GnRH neurons. We show in this computational study that the features of coordinated KNDy neuron activity can be explained by a neural network in which connectivity among neurons is modular. That is, a network structure consisting of clusters of highly-connected neurons with sparse coupling among the clusters. This modular structure, with distinct parameters for intracluster and intercluster coupling, also yields predictions for the differential effects on synchronization of changes in the coupling strength within clusters versus between clusters.

Design of a bi-level PSO based modular neural network for multi-step time series prediction

Design of a bi-level PSO based modular neural network for multi-step time series prediction

Modular stochastic configuration network prediction interval for furnace temperature in municipal solid waste incineration

The high dimensionality of characteristic variables and the presence of numerous uncertain factors affecting furnace temperature during municipal solid waste incineration can lead to poor accuracy and generalization ability for furnace temperature prediction. This paper adopts the modular neural network modeling approach and incorporates Gaussian process regression analysis into stochastic configuration networks to propose a method for establishing a furnace temperature prediction interval model. First, a Gaussian mixture model is used to decompose the complex task into several subtasks. Then, considering the differences among the subtasks, Gaussian process regression with different kernel functions is combined with a stochastic configuration network to form corresponding base models, which are trained and learned. The prediction interval results are obtained through blending ensemble methods. Finally, the effectiveness of the proposed method is tested using historical data obtained from the municipal solid waste incineration process. The results indicate that the furnace temperature prediction model demonstrates advantages in terms of accuracy and generalization ability, making it applicable to the field of process parameter modeling.

The Profile of Network Spontaneous Activity and Functional Organization Interplay in Hierarchically Connected Modular Neural Networks In Vitro.

Modern microtechnology methods are widely used to create neural networks on a chip with a connection architecture demonstrating properties of modularity and hierarchy similar to brain networks. Such in vitro networks serve as a valuable model for studying the interplay of functional architecture within modules, their activity, and the effectiveness of inter-module interaction. In this study, we use a two-chamber microfluidic platform to investigate functional connectivity and global activity in hierarchically connected modular neural networks. We found that the strength of functional connections within the module and the profile of network spontaneous activity determine the effectiveness of inter-modular interaction and integration activity in the network. The direction of intermodular activity propagation configures the different densities of inhibitory synapses in the network. The developed microfluidic platform holds the potential to explore function-structure relationships and efficient information processing in two- or multilayer neural networks, in both healthy and pathological states.

Bridging the gap between machine learning and particle accelerator physics with high-speed, differentiable simulations

Machine learning has emerged as a powerful solution to the modern challenges in accelerator physics. However, the limited availability of beam time, the computational cost of simulations, and the high dimensionality of optimization problems pose significant challenges in generating the required data for training state-of-the-art machine learning models. In this work, we introduce heetah, a yorch-based high-speed differentiable linear beam dynamics code. heetah enables the fast collection of large datasets by reducing computation times by multiple orders of magnitude and facilitates efficient gradient-based optimization for accelerator tuning and system identification. This positions heetah as a user-friendly, readily extensible tool that integrates seamlessly with widely adopted machine learning tools. We showcase the utility of heetah through five examples, including reinforcement learning training, gradient-based beamline tuning, gradient-based system identification, physics-informed Bayesian optimization priors, and modular neural network surrogate modeling of space charge effects. The use of such a high-speed differentiable simulation code will simplify the development of machine learning-based methods for particle accelerators and fast-track their integration into everyday operations of accelerator facilities. Published by the American Physical Society 2024

Multiaxial fatigue life prediction based on modular neural network pretrained with uniaxial fatigue data

PurposeData-driven models are increasingly being used to predict the fatigue life of many engineering components exposed to multiaxial loading. However, owing to their high data requirements, they are cost-prohibitive and underperforming for application scenarios with limited data. Therefore, it is essential to develop an advanced model with good applicability to small-sample problems for multiaxial fatigue life assessment.Design/methodology/approachDrawing inspiration from the modeling strategy of empirical multiaxial fatigue models, a modular neural network-based model is proposed with assembly of three sub-networks in series: the first two sub-networks undergo pretraining using uniaxial fatigue data and are then connected to a third sub-network trained on a few multiaxial fatigue data. Moreover, general material properties and necessary loading parameters are used as inputs in place of explicit damage parameters, ensuring the universality of the proposed model.FindingsBased on extensive experimental evaluations, it is demonstrated that the proposed model outperforms empirical models and conventional data-driven models in terms of prediction accuracy and data demand. It also holds good transferability across various multiaxial loading cases.Originality/valueThe proposed model explores a new avenue to incorporate uniaxial fatigue data into the data-driven modeling of multiaxial fatigue life, which can reduce the data requirement under the promise of maintaining good prediction accuracy.

Cost Analysis and Prediction of Railroad Level Crossing Accidents for Indian Railways

With the tremendous increase in the number of vehicles, the dense traffic created can lead to accidents and fatalities. In a traffic system, the costs for accidents are immeasurable. Numerous studies have been carried out to predict the cost of fatal accidents but have provided the actual values. Therefore, in this study, a monkey-based modular neural system (MbMNS) is developed to identify accident cost. The accident cases and cost data were collected and preprocessed to remove the noise, and the required features were extracted using the spider monkey function. Based on the extracted features, the accidents and the costs were identified. For rail engineering, this will support evaluating the number of railroad crossing accidents with different time intervals. The impact of every accident was also measured with different cost analysis constraints, including insurance, medical, and legal and administrative costs. Therefore, the present study contributes to the field by collecting and organizing the present railroad level crossing accident data from crossing inventory dashboards. Then, the introduction of a novel MbMNS for the cost analysis is the primary contribution of this study to further enrich the railroad level crossing protection system. The third contribution is the tuning of the prediction layer of a modular neural network to the desired level to achieve the highest predictive exactness score. Hence, the designed MbMNS was tested in the Python environment, and the results were validated with regard to recall, accuracy, F-measure, precision, and error values; a comparative analysis was also conducted to confirm the improvement. The novel MbMNS recorded high accuracy of 96.29% for accident and cost analysis, which is better than that reported for other traditional methods.

Modular neural network for Edge-based Detection of early-stage IoT Botnet

The Internet of Things (IoT) has led to rapid growth in smart cities. However, IoT botnet-based attacks against smart city systems are becoming more prevalent. Detection methods for IoT botnet-based attacks have been the subject of extensive research, but the identification of early-stage behaviour of the IoT botnet prior to any attack remains a largely unexplored area that could prevent any attack before it is launched. Few studies have addressed the early stages of IoT botnet detection using monolithic deep learning algorithms that could require more time for training and detection. We, however, propose an edge-based deep learning system for the detection of the early stages of IoT botnets in smart cities. The proposed system, which we call EDIT (Edge-based Detection of early-stage IoT Botnet), aims to detect abnormalities in network communication traffic caused by early-stage IoT botnets based on the modular neural network (MNN) method at multi-access edge computing (MEC) servers. MNN can improve detection accuracy and efficiency by leveraging parallel computing on MEC. According to the findings, EDIT has a lower false-negative rate compared to a monolithic approach and other studies. At the MEC server, EDIT takes as little as 16 ms for the detection of an IoT botnet.

An adaptive evolutionary modular neural network with intermodule connections

An adaptive evolutionary modular neural network with intermodule connections

Survival risk prediction of gastric cardia cancer-based on a dynamic modular neural network

Gastric cardia cancer is a high-incidence malignant tumour, which seriously endangers human health and life safety. The patient prognosis of gastric cardia cancer is affected by diet, physical condition, regional environment, medical history and other factors. Traditional prediction methods cannot fully reflect the prognosis characteristics and survival risks of all patients. Therefore, this paper proposes a data-driven method for the survival risk of cardiac cancer based on an adaptive particle swarm optimization algorithm (APSO) and a dynamic modular neural network (DMNN). First, the article uses density clustering to cluster 293 patients’ blood characteristics and generate different sub-networks. Second, the weight is calculated through the APSO algorithm and the sub-network output is obtained by the integration algorithm. At last, the effectiveness of this network is verified through a 50% cross-validation of training sets and test sets. The results show that the survival prediction based on the APSO-DMNN data-driven method shows good classification performance and accuracy.

Bayesian inference is facilitated by modular neural networks with different time scales.

Various animals, including humans, have been suggested to perform Bayesian inferences to handle noisy, time-varying external information. In performing Bayesian inference by the brain, the prior distribution must be acquired and represented by sampling noisy external inputs. However, the mechanism by which neural activities represent such distributions has not yet been elucidated. Our findings reveal that networks with modular structures, composed of fast and slow modules, are adept at representing this prior distribution, enabling more accurate Bayesian inferences. Specifically, the modular network that consists of a main module connected with input and output layers and a sub-module with slower neural activity connected only with the main module outperformed networks with uniform time scales. Prior information was represented specifically by the slow sub-module, which could integrate observed signals over an appropriate period and represent input means and variances. Accordingly, the neural network could effectively predict the time-varying inputs. Furthermore, by training the time scales of neurons starting from networks with uniform time scales and without modular structure, the above slow-fast modular network structure and the division of roles in which prior knowledge is selectively represented in the slow sub-modules spontaneously emerged. These results explain how the prior distribution for Bayesian inference is represented in the brain, provide insight into the relevance of modular structure with time scale hierarchy to information processing, and elucidate the significance of brain areas with slower time scales.

Directional ΔG Neural Network (DrΔG-Net): A Modular Neural Network Approach to Binding Free Energy Prediction.

The protein-ligand binding free energy is a central quantity in structure-based computational drug discovery efforts. Although popular alchemical methods provide sound statistical means of computing the binding free energy of a large breadth of systems, they are generally too costly to be applied at the same frequency as end point or ligand-based methods. By contrast, these data-driven approaches are typically fast enough to address thousands of systems but with reduced transferability to unseen systems. We introduce DrΔG-Net (or simply Dragnet), an equivariant graph neural network that can blend ligand-based and protein-ligand data-driven approaches. It is based on a 3D fingerprint representation of the ligand alone and in complex with the protein target. Dragnet is a global scoring function to predict the binding affinity of arbitrary protein-ligand complexes, but can be easily tuned via transfer learning to specific systems or end points, performing similarly to common 2D ligand-based approaches in these tasks. Dragnet is evaluated on a total of 28 validation proteins with a set of congeneric ligands derived from the Binding DB and one custom set extracted from the ChEMBL Database. In general, a handful of experimental binding affinities are sufficient to optimize the scoring function for a particular protein and ligand scaffold. When not available, predictions from physics-based methods such as absolute free energy perturbation can be used for the transfer learning tuning of Dragnet. Furthermore, we use our data to illustrate the present limitations of data-driven modeling of binding free energy predictions.

A modular neural network to quickly approximate the modal dispersion in coastal waters

Low-frequency acoustic propagation modeling in coastal waters usually relies on numerical models based on modal theory such as Kraken and Orca. These models compute the modal parameters (e.g., modal wavenumbers and depth functions) that can be used in the calculation of the acoustic field. Their repeated use in broadband applications, or for inversion purposes, comes with a notable computational cost. To mitigate this, a modular neural network (NN) was trained to approximate modal parameters for varying modes and frequencies, across diverse environments, with variable water sound speed profile and variable seabed geoacoustic parameters. The training dataset is generated using Kraken and the NN is evaluated on many environments not seen during training. Once trained, the NN can make broadband predictions without prior knowledge on the number of modes, even when the number of modes changes over the frequency-band of interest. This approach reduces computation time compared to the original forward propagation model, while maintaining high precision. The effectiveness of our method is demonstrated through transmission loss calculations and a simulated geoacoustic inversion scenario.

Automated modeling of nonlinear systems using fuzzy modular neural network

Modular neural networks (MNNs) have garnered substantial attention in the field of nonlinear system modeling. However, even though MNNs require fewer hyperparameters due to their hierarchical structure compared to traditional NNs, determining the optimal module arrangement remains challenging. To address these issues, a novel approach named fuzzy modular neural networks (FMNN) is introduced. This method employs conditional fuzzy clustering and incremental radial basis function (RBF) neural networks to automatically construct sub-modules within the MNN framework. The resultant sub-modules are chosen utilizing a distance-based fuzzy integrative strategy, effectively diminishing the necessity for manual intervention. To showcase the superiority of the FMNN approach, a series of experiments are carried out employing three benchmark examples. These experiments encompass a comparison of modeling accuracy against other extensively employed neural network models. The experimental findings illustrate that FMNN surpasses alternative neural network models in terms of model precision.

Power systems disturbance classification using modular neural networks with multilayers experts

This work evaluates the effectiveness of a new hybrid algorithm in the training of modular neural networks with multilayers specialists. The algorithm is applied on the classification of electric disturbances previously characterized and pre-processed, making possible the comparison between its performance with multilayers perceptrons networks. It presents the algorithm’s development and inclusion of delta-bar-delta rule, such as the data acquisition methodology and its preprocessing.

A neuro-fuzzy modular system for modeling nonlinear systems

The real world is nonlinear and in the control application field, this aspect needs to be resolved to build models so we need to refer to nonlinear system modeling techniques. Neuro-fuzzy systems and modular neural networks (NNs) are among the best modeling approaches for nonlinear systems. The combined features of both approaches provide better models. Thus, we propose in this paper a neuro-fuzzy modular architecture for modeling nonlinear systems. The modular architecture consists of dividing a nonlinear problem into several simpler subproblems. We assigned to each subproblem an NN. Each NN provides individual solutions that will be combined to provide a general solution to the original problem. In this respect, the decomposition of the original problem is based on a fuzzy decision mechanism. This mechanism consists of a set of fuzzy rules for processing nonlinear problems using two different strategies. The first involves training only the network weights, and the second adds the fuzzy set parameters to the training step. A comparative study of both strategies reveals the competence of the second strategy in providing better accuracy and simplicity. Using the neuro-fuzzy combination among the modular NNs reduces the complexity of the original problem and achieves much better performance. The proposed architecture is evaluated by two second-order nonlinear systems, a numerical system and a real system called “the chemical reactor,” which is used to carry out a chemical reaction not only in chemical and biochemical engineering, but also in the petrochemical industry. For both systems, the proposed approach provides better performance in terms of the learning time, learning error, and number of neurons.

Creating Distributed Artificial Neural Networks Based on Orthogonal Transformations

The article addresses the issue of separating input information of artificial neural networks into modules using orthogonal transformations. This separation enables modular organization of neural networks with layer separation, facilitating the use of the proposed approach for distributed computing. Such an approach is required for organizing the operation of neural networks in fog and edge computing environments, as well as for high-performance computing across multiple low-performance computational nodes. The possibility of cross-layer separation of artificial neural networks using orthogonal transformations is theoretically substantiated, and practical examples of such an approach are provided. A comparison of the characteristics of modular neural networks using various types of orthogonal transformations, including the Haar wavelet transform, is conducted.

Fractal Dimension on CBCT Images and Modular Neural Networks to Identify Reduced Bone Mineral Density in Women

This paper provides two different methods to diagnose osteoporosis in women; the first method is the fractal analysis evaluated by CBCT at two bone locations (the mandible and the second cervical vertebrae) to see if there is any correlation between the two. At the same time, the second method is deep convolutional neural networks (DCNNs). One hundred eighty-eight patients' mandibular CBCT images were used, and DCNN models based on the ResNet-101 framework were employed. Dual X-ray absorptiometry of the hip and lumbar spine revealed that 139 of the 188 postmenopausal women tested had osteoporosis, whereas 49 had average bone mineral density. The second cervical vertebra and the mandible were selected as locations of interest for FD analysis on the CBCT images. Measurement accuracy, both within and between observers' agreements, and correlations between two data sets were all calculated. To evaluate osteoporosis, we used a segmented, three-phase approach. Stage 1 was devoted to the identification of mandibular bone slices. In Stage 2, the coordinates for the mandible's cross-sectional views were established, and Stage 3 calculated the thickness of the mandible bone, emphasizing osteoporotic variations. The average FD values within the interest area of the mandible were significantly lower in people with osteoporosis than in those with average bone mineral density. At the same time, the two groups had no significant difference in FD values at the second cervical vertebra. For the mandibular site, areas beneath the curve were 0.644 (P = 0.008), while the area under the curve for the vertebral site was 0.531 (P = 0.720). DCNN training in the first stage yielded an astounding 98.85% training accuracy, the second stage decreased L1 loss to a meager 1.02 pixels, and the bone thickness computation method used in the last stage had a mean squared error of 0. 8377. We concluded that FD was underutilized even though it distinguished between women with normal BMD and those with osteoporosis in the mandibular area. Additionally, even with small mandibular CBCT datasets, the results show the value of a modular transfer learning approach for osteoporosis detection.