Multiple Neural Networks Research Articles

Multi-level physics informed deep learning for solving partial differential equations in computational structural mechanics

Physics-informed neural network has emerged as a promising approach for solving partial differential equations. However, it is still a challenge for the computation of structural mechanics problems since it involves solving higher-order partial differential equations as the governing equations are fourth-order nonlinear equations. Here we develop a multi-level physics-informed neural network framework where an aggregation model is developed by combining multiple neural networks, with each one involving only first-order or second-order partial differential equations representing different physics information such as geometrical, constitutive, and equilibrium relations of the structure. The proposed framework demonstrates a remarkable advancement over the classical neural networks in terms of the accuracy and computation time. The proposed method holds the potential to become a promising paradigm for structural mechanics computation and facilitate the intelligent computation of digital twin systems.

Multiple Neural Networks Originating from the Lateral Parabrachial Nucleus Modulate Cough-like Behavior and Coordinate Cough with Pain.

It has been reported that experimental pain could diminish cough sensitivity, the lateral parabrachial nucleus (LPBN) coordinated pain with breathing, whether LPBN regulates cough-like behaviors and pain-induced changes in cough sensitivity remains elusive. We investigated the roles of LPBN γ-aminobutyric acidergic (GABAergic) and glutamatergic neurons in the regulation of cough sensitivity and its relationship with pain in mice via chemogenetic approaches. Adeno-associated virus (AAV) tracing combined with chemogenetics was used to map the projections of LPBN GABAergic and glutamatergic neurons to the periaqueductal gray (PAG). LPBN neurons were activated by cough challenge, and nonspecific inhibition of LPBN neurons suppressed cough-like behavior. Chemogenetic suppression of LPBN GABAergic neurons reduced cough sensitivity in mice, whereas suppression of LPBN glutamatergic neurons counteracted the pain-driven decrease in cough sensitivity, so did silencing LPBN glutamatergic neurons projecting to the PAG. Our data suggest that GABAergic and glutamatergic neurons in the LPBN critically are involved in cough sensitivity and coordinate pain with cough through inhibitory or activating mechanisms at the midbrain level.

BENN: Balanced Ensemble Neural Network for Handling Class Imbalance in Big Data

ABSTRACTClass imbalance is a critical challenge in big data analytics, often leading to biased predictive models. This imbalance can lead to biased models that perform well on the majority class but poorly on the minority class. Many machine learning models tend to be biased towards the majority class because they aim to minimise overall error, often leading to poor performance on the minority class. This paper presents the balanced ensemble neural network, a novel solution to effectively address class imbalance in big data. Balanced ensemble neural network combines the robust capabilities of neural networks with the power of ensemble learning, incorporating class balancing strategies to ensure fair representation of minority classes. The methodology involves integrating multiple neural networks, each trained on balanced subsets of data using techniques like Synthetic Minority Over‐sampling Technique and Random Undersampling. This integration aims to leverage the strengths of individual networks while reducing their inherent biases. Our extensive experiments across various datasets reveal that BENN achieves an AUC‐ROC score of 0.94, surpassing other models such as random forest (0.88), support vector (0.84) and single neural net (0.80). It was also observed that BENN's performance is better compared to traditional neural network models and standard ensemble methods in key metrics like accuracy, precision, recall, F1‐score and AUC‐ROC. The results specifically highlight BENN's effectiveness in accurately classifying instances of minority classes, a notable challenge in many existing models. These findings underscore BENN's potential as a substantial advancement in handling class imbalance within big data environments, offering a promising direction for future research and application in machine learning.

Development of an evaluation method for addictive compounds based on electrical activity of human iPS cell-derived dopaminergic neurons using microelectrode array.

Addiction is known to occur through the consumption of substances such as pharmaceuticals, illicit drugs, food, alcohol and tobacco. These addictions can be viewed as drug addiction, resulting from the ingestion of chemical substances contained in them. Multiple neural networks, including the reward system, anti-reward/stress system and central immune system in the brain, are believed to be involved in the onset of drug addiction. Although various compound evaluations using microelectrode array (MEA) as an in vitro testing methods to evaluate neural activities have been conducted, methods for assessing addiction have not been established. In this study, we aimed to develop an in vitro method for assessing the addiction of compounds, as an alternative to animal experiments, using human iPS cell-derived dopaminergic neurons with MEA measurements. MEA data before and after chronic exposure revealed specific changes in addictive compounds compared to non-addictive compounds, demonstrating the ability to estimate addiction of compound. Additionally, conducting gene expression analysis on cultured samples after the tests revealed changes in the expression levels of various receptors (nicotine, dopamine and GABA) due to chronic administration of addictive compounds, suggesting the potential interpretation of these expression changes as addiction-like responses in MEA measurements. The addiction assessment method using MEA measurements in human iPS cell-derived dopaminergic neurons conducted in this study proves effective in evaluating addiction of compounds on human neural networks.

Identification of perceived sentences using deep neural networks in EEG

Objetive. Decoding speech from brain activity can enable communication for individuals with speech disorders. Deep neural networks (DNNs) have shown great potential for speech decoding applications. However, the limited availability of large datasets containing neural recordings from speech-impaired subjects poses a challenge. Leveraging data from healthy participants can mitigate this limitation and expedite the development of speech neuroprostheses while minimizing the need for patient-specific training data.Approach. In this study, we collected a substantial dataset consisting of recordings from 56 healthy participants using 64 EEG channels. Multiple neural networks were trained to classify perceived sentences in the Spanish language using subject-independent, mixed-subjects, and fine-tuning approaches. The dataset has been made publicly available to foster further research in this area.Main results. Our results demonstrate a remarkable level of accuracy in distinguishing sentence identity across 30 classes, showcasing the feasibility of training DNNs to decode sentence identity from perceived speech using EEG. Notably, the subject-independent approach rendered accuracy comparable to the mixed-subjects approach, although with higher variability among subjects. Additionally, our fine-tuning approach yielded even higher accuracy, indicating an improved capability to adapt to individual subject characteristics, which enhances performance. This suggests that DNNs have effectively learned to decode universal features of brain activity across individuals while also being adaptable to specific participant data. Furthermore, our analyses indicate that EEGNet and DeepConvNet exhibit comparable performance, outperforming ShallowConvNet for sentence identity decoding. Finally, our Grad-CAM visualization analysis identifies key areas influencing the network's predictions, offering valuable insights into the neural processes underlying language perception and comprehension.Significance. These findings advance our understanding of EEG-based speech perception decoding and hold promise for the development of speech neuroprostheses, particularly in scenarios where subjects cannot provide their own training data.

A Probabilistic Descent Ensemble for Malware Prediction Using Deep Learning

INTRODUCTION: Introducing a Probabilistic Descent Ensemble (PDE) approach for enhancing malware prediction through deep learning leverages the power of multiple neural network models with distinct architectures and training strategies to achieve superior accuracy while minimizing false positives. OBJECTIVES: Combining Stochastic Gradient Descent (SGD) with early stopping is a potent approach to optimising deep learning model training. Early stopping, a vital component, monitors a validation metric and halts training if it stops improving or degrades, guarding against overfitting. METHODS: This synergy between SGD and early stopping creates a dynamic framework for achieving optimal model performance adaptable to diverse tasks and datasets, with potential benefits including reduced training time and enhanced generalization capabilities. RESULTS: The proposed work involves training a Gaussian NB classifier with SGD as the optimization algorithm. Gaussian NB is a probabilistic classifier that assumes the features follow a Gaussian (normal) distribution. SGD is an optimization algorithm that iteratively updates model parameters to minimize a loss function. CONCLUSION: The proposed work gives an accuracy of 99% in malware prediction and is free from overfitting and local minima.

Physics-informed neural network for simulating magnetic field of permanent magnet

Abstract With the rapid development of deep learning, its application in physical field simulation has been widely concerned, and it has begun to lead a new model of meshless simulation. In this paper, research based on physics-informed neural networks is carried out to solve partial differential equations related to the physical laws of electromagnetism. Then the magnetic field simulation is realized. In this method, the governing equation and the boundary conditions containing physical information are embedded into the neural network loss function as constraints, and the backpropagation of neural networks is realized based on automatic differentiation to solve partial differential equations. The high-precision simulation of tile-shaped and rectangular permanent magnet magnetic fields of permanent magnet motors based on physical information neural network is studied, and the error is within 5%. We consider the simulation of magnetic field in two coordinate systems, and realize the joint training of multiple neural networks in multiple sub-domains and different media.

State Estimation of Radial Distribution Systems Based on Multiple Legendre Neural Networks

The conventional weighted least square (WLS) method is the most effective technique used in the state estimation of high voltage transmission system. Unfortunately, the application of WLS in radial distribution network encounter difficulties due to the inherent characteristics of these systems, such as the low measurement redundancy and high r/x ratio of the distribution systems. Given the structure of bulky systems that require a bulky number of measurements, the use of artificial neural networks is considered an effective alternative to estimate these values using a lesser number of measurements than conventional techniques. Due to state estimation based on ANN technique, the time-consuming gain matrix manipulation and pseudo measurements required in the conventional WLS method are no longer necessary. The efficiency of deep learning neural networks such as multi-layer perceptron (MLP) and Legendre neural network (LeNN) depends on the position of the measurements and the number of neural networks applied. Determining the applicable number of neural networks to ensure high estimation accuracy plays an important role in the estimation process. This aspect is addressed in this study, where multiple neural networks are used to improve performance compared to a single neural network. The results obtained indicate that determining the applicable number of neural networks depends on several factors such as the position of the measurements and the diversity of the data. The application of LeNN on state estimation of a 69-bus radial distribution network is used as an illustrative example to explain the distinctive feature of the proposed technique.

An Agent-Based Method for Feature Recognition and Path Optimization of Computer Numerical Control Machining Trajectories.

In recent years, artificial intelligence technology has seen increasingly widespread application in the field of intelligent manufacturing, particularly with deep learning offering novel methods for recognizing geometric shapes with specific features. In traditional CNC machining, computer-aided manufacturing (CAM) typically generates G-code for specific machine tools based on existing models. However, the tool paths for most CNC machines consist of a series of collinear motion commands (G01), which often result in discontinuities in the curvature of adjacent tool paths, leading to machining defects. To address these issues, this paper proposes a method for CNC system machining trajectory feature recognition and path optimization based on intelligent agents. This method employs intelligent agents to construct models and analyze the key geometric information in the G-code generated during CNC machining, and it uses the MCRL deep learning model incorporating linear attention mechanisms and multiple neural networks for recognition and classification. Path optimization is then carried out using mean filtering, Bézier curve fitting, and an improved novel adaptive coati optimization algorithm (NACOA) according to the degree of unsmoothness of the path. The effectiveness of the proposed method is validated through the optimization of process files for gear models, pentagram bosses, and maple leaf models. The research results indicate that the CNC system machining trajectory feature recognition and path optimization method based on intelligent agents can significantly enhance the smoothness of CNC machining paths and reduce machining defects, offering substantial application value.

Machine vision-based detection of forbidden elements in the high-speed automatic scrap sorting line

Highly efficient industrial sorting lines require fast and reliable classification methods. Various types of sensors are used to measure the features of an object to determine which output class it belongs to. One technique involves the use of an RGB camera and a machine learning classifier. The paper is focused on protecting the sorting process against prohibited and dangerous items potentially present in the sorted material that pose a threat to the sorting process or the subsequent metallurgical process. To achieve this, a convolutional neural network classifier was applied under real-life conditions to detect forbidden elements in copper-based metal scrap. A laboratory stand simulating the working conditions in a high-speed scrap sorting line was prepared. Using this custom stand, training and test sets for machine learning were gathered and labeled. An image preprocessing algorithm was designed to increase the robustness of the resulting forbidden element detector system. The performance of multiple neural network architectures and data set augmentations was analyzed. The highest accuracy of 98.03% and F1-score of 97.16% were achieved with a DenseNet-based classifier. The results of this paper show the feasibility of using the presented solution on a high-speed industrial line.

An explainable AI for green hydrogen production: A deep learning regression model

Currently, hydrogen generation is considered a crucial aspect of sustainable energy production. This paper disscusses the hypothesis that hydrogen generation occurs during the operation of the membrane electrode assembly and is an acceptable and eco-friendly method for producing hydrogen without carbon dioxide. This paper considers the hypothesis by simulating hydrogen generation in fuel cells that rely on fossil fuels using proton fuel cells. The Optimized Hydrogen Generation-based Regression (OHGR) model is based on a deep learning architecture. The architecture is based on stacking multiple neural networks with crossover connections that feed information through a pool of activation functions. The OHGR predicts the amount of hydrogen generated in 1 mA cm−2 and can exceed 10–20 mA cm−2 after long-term operation. For given temperatures, pressures, and humidity, which are considered fundamental factors in the hydrogen crossing phenomenon.The OHGR uses a stochastic gradient as an optimization engine that informs the most-recommended values for temperature, pressure, and humidity to generate the optimal amount of hydrogen. In addition, the OHGR model is interoperated to accept the same number of most significant features obtained from the principal component analysis. The OHGR was evaluated using empirical regression evaluation metrics, including the root mean square error, R2, Mean Square Error, and Mean Absolute Error. The optimization process was designed to include the hypothesis and hyperparameters to determine the most significant values for the OGHR and its outcomes. The introduced OHGR model is sufficiently efficient to predict the generated hydrogen with RMSE = 0.220 and R2 = 0.564, indicating an enhancement of the OHGR model compared to recently reported models. Since the sustainability of hydrogen generation is important for energy availability and reducing climate change, the OHGR model is justified using the SHapley Additive exPlanations (SHAP) method to ensure that the model is transparent, reliable, and trustworthy.

Research on autonomous decision-making of UAV swarm based on neural network algorithm

This article focuses on typical unmanned aerial vehicle (UAV) cluster autonomous collaborative reconnaissance mission scenarios, constructs UAV classes in Matlab, and effectively constructs UAV autonomous decision-making models using two different algorithms; Using Extreme Learning Machines (ELM) and introducing neural network methods to solve autonomous decision-making problems for unmanned aerial vehicles. Test the performance of the model under multiple neural network algorithms, basically achieving unmanned aerial vehicles to complete flight targets based on multiple sensor parameters in unknown environments, using reinforcement learning algorithms to achieve autonomous decision-making of unmanned aerial vehicles, achieving multi parameter fusion (with parameters greater than 4) decision-making, with a decision-making time of less than 100ms, which can ensure timely decision-making while also considering the rationality of the model.

Transcriptionally Conditional Recurrent Neural Network for De Novo Drug Design.

Computational molecular generation methods that generate chemical structures from gene expression profiles have been actively developed for de novo drug design. However, most omics-based methods involve complex models consisting of multiple neural networks, which require pretraining. In this study, we propose a straightforward molecular generation method called GxRNN (gene expression profile-based recurrent neural network), employing a single recurrent neural network (RNN) that necessitates no pretraining for omics-based drug design. Specifically, our method utilizes the desired gene expression profile as input for the RNN, conditioning it to generate molecules likely to induce a similar profile. In a case study involving ten target proteins, GxRNN exhibited superior structural reproducibility of known ligands, surpassing several existing methods. This advancement positions our proposed method as a promising tool for facilitating de novo drug design.

Multi-Source Information Graph Embedding with Ensemble Learning for Link Prediction

Link prediction is a key technique for connecting entities and relationships in a graph reasoning field. It leverages known information about the graph structure data to predict missing factual information. Previous studies have either focused on the semantic representation of a single triplet or on the graph structure data built on triples. The former ignores the association between different triples, and the latter ignores the true meaning of the node itself. Furthermore, common graph-structured datasets inherently face challenges, such as missing information and incompleteness. In light of this challenge, we present a novel model called Multi-source Information Graph Embedding with Ensemble Learning for Link Prediction (EMGE), which can effectively improve the reasoning of link prediction. Ensemble learning is systematically applied throughout the model training process. At the data level, this approach enhances entity embeddings by integrating structured graph information and unstructured textual data as multi-source information inputs. The fusion of these inputs is effectively addressed by introducing an attention mechanism. During the training phase, the principle of ensemble learning is employed to extract semantic features from multiple neural network models, facilitating the interaction of enriched information. To ensure effective model learning, a novel loss function based on contrastive learning is devised, effectively minimizing the discrepancy between predicted values and the ground truth. Moreover, to enhance the semantic representation of graph nodes in link prediction, two rules are introduced during the aggregation of graph structure information. These rules incorporate the concept of spreading activation, enabling a more comprehensive understanding of the relationships between nodes and edges in the graph. During the testing phase, the EMGE model is validated on three datasets, including WN18RR, FB15k-237, and a private Chinese financial dataset. The experimental results demonstrate a reduction in the mean rank (MR) by 0.2 times, an improvement in the mean reciprocal rank (MRR) by 5.9%, and an increase in the Hit@1 by 12.9% compared to the baseline model.

Particle identification with machine learning from incomplete data in the ALICE experiment

The ALICE experiment at the LHC measures properties of the strongly interacting matter formed in ultrarelativistic heavy-ion collisions. Such studies require accurate particle identification (PID). ALICE provides PID information via several detectors for particles with momentum from about 100 MeV/c up to 20 GeV/c. Traditionally, particles are selected with rectangular cuts. A much better performance can be achieved with machine learning (ML) methods. Our solution uses multiple neural networks (NN) serving as binary classifiers. Moreover, we extended our particle classifier with Feature Set Embedding and attention in order to train on data with incomplete samples. We also present the integration of the ML project with the ALICE analysis software, and we discuss domain adaptation, the ML technique needed to transfer the knowledge between simulated and real experimental data.

Functional connectivity changes in the brain of adolescents with internet addiction: A systematic literature review of imaging studies

Internet usage has seen a stark global rise over the last few decades, particularly among adolescents and young people, who have also been diagnosed increasingly with internet addiction (IA). IA impacts several neural networks that influence an adolescent’s behaviour and development. This article issued a literature review on the resting-state and task-based functional magnetic resonance imaging (fMRI) studies to inspect the consequences of IA on the functional connectivity (FC) in the adolescent brain and its subsequent effects on their behaviour and development. A systematic search was conducted from two databases, PubMed and PsycINFO, to select eligible articles according to the inclusion and exclusion criteria. Eligibility criteria was especially stringent regarding the adolescent age range (10–19) and formal diagnosis of IA. Bias and quality of individual studies were evaluated. The fMRI results from 12 articles demonstrated that the effects of IA were seen throughout multiple neural networks: a mix of increases/decreases in FC in the default mode network; an overall decrease in FC in the executive control network; and no clear increase or decrease in FC within the salience network and reward pathway. The FC changes led to addictive behaviour and tendencies in adolescents. The subsequent behavioural changes are associated with the mechanisms relating to the areas of cognitive control, reward valuation, motor coordination, and the developing adolescent brain. Our results presented the FC alterations in numerous brain regions of adolescents with IA leading to the behavioural and developmental changes. Research on this topic had a low frequency with adolescent samples and were primarily produced in Asian countries. Future research studies of comparing results from Western adolescent samples provide more insight on therapeutic intervention.

A Hybrid Neural Network-Based Fast Financial Fraud Detection Model

With the increasing number of financial transactions, financial fraud has become increasingly serious for financial institutions and the public. The core idea of this model is to integrate multiple neural network structures and utilize their respective advantages to improve the performance of fraud detection. Firstly, we employed the convolutional neural network with interpretable blocks (CNNIB) convolutional neural network (CNN) to extract key features from the data to capture patterns and patterns in fraud cases. Secondly, we introduced the autoencoder generative adversarial network (AE-GAN) adversarial network to perform feature analysis on sequence data to capture temporal features in transaction sequences. Finally, we used differential detection for classification to determine whether transactions were fraudulent. An independent detection module was established to accelerate the recognition of financial fraud, and parameter indicators were optimized. Finally, a hybrid neural network model was established. The experimental results indicate that our model has achieved significant results in quickly detecting financial fraud; compared with traditional single neural network models, hybrid neural network models have significant improvements in accuracy and efficiency. In addition, we conducted in-depth analysis of the model and revealed its performance stability under different training set sizes and data distributions. Our research findings provide an effective tool for financial institutions to quickly identify financial fraud.

Active Control of Longitudinal Combustion Instability in Bluff-Body Stabilized Premixed Flames with Multiple Neural Network Controller

ABSTRACT The suppression performance of multiple neural network controller on the longitudinal combustion instability is investigated in the Volvo bluff-body stabilized premixed flame. Two controlled plant models, e.g. one-dimensional (1D) reduced order acoustic network model and three-dimensional (3D) large eddy simulation model are adopted to verify this control strategy. In order to build the reduced order Volvo acoustic network model, large eddy simulation is adopted to extract the characteristics of system heat release under wide bandwidth and construct the flame transfer functions. For the 1D acoustic network model, this controller can completely eliminate the pressure oscillation using fuel valve as actuator. For the 3D large eddy simulation model that mimics the real-time oscillating characteristics, clustered prediction method and amplitude limiting method are adopted in multiple neural network control strategy to better account for the nonlinear characteristics of the 3D Volvo flame. This control strategy enables the modulation of the phase relationship between the heat release rate oscillation and pressure oscillation by introducing small perturbations to the inlet equivalence ratio, and attenuates the amplitude of the second harmonic of the unstable mode effectively.

Multi-modal domain adaptation for revealing spatial functional landscape from spatially resolved transcriptomics.

Spatially resolved transcriptomics (SRT) has emerged as a powerful tool for investigating gene expression in spatial contexts, providing insights into the molecular mechanisms underlying organ development and disease pathology. However, the expression sparsity poses a computational challenge to integrate other modalities (e.g. histological images and spatial locations) that are simultaneously captured in SRT datasets for spatial clustering and variation analyses. In this study, to meet such a challenge, we propose multi-modal domain adaption for spatial transcriptomics (stMDA), a novel multi-modal unsupervised domain adaptation method, which integrates gene expression and other modalities to reveal the spatial functional landscape. Specifically, stMDA first learns the modality-specific representations from spatial multi-modal data using multiple neural network architectures and then aligns the spatial distributions across modal representations to integrate these multi-modal representations, thus facilitating the integration of global and spatially local information and improving the consistency of clustering assignments. Our results demonstrate that stMDA outperforms existing methods in identifying spatial domains across diverse platforms and species. Furthermore, stMDA excels in identifying spatially variable genes with high prognostic potential in cancer tissues. In conclusion, stMDA as a new tool of multi-modal data integration provides a powerful and flexible framework for analyzing SRT datasets, thereby advancing our understanding of intricate biological systems.

Stacked neural network for predicting polygenic risk score

In recent years, the utility of polygenic risk scores (PRS) in forecasting disease susceptibility from genome-wide association studies (GWAS) results has been widely recognised. Yet, these models face limitations due to overfitting and the potential overestimation of effect sizes in correlated variants. To surmount these obstacles, we devised the Stacked Neural Network Polygenic Risk Score (SNPRS). This novel approach synthesises outputs from multiple neural network models, each calibrated using genetic variants chosen based on diverse p-value thresholds. By doing so, SNPRS captures a broader array of genetic variants, enabling a more nuanced interpretation of the combined effects of these variants. We assessed the efficacy of SNPRS using the UK Biobank data, focusing on the genetic risks associated with breast and prostate cancers, as well as quantitative traits like height and BMI. We also extended our analysis to the Korea Genome and Epidemiology Study (KoGES) dataset. Impressively, our results indicate that SNPRS surpasses traditional PRS models and an isolated deep neural network in terms of accuracy, highlighting its promise in refining the efficacy and relevance of PRS in genetic studies.