Neural Network Structure Research Articles

Coal-gangue sound recognition using hybrid multi-branch CNN based on attention mechanism fusion in noisy environments

The coal-gangue recognition technology plays an important role in the intelligent realization of fully mechanized caving face and the improvement of coal quality. Although great progress has been made for the coal-gangue recognition in recent years, most of them have not taken into account the impact of the complex environment of top coal caving on recognition performance. Herein, a hybrid multi-branch convolutional neural network (HMBCNN) is proposed for coal-gangue recognition, which based on improved Mel Frequency Cepstral Coefficient (MFCC) as well as Mel spectrogram, and attention mechanism. Firstly, the MFCC and its smooth feature matrix are input into each branch of one-dimensional multi-branch convolutional neural network, and the spliced features are extracted adaptively through multi-head attention mechanism. Secondly, the Mel spectrogram and its first-order derivative are input into each branch of the two-dimensional multi-branch convolutional neural network respectively, and the effective time-frequency information is paid attention to through the soft attention mechanism. Finally, at the decision-making level, the two networks are fused to establish a model for feature fusion and classification, obtaining optimal fusion strategies for different features and networks. A database of sound pressure signals under different signal-to-noise ratios and equipment operations is constructed based on a large amount of data collected in the laboratory and on-site. Comparative experiments and discussions are conducted on this database with advanced algorithms and different neural network structures. The results show that the proposed method achieves higher recognition accuracy and better robustness in noisy environments.

Neural Network Algorithm for Training a Mobile Robot in the Task of Following a Target

The article is devoted to the research, development and implementation of an artificial neural network with reinforcement for controlling a mobile robot with a differential drive to solve the task of following a target. The authors of the study present a detailed description of the architecture of the neural network, its training using a deep deterministic gradient policy algorithm, and integration with the robot control system. A distributed neural network structure was selected for specialized generation of controls, for which the authors chose the angular and linear speed of the robot. A system of rules has been synthesized that takes into account changes in the distance between the robot and the target object and tracking of the heading angle in real time. A mathematical model of a robot with a differential drive is considered, on the basis of which a simulation program is implemented for intermediate training of a neural network, as a result of which the initial weighting coefficients are formed. Using this program, you can get by with less energy and time costs at the initial stage of the study. Experiments to test the effectiveness of the developed neural network were carried out in the Gazebo simulation environment using the ROS2 communication interface. The article describes the process of integrating a neural network with a robot control system in a simulation environment, as well as test results and analysis of the obtained data. The results of experiments are presented for three scenarios in which the initial position of the robot is the same, the remaining parameters are generated according to the rules described by the authors. The effectiveness of using such a solution for trajectory planning has been confirmed. The research contributes to the field of autonomous systems and demonstrates the potential of artificial reinforcement neural networks in robotics.

Physics-informed neural networks for efficient aeroacoustic computational based on two time sequence prediction models

Physics-informed neural networks (PINNs) have been proven powerful for solving partial differential equations (PDEs). However, conventional PINNs fail handling time/space variables in the training data distinctly and overlook the temporal relationships among data points, which is critical for accurately modeling dynamic systems. To address this issue, this work proposes two novel PINN models based on the transformer and the convolutional long short-term memory (ConvLSTM) architectures, respectively. An encoder-decoder neural network structure is adopted to enhance handling of time-space variables and a high-order finite-difference method is used to compute partial derivatives. These computations are implemented using the convolution within the PyTorch framework that incorporates both physical and mathematical information into the neural networks. The efficacy of the proposed methods has been evaluated for the wave equation and the Reynolds-Averaged Navier-Stokes equations. Results suggest that the transformer-based PINN outperforms both the conventional and ConvLSTM-based PINN in terms of accuracy, efficiency and generalizability.

An artificial neural network based approach for harmonic component prediction in a distribution line

With the increasing use of nonlinear devices in both generation and consumption of power, it is essential that we develop accurate and quick control for active filters to suppress harmonics. Time delays between input and output are catastrophic for such filters which rely on real-time operation. Artificial Neural Networks (ANNs) are capable of modeling complex nonlinear systems through adjustments in their learned parameters. Once properly trained, they can produce highly accurate predictions at an instantaneous time frame. Leveraging these qualities, various complex control systems may be replaced or aided by neural networks to provide quick and precise responses. This paper proposes an ANN-based approach for the prediction of individual harmonic components using minimal inputs. By extracting and analyzing the nature of harmonic component magnitudes obtained from the survey of a particular area through real-time measurements, a sequential pattern in their occurrence is observed. Various neural network architectures are trained using the collected data and their performances are evaluated. The best-performing model, whose losses are minimal, is then used to observe the harmonic cancellation for multiple unseen cases through a simplified simulation in hardware-in-the-loop. These neural network structures, which produce instantaneous and accurate outputs, are effective in harmonic filtering.

Analysis and prediction of sputtering yield using combined hierarchical clustering analysis and artificial neural network algorithms

Sputtering is a crucial technology in fields such as electric propulsion, materials processing and semiconductors. Modeling of sputtering is significant for improving thruster design and designing material processing control algorithms. In this study we use the hierarchical clustering analysis algorithm to perform cluster analysis on 17 descriptors related to sputtering. These descriptors are divided into four fundamental groups, with representative descriptors being the mass of the incident ion, the formation energy of the incident ion, the mass of the target and the formation energy of the target. We further discuss the possible physical processes and significance involved in the classification process, including cascade collisions, energy transfer and other processes. Finally, based on the analysis of the above descriptors, several neural network models are constructed for the regression of sputtering threshold E th, maximum sputtering energy E max and maximum sputtering yield SY max. In the regression model based on 267 samples, the four descriptor attributes showed higher accuracy than the 17 descriptors (R 2 evaluation) in the same neural network structure, with the 5×5 neural network structure achieving the highest accuracy, having an R 2 of 0.92. Additionally, simple sputtering test data also demonstrated the generalization ability of the 5×5 neural network model, the error in maximum sputtering yield being less than 5%.

Fusion innovation: Multi-scale dilated collaborative model of ConvNeXt and MSDA for fault diagnosis

Bearings and gears are critical components in modern industry, and cross-domain diagnosis of these elements is of great significance. However, in practical applications, challenges such as insufficient training data and variability between equipment arise. To address this issue, this study proposes an innovative neural network structure, ConvNeXt, and a Multi-Scale Dilated Attention (MSDA) mechanism to improve the accuracy problem caused by inadequate feature extraction. ConvNeXt improves upon traditional convolutional neural networks by introducing a multi-scale attention mechanism to enhance the model's performance and expressiveness. Through parallel multi-channel convolution operations, ConvNeXt can capture dependencies between different channels and reduce the number of parameters. Meanwhile, the MSDA mechanism allows signals to interact and exchange information at different scales, effectively extracting complex features in one-dimensional signals. Experimental results demonstrate a significant performance improvement in one-dimensional signal processing using ConvNeXt and MSDA, better capturing relationships between global and local features in one-dimensional signals and enhancing model accuracy. The joint application of ConvNeXt and MSDA brings new solutions to one-dimensional signal processing, offering potential opportunities for effective monitoring of critical components in rotating machinery. Experimental results show that this method achieves high diagnostic accuracy in various transfer tasks, with an average accuracy of 94.28%, providing reliable support for bearing fault diagnosis.

Deformation control of piezoelectric intelligent curved shell structure with multiple actuators based on neural network optimized by genetic algorithm

In aerospace engineering, it is important to achieve precise deformation control of curved shell. In this paper, the piezoelectric intelligent curved shell structure (PICSS) with multiple actuators was selected as the research object. Firstly, a finite element model of the structure was constructed, and the sample data was obtained based on simulation method for the back propagation (BP) neural network. Next, the structure of the BP neural network was determined as ‘6–13-24-3′ with the goal of obtaining the minimum mean square error. Then genetic algorithm was used to optimize the initial weights and thresholds of the BP neural network, and the input voltage of each actuator was predicted. The accuracy of voltage prediction was confirmed through collaborative simulation technology by testing the BP neural network before and after optimization. Finally, the deformation perception and control of the piezoelectric curved shell were verified through the PICSS.

Transparent sparse graph pathway network for analyzing the internal relationship of lung cancer.

While it is important to find the key biomarkers and improve the accuracy of disease models, it is equally important to understand their interaction relationships. In this study, a transparent sparse graph pathway network (TSGPN) is proposed based on the structure of graph neural networks. This network simulates the action of genes in vivo, adds to prior knowledge, and improves the model's accuracy. First, the graph connection was constructed according to protein-protein interaction networks and competing endogenous RNA (ceRNA) networks, from which some noise or unimportant connections were spontaneously removed based on the graph attention mechanism and hard concrete estimation. This realized the reconstruction of the ceRNA network representing the influence of other genes in the disease on mRNA. Next, the gene-based interpretation was transformed into a pathway-based interpretation based on the pathway database, and the hidden layer was added to realize the high-dimensional analysis of the pathway. Finally, the experimental results showed that the proposed TSGPN method is superior to other comparison methods in F1 score and AUC, and more importantly, it can effectively display the role of genes. Through data analysis applied to lung cancer prognosis, ten pathways related to LUSC prognosis were found, as well as the key biomarkers closely related to these pathways, such as HOXA10, hsa-mir-182, and LINC02544. The relationship between them was also reconstructed to better explain the internal mechanism of the disease.

An artificial neural network visible mathematical model for predicting slug liquid holdup in low to high viscosity multiphase flow for horizontal to vertical pipes

AbstractSlug liquid holdup (SLH) is a critical requirement for accurate pressure drop prediction during multiphase pipe flows and by extension optimal gas lift design and production optimization in wellbores. Existing empirical correlations provide inaccurate predictions because they were developed with regression analysis and data measured for limited ranges of flow conditions. Existing SLH machine learning models provide accurate predictions but are published without any equations making their use by other researchers difficult. The only existing ML model published with actual equations cannot be considered optimum because it was selected by considering artificial neural network (ANN) structures with only one hidden layer. In this study, an ANN-based model for SLH prediction with actual equations is presented. A total of 2699 data points randomly divided into 70%, 15%, and 15% for training, validation, and testing was used in constructing 71 different network structures with 1, 2, and 3 hidden layers respectively. Sensitivity analysis revealed that the optimum network structure has 3 hidden layers with 20, 5, and 15 neurons in the first, second, and third hidden layers, respectively. The optimum network structure was translated into actual equations with the aid of the weights, biases, and activation functions. Trend analysis revealed that this study’s model reproduced the expected effects of inputs on SLH. Test against measured data revealed that this study’s model is in agreement with measured data with coefficient of determinations of 0.9791, 0.9727, 0.9756, and 0.9776 for training, testing, validation, and entire datasets, respectively. Comparative study revealed that this study’s model outperformed existing models with a relative performance factor of 0.002. The present model is presented with visible mathematical equations making its implementation by any user easy and without the need for any ML framework. Unlike existing ANN-based models developed with one hidden layered ANN structures, the present model was developed by considering ANN structures with one, two, and three hidden layers, respectively.

Improving I-ELM structure through optimal addition of hidden nodes: Compact I-ELM

Incremental extreme learning machines (I-ELMs) can automatically determine the structure of neural networks and achieve high learning speeds. However, during the process of adding hidden nodes, unnecessary hidden nodes that have little relevance to the target may be added. Several studies have proposed methods to overcome this problem by measuring the relevance between hidden nodes and outputs and adding or removing hidden nodes accordingly. Random hidden nodes have the advantage of creating diverse patterns, but they encounter a problem in which hidden nodes that generate patterns with little or no relevance to the target can be added, thereby increasing the number of hidden nodes. Unlike in existing I-ELMs, which use random hidden nodes, we propose a compact I-ELM algorithm that initially adds linear regression nodes and subsequently applies a method to ensure that the hidden nodes have patterns differing from the existing ones. Based on benchmark data, we confirmed that the proposed method constructs a compact neural network structure with fewer hidden nodes compared to the existing I-ELM systems.

Predicting Binge Eating Disorder Using Machine Learning Methods

Eating disorders are enduring conditions characterized by elevated rates of mortality and morbidity, presenting a serious threat to life. Among these disorders, binge eating disorder is the most prevalent. Therefore, it is an important health problem that often results in obesity worldwide. This study was conducted to evaluate the eating attitudes and behaviors of university students and predict binge eating disorder using machine learning methods. The study was carried out on 306 individuals (117 males, 189 females). Individuals' personal characteristics were questioned with the questionnaire form. The Bulimic Investigatory Test Edinburgh (BITE) test was used to determine whether individuals taking part in the study had binge eating disorder. In this study, in which binge eating disorder was classified, different artificial neural network models were created by changing the basic parameters, and the optimum model was assessed accordingly. Among the models created with different layers and activation functions, the optimum results were obtained using the number of fully connected layers as 2, first and second layers' sizes as 10, and ReLU, a non-linear activation function, in the Bilayered Neural Network structure. This study is the first trial in which binge eating disorder is predicted using machine learning methods, and we believe that machine learning is an important tool to help researchers and clinicians diagnose, prevent, and treat eating disorders at an early stage.

Physics-informed Neural Network for Force-free Magnetic Field Extrapolation

In this paper, we propose a physics-informed neural network extrapolation method that leverages machine learning techniques to reconstruct coronal magnetic fields. We enhance the classical neural network structure by introducing the concept of a quasi-output layer to address the challenge of preserving physical constraints during the neural network extrapolation process. Furthermore, we employ second-order optimization methods for training the neural network, which are more efficient compared to the first-order optimization methods commonly used in classical machine learning. Our approach is evaluated on the widely recognized semi-analytical model proposed by Low and Lou. The results demonstrate that the deep learning method achieves high accuracy in reconstructing the semi-analytical model across multiple evaluation metrics. In addition, we validate the effectiveness of our method on the observed magnetogram of active region.

ADP-Based Fault-Tolerant Control for Multiagent Systems With Semi-Markovian Jump Parameters.

This article analyzes and validates an approach of integration of adaptive dynamic programming (ADP) and adaptive fault-tolerant control (FTC) technique to address the consensus control problem for semi-Markovian jump multiagent systems having actuator bias faults. A semi-Markovian process, a more versatile stochastic process, is employed to characterize the parameter variations that arise from the intricacies of the environment. The reliance on accurate knowledge of system dynamics is overcome through the utilization of an actor-critic neural network structure within the ADP algorithm. A data-driven FTC scheme is introduced, which enables online adjustment and automatic compensation of actuator bias faults. It has been demonstrated that the signals generated by the controlled system exhibit uniform boundedness. Additionally, the followers' states can achieve and maintain consensus with that of the leader. Ultimately, the simulation results are given to demonstrate the efficacy of the designed theoretical findings.

RGP: Neural Network Pruning Through Regular Graph With Edges Swapping.

Deep learning technology has found a promising application in lightweight model design, for which pruning is an effective means of achieving a large reduction in both model parameters and float points operations (FLOPs). The existing neural network pruning methods mostly start from the consideration of the importance of model parameters and design parameter evaluation metrics to perform parameter pruning iteratively. These methods were not studied from the perspective of network model topology, so they might be effective but not efficient, and they require completely different pruning for different datasets. In this article, we study the graph structure of the neural network and propose a regular graph pruning (RGP) method to perform a one-shot neural network pruning. Specifically, we first generate a regular graph and set its node-degree values to meet the preset pruning ratio. Then, we reduce the average shortest path-length (ASPL) of the graph by swapping edges to obtain the optimal edge distribution. Finally, we map the obtained graph to a neural network structure to realize pruning. Our experiments demonstrate that the ASPL of the graph is negatively correlated with the classification accuracy of the neural network and that RGP has a strong precision retention capability with high parameter reduction (more than 90%) and FLOPs reduction (more than 90%) (the code for quick use and reproduction is available at https://github.com/Holidays1999/Neural-Network-Pruning-through-its-RegularGraph-Structure).

A Novel Detection and Classification Framework for Diagnosing of Cerebral Microbleeds Using Transformer and Language.

The detection of Cerebral Microbleeds (CMBs) is crucial for diagnosing cerebral small vessel disease. However, due to the small size and subtle appearance of CMBs in susceptibility-weighted imaging (SWI), manual detection is both time-consuming and labor-intensive. Meanwhile, the presence of similar-looking features in SWI images demands significant expertise from clinicians, further complicating this process. Recently, there has been a significant advancement in automated detection of CMBs using a Convolutional Neural Network (CNN) structure, aiming at enhancing diagnostic efficiency for neurologists. However, existing methods still show discrepancies when compared to the actual clinical diagnostic process. To bridge this gap, we introduce a novel multimodal detection and classification framework for CMBs' diagnosis, termed MM-UniCMBs. This framework includes a light-weight detection model and a multi-modal classification network. Specifically, we proposed a new CMBs detection network, CMBs-YOLO, designed to capture the salient features of CMBs in SWI images. Additionally, we design an innovative language-vision classification network, CMBsFormer (CF), which integrates patient textual descriptions-such as gender, age, and medical history-with image data. The MM-UniCMBs framework is designed to closely align with the diagnostic workflow of clinicians, offering greater interpretability and flexibility compared to existing methods. Extensive experimental results show that MM-UniCMBs achieves a sensitivity of 94% in CMBs' classification and can process a patient's data within 5 s.

Weighted Expectile Regression Neural Networks for Right Censored Data.

As a favorable alternative to the censored quantile regression, censored expectile regression has been popular in survival analysis due to its flexibility in modeling the heterogeneous effect of covariates. The existing weighted expectile regression (WER) method assumes that the censoring variable and covariates are independent, and that the covariates effects has a global linear structure. However, these two assumptions are too restrictive to capture the complex and nonlinear pattern of the underlying covariates effects. In this article, we developed a novel weighted expectile regression neural networks (WERNN) method by incorporating the deep neural network structure into the censored expectile regression framework. To handle the random censoring, we employ the inverse probability of censoring weighting (IPCW) technique in the expectile loss function. The proposed WERNN method is flexible enough to fit nonlinear patterns and therefore achieves more accurate prediction performance than the existing WER method for right censored data. Our findings are supported by extensive Monte Carlo simulation studies and a real data application.

Prediction of Tsunamis in Indonesia Using an Optimized Neural Network with SMOTE

Tsunamis have the potential to have a large impact on the environment, therefore early detection and preparation for tsunami need to be carried out to reduce the impact of casualties and losses incurred. This research aims to predict tsunami events due to large earthquakes in Indonesia as a form of early detection. The optimized neural network method is used in research to classify tsunami events in Indonesia in 2000-2023 for large earthquakes with strength more than 5 magnitudes. The research results show that the neural network structure formed consists of an input layer, a hidden layer, and an output layer. The results of the evaluation of the neural network model with SMOTE obtained an accuracy value of 99.43%, precision of 96.31%, and an F1 score of 97.86%, which means the resulting model is good. Therefore, an optimized neural networks can be applied as a warning system in various regions to detect potential tsunami events in the future.

Analysis of rogue wave in the mid-infrared supercontinuum under femtosecond weak seed pulse conditions based on deep learning

The generation process of rogue wave (RW) is affected by noise, which is an unstable state, and the existence of RW will reduce the stability of mid-infrared supercontinuum. However, the process of studying RW requires a large amount of data simulation and statistics, and traditional methods are time-consuming and inefficient. Therefore, this paper adopts long short-term memory (LSTM) neural network to obtain the spectrum information after transmission for a certain distance according to the waveform information of the incident pulse. The results show that the LSTM neural network structure can train and predict the peak power, time deviation information, time intensity evolution and spectrum evolution of RW after 10 cm propagation with only changing the number of internal units. And it performs well on both large and small data sets.

Motion coprime array-based DOA estimation considering phase disturbance of sensor array

To address the phase disturbance issue faced by sensor arrays in practical applications, a cascaded deep convolutional neural network structure is proposed to achieve direction-of-arrival (DOA) estimation for motion coprime arrays. Firstly, the synthesized covariance matrix obtained after motion is inputted into the first-level network for estimating the phase disturbance matrix. Then, we analyze the impact of phase perturbation on the synthesized covariance matrix and utilize the estimated disturbance phase to obtain an undisturbed synthesized covariance matrix. Finally, after phase compensation, the synthesized covariance matrix performs DOA estimation through the second-level network. Furthermore, to acquire three times the virtual unique lags of the coprime array, the synthesis condition about moving distance and unique lags is derived. The proposed method is shown to be effective and superior through the experiment results.

On examining the predictive capabilities of two variants of the PINN in validating localized wave solutions in the generalized nonlinear Schrödinger equation

We introduce a novel neural network structure called strongly constrained theory-guided neural network (SCTgNN), to investigate the behaviour of the localized solutions of the generalized nonlinear Schrödinger (NLS) equation. This equation comprises four physically significant nonlinear evolution equations, namely, the NLS, Hirota, Lakshmanan–Porsezian–Daniel and fifth-order NLS equations. The generalized NLS equation demonstrates nonlinear effects up to quintic order, indicating rich and complex dynamics in various fields of physics. By combining concepts from the physics-informed neural network and theory-guided neural network (TgNN) models, the SCTgNN aims to enhance our understanding of complex phenomena, particularly within nonlinear systems that defy conventional patterns. To begin, we employ the TgNN method to predict the behaviour of localized waves, including solitons, rogue waves and breathers, within the generalized NLS equation. We then use the SCTgNN to predict the aforementioned localized solutions and calculate the mean square errors in both the SCTgNN and TgNN in predicting these three localized solutions. Our findings reveal that both models excel in understanding complex behaviour and provide predictions across a wide variety of situations.