Neural Network Structure Research Articles

Physics-informed Neural Network for Force-free Magnetic Field Extrapolation

In this paper, we propose a physics-informed neural network extrapolation method that leverages machine learning techniques to reconstruct coronal magnetic fields. We enhance the classical neural network structure by introducing the concept of a quasi-output layer to address the challenge of preserving physical constraints during the neural network extrapolation process. Furthermore, we employ second-order optimization methods for training the neural network, which are more efficient compared to the first-order optimization methods commonly used in classical machine learning. Our approach is evaluated on the widely recognized semi-analytical model proposed by Low and Lou. The results demonstrate that the deep learning method achieves high accuracy in reconstructing the semi-analytical model across multiple evaluation metrics. In addition, we validate the effectiveness of our method on the observed magnetogram of active region.

ADP-Based Fault-Tolerant Control for Multiagent Systems With Semi-Markovian Jump Parameters.

This article analyzes and validates an approach of integration of adaptive dynamic programming (ADP) and adaptive fault-tolerant control (FTC) technique to address the consensus control problem for semi-Markovian jump multiagent systems having actuator bias faults. A semi-Markovian process, a more versatile stochastic process, is employed to characterize the parameter variations that arise from the intricacies of the environment. The reliance on accurate knowledge of system dynamics is overcome through the utilization of an actor-critic neural network structure within the ADP algorithm. A data-driven FTC scheme is introduced, which enables online adjustment and automatic compensation of actuator bias faults. It has been demonstrated that the signals generated by the controlled system exhibit uniform boundedness. Additionally, the followers' states can achieve and maintain consensus with that of the leader. Ultimately, the simulation results are given to demonstrate the efficacy of the designed theoretical findings.

Application of Seismic Attribute Clustering Method Based on PCA-SOM in Identification of Paleochannels

Abstract Paleochannels are crucial reservoir types in continental oil and gas fields. The P oil layer in the Sanzhao Depression of the Songliao Basin is a typical example of river-controlled delta-phase deposition. The underwater diversion channel is highly developed and serves as the primary unit of the oil and gas reservoir. The stacking of the river channel creates a complex relationship. Identifying river channels has always been a challenging aspect of predicting sweet spot reservoirs due to the thinness of the sand, fault cutting, seismic acquisition and processing noise, low-resolution constraints, and other factors. This paper proposes a method for identifying river channels by combining principal component analysis (PCA) and self-organised mapping network (SOM) with seismic attribute clustering. The aim is to address the issue of difficult identification of paleochannels. The seismic attribute set, which contains multiple types of attributes, is first dimensionality reduced using principal component analysis. Then, a few principal components that can characterise the attribute set are extracted and used as inputs to the self-organising map (SOM). This reduces the dimensionality of the training sample set and simplifies the network structure. The self-organising neural network structure is designed based on the input sample features. Training is then carried out, and the network weights are updated. Attribute clustering is performed to obtain the river identification results. The effectiveness of this method for paleochannel recognition is verified by the clear morphology of the recognised paleochannels in the Songliao Basin, their conformity to geological understanding, and their matching with drilling information.

RGP: Neural Network Pruning Through Regular Graph With Edges Swapping.

Deep learning technology has found a promising application in lightweight model design, for which pruning is an effective means of achieving a large reduction in both model parameters and float points operations (FLOPs). The existing neural network pruning methods mostly start from the consideration of the importance of model parameters and design parameter evaluation metrics to perform parameter pruning iteratively. These methods were not studied from the perspective of network model topology, so they might be effective but not efficient, and they require completely different pruning for different datasets. In this article, we study the graph structure of the neural network and propose a regular graph pruning (RGP) method to perform a one-shot neural network pruning. Specifically, we first generate a regular graph and set its node-degree values to meet the preset pruning ratio. Then, we reduce the average shortest path-length (ASPL) of the graph by swapping edges to obtain the optimal edge distribution. Finally, we map the obtained graph to a neural network structure to realize pruning. Our experiments demonstrate that the ASPL of the graph is negatively correlated with the classification accuracy of the neural network and that RGP has a strong precision retention capability with high parameter reduction (more than 90%) and FLOPs reduction (more than 90%) (the code for quick use and reproduction is available at https://github.com/Holidays1999/Neural-Network-Pruning-through-its-RegularGraph-Structure).

A Novel Detection and Classification Framework for Diagnosing of Cerebral Microbleeds Using Transformer and Language.

The detection of Cerebral Microbleeds (CMBs) is crucial for diagnosing cerebral small vessel disease. However, due to the small size and subtle appearance of CMBs in susceptibility-weighted imaging (SWI), manual detection is both time-consuming and labor-intensive. Meanwhile, the presence of similar-looking features in SWI images demands significant expertise from clinicians, further complicating this process. Recently, there has been a significant advancement in automated detection of CMBs using a Convolutional Neural Network (CNN) structure, aiming at enhancing diagnostic efficiency for neurologists. However, existing methods still show discrepancies when compared to the actual clinical diagnostic process. To bridge this gap, we introduce a novel multimodal detection and classification framework for CMBs' diagnosis, termed MM-UniCMBs. This framework includes a light-weight detection model and a multi-modal classification network. Specifically, we proposed a new CMBs detection network, CMBs-YOLO, designed to capture the salient features of CMBs in SWI images. Additionally, we design an innovative language-vision classification network, CMBsFormer (CF), which integrates patient textual descriptions-such as gender, age, and medical history-with image data. The MM-UniCMBs framework is designed to closely align with the diagnostic workflow of clinicians, offering greater interpretability and flexibility compared to existing methods. Extensive experimental results show that MM-UniCMBs achieves a sensitivity of 94% in CMBs' classification and can process a patient's data within 5 s.

Weighted Expectile Regression Neural Networks for Right Censored Data.

As a favorable alternative to the censored quantile regression, censored expectile regression has been popular in survival analysis due to its flexibility in modeling the heterogeneous effect of covariates. The existing weighted expectile regression (WER) method assumes that the censoring variable and covariates are independent, and that the covariates effects has a global linear structure. However, these two assumptions are too restrictive to capture the complex and nonlinear pattern of the underlying covariates effects. In this article, we developed a novel weighted expectile regression neural networks (WERNN) method by incorporating the deep neural network structure into the censored expectile regression framework. To handle the random censoring, we employ the inverse probability of censoring weighting (IPCW) technique in the expectile loss function. The proposed WERNN method is flexible enough to fit nonlinear patterns and therefore achieves more accurate prediction performance than the existing WER method for right censored data. Our findings are supported by extensive Monte Carlo simulation studies and a real data application.

Prediction of Tsunamis in Indonesia Using an Optimized Neural Network with SMOTE

Tsunamis have the potential to have a large impact on the environment, therefore early detection and preparation for tsunami need to be carried out to reduce the impact of casualties and losses incurred. This research aims to predict tsunami events due to large earthquakes in Indonesia as a form of early detection. The optimized neural network method is used in research to classify tsunami events in Indonesia in 2000-2023 for large earthquakes with strength more than 5 magnitudes. The research results show that the neural network structure formed consists of an input layer, a hidden layer, and an output layer. The results of the evaluation of the neural network model with SMOTE obtained an accuracy value of 99.43%, precision of 96.31%, and an F1 score of 97.86%, which means the resulting model is good. Therefore, an optimized neural networks can be applied as a warning system in various regions to detect potential tsunami events in the future.

Analysis of rogue wave in the mid-infrared supercontinuum under femtosecond weak seed pulse conditions based on deep learning

The generation process of rogue wave (RW) is affected by noise, which is an unstable state, and the existence of RW will reduce the stability of mid-infrared supercontinuum. However, the process of studying RW requires a large amount of data simulation and statistics, and traditional methods are time-consuming and inefficient. Therefore, this paper adopts long short-term memory (LSTM) neural network to obtain the spectrum information after transmission for a certain distance according to the waveform information of the incident pulse. The results show that the LSTM neural network structure can train and predict the peak power, time deviation information, time intensity evolution and spectrum evolution of RW after 10 cm propagation with only changing the number of internal units. And it performs well on both large and small data sets.

Motion coprime array-based DOA estimation considering phase disturbance of sensor array

To address the phase disturbance issue faced by sensor arrays in practical applications, a cascaded deep convolutional neural network structure is proposed to achieve direction-of-arrival (DOA) estimation for motion coprime arrays. Firstly, the synthesized covariance matrix obtained after motion is inputted into the first-level network for estimating the phase disturbance matrix. Then, we analyze the impact of phase perturbation on the synthesized covariance matrix and utilize the estimated disturbance phase to obtain an undisturbed synthesized covariance matrix. Finally, after phase compensation, the synthesized covariance matrix performs DOA estimation through the second-level network. Furthermore, to acquire three times the virtual unique lags of the coprime array, the synthesis condition about moving distance and unique lags is derived. The proposed method is shown to be effective and superior through the experiment results.

On examining the predictive capabilities of two variants of the PINN in validating localized wave solutions in the generalized nonlinear Schrödinger equation

We introduce a novel neural network structure called strongly constrained theory-guided neural network (SCTgNN), to investigate the behaviour of the localized solutions of the generalized nonlinear Schrödinger (NLS) equation. This equation comprises four physically significant nonlinear evolution equations, namely, the NLS, Hirota, Lakshmanan–Porsezian–Daniel and fifth-order NLS equations. The generalized NLS equation demonstrates nonlinear effects up to quintic order, indicating rich and complex dynamics in various fields of physics. By combining concepts from the physics-informed neural network and theory-guided neural network (TgNN) models, the SCTgNN aims to enhance our understanding of complex phenomena, particularly within nonlinear systems that defy conventional patterns. To begin, we employ the TgNN method to predict the behaviour of localized waves, including solitons, rogue waves and breathers, within the generalized NLS equation. We then use the SCTgNN to predict the aforementioned localized solutions and calculate the mean square errors in both the SCTgNN and TgNN in predicting these three localized solutions. Our findings reveal that both models excel in understanding complex behaviour and provide predictions across a wide variety of situations.

Non-Contact Human Body Dimension Estimation Methods Based on Deep Learning: A Critical Analysis

With the advancement of information technology, Human Body Dimension Estimation (HBDE) is moving towards digitalization and non-contact methods. This paper critically reviews non-contact HBDE methods based on deep learning, primarily covering image-based methods, the Adaptive Body Structure Segmentation (ABSS) algorithm with height and weight measurements, and 3D meshes or point cloud data methods. Convolutional Neural Networks (CNNs) excel in image measurement, while the ABSS algorithm with adaptive segmentation achieves high accuracy. Recent studies, such as the Grey Wolf Optimizer-Elman Neural Network (GBWO-ENN) model, have significantly improved prediction accuracy by optimizing neural network structures. Additionally, 3D point cloud data combined with the Skinned Multi-Person Linear (SMPL) model has proven effective in complex scenarios. This paper summarizes the performance of various methods, analyzes common datasets, and explores future development directions for non-contact HBDE. Despite significant progress, this field still faces challenges such as data quality, scarcity of extreme samples, and privacy issues. Future research should focus on building high-quality datasets and developing lightweight measurement networks to meet practical application needs.

A Multi-Scale GNN-Based Personalized Recommender System for Online Consumption Decision

In an era of consumer electronics, effective consumption recommendation contributes a lot to improving benefits for online shopping operators. However, consumers and products constitute a complex heterogeneous information network, in which extraction of structural features is essential. To construct more fine-grained feature space for modeling, we introduce the graph neural network (GNN) theory for this purpose. Accordingly, a multi-scale GNN-based personalized recommender system for online consumption decision is proposed in this paper. Firstly, we set up an encoder for consumption decision based on a multi-scale GNN structure, and define the loss function. Two realistic datasets are utilized as the research scenario, in order to complete the feature combination of online consumption. Then, personalized recommendation settings are completed based on a recurrent neural network structure. And a graph learning module is embedded in it to integrate cross-attention mechanism to establish the consumption decision algorithm. Finally, the proposed method is tested by comparing with relevant research methods under several metrics: adaptability, success rate, and stability. The results show that the proposed method has achieved some improvement in accuracy, coverage, and recommendation speed.

Soft sensor modeling based on self-organizing fuzzy neural network with clustering, merging, and splitting scheme

Abstract As a pivotal role in the control, optimization, and monitoring of contemporary industrial processes, soft sensors are frequently employed in the prediction of key quality variables. To achieve accurate prediction of key quality variables in industrial processes, a soft sensor modeling method based on the self-organizing fuzzy neural network with the clustering, merging, and splitting scheme (SOFNN-CMS) is proposed. First, the supervised fuzzy C-means clustering algorithm is proposed to identify the appropriate initial center and width of the fuzzy neural network, obtaining appropriate initial fuzzy rules. Then, a neuron merging and splitting strategy is designed to adjust the structure of the fuzzy neural network, by merging and splitting the hidden neurons according to the distance of clusters, increasing the adaptability of the fuzzy neural network. Besides, to accelerate the convergence of estimation errors, an improved Levenberg Marquardt algorithm is utilized to update neural network parameters in the training phase, realizing the soft sensor modeling of key quality variables. The effectiveness of the proposed SOFNN-CMS neural network is demonstrated on two benchmark problems and an industrial debutanizer column. Finally, the experiments showcase that the proposed SOFNN-CMS neural network can obtain better soft sensor modeling performance with a compact structure.

Multi-agent reinforcement learning for task offloading with hybrid decision space in multi-access edge computing

Multi-access Edge Computing (MEC) has become a significant technology for supporting the computation-intensive and time-sensitive applications on the Internet of Things (IoT) devices. However, it is challenging to jointly optimize task offloading and resource allocation in the dynamic wireless environment with constrained edge resource. In this paper, we investigate a multi-user and multi-MEC servers system with varying task request and stochastic channel condition. Our purpose is to minimize the total energy consumption and time delay by optimizing the offloading decision, offloading ratio and computing resource allocation simultaneously. As the users are geographically distributed within an area, we formulate the problem of task offloading and resource allocation in MEC system as a partially observable Markov decision process (POMDP) and propose a novel multi-agent deep reinforcement learning (MADRL) -based algorithm to solve it. In particular, two aspects have been modified for performance enhancement: (1) To make fine-grained control, we design a novel neural network structure to effectively handle the hybrid action space arisen by the heterogeneous variables. (2) An adaptive reward mechanism is proposed to reasonably evaluate the infeasible actions and to mitigate the instability caused by manual configuration. Simulation results show the proposed method can achieve 7.12%−20.97% performance enhancements compared with the existing approaches.

Temperature compensation method for impedance signals of bolt loosening under temperature varying conditions using TransUnet

Abstract Piezoelectric transducer based electro-mechanical impedance technology is an effective and reliable detection method for bolt structure loosening under external load. However, temperature change usually causes some non-loosening factors to change the impedance spectrum of bolt structure. In order to reduce the misjudgment of loosening damage caused by temperature change, it is necessary to construct a temperature compensation model for impedance spectrum. In this paper, the convolutional structure in U-net and Transformer are effectively combined to form a TransUnet deep neural network structure suitable for input and output of one-dimensional data. Using impedance data between 10 °C and 50 °C and temperature as input to the network. After convolution operation, the convolutional block attention module is embedded in the U-net to optimize the encoder transmission characteristics and enhance the performance of the skip connection in the traditional U-shaped structure. The temperature compensation rate (TCR) is defined to measure the effect of temperature compensation. Then the lightweight convolutional neural network structure is used to recognize the bolt loosening damage of the compensated impedance signal. The generalization ability of the TransUnet was tested using impedance data that is not in the training dataset. The results show that the TransUnet proposed in the paper can realize the temperature compensation of multi-peak impedance signals. The TCR and recognition accuracy of bolt loose damage reaches reach 0.003 and 95.7%, respectively, which is 11% higher than that without temperature compensation. At 60 °C, the TCR and the identification accuracy of loosening damage can still reach 0.0055 and 90.2%, respectively, which show that the TransUnet has strong generalization ability.

Adaptive Neural Network Control of Four-Switch Buck–Boost Converters

Based on the adaptive control structure of neural networks, this paper proposes a novel output voltage control strategy for DC converters. The strategy regulates the inductor current to maintain a constant voltage by adjusting the duty cycle of four-switch buck—boost (FSBB) converters. A nonlinear average model for the FSBB converter, derived from its energy consumption, is introduced, and its effectiveness is demonstrated through simulations. The simulations confirm that the FSBB converter enables zero-voltage switching (ZVS) of the four switches across the entire operating voltage range. The comparative simulation results show that the proposed control strategy achieves faster voltage regulation while ensuring ZVS, leading to improved converter performance across the full power range.

EfficientUNetViT: Efficient Breast Tumor Segmentation Utilizing UNet Architecture and Pretrained Vision Transformer.

This study introduces a sophisticated neural network structure for segmenting breast tumors. It achieves this by combining a pretrained Vision Transformer (ViT) model with a UNet framework. The UNet architecture, commonly employed for biomedical image segmentation, is further enhanced with depthwise separable convolutional blocks to decrease computational complexity and parameter count, resulting in better efficiency and less overfitting. The ViT, renowned for its robust feature extraction capabilities utilizing self-attention processes, efficiently captures the overall context within images, surpassing the performance of conventional convolutional networks. By using a pretrained ViT as the encoder in our UNet model, we take advantage of its extensive feature representations acquired from extensive datasets, resulting in a major enhancement in the model's ability to generalize and train efficiently. The suggested model has exceptional performance in segmenting breast cancers from medical images, highlighting the advantages of integrating transformer-based encoders with efficient UNet topologies. This hybrid methodology emphasizes the capabilities of transformers in the field of medical image processing and establishes a new standard for accuracy and efficiency in activities related to tumor segmentation.

Deep learning to improve the sensitivity of Di-Higgs searches in the 4b channel

The study of di-Higgs events, both resonant and non-resonant, plays a crucial role in understanding the fundamental interactions of the Higgs boson. In this work we consider di-Higgs events decaying into four b-quarks and propose to improve the experimental sensitivity by utilizing a novel machine learning algorithm known as Symmetry Preserving Attention Network (Spa-Net) — a neural network structure whose architecture is designed to incorporate the inherent symmetries in particle reconstruction tasks. We demonstrate that the Spa-Net can enhance the experimental reach over baseline methods such as the cut-based and the Dense Neural Network-based analyses. At the Large Hadron Collider, with a 14-TeV center-of-mass energy and an integrated luminosity of 300 fb−1, the Spa-Net allows us to establish 95% C.L. upper limits in resonant production cross-sections that are 10% to 45% stronger than baseline methods. For non-resonant di-Higgs production, Spa-Net enables us to constrain the self-coupling that is 9% more stringent than the baseline method.

Mapless navigation via Hierarchical Reinforcement Learning with memory-decaying novelty

Hierarchical Reinforcement Learning (HRL) has shown superior performance for mapless navigation tasks. However, it remains limited in unstructured environments that might contain terrains like long corridors and dead corners, which can lead to local minima. This is because most HRL-based mapless navigation methods employ a simplified reward setting and exploration strategy. In this work, we propose a novel reward function for training the high-level (HL) policy, which contains two components: extrinsic reward and intrinsic reward. The extrinsic reward encourages the robot to move towards the target location, while the intrinsic reward is computed based on novelty, episode memory and memory decaying, making the agent capable of accomplishing spontaneous exploration. We also design a novel neural network structure that incorporates an LSTM network to augment the agent with memory and reasoning capabilities. We test our method in unknown environments and specific scenarios prone to the local minimum problem to evaluate the navigation performance and local minimum resolution ability. The results show that our method significantly increases the success rate when compared to advanced RL-based methods, achieving a maximum improvement of nearly 28%. Our method demonstrates effective improvement in addressing the local minimum issue, especially in cases where the baselines fail completely. Additionally, numerous ablation studies consistently confirm the effectiveness of our proposed reward function and neural network structure.

Deep learning Methods for ECG-Based Heart Disease Detection

Cardiovascular disease (CVD) continues to be a primary cause of death globally, and early detection plays a critical role in improving patient outcomes. This research introduces the development of a deep learning model designed to automatically categorize heart diseases using Electrocardiogram (ECG) data. The model utilizes a 1D Convolutional Neural Network (CNN) structure and makes use of the MIT-BIH Arrhythmia dataset from Physionet. The dataset was split into training, validation, and testing subsets. Our proposed design incorporates convolutional layers, max-pooling, ReLU activation functions, and dropout layers to prevent overfitting. Comparative assessment against conventional methods such as logistic regression and Support Vector Machines (SVM) shows superior performance, achieving an accuracy of 98.29%, recall of 87.60%, precision of 93.75%, and F1 score of 90.37%. The potential of deep learning to enhance the accuracy and efficiency of diagnosing CVD from ECG data is highlighted in this study, introducing a reliable tool for clinical application.