Neural Network Structure Research Articles

Application Research of 3D Virtual Interactive Technology in Interactive Teaching of Arts and Crafts

In the virtual reality teaching of arts and crafts, the uploaded image data often contains a lot of noise, which significantly reduces the quality of uploaded images and seriously affects the teaching effect of interactive teaching of arts and crafts based on virtual reality. For this reason, the research proposes an image denoising model based on convolutional neural network and wavelet transform, which adopts residual neural network structure and batch normalization algorithm, aiming at gradient explosion and gradient disappearance that may be caused by convolutional neural network, And the problem of low training efficiency has been optimized. The results show that when the noise standard deviation is 50, model 1 can achieve the target accuracy with only 82 iterations, while model 2 requires as many as 56 iterations. For images of different types and noise intensities, the average PSNR values of model 1 are 29.3dB, 30.5dB and 28.9dB, respectively. In addition, by applying Model 1 to VR teaching of arts and crafts, the average score of class 1 is 9.3 points higher than that of class 2. The proposed model effectively optimizes the virtual reality teaching process of arts and crafts, and provides a useful contribution to the application of virtual reality in the field of education.

Condition Monitoring in Marine Oil Separation Systems Using Wavelet Packet Transform and Genetic Technique

Condition Monitoring is key to predictive maintenance and especially in the operational efficiency of the Marine Oil Separation System. These systems are crucial for environmental protection and compliance with international maritime regulations. Therefore, it is necessary to design a technique capable of analyzing the signals from sensors and estimating the remaining useful life in order to avoid breakage or unnecessary replacement. This work presents an intelligent method with signal processing based on Wavelet Packets Transform that provides energy data from vibration measurements as characteristic parameters. These values can be related to its RUL, and they are used as inputs for the training process. In particular, a Genetic Neuro-Fuzzy system is proposed as an intelligent classification technique. Once the training process is completed, it can be concluded that a good classifier has been built, since it relates the energy state of the oil separation system with its remaining useful life, and therefore, the necessary information for efficient predictive maintenance is achieved. Furthermore, a mechanism to obtain the final set of fuzzy rules has been developed, showing the correspondence between these fuzzy rules and the neural network structure.

Integrating fast iterative filtering and ensemble neural network structure with attention mechanism for carbon price forecasting

Accurate carbon price forecasts are crucial for policymakers and enterprises to understand the dynamics of carbon price fluctuations, enabling them to formulate informed policies and investment strategies. However, due to the non-linear and non-stationary nature of carbon price, traditional models often struggle to achieve high prediction accuracy. To address this challenge, this study proposes a novel integrated prediction framework designed to enhance forecast accuracy. First, the carbon price series is decomposed into a series of smoother subsequences using fast iterative filtering (FIF). Subsequently, an integrated prediction model, AM-TCN-LSTM, is constructed, incorporating the attention mechanism (AM), temporal convolutional networks (TCN), and long short-term memory (LSTM) neural networks. The attention mechanism adaptively captures complex features from multiple factors, while the TCN-LSTM efficiently extracts temporal features from the sequences. Finally, the results from each subsequence are aggregated to generate the final prediction. Five carbon markets in china: Guangdong, Hubei, Shenzhen, Beijing, and Shanghai were selected to verify the validity of the proposed model. Various comparative models and evaluation metrics were employed to assess performance. The results demonstrate that: (1) the TCN-LSTM model achieves higher prediction accuracy compared to single models. (2) FIF is a more effective decomposition method with superior performance compared to EMD-based methods. (3) The proposed model exhibits the highest predictive capability, with MAE values of 0.0964, 0.1403, 1.9476, 2.0848, and 0.5029 for the five carbon markets, significantly outperforming comparison models. (4) The attention mechanism effectively captures the influence of multiple factors on carbon price, particularly within the short-term components.

Autonomous Systems Control Design Using Neuro Evolution

This paper explores the application of neuro-evolution techniques in the design of control systems for autonomous systems. Neuro-evolution merges neural networks with evolutionary algorithms to develop adaptive, real-time control strategies for autonomous platforms. This approach is particularly effective in dynamic and complex environments, where traditional control methods struggle to ensure optimal performance. By evolving neural network structures, neuro-evolution provides robust and flexible control solutions, with promising applications in fields such as robotics, autonomous vehicles, and drone navigation. The design of autonomous systems has rapidly advanced, fueled by the need for intelligent, adaptive control in environments that are complex and unpredictable. Traditional control systems often lack the flexibility required to handle dynamic changes, making them less effective in highly variable conditions. Neuro-evolution, which combines neural networks with evolutionary algorithms, offers a promising solution by enabling autonomous systems to develop and refine control strategies on their own. This approach leverages evolutionary principles to optimize neural network architectures, producing control systems that adapt in real-time and improve through experience. Neuro-evolution has gained significant attention for applications in robotics, self- driving vehicles, and aerospace, where autonomous, robust control is essential. This paper explores the role of neuro-evolution in enhancing control design, focusing on its advantages over conventional methods and its potential to drive further innovations in autonomous systems.

Method of Motion Path Planning Based on a Deep Neural Network with Vector Input

The article deals with the problem of path planning in a two-dimensional environment based on deep learning neural networks. Deep neural networks require large amounts of data and place high computational requirements on computing tools. The lack of sufficient data leads to a decrease in the accuracy of the neural network, and high computational requirements at the learning stage limit the use of this technology in engineering practice. In this paper, the forms of representation of the environment for the input of a neural network are studied. Vector form allows to reduce the amount of information supplied to the input of a neural network, but it leads to the need to use more complex neural networks. In this article, a combined form of representation is proposed, including a vector global and local map layout. The vector part of the map includes the position of the robot, the position of the target point and a description of obstacles. The local raster map describes the area closest to the robot. Using numerical research, the effectiveness of this form of data representation for a precise neural network is shown, compared with the raster representation of the map. In this article, two structures of neural networks are studied, one of which uses 8 possible directions of movement, and the other uses 3 possible directions of movement. It is shown that when using 3 possible directions, the cycling of trajectories planned by the neural network is eliminated, which leads to an increase in accuracy.

Deep reinforcement learning for collision avoidance of autonomous ships in inland river

Due to the characteristics of large inertia, large time delay, and nonlinearity in ship motion, the maneuverability of ships is an important issue related to the safety of ship navigation. The manipulation of ships significantly affects the safety of ship navigation, and its importance will gradually increase with the development of autonomous ships. Collision accidents are common due to the complexity of inland waterways and density of the ships. Herein, an autonomous learning framework with deep reinforcement learning (DRL) is constructed for autonomous driving tasks. The state space, action space, reward function and neural network structure of DRL are designed based on the ship’s maneuvering characteristics and control requirements. A DDPG algorithm is then used to implement the controller. Finally, some representative route segments of the inland waterway is selected for simulation research based on the virtual simulation environment. The designed DRL controller can quickly converge from the training and learning process to meet the control requirements. By comparing with the previous experimental results, the effectiveness of this algorithm is verified. Therefore, this study provides a reference for future research on the collision avoidance technology of autonomous ships in inland rivers.

Convolutional neural networks uncover the dynamics of human visual memory representations over time.

The ability to accurately retrieve visual details of past events is a fundamental cognitive function relevant for daily life. While a visual stimulus contains an abundance of information, only some of it is later encoded into long-term memory representations. However, an ongoing challenge has been to isolate memory representations that integrate various visual features and uncover their dynamics over time. To address this question, we leveraged a novel combination of empirical and computational frameworks based on the hierarchal structure of convolutional neural networks and their correspondence to human visual processing. This enabled to reveal the contribution of different levels of visual representations to memory strength and their dynamics over time. Visual memory strength was measured with distractors selected based on their shared similarity to the target memory along low or high layers of the convolutional neural network hierarchy. The results show that visual working memory relies similarly on low and high-level visual representations. However, already after a few minutes and on to the next day, visual memory relies more strongly on high-level visual representations. These findings suggest that visual representations transform from a distributed to a stronger high-level conceptual representation, providing novel insights into the dynamics of visual memory over time.

Medical image recognition based on multilayer neural network

As a common accidental injury, accurate identification of burns and scalded injuries is of great significance to the development of treatment programs and prognosis assessment. Traditional identification methods are subjective and inaccurate, this paper proposes a burns and scalds grade identification technique based on multilayer neural network. The three-degree classification standard of burns and scalds is described, and the structure and principle of multilayer neural network are introduced, including the network structure composed of input layer, hidden layer and output layer, forward propagation, back propagation and training process. The implementation steps of burns and scald grade recognition technology based on multi-layer neural network are discussed in detail, such as data acquisition, image pre-processing, feature extraction, classifier design, model training and evaluation. Through experiments, the model is tested using a dataset containing 362 burns and scalds images, and the model achieves more than 90% accuracy on the test set with high accuracy and reliability. The multi-layer neural network model is used to classify the burn and scald data, which improves the diagnostic accuracy, shortens the delay time of the disease, reduces the pressure of professional doctors, and helps patients have a preliminary understanding of the degree of burn.

Scalable parallel photonic processing unit for various neural network accelerations

In recent years, integrated optical processing units (IOPUs) have demonstrated advantages in energy efficiency and computational speed for neural network inference applications. However, limited by optical integration technology, the practicality and versatility of IOPU face serious challenges. In this work, a scalable parallel photonic processing unit (SPPU) for various neural network accelerations based on high-speed phase modulation is proposed and implemented on a silicon-on-insulator platform, which supports parallel processing and can switch between multiple computational paradigms simply and without latency to infer different neural network structures, enabling to maximize the utility of on-chip components. The SPPU adopts a scalable and process-friendly architecture design, with a preeminent photonic-core energy efficiency of 0.83 TOPS/W, two to ten times higher than existing integrated solutions. In the proof-of-concept experiment, a convolutional neural network (CNN), a residual CNN, and a recurrent neural network (RNN) are all implemented on our photonic processor to handle multiple tasks of handwritten digit classification, signal modulation format recognition, and review emotion recognition. The SPPU achieves multi-task parallel processing capability, serving as a promising and attractive research route to maximize the utility of on-chip components under the constraints of integrated technology, which helps to make IOPU more practical and universal.

Investigating neural networks with groundwater flow equation loss

Physics-Informed Neural Networks (PINNs) are considered a powerful tool for solving partial differential equations (PDEs), particularly for the groundwater flow (GF) problem. In this paper, we investigate how the deep learning (DL) architecture, within the PINN framework, is connected to the ability to compute a more or less accurate numerical GF solution, so the link ‘PINN architecture - numerical performance’ is explored. Specifically, this paper explores the effect of various DL components, such as different activation functions and neural network structures, on the computational framework. Through numerical results and on the basis of some theoretical foundations of PINNs, this research aims to improve the explicability of PINNs to resolve, in this case, the one-dimensional GF equation. Moreover, our problem involves source terms described by a Dirac delta function, providing insights into the role of DL architecture in solving complex PDEs.

Split tensile strength of fiber-reinforced coral aggregate concrete: Deep learning model and experimental validation

Machine learning methods are increasingly becoming an important research field for solving the complex mechanical behavior of concrete materials. A reliable deep learning model was developed based on a dataset consisting of 319 experimental data points to predict the split tensile strength of fiber-reinforced coral aggregate concrete (FRCAC). The feature selection was carried out using correlation analysis, and the neural network structure was optimized by adjusting hyperparameters based on the 10-fold cross-validation (CV). Subsequently, six statistical indicators were employed to evaluate the accuracy and generalization ability of the model. The deep neural network (DNN) model demonstrates excellent performance, with R2, MAE, and MAPE of 0.98, 0.181 MPa, and 0.037 for the training dataset and 0.95, 0.286 MPa, and 0.074 for the test dataset, respectively. Additionally, 24 sets of concrete proportions are designed using the trained DNN model, and the results indicate that the absolute error and relative error between predicted and experimental values are less than 0.4 MPa and 6.8 %, respectively. The feature analysis based on SHapley Additive exPlanations (SHAP) reveals that volume fraction, curing age, water-binder ratio, and the tensile strength of fiber have the highest contribution in estimating the split tensile strength of FRCAC. This research can provide a reference for optimizing the design of FRCAC.

Research on trajectory planning based on deep reinforcement learning

Abstract For trajectory planning in a three-dimensional continuous space with multiple complex constraints, traditional optimal control methods face significant challenges, such as long computation times and poor dynamic performance. Using deep reinforcement learning algorithms for training agents in flight trajectory planning can effectively improve the dynamic performance of the planning algorithms. This paper focuses on training agents for UAV emergency landing and obstacle avoidance trajectory planning based on deep reinforcement learning algorithms. Within the reinforcement learning framework, we design the training environment, state space, action space, reward function, and neural network structure. To address common issues in reinforcement learning, such as poor generalization and slow training speed, we improve the training process using the concept of curriculum learning. Comparisons with pseudospectral methods demonstrate that deep reinforcement learning can effectively handle trajectory planning tasks in real flight environments, achieving trajectory quality close to that of pseudospectral methods while offering superior real-time performance.

A novel Taguchi-based approach for optimizing neural network architectures: application to elastic short fiber composites

This study presents an innovative application of the Taguchi design of experiment method to optimize the structure of an Artificial Neural Network (ANN) model for the prediction of elastic properties of short fiber reinforced composites. The main goal is to minimize the computational effort required for hyperparameter optimization while enhancing the prediction accuracy. By utilizing a robust experimental design framework, the structure of an ANN model is optimized. This approach involves identifying a combination of hyperparameters that provides optimal predictive accuracy with the fewest algorithmic runs, thereby significantly reducing the required computational effort. The results suggested that the Taguchi-based developed ANN model with three hidden layers, 20 neurons in each hidden layer, elu activation function, Adam optimizer, and a learning rate of 0.001 can predict the anisotropic elastic properties of short fiber reinforced composites with a prediction accuracy of 97.71%. Then, external validation of the proposed ANN model was done using experimental data, and differences of less than 10% were obtained, indicating an appropriate predictive performance of the proposed ANN algorithm. Our findings demonstrate that the Taguchi method not only streamlines the hyperparameter tuning process but also substantially improves the algorithm's performance. These results highlight the potential of the Taguchi method as a powerful tool for optimizing machine learning algorithms, especially in scenarios where computational resources are limited. The implications of this study are far-reaching, offering insights for future research in the optimization of different algorithms for improved accuracies and computational efficiencies.

An effective multi-scale interactive fusion network with hybrid Transformer and CNN for smoke image segmentation

Smoke has visually elusive appearances, especially in low-light conditions, so it is quite difficult to quickly and accurately detect smoke from images. To address these challenges, we design a dual-encoder structure of Transformer and Convolutional Neural Network (CNN) to propose an effective Multi-scale Interactive Fusion Network (MIFNet) for smoke image segmentation. To enhance the presentation of features, we propose a Local Feature Enhancement Propagation (LFEP) module to enhance spatial details. To optimize global and local features for efficient fusion, we integrate LFEP into the original Transformer to replace the traditional multi-head self-attention mechanism. Then, we propose a Multi-level Attention Coupled Module (MACM) to fuse Transformer and CNN features of the dual-encoder. MACM can flexibly focus on information interaction between different levels of two encoding paths. Finally, we design a Prior-guided Multi-scale Fusion Decoder (PMFD), which combines prior knowledge with a multi-scale feature fusion strategy to improve the performance of segmentation. Experimental results demonstrate that MIFNet substantially outperforms the state-of-the-art methods. MIFNet achieves a mean Intersection over Union (mIoU) of 81.6% on the synthetic smoke (SYN70K) dataset, and a remarkable accuracy of 98.3% on the forest smoke dataset.

Online Q-learning for stochastic linear systems with state and control dependent noise

For continuous-time (CT) systems that are characterized by stochastic differential equations (SDEs) with completely unknown dynamics parameters, a reinforcement learning (RL)-based optimal control framework is presented in this paper. To obtain the near-optimal control policy, an online Q-learning algorithm is proposed by learning the data sampled from the system state trajectory, while an integral reinforcement learning (IRL) approach is developed in stochastic situation so as to formulate the Q-learning iterative algorithm. In particular, an actor/critic neural network (NN) structure is applied in iteration, where a critic approximator is used for estimating the designed Q-function while an actor approximator is for estimating the optimal control policy online. To ensure the convergence of iteration, the tuning laws of two neural networks are designed, respectively, by a gradient descent scheme. Moreover, the mean-square stability of the closed-loop system is proved through rigorous analysis, and the convergence to optimal solution is guaranteed as well.

Deep learning aided underwater acoustic OFDM receivers: Model-driven or data-driven?

The Underwater Acoustic (UWA) channel is bandwidth-constrained and experiences doubly selective fading. It is challenging to acquire perfect channel knowledge for Orthogonal Frequency Division Multiplexing (OFDM) communications using a finite number of pilots. On the other hand, Deep Learning (DL) approaches have been very successful in wireless OFDM communications. However, whether they will work underwater is still a mystery. For the first time, this paper compares two categories of DL-based UWA OFDM receivers: the Data-Driven (DD) method, which performs as an end-to-end black box, and the Model-Driven (MD) method, also known as the model-based data-driven method, which combines DL and expert OFDM receiver knowledge. The encoder-decoder framework and Convolutional Neural Network (CNN) structure are employed to establish the DD receiver. On the other hand, an unfolding-based Minimum Mean Square Error (MMSE) structure is adopted for the MD receiver. We analyze the characteristics of different receivers by Monte Carlo simulations under diverse communications conditions and propose a strategy for selecting a proper receiver under different communication scenarios. Field trials in the pool and sea are also conducted to verify the feasibility and advantages of the DL receivers. It is observed that DL receivers perform better than conventional receivers in terms of bit error rate.

Wheat Flour Discrimination Using Two-Dimensional Correlation Spectroscopy and Deep Learning.

The continuous evolution of deep learning has garnered significant attention in spectroscopy. This study focuses on identifying wheat flour, presenting a more efficient and accurate method by combining two-dimensional correlation spectroscopy (2D-COS) and deep learning techniques. A data set of 316 near-infrared (NIR) spectral samples of four types of wheat flour was collected. By applying three disparate 2D-COS techniques, i.e., synchronous, asynchronous, and integrated, we crafted 948 2D-COS images. These images, obtained by transforming the original one-dimensional spectra into 2D representations, offer richer information for deep learning analysis. The study introduced an 18-layer residual network incorporating a convolutional attention mechanism, specifically tailored for the 2D-COS analysis of wheat flour, aimed at enhancing the model's discriminative capabilities by refining the residual neural network's structure. Achieving an unprecedented recognition accuracy of 100% through methodical optimization and rigorous training on the synchronous 2D-COS data set of wheat flour, the proposed model is a testament to the efficacy of deep learning in spectroscopic analysis. To further exhibit the confluence of 2D-COS with deep learning, t-distributed stochastic neighbor embedding was employed to visualize the distinctive 2D-COS features within the deep learning architecture. Additionally, the model's performance was juxtaposed with prevailing NIR spectral recognition methods, including random forest, gradient boosting decision tree, and artificial neural network. This comparison cemented the proposed approach's superiority in wheat flour categorization. The findings of this study not only introduce a novel and efficient solution for wheat flour quality analysis but also underscore the significant potential of deep learning techniques in spectroscopy applications.

From CNN to GAN: The evolution of deep learning techniques in image processing

In recent years, the technique for image processing has developed to improve the visual effects of images. This paper introduces ways to express images and basic operations. For the rest part of this paper, we elaborate on the current techniques for image processing including CNNs, RNNs, and GANs. each of them has different benefits and drawbacks. This paper also have further introduction about the theory and structure of neural networks above, and the reason for creating GANs, which is two adversarial network to improve the visual effects.

Numerical investigation of the effective receptive field and its relationship with convolutional kernels and layers in convolutional neural network

The receptive field (RF) plays a crucial role in convolutional neural networks (CNNs) because it determines the amount of input information that each neuron in a CNN can perceive, which directly affects the feature extraction ability. As the number of convolutional layers in CNNs increases, there is a decay of the RF according to the two-dimensional Gaussian distribution. Thus, an effective receptive field (ERF) can be used to characterize the available part of the RF. The ERF is calculated by the kernel size and layer number within the neural network architecture. Currently, ERF calculation methods are typically applied to single-channel input data that are both independent and identically distributed. However, such methods may result in a loss of effective information if they are applied to more general (i.e., multi-channel) datasets. Therefore, we proposed a multi-channel ERF calculation method. By conducting a series of numerical experiments, we determined the relationship between the ERF and the convolutional kernel size in conjunction with the layer number. To validate the new method, we used the recently published global wave surrogate model for climate simulation (GWSM4C) and its accompanying dataset. According to the newly established relationship, we refined the kernel size and layer number in each neural network of the GWSM4C to produce the same ERF but lower RF attenuation rates than those of the original version. By visualizing the gradient map at several points in West African and East Pacific areas, the high gradient value regions confirmed the known swell sources, which indicated effective feature extraction in these areas. Furthermore, the new version of the GWSM4C yielded better prediction accuracy for significant wave height in global swell pools. The root mean square errors in the West African and East Pacific regions reduced from approximately 0.3 m, in the original model to about 0.15 m, in the new model. Moreover, these improvements were attributed to the higher efficiency of the newly modified neural network structure that allows the inclusion of more historical winds while maintaining acceptable computational consumption.

Comparison of transformer, LSTM and coupled algorithms for soil moisture prediction in shallow-groundwater-level areas with interpretability analysis

Accurate quantification of soil moisture is essential for understanding water and energy exchanges between the atmosphere and the Earth’s surface, as well as for agricultural applications. Predicting soil moisture content is vital for efficient water management, irrigation scheduling, and drought monitoring. Traditional forecasting methods, such as numerical regression models, often struggle due to various influencing factors and poor observation data quality. In contrast, deep learning algorithms, particularly recurrent and convolutional neural networks, show promise in predicting nonlinear data like soil moisture. This study focuses on shallow groundwater regions, using groundwater levels and meteorological data as features while coupling the Transformer model with other neural network structures. We investigate the potential of attention-based neural networks for soil moisture time series prediction. Our findings demonstrate that the Transformer model achieves an average R2 of 0.523 across different time lags, outperforming the LSTM model with an R2 of 0.485. The introduction of LSTM enhances the Transformer’s stability in handling temporal changes. Additionally, we verified the importance of groundwater for soil moisture prediction. This study introduces new methods for soil moisture prediction and offers new insights and recommendations for the development of artificial intelligence technology for soil moisture prediction.