Structure Of Neural Network Model Research Articles

A novel Taguchi-based approach for optimizing neural network architectures: application to elastic short fiber composites

This study presents an innovative application of the Taguchi design of experiment method to optimize the structure of an Artificial Neural Network (ANN) model for the prediction of elastic properties of short fiber reinforced composites. The main goal is to minimize the computational effort required for hyperparameter optimization while enhancing the prediction accuracy. By utilizing a robust experimental design framework, the structure of an ANN model is optimized. This approach involves identifying a combination of hyperparameters that provides optimal predictive accuracy with the fewest algorithmic runs, thereby significantly reducing the required computational effort. The results suggested that the Taguchi-based developed ANN model with three hidden layers, 20 neurons in each hidden layer, elu activation function, Adam optimizer, and a learning rate of 0.001 can predict the anisotropic elastic properties of short fiber reinforced composites with a prediction accuracy of 97.71%. Then, external validation of the proposed ANN model was done using experimental data, and differences of less than 10% were obtained, indicating an appropriate predictive performance of the proposed ANN algorithm. Our findings demonstrate that the Taguchi method not only streamlines the hyperparameter tuning process but also substantially improves the algorithm's performance. These results highlight the potential of the Taguchi method as a powerful tool for optimizing machine learning algorithms, especially in scenarios where computational resources are limited. The implications of this study are far-reaching, offering insights for future research in the optimization of different algorithms for improved accuracies and computational efficiencies.

Review of Convolutional Neural Network Models and Image Classification

With the arrival of the era of big data and the improvement of computing power, deep learning has swept the world. Traditional image classification methods are difficult to deal with the huge image data, and cannot meet the requirements of people on the accuracy and speed of image classification, the image classification method based on convolutional neural network breaks through the bottleneck of the traditional image classification method, and becomes the mainstream algorithm of image classification, how to effectively use convolutional neural network to classify images has become a hot spot of research in the field of computer vision at home and abroad. In this paper, we review the research background, significance and current research status of convolutional neural network model and image classification, study two image classification methods based on ResNet and ShuffleNet, and provide a comprehensive review of the construction methods and characteristics of the two deep convolutional neural network model structures, and finally compare and analyse the performance of the two classification models.

Discriminant Decision Making of Cardiovascular Diseases Based on Cloud-Based Convolutional Attention Network

Cardiovascular diseases (CVDs) have become the number one killer affecting human health. In order to reduce the burden of medical workers, facilitate government screening of the population and enable patients to conduct their own health status checks, there is an urgent need for a complementary diagnostic system to predict the occurrence of CVD. In this study, a new cloud-based convolutional attention network (C-CAN) model is proposed for the discriminant decision making of CVD. In this model, the indicator data for discriminant decision making of CVD are trained using an improved one-dimensional convolutional neural network (1D CNN) model structure based on the correlation of factors influencing CVD given by decision-making trial and evaluation laboratory (DEMATEL) and cloud models. This 1D CNN model consists of a convolutional pooling module, an attention module and a fully connected module. The cloud model is used to process the original data based on the discriminating opinion of experts, so as to select the important factors that affect CVD. The attention mechanism is effective in augmenting attention to the essential elements of the data and reducing attention to the less important features. Both have similarities in that they are effective in augmenting the important features in the data and combine with each other to achieve better results. Moreover, the C-CAN is compared with decision tree (DT), [Formula: see text]-nearest neighbors (KNN), random forests (RF) and normal CNN according to the CVD dataset from the Kaggle platform. The results show that the classification accuracy, precision, recall and F1 value of C-CAN are all higher than that of all compared models. Further, the proposed model is further externally validated using other imbalanced datasets, and the results indicate that C-CAN has good resilience for imbalanced data. Our findings suggest that C-CAN represents a promising new approach that may somehow address the challenges associated with deep learning (DL) in the medical field.

Ultrasensitive Online NO Sensor Based on a Distributed Parallel Self-Regulating Neural Network and Ultraviolet Differential Optical Absorption Spectroscopy for Exhaled Breath Diagnosis.

The concentration of fractional exhaled nitric oxide (FeNO) is closely related to human respiratory inflammation, and the detection of its concentration plays a key role in aiding diagnosing inflammatory airway diseases. In this paper, we report a gas sensor system based on a distributed parallel self-regulating neural network (DPSRNN) model combined with ultraviolet differential optical absorption spectroscopy for detecting ppb-level FeNO concentrations. The noise signals in the spectrum are eliminated by discrete wavelet transform. The DPSRNN model is then built based on the separated multipeak characteristic absorption structure of the UV absorption spectrum of NO. Furthermore, a distributed parallel network structure is built based on each absorption feature region, which is given self-regulating weights and finally trained by a unified model structure. The final self-regulating weights obtained by the model indicate that each absorption feature region contributes a different weight to the concentration prediction. Compared with the regular convolutional neural network model structure, the proposed model has better performance by considering the effect of separated characteristic absorptions in the spectrum on the concentration and breaking the habit of bringing the spectrum as a whole into the model training in previous related studies. Lab-based results show that the sensor system can stably achieve high-precision detection of NO (2.59-750.66 ppb) with a mean absolute error of 0.17 ppb and a measurement accuracy of 0.84%, which is the best result to date. More interestingly, the proposed sensor system is capable of achieving high-precision online detection of FeNO, as confirmed by the exhaled breath analysis.

Research on the application of neural networks in macroeconomic forecasting and strategic management decision-making

Abstract Traditional macroeconomic forecasting models have some limitations in utilizing large-scale variables, screening appropriate variables, and forecasting accuracy. Based on the theories of economic fluctuation and New Keynes, this paper uses monetary policy, investment behavior, and consumption behavior as predictors of macroeconomic fluctuation. Then, it explores the principles and steps of the BP network applied to forecasting, the design of BP neural network structure, the selection of training algorithm and training parameters, and finally establishes a reasonable BP network structure model. Finally, using the prediction of GDP growth rate as an example, an empirical comparative analysis of model prediction accuracy is carried out. The empirical results of the model in the training set and the test set show that the structure of the BP neural network model is 3-48-1, and the model obtains the optimal learning error (6.207×10−5) at 500 iterations. In the experimental set, the root-mean-square prediction error between the output predicted value and the actual value is less than ±0.2 . The model accurately predicts the GDP growth rate, which can provide a theoretical basis for proposing macroeconomic strategic management decisions.

Innovative Research on Industrial Upgrading and Transformation Path of Regional Economy in the Perspective of Digital Economy

Abstract This paper utilizes the digital economy, regional spatial distribution digital mining technology, establishes the indicator system, assigns values to the indicators, and applies the indicator system to evaluate the development of industrial structure upgrading and transformation of the regional economy. According to the research results of the basic structure and algorithm of the self-organized neural network model, combined with the classification layer smoothing algorithm, the regional spatial clustering model is constructed. Using the constructed model, the regional economic spatial dynamic data is mined, and the path of industrial transformation and upgrading in the regional spatial area is dynamically studied. Furthermore, using the CA algorithm, the evolution law of land use structure in the region is predicted, and the development of the regional economy is inferred. At the same time, through the spatial autocorrelation regional prediction model, the relevant regional spatial expansion prediction method is proposed, the three research hypotheses of this paper are made, and the constructed model is utilized to innovate the exploration of industrial transformation and upgrading paths from the spatial dimension. Using empirical research methods, descriptive statistics, and mediating effect heterogeneity analysis are carried out to analyze the factors that influence regional economic variables in place A. The empirical analysis shows that the fluctuation range interval of industrial structure is between [0.811,10.2805], which brings a large positive impact on the regional economy.2017 national digital economic growth rate changed from -0.17446 to 0.17143 to realize a positive growth rate, and at the same time, there is a heterogeneity in the development of the east-west and central regions of A. The hypotheses presented in this paper are all valid.

Mixed pesticide recognition based on three-dimensional fluorescence spectroscopy and a convolutional neural network.

Kasugamycin, spinosad, and lambda-cyhalothrin are common organic pesticides that are widely used to control and prevent diseases and pests in fruits and vegetables. However, the unreasonable use of pesticides will cause great harm to the natural environment and human health. Pesticides often exist in the form of mixtures in nature. Establishing recognition models for mixed pesticides in large-scale sample testing can provide guidance for further precise analysis and reduce resource waste and time. Therefore, finding a fast and effective identification method for mixed pesticides is of great significance. This paper applies three-dimensional fluorescence spectroscopy to detect mixed pesticides and introduces a convolutional neural network (CNN) model structure based on an improved LeNet-5 to classify mixed pesticides. The input part of the model corresponds to fluorescence spectrum data at excitation wavelengths of 250-306nm and emission wavelengths of 300-450nm, and the mixed pesticides are divided into three categories. The research results show that when the learning rate is set to 1 and the number of iterations is 300, the CNN classification model has ideal performance (with a recognition accuracy of 100%) and is superior to the performance of the support vector machine method. This paper provides a certain methodological basis for the rapid identification of mixed pesticides.

Applying multi-layer perceptron and ResNet for handwritten digits recognition

The task of handwritten digit recognition is to recognizing the handwritten digits from pictures. Applying machine learning based models to automatically perform handwritten digit recognition task can significantly improve efficiency. This paper applies two machine learning based models, including multi-layer perceptron and residual neural network, for such a task. Firstly, this paper introduces the basic concept of the simple multi-layer perceptron model and then presents the structure of the residual neural network model. Subsequently, such two models are trained on the MNIST corpus, one of the classical dataset for the handwritten digit recognition task. The data pre-processing, like the splitting of training and test set, is described. Also, the processes of testing and training of the two models are presented. According to the experiments on the test set of MNIST, it is observed that the residual neural network can achieve better performance where the accuracy score is 99.240%, while the accuracy score of the multi-layer perceptron model is 97.260%.

TURBO: The Swiss Knife of Auto-Encoders.

We present a novel information-theoretic framework, termed as TURBO, designed to systematically analyse and generalise auto-encoding methods. We start by examining the principles of information bottleneck and bottleneck-based networks in the auto-encoding setting and identifying their inherent limitations, which become more prominent for data with multiple relevant, physics-related representations. The TURBO framework is then introduced, providing a comprehensive derivation of its core concept consisting of the maximisation of mutual information between various data representations expressed in two directions reflecting the information flows. We illustrate that numerous prevalent neural network models are encompassed within this framework. The paper underscores the insufficiency of the information bottleneck concept in elucidating all such models, thereby establishing TURBO as a preferable theoretical reference. The introduction of TURBO contributes to a richer understanding of data representation and the structure of neural network models, enabling more efficient and versatile applications.

A Study on Improving Musical Expression through Optimization of Artificial Neural Network Model Structure

A Study on Improving Musical Expression through Optimization of Artificial Neural Network Model Structure

Diagnosis of Loose Core Fault in Saturable Reactor of Thyristor Valve Based on Vibration Signal Time-Frequency Analysis and CNN

The saturable reactor is one of the core components of the thyristor valve, which plays a role in suppressing current changes and balancing voltage distribution. Due to long-term operation under complex working conditions, the vibration of the reactor can cause the core to loosen. To detect faults in time, a fault diagnosis method is proposed for loose cores in saturable reactors of thyristor valves based on time-frequency analysis of vibration signals and convolutional neural networks. The two-dimensional time-frequency spectrum of the vibration signal is obtained using the short-time Fourier transform method. A convolutional neural network model structure is designed to extract time-frequency spectrum features and diagnose different levels of core looseness faults through learning and training. An experiment was carried out to collect core fault information under variable working conditions. The effectiveness of the method proposed in this paper was verified based on experimental data validation. Through comparative experiments, it was found that the proposed method is superior to existing fault diagnosis methods.

VFSN: a ResNet Multi-scale Fusion Network for Metal Defect Detection

Defect detection is an imperative step in ensuring the quality of steel products. To overcome the problem of low accuracy and poor detection of subtle features in current detection methods, a revised neural network model structure has been proposed. VFSN’s main convolutional layer structure is VGG with residuals, where two convolutional blocks are designed. It’s connected to a spatial pyramid pool (SPP) after reducing the aliasing effects and detecting defect features at various scales by constructing multibranch inputs and multilevel feature overlays and fusions. Research on the open-source data set NEU-DET has shown high-accuracy recognition, with the recognition accuracy rate reaching 75.1% and an average accuracy rate improving by 2.7%. The F1 score improved by 1.3%. There’s a significant improvement in detecting small defects using the proposed network structure.

A new combined approach of neural-metaheuristic algorithms for predicting and appraisal of landslide susceptibility mapping.

In this research, to predict landslide susceptibility mapping (LSM), we have studied and optimized an artificial neural network (ANN) by utilizing the backtracking search algorithm (BSA) as well as the Cuckoo optimization algorithm (COA). Multiple research studies have shown that ANN-based techniques can be used to figure out the LSM. Still, ANN computing models have big problems, like slow system learning and getting stuck in their local minimums. Optimization strategies may improve ANN performance results. Existing uses of the BSA and COA models in ANN training have not been used to map landslides, nor have the best ways to set up networks or other factors that affect this problem been examined. Consequently, the present research focuses on predicting landslide susceptibility for hazardous mapping using hybrid BSA and COA-based ANN algorithms (BSA-MLP and COA). A large data set was provided from an area in the province of Kurdistan, west of Iran, to provide training and testing datasets for the algorithms. All of the BSA and COA algorithms' parameters and weights, for instance, were fine-tuned to make the utmost accurate maps of landslide risk. The input dataset consists of elevation, slope angle, slope orientation, NDVI, fault tolerance, profile curvature, plan curvature, distance to the river, rainfall, far from the road, SPI, STI, TRI, TWI, land use, and geology; the output is landslide susceptibility value. In the testing phase, the AUC rose significantly from 0.701 to 0.864 for BSA-MLP and 0.738 to 0.822 for COA-MLP after using the abovementioned techniques. We have used the area under the curve (AUC) to evaluate how well the probabilistic models worked. In addition, the computed AUCs for the BSA-MLP available databases and the actual AUCs were 0.864, 0.857, 0.833, 0.778, 0.777, 0.769, 0.763, 0.758, 0.727, and 0.701 and 0.822, 0.808, 0.807, 0.805, 0.804, 0.777, and 0.769 for the COA-MLP combination. The integrated models can produce beneficial results for this area of research. The results suggest that the BSA-ANN model is better than the COA-ANN in optimizing an artificial neural network model's structure and computational parameters. The collected landslide susceptibility maps are significant for figuring out how dangerous landslides are in the studied area.

An intelligent model for predicting the dressed weight of pigs using morphometric measurements.

Determining the slaughter weight of pigs is crucial to the profitability of swine production farms. Unfortunately, in developing countries, the basic infrastructure for weight measurement may not always be available, affecting farmers' income. This study presents a machine learning-based approach to determine the dressed weight of pigs using four morphometric dimensions: paunch girth (PG), heart girth (HG), body length and wither height, which can be measured in situ. Different neural network model structures were constructed taking LM, GDX and BR training algorithms, tansigmoid/logsigmoid hidden layer transfer functions and 5-30 hidden layer neurons (HLNs). Results showed that LM training algorithm with logsigmoidal transfer function and 20 HLNs resulted in 99.8% accuracy in determining the pigdressed weight. Further, the number of morphometric parameters as inputs was gradually reduced and it was found that 99% accuracy can still be achieved using just PG and HG, thereby reducing the measurement time.

Research of intelligent reasoning system of Arabidopsis thaliana phenotype based on automated multi-task machine learning.

Traditional machine learning in plant phenotyping research requires the assistance of professional data scientists and domain experts to adjust the structure and hy-perparameters tuning of neural network models with much human intervention, making the model training and deployment ineffective. In this paper, the automated machine learning method is researched to construct a multi-task learning model for Arabidopsis thaliana genotype classification, leaf number, and leaf area regression tasks. The experimental results show that the genotype classification task's accuracy and recall achieved 98.78%, precision reached 98.83%, and classification F 1 value reached 98.79%, as well as the R 2 of leaf number regression task and leaf area regression task reached 0.9925 and 0.9997 respectively. The experimental results demonstrated that the multi-task automated machine learning model can combine the benefits of multi-task learning and automated machine learning, which achieved more bias information from related tasks and improved the overall classification and prediction effect. Additionally, the model can be created automatically and has a high degree of generalization for better phenotype reasoning. In addition, the trained model and system can be deployed on cloud platforms for convenient application.

Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes

The human-robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human's state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applications.

Physics-Informed Hybrid GRU Neural Networks for MPC Prediction

This work describes a novel Physics-Informed Hybrid Neural Network (PIHNN) model structure based on the Gated Recurrent Unit (GRU) neural network that combines the first principle and black-box data-driven approaches. We recommend the presented modeling method when the measurement of process variables is possible, but only in some limited neighborhood of the operating points, and the available first principle model describing the process is generally correct but not accurate. We discuss three methods of data fusion. Two of them utilize the fuzzy approach, while the third one relies on an additional neural network of the Multi-Layer Perceptron (MLP) type. The first method assumes a simplified data fusion block, while the next two use machine learning to minimize the overall model error. The efficiency of the presented approach and data fusion techniques is demonstrated for a benchmark chemical process (the pH reactor). Finally, we consider applying the PIHNN model in Model Predictive Control (MPC). It gives much better control quality than the MPC controller relying on the entirely black-box GRU model.

Convolutional Neural Network Structure Optimization based on Network Pruning

Recently, the number of layers of neural network model is deeper and deeper, the number of parameters is more and more, and the calculation scale is also larger and larger. This improves the use conditions of some excellent models, which is not conducive to the wide application of deep learning methods in more fields. In view of this trend of increasing the size of neural network models, in this paper, we optimize the structure of a convolutional neural network model for image super-resolution, which reduces the size of the model. The model structure optimization method we use is network pruning, which simplifies the number of layers and parameters of the model, improves the effect of the model and reduces the computational consumption of the model. The key insight of network pruning is to remove the relatively redundant and unimportant parts of the network to make the original network sparser and more streamlined. And the simplified model can keep the original performance. The original model used a cascade structure for multiple sampling, resulting in the increase of the scale of the neural network. By removing the redundant sampling structure and retaining only one sampling process, the number of layers of the model is reduced to 1/3 of the original. Under the same data set (BSD300) training, the PSNR (evaluation index of model effect) of the model is improved from 24.471 db to 24.490 db, and the training time is reduced by 13.8% of the original.

Study on dynamic tension estimation for the underwater soft yoke mooring system with LSTM-AM neural network

It is significant to estimate the mooring tension for the safety, operation and maintenance of single point mooring system of floating production, storage and offloading unit (FPSO). In this study, a neural network model named long-short term memory combined with attention mechanism (LSTM-AM) is adopted to estimate the mooring legs dynamic tension of an underwater soft yoke mooring system in time domain. The mooring legs tension and FPSO motions are set as the training features in the LSTM-AM neural network, together with the corresponding first-order and second-order central moments. The training data collection is implemented with the numerical hydrodynamic coupled analysis of FPSO and an underwater soft yoke mooring system. Different variables are studied to determine the optimal structure of the LSTM-AM neural network model, including the time window, the layer numbers, the neural units and the optimizer. Through cases studied, it is proved that the LSTM-AM neural network model is suitable to estimate the mooring legs tension of FPSO and the underwater soft yoke mooring system in different sea states.

Estimation of Final Product Concentration in Metalic Ores Using Convolutional Neural Networks

Although artificial neural networks are widely used in various fields, including mining and mineral processing, they can be problematic for appropriately choosing the model architecture and parameters. In this article, we describe a procedure for the optimization of the structure of a convolutional neural network model developed for the purposes of metallic ore pre-concentration. The developed model takes as an input two-band X-ray scans of ore grains, and for each scan two values corresponding to concentrations of zinc and lead are returned by the model. The whole process of sample preparation and data augmentation, optimization of the model hyperparameters and training of selected models is described. The ten best models were trained ten times each in order to select the best possible one. We were able to achieve a Pearson coefficient of R = 0.944 for the best model. The detailed results of this model are shown, and finally, its applicability and limitations in real-world scenarios are discussed.