Number Of Neural Network Layers Research Articles

A deep neural network model with GCN and 3D convolutional network for short‐term metro passenger flow forecasting

AbstractRail transit has many advantages, such as large passenger capacity, convenience, safety, and environmental protection, making it the preferred travel mode for most passengers. Deep learning has become an effective method for short‐term rail passenger flow prediction. A deep learning model which combines a graph convolutional network and a three‐dimensional Convolutional Neural Network improved by a residual module and an attention mechanism (ARConv‐CGN) is proposed. First, historical passenger inflow and outflow data are aggregated into three patterns: recent pattern, daily pattern, and weekly pattern separately. The GCN is applied to capture the spatiotemporal and topological information of passenger flows in each pattern. Second, a three‐dimensional convolutional neural network is used to deeply integrate three patterns of passenger flow information. Additionally, a residual module is used to increase the number of neural network layers and prevent the gradient of the deep neural network from disappearing. Finally, an attention mechanism is also introduced to adjust the importance of passenger flow in different pattern, so that improving the performance of the model. Training with passenger flow data from automatic fare collection on weekdays in Beijing and Xiamen provides a good demonstration of the superiority of ARConv‐CGN in metro passenger flow forecasting.

Heterogeneous Defect Prediction Algorithm Combined With Federated Sparse Compression

Heterogeneous defect prediction (HDP) constructs a defect prediction model through the source project to realize the defect tendency prediction of the target project. HDP based on federated learning can combine multi-party defect data to improve the defect prediction effect under the condition of protecting privacy. However, with the increase of task complexity and model performance requirements, the number of neural network layers gradually deepens, and the model parameters also increase. increase accordingly. In a federated learning scenario with multiple model parameters, limited communication bandwidth, and multiple clients, the amount of data received by the server is very large, causing huge communication pressure and seriously affecting the overall training efficiency. In order to reduce the communication cost, this paper proposes a federated sparse compression (FSC) algorithm. First, in order to improve the generalization performance of the model, each participant performs local training based on CapsNet, and achieves better prediction performance by calculating the relative position information of feature combinations. Then the gradient parameters are encrypted based on differential privacy to ensure data security. In order to reduce the amount of communication bits, the encrypted model parameters are subjected to sparse binary compression, the model training is converted from dense calculation to sparse calculation, and Golomb encoding is selected to be sent to the server for aggregation. Finally, the server side decodes the received data, performs sparse binary compression and encoding and sends it back to each participant. Experiments on nine projects in three public databases (Relink, NASA, and AEEEM) show that FSC can effectively reduce the amount of communication bits.

VARIANTS OF NEURAL NETWORKS: A REVIEW

Machine learning (ML) techniques are part of artificial intelligence. ML involves imitating human behavior in solving different problems, such as object detection, text handwriting recognition, and image classification. Several techniques can be used in machine learning, such as Neural Networks (NN). The expansion in information technology enables researchers to collect large amounts of various data types. The challenging issue is to uncover neural network parameters suitable for object detection problems. Therefore, this paper presents a literature review of the latest proposed and developed components in neural network techniques to cope with different sizes and data types. A brief discussion is also introduced to demonstrate the different types of neural network parameters, such as activation functions, loss functions, and regularization methods. Moreover, this paper also uncovers parameter optimization methods and hyperparameters of the model, such as weight, the learning rate, and the number of iterations. From the literature, it is notable that choosing the activation function, loss function, number of neural network layers, and data size is the major factor affecting NN performance. Additionally, utilizing deep learning NN resulted in a significant improvement in model performance for a variety of issues, which became the researcher's attention.

Analysis of deep learning models for network anomaly detection in Internet of Things

Introduction: The article discusses the problem of choosing deep learning models for detecting anomalies in Internet of Things (IoT) network traffic. This problem is associated with the necessity to analyze a large number of security events in order to identify the abnormal behavior of smart devices. A powerful technology for analyzing such data is machine learning and, in particular, deep learning. Purpose: Development of recommendations for the selection of deep learning models for anomaly detection in IoT network traffic. Results: The main results of the research are comparative analysis of deep learning models, and recommendations on the use of deep learning models for anomaly detection in IoT network traffic. Multilayer perceptron, convolutional neural network, recurrent neural network, long short-term memory, gated recurrent units, and combined convolutional-recurrent neural network were considered the basic deep learning models. Additionally, the authors analyzed the following traditional machine learning models: naive Bayesian classifier, support vector machines, logistic regression, k-nearest neighbors, boosting, and random forest. The following metrics were used as indicators of anomaly detection efficiency: accuracy, precision, recall, and F-measure, as well as the time spent on training the model. The constructed models demonstrated a higher accuracy rate for anomaly detection in large heterogeneous traffic typical for IoT, as compared to conventional machine learning methods. The authors found that with an increase in the number of neural network layers, the completeness of detecting anomalous connections rises. This has a positive effect on the recognition of unknown anomalies, but increases the number of false positives. In some cases, preparing traditional machine learning models takes less time. This is due to the fact that the application of deep learning methods requires more resources and computing power. Practical relevance: The results obtained can be used to build systems for network anomaly detection in Internet of Things traffic.

Research on the online forecasting of algal kinetics based on time-series data and LSTM neural network:Taking Three Gorges Reservoir as an example

三峡水库在不同水位调控期支流回水区末端水深变化幅度较大，加之复杂水动力变化产生的生境异质性，塑造出有别于浅水湖泊的水华暴发特征.本研究基于库区4条支流——香溪河、澎溪河、大宁河及草堂河部署的自动监测数据，利用小波变换（WT）和长短期记忆网络（LSTM）构建藻类时序变化预测模型，并探讨神经网络层数、每层隐藏神经元数、时间步长数等关键参数的最优组合.结果表明：WT-LSTM模型可有效预测在线获取的叶绿素a浓度变化，模型在4条支流的均方根误差（RMSE）为0.049~0.221 μg/L，平均相对误差（MRE）为0.43%~1.12%；预测结果揭示深度神经网络方法可有效地提取在线藻类时序数据特征，而相较于深度置信网络（DBN），LSTM在4条支流叶绿素a预测的平均RMSE和MRE分别降低了9.20%和3.06%；在线监测数据的小波降噪并未影响叶绿素a的变化趋势，且WT-LSTM模型对叶绿素a预测效果显著提升于WT-DBN，平均RMSE和MRE分别降低了51.72%和59.24%；通过设置不同时间步长的预测实验，证实24 h内模型精度会随着预测步长的增加而降低，但模型平均相对误差可保持在13%以内，且对区间内叶绿素a极大值的预测精度要优于其平均值.本研究为水华预测上耦合在线监测与深度学习提供了研究范例，通过4个站点数据的交叉验证实验，亦证实具有统计学关联性的不同空间数据合并后可延展时序模型的学习样本，增强模型在实际应用中的稳健性.;The water depth at the end of the tributary backwater area in the Three Gorges Reservoir (TGR) varies greatly, coupled with the habitat heterogeneity caused by complex hydrodynamic changes, shaping the characteristics of water bloom outbreaks that are different from shallow lakes. Based on the online monitoring system deployed in four tributaries of the TGR, this study uses wavelet transform (WT) and long short-term memory network (LSTM) to build the time-series forecasting model of algal kinetics, and discusses the optimal combination of key parameters such as the number of neural network layers, the number of hidden neurons in each layer, and the time steps. The results show that: WT-LSTM model can effectively predict the change of chlorophyll-a concentrations in four tributaries, the root mean square error (RMSE) is 0.049-0.221 μg/L, and the mean relative error (MRE) is 0.43%-1.12%. This study confirms that the deep learning model can learn inherent patterns of high-frequency monitoring data, and the mean RMSE and MRE in the four tributaries are decreased by 9.20% and 3.06%, respectively. After online data processing with the wavelet transform, the prediction performance of WT-LSTM is also better than WT-DBN, and the mean RMSE and MRE decreased by 51.72% and 59.24%, respectively. Comparison experiments with different time steps confirm that the accuracy of the model decreases with the improvement of the prediction time. While the mean relative error of the prediction task within 24 hours is less than 13%, and the prediction ability of the model to chlorophyll-a concentration of the interval maximum is better than the average. This study provides a research example for the combination of automatic monitoring data and deep neural network models to forecast harmful algal blooms. Through cross-validation experiments on data from four sites, it is confirmed that statistically relevant data can be extended for model training and testing samples, enhancing the stability of machine learning models in practical applications.

A residual neural network based method for the classification of tobacco cultivation regions using near-infrared spectroscopy sensors

Near-infrared (NIR) spectroscopy techniques have been widely used to classify tobacco cultivation regions. NIR spectroscopy of tobacco leaves involves a large number of correlated features, so it is difficult to find the connection between spectral data and tobacco cultivation regions. This paper proposes a novel classification model of tobacco cultivation regions that integrates residual network (ResNet) and NIR spectroscopy techniques. As the number of neural network layers increases, the network may have issues such as network degradation, gradient disappearance, and the reduction of sample recognition rate. The proposed model applies a residual module to a neural network which effectively solves or alleviates the vanishing gradient issues caused by the increase of network depth. This paper also adds balance and suppression factors to the loss function to solve the issues caused by uneven sizes of tobacco samples collected from different cultivation regions in the training process. In the proposed method, tobacco samples are marked as internal and external samples respectively during the training process. Internal samples are collected from the corresponding cultivation regions in the north, northeast, and northwest of Guizhou Province, China. External samples are collected from other cultivation regions. The weight distributions of internal and external samples can be adjusted by experimental results to improve the identification accuracy of the proposed solution. The size of training samples determines the generalization ability of the network and affects the experimental results. A parametric rectified linear unit (PReLU) function is integrated into the network, in which the parameters of a linear unit are adaptively learned to further improve the identification accuracy of the proposed solution. Compared with current mainstream methods, the experimental results confirm that the proposed model is superior in accurately identifying different cultivation regions of tobacco leaves.

A Novel and Effective Image Super-Resolution Reconstruction Technique via Fast Global and Local Residual Learning Model

The principle of image super-resolution reconstruction (SR) is to pass one or more low-resolution (LR) images through information processing technology to obtain the final high-resolution (HR) image. Convolutional neural networks (CNN) have achieved better results than traditional methods in the process of an image super-resolution reconstruction. However, if the number of neural network layers is increased blindly, it will cause a significant increase in the amount of calculation, increase the difficulty of training the network, and cause the loss of image details. Therefore, in this paper, we use a novel and effective image super-resolution reconstruction technique via fast global and local residual learning model (FGRLR). The principle is to directly train a low-resolution small image on a neural network without enlarging it. This will effectively reduce the amount of calculation. In addition, the stacked local residual block (LRB) structure is used for non-linear mapping, which can effectively overcome the problem of image degradation. After extracting features, use 1 × 1 convolution to perform dimensional compression, and expand the dimensions after non-linear mapping, which can reduce the calculation amount of the model. In the reconstruction layer, deconvolution is used to enlarge the image to the required size. This also reduces the number of parameters. We use skip connections to use low-resolution information for reconstructing high-resolution images. Experimental results show that the algorithm can effectively shorten the running time without affecting the quality of image restoration.

Artificial Intelligent techniques for Flow Bottom Hole Pressure Prediction

This paper proposes Radial Basis and Feed-forward Neural Networks to predict the flowing bottom-hole pressure in vertical oil wells. The developed neural network models rely on a large amount of available historical data measured from actual different oil fields. The unsurpassed number of neural network layers, the number of neurons per layer, and the number of trained samples required to get an outstanding performance have been obtained. Intensive experiments have been conducted and the standard statistical analysis has been accomplished on the achieved results to validate the models’ prediction accuracy. For the sake of qualitative comparison, empirical modes have been developed. The obtained results show that the proposed Feed-Forward Neural Network models outperforms and capable of estimating the FBHPaccurately.The paper showed that the accuracy of FBHP estimation using FFNN with two hidden layer model is better than FFNN with single hidden layer model, Radial Basis neural network, and the empirical model in terms of data set used, mean square error, and the correlation coefficient error. With best results of 1.4 root mean square error (RMSE), 1.4 standard deviation of relative error (STD), correlation coefficient (R) 1.0 and 99.4% of the test data sets achieved less than 5% error. The minimum sufficient number of data sets used in training ANN model can be low as 375 sets only to give a 3.4 RMES and 97% of the test data achieved 90% accuracy.

Neural Networks for Flow Bottom Hole Pressure Prediction

Installation of down-hole gauges in oil wells to determine Flowing Bottom-Hole Pressure (FBHP) is a dominant process especially in wells lifted with electrical submersible pumps. However, intervening a well occasionally is an exhaustive task, associated with production risk, and interruption. The previous empirical correlations and mechanistic models failed to provide a satisfactory and reliable tool for estimating pressure drop in multiphase flowing wells. This paper aims to find the optimum parameters of Feed-Forward Neural Network (FFNN) with back-propagation algorithm to predict the flowing bottom-hole pressure in vertical oil wells. The developed neural network models rely on a large amount of available historical data measured from actual different oil fields. The unsurpassed number of neural network layers, the number of neurons per layer, and the number of trained samples required to get an outstanding performance have been obtained. Intensive experiments have been conducted and for the sake of qualitative comparison, Radial Basis neural and network and the empirical modes have been developed. The paper showed that the accuracy of FBHP estimation using FFNN with two hidden layer model is better than FFNN with single hidden layer model, Radial Basis neural network, and the empirical model in terms of data set used, mean square error, and the correlation coefficient error. With best results of 1.4 root mean square error (RMSE), 1.4 standard deviation of relative error (STD), correlation coefficient (R) 1.0 and 99.4% of the test data sets achieved less than 5% error. The minimum sufficient number of data sets used in training ANN model can be low as 12.5% of the total data sets to give 3.4 RMSE and 97% of the test data achieved 90% accuracy.

Neural Networks for Flow Bottom Hole Pressure Prediction

Installation of down-hole gauges in oil wells to determine Flowing Bottom-Hole Pressure (FBHP) is a dominant process especially in wells lifted with electrical submersible pumps. However, intervening a well occasionally is an exhaustive task, associated with production risk, and interruption. The previous empirical correlations and mechanistic models failed to provide a satisfactory and reliable tool for estimating pressure drop in multiphase flowing wells. This paper aims to find the optimum parameters of Feed-Forward Neural Network (FFNN) with back-propagation algorithm to predict the flowing bottom-hole pressure in vertical oil wells. The developed neural network models rely on a large amount of available historical data measured from actual different oil fields. The unsurpassed number of neural network layers, the number of neurons per layer, and the number of trained samples required to get an outstanding performance have been obtained. Intensive experiments have been conducted and for the sake of qualitative comparison, Radial Basis neural and network and the empirical modes have been developed. The paper showed that the accuracy of FBHP estimation using FFNN with two hidden layer model is better than FFNN with single hidden layer model, Radial Basis neural network, and the empirical model in terms of data set used, mean square error, and the correlation coefficient error. With best results of 1.4 root mean square error (RMSE), 1.4 standard deviation of relative error (STD), correlation coefficient (R) 1.0 and 99.4% of the test data sets achieved less than 5% error. The minimum sufficient number of data sets used in training ANN model can be low as 12.5% of the total data sets to give 3.4 RMSE and 97% of the test data achieved 90% accuracy.