NN Model Research Articles

Linear and nonlinear influences of climatic changes on migration flows: a case study for the ‘Mediterranean bridge’

The influence of climatic changes on migrations around the world is a topic widely discussed in the scientific literature, but investigations are often limited to particular regions. The possible causes of migration flows from Africa to Europe due to landings in Italy, a peninsula which can be considered as a ‘bridge’ between these two continents, have not been investigated in detail even if, at present, this problem is at the top of the political agenda in Europe. Here a simple linear model and a fully nonlinear one (neural networks—NNs) are applied to the study of possible climatic influences on migration flows from the Sahel to Italy during the period 1995–2009. The results show the ability of the NN model to explain the majority of the variance found in the data and permit the identification of the major climatic drivers affecting the amount of yields in Sahelian countries and the migrations flows from them to Italy. In particular, the use of a NN model fully identifies both linear and nonlinear influences. We can explain much of the variance in the migration data (R2 = 0.775). Agriculture (harvest yields) is shown to link climatic changes and migration, and poor yields can enhance this latter phenomenon. Heat waves (during the cereal growing season) have an important nonlinear role. The annual temperature, however, is most likely the dominant climatic factor influencing migrations in this region.

Incorporating Label Co-Occurrence Into Neural Network-Based Models for Multi-Label Text Classification

Multi-label text classification (MLTC) addresses a fundamental problem in natural language processing, which assigns multiple relevant labels to each document. In recent years, Neural Network-based models (NN models) for MLTC have attracted much attention. In addition, NN models achieve favorable performances because they can exploit label correlations in the penultimate layer. To further capture and explore label correlations, we propose a novel initialization to incorporate label co-occurrence into NN models. First, we represent each class as a column vector of the weight matrix in the penultimate layer, which we name the class embedding matrix. Second, we deduce an equation for correlating the class embedding matrix with the label co-occurrence matrix, ensuring that relevant classes are denoted by vectors with large correlations. Finally, we provide a theoretical analysis of the equation, and propose an algorithm to calculate the initial values of the class embedding matrix from the label co-occurrence matrix. We evaluate our approach with various text extractors, such as Recurrent Neural Network (RNN), Convolutional Neural Network (CNN) and Transformer on four public datasets. The experimental results demonstrate that our approach markedly improves the performance of existing NN models.

An LSTM Short-Term Solar Irradiance Forecasting Under Complicated Weather Conditions

Complicated weather conditions lead to intermittent, random and volatility in photovoltaic (PV) systems, which makes PV predictions difficult. A recurrent neural network (RNN) is considered to be an effective tool for time-series data prediction. However, when the weather changes intensely, the long-term sequence of multivariate may cause gradient vanishing (exploding) during the training of RNN, leading the prediction results to local optimum. Long short-term memory (LSTM) network is the deep structure of RNN. Due to its special hidden layer unit structure, it can preserve the trend information contained in the long-term sequence, which is allowed to solve the problems of RNN and improve performance. An LSTM-based approach is applied for short-term predictions in this study based on a timescale that encompasses global horizontal irradiance (GHI) one hour in advance and one day in advance. Inaccurate forecasts usually occur on cloudy days, and the results of ANN and SVR in the literature prove this. To improve prediction accuracy on cloudy days, the clearness-index was introduced as an input data for the LSTM model and to classify the type of weather by k-means during the data processing, where cloudy days are classified as the cloudy and the mixed(partially cloudy). NN models are established to compare the accuracy of different approaches and the cross-regional study is to prove whether the method can be generalizable. From the results of hourly forecast, the R 2 coefficient of LSTM on cloudy days and mixed days is exceeding 0.9, while the R 2 of RNN is only 0.70 and 0.79 in Atlanta and Hawaii. From the results of daily forecast, All R 2 on cloudy days is about 0.85. However, the LSTM is still very effective in improving of RNN and more accurate than other models.

A Novel Hybrid Model for Wind Speed Prediction Based on VMD and Neural Network Considering Atmospheric Uncertainties

Considering the strong fluctuation and the nonlinearity of wind speed, and atmospheric uncertainties, wind speed prediction based on the hybrid model is presented, which is composed of the neural network, variational mode decomposition (VMD), and Lorenz disturbance. First, the VMD is used to process the data to get several intrinsic mode functions (IMFs). Second, the neural network model (NN model) can be established by these IMFs of the training set, and the validation set is used to adjust the model parameters. Subsequently, given the nonlinearity of wind speed, Lorenz disturbance is added to determine the finial model, and the best Lorenz disturbance parameter and the best Lorenz disturbance sequence can be obtained by minimizing the mean absolute error of validation set. At last, the wind speed can be forecasted by the hybrid model. Taking Sotavento wind farm in Spain as an example, the results show that, the hybrid model has stable prediction performance, and the distribution characteristics of its results are consistent with the actual wind speed. The general model only focuses on improving prediction accuracy. However, on the basis of improving the forecasting accuracy, the proposed model not only enhances the prediction stability, but also restores the characteristics of wind speed. This research work provides a more scientific basis for wind power dispatching arrangement, and it is of great significance to improve the utilization rate of wind power.

Employment of PSO algorithm to improve the neural network technique for radial distribution system state estimation

Abstract This study presented a new hybrid algorithm to improve the state estimation (SE) of radial distribution power systems (PSs). The proposed particle swarm optimization–neural network (PSO–NN) algorithm constructed an independent and fast monitoring system with high accuracy that can detect abnormal conditions or failures in a PS. In this study, PSO was adopted to determine the appropriate weights of the NN model. The speed and accuracy of PSO with the NN model were evaluated in the SE of power system networks (PSNs). The information obtained through SE was used to enhance the operations and customer service delivery in terms of energy consumption and power quality in PSNs. Capacitor banks were installed to reduce the losses and improve the voltage profiles. The PSO–NN algorithm was assessed on IEEE (9, 33, and 69) bus standards. Simulation results proved that the new technique can be tested on any distribution network because of its accurate and efficient SE. Results indicated that the PSO–NN algorithm had better performance than the phasor measurement units.

Parameter Prediction of Stretch-Blow Molding Process of PET Using Neural Networks

This study presents a parameter selection strategy developed for the Stretch-Blow Molding (SBM) process to minimize the weight of preforms used. The method is based on a predictive model developed using Neural Networks. The temperature distribution model of the preform was predicted using a 3-layer NN model with supervised backpropagation learning. In addition, the model was used to predict the uniform air pressure applied inside the preform, taking into account the relationship between the internal air pressure and the volume of the preform. Parameters were validated using in situ tests and measurements performed on several weights and lengths of a 0.330 Liter Polyethylene Terephthalate (PET) bottles. Tests showed that the model adequately predicts both the blowing kinematics, mainly zone temperatures and blowing and stretching pressures along the walls of the bottle while maintaining the bottle strength and top load requirements. In the second step, the model was combined to automatically compute the lowest preform weight that can be used for a particular 330 ml bottle design providing a uniform wall thickness distribution.

Learning Human-Like Trajectory Planning on Urban Two-Lane Curved Roads From Experienced Drivers

In the coming decades, it is a universal consensus that autonomous vehicles (AVs) and human-driven vehicles will share the traffic roads. Trajectory planning of AVs has been extensively studied from the perspective of driving safety and riding comfort. However, human-like trajectory planning has rarely been studied. In this paper, we characterize and model human driving trajectories using real vehicle field test data collected on five two-way and two-lane urban curved roads with 20 experienced drivers and 3 experimental vehicles. A differential global positioning system (GPS) and an inertial navigation system (INS) are used to measure the vehicle positions and velocities in high precision. We study the trajectory characteristics of experienced drivers on curved two-lane roads, especially the relationships between the vehicle trajectories on bidirectional two lanes. Based on long short-term memory neural network (LSTM NN), we develop a data-driven trajectory model to generate human-like driving trajectories. By comparing with other three modeling methods, the LSTM NN model was validated and tested in various cases with promising performance.

Non-linear Dimensionality Reduction-based Intrusion Detection using Deep Autoencoder

The intrusion detection has become core part of any network of computers due to increasing amount of digital content available. In parallel, the data breaches and malware attacks have also grown in large numbers which makes the role of intrusion detection more essential. Even though many existing techniques are successfully used for detecting intruders but new variants of malware and attacks are being released every day. To counterfeit these new types of attacks, intrusion detection must be designed with state of art techniques such as Deep learning. At present the Deep learning techniques have a strong role in Natural Language Processing, Computer Vision and Speech Processing. This paper is focused on reviewing the role of deep learning techniques for intrusion detection and proposing an efficient deep Auto Encoder (AE) based intrusion detection technique. The intrusion detection is implemented in two stages with a binary classifier and multiclass classification algorithm (dense neural network). The performance of the proposed approach is presented and compared with parallel methods used for intrusion detection. The reconstruction error of the AE model is compared with the PCA and the performance of both anomaly detection and the multiclass classification is analyzed using metrics such as accuracy and false alarm rate. The compressed representation of the AE model helps to lessen the false alarm rate of both anomaly detection and attack classification using SVM and dense NN model respectively.

Accurate Identification of the Sex and Species of Silkworm Pupae Using Near Infrared Spectroscopy

The present study proposes a novel method to discriminate the sex and species of silkworm pupae using NIR spectroscopy (800–2778 nm). The spectra from 840 silkworm pupae were collected then divided into a calibration set (700) and a test set (140) using the Kennard–Stone (KS) algorithm. The recognition models were built using the radial basis function and neural network (RBF–NN) and support vector machine (SVM) approaches. The species and sex identification results using the RBF–NN and SVM models based on full spectral data achieved a low accuracy of 5% and 33.57%, respectively. To improve the accuracy and decrease the processing time, both principal component analysis (PCA) and linear discriminant analysis (LDA) were used to reduce the data dimensions. The performance of the optimized SVM model (92.14%) was much better than the RBF–NN model (19.29%) based on PCA. Overall, the best discrimination results were obtained using the RBF–NN and SVM models based on LDA, providing an accuracy of 100%. These promising results have shown that the LDA–SVM and LDA–RBF–NN models can accurately recognize the sex and species of silkworm pupae using NIR spectroscopy.

Multi-objective optimization of the trimming operation of CFRPs using sensor-fused neural networks and TOPSIS

This work presents the results of an optimization study of trimming carbon fiber composite by technique of order preference by similarity to ideal solution (TOPSIS). Edge trimming was conducted at different levels of cutting speed, feed speed, and radial depth of cut using a 2k factorial design. The effects of cutting parameters on tool wear, surface roughness and tool temperature were analyzed and discussed. Data generalization was performed by training multiple neural network structures (NN) where the input vector of the neural networks contained sensor data in addition to typical process parameters. TOPSIS was used to select the optimal cutting conditions that could generate minimum tool wear, surface roughness, and tool temperature simultaneously while maintaining high production rates. The results show that the NN models offer better results when sensor data is fused in the input vector. TOPSIS optimum conditions were confirmed by validation experiments and were found to be accurate.

The Permanent Downshifting at Later Stages of Benjamin–Feir Instability of Waves

Some of the prominent features in later stages of the Benjamin–Feir (BF) instability development are still not well clarified: What are the conditions for a permanent downshifting in the wave energy spectrum, is this process inevitable accompanied by wave breaking dissipation? What is the mechanism of multiple downshifting and discrete energy spreading to higher frequencies, how does it depend on the initial steepness and frequency space of waves? We conducted experimental and theoretical studies on this issue and revealed a number of new features of the development of instability in the late stages of wave’s evolution. We employ the reduced (truncated) version of Zakharov equations, the multi-wave near-neighbor resonance model (NN model), which takes into account the most effective quasi-resonances with minimum detuning from the exact resonance conditions. We show that the near-neighbor model of wave interactions can adequately describe the number of new characteristic features of BF instability. NN model is much simpler than Zakharov equation for computation and analysis. The dissipation version of Zakharov equations based on the Tulin’s semi-empirical model is employed for the description of breaking wave’s propagation. We verify the new characteristic features of BFI for various initial wave conditions by experiments conducted in Tainan Hydraulics Laboratory and confirm the theoretical predictions of NN model. A strong permanent downshifting of the spectral maximum for gentle waves without wave breaking is revealed for initially two-time narrower spectral width in comparison with the most unstable case. For steep waves, a multiple downshifting regime is detected, accompanied by a wave breaking. The discrete energy flow to higher spectral components takes place in the breaking and no-breaking regimes. Results of numerical simulations of Zakharov and NN models reasonably correspond to each other and to our experimental and field observations on wave modulation.

Privacy in Neural Network Learning: Threats and Countermeasures

Algorithmic breakthroughs, the feasibility of collecting huge amount of data, and increasing computational power, contribute to the remarkable achievements of NNs. In particular, since Deep Neural Network (DNN) learning presents astonishing results in speech and image recognition, the amount of sophisticated applications based on it has exploded. However, the increasing number of instances of privacy leakage has been reported, and the corresponding severe consequences have caused great worry in this area. In this article, we focus on privacy issues in NN learning. First, we identify the privacy threats during NN training, and present privacy-preserving training schemes in terms of using centralized and distributed approaches. Second, we consider the privacy of prediction requests, and discuss the privacy-preserving protocols for NN prediction. We also analyze the privacy vulnerabilities of trained models. Three types of attacks on private information embedded in trained NN models are discussed, and a differential privacy-based solution is introduced.

Reliable modeling of constant volume depletion (CVD) behaviors in gas condensate reservoirs

It is important to access a clear understanding about the phase behavior of gas condensate reservoirs in order to forecast the future performance of such reservoirs. In this communication, different models based on multilayer perceptron network (MLP NN), least square support vector machine (LSSVM), adaptive neuro inference system (ANFIS) and radial basis function networks optimized by genetic algorithm (GA-RBF NN) were developed for estimation of amount of produced gas using constant volume depletion (CVD) tests of retrograde gas condensate reservoirs. Results show that the developed models are capable of accurately estimating the cumulative produced gas (Gp) as an output parameter by utilizing various input parameters including temperature, pressure, composition of gas, and properties of plus fraction. The analysis of results reveals that the GA-RBF NN presents more accurate results in comparison with MLP NN, ANFIS and LSSVM models. Moreover, comparison between GA-RBF NN model as the most accurate model developed in the present work and two literature models shows the superiority of GA-RBF NN. Results of this study can be used in PVT softwares to enhance the accuracy and precision of CVD modeling of gas condensate reservoirs.

Spam filtering using integrated distribution-based balancing approach and regularized deep neural networks

Rapid growth in the volume of unsolicited and unwanted messages has inspired the development of many anti-spam methods. Supervised anti-spam filters using machine-learning methods have been particularly effective in categorizing spam and non-spam messages. These automatically integrate spam corpora pre-processing, appropriate word lists selection, and the calculation of word weights, usually in a bag-of-words fashion. To develop an accurate spam filter is challenging because spammers attempt to decrease the probability of spam detection by using legitimate words. Complex models are therefore needed to solve such a problem. However, existing spam filtering methods usually converge to a poor local minimum, cannot effectively handle high-dimensional data and suffer from overfitting issues. To overcome these problems, we propose a novel spam filter integrating an N-gram tf.idf feature selection, modified distribution-based balancing algorithm and a regularized deep multi-layer perceptron NN model with rectified linear units (DBB-RDNN-ReL). As demonstrated on four benchmark spam datasets (Enron, SpamAssassin, SMS spam collection and Social networking), the proposed approach enables capturing more complex features from high-dimensional data by additional layers of neurons. Another advantage of this approach is that no additional dimensionality reduction is necessary and spam dataset imbalance is addressed using a modified distribution-based algorithm. We compare the performance of the approach with that of state-of-the-art spam filters (Minimum Description Length, Factorial Design using SVM and NB, Incremental Learning C4.5, and Random Forest, Voting and Convolutional Neural Network) and several machine learning algorithms commonly used to classify text. We show that the proposed model outperforms these other methods in terms of classification accuracy, with fewer false negatives and false positives. Notably, the proposed spam filter classifies both major (legitimate) and minor (spam) classes well on personalized / non-personalized and balanced / imbalanced spam datasets. In addition, we show that the proposed model performs better than the results reported by previous studies in terms of accuracy. However, the high computational expenses related to additional hidden layers limit its application as an online spam filter and make it difficult to overcome the problem of concept drift.

Spatial Transferability of Neural Network Models in Travel Demand Modeling

AbstractNeural network (NN) models have been widely used in travel demand modeling in recently years. However, there are few studies about the spatial transferability of NN models. In this paper, t...

A neural network model for predicting weighted mean temperature

Water vapor is an important element of the Earth’s atmosphere, and most of it concentrates at the bottom of the troposphere. Knowledge of the water vapor measured by Global Navigation Satellite Systems (GNSS) is an important direction of GNSS research. In particular, when the zenith wet delay is converted to precipitable water vapor, the weighted mean temperature $$T_\mathrm{m}$$ is a variable parameter to be determined in this conversion. The purpose of the study is getting a more accurate $$T_\mathrm{m}$$ model for global users by a combination of two different characteristics of $$T_\mathrm{m}$$ (i.e., the $$T_\mathrm{m}$$ seasonal variations and the relationships between $$T_\mathrm{m}$$ and surface meteorological elements). The modeling process was carried out by using the neural network technology. A multilayer feedforward neural network model (the NN) was established. The NN model is used with measurements of only surface temperature $$T_\mathrm{S}$$ . The NN was validated and compared with four other published global $$T_\mathrm{m}$$ models. The results show that the NN performed better than any of the four compared models on the global scale.

A modified damage law with the fuzzy neural network method for crack initiation life prediction of notched specimen

It is difficult to predict the fatigue life of notched specimen due to the complexity of notch effect. Based on the continuum damage mechanics theory of Lemaitre, a modified damage law for fatigue life prediction of notched specimen combined with the fuzzy neural network method is proposed to obtain a more reliable life. According to the fatigue experimental data, the material parameters in the damage evolution equation are demarcated. Then the fatigue life prediction of notched specimen is conducted. At last, after the stage pf experiment data collection and NN training , the established NN model will be able to reduce the relative error from 20.85% to about %7.12 in the verification test. Although the life prediction after being modified may not be accurate in every local region, the combination between the theoretical method and the fuzzy neural network method will obtain a more reliable material life in the whole. DOI: http://dx.doi.org/10.5755/j01.mech.23.6.14862

Long short-term memory neural network for air pollutant concentration predictions: Method development and evaluation

Air pollutant concentration forecasting is an effective method of protecting public health by providing an early warning against harmful air pollutants. However, existing methods of air pollutant concentration prediction fail to effectively model long-term dependencies, and most neglect spatial correlations. In this paper, a novel long short-term memory neural network extended (LSTME) model that inherently considers spatiotemporal correlations is proposed for air pollutant concentration prediction. Long short-term memory (LSTM) layers were used to automatically extract inherent useful features from historical air pollutant data, and auxiliary data, including meteorological data and time stamp data, were merged into the proposed model to enhance the performance. Hourly PM2.5 (particulate matter with an aerodynamic diameter less than or equal to 2.5 μm) concentration data collected at 12 air quality monitoring stations in Beijing City from Jan/01/2014 to May/28/2016 were used to validate the effectiveness of the proposed LSTME model. Experiments were performed using the spatiotemporal deep learning (STDL) model, the time delay neural network (TDNN) model, the autoregressive moving average (ARMA) model, the support vector regression (SVR) model, and the traditional LSTM NN model, and a comparison of the results demonstrated that the LSTME model is superior to the other statistics-based models. Additionally, the use of auxiliary data improved model performance. For the one-hour prediction tasks, the proposed model performed well and exhibited a mean absolute percentage error (MAPE) of 11.93%. In addition, we conducted multiscale predictions over different time spans and achieved satisfactory performance, even for 13–24 h prediction tasks (MAPE = 31.47%).

Neural network adaptive real-time optimizing control of industrial processes

<p>Real-time optimization (RTO) has attracted considerable interest among researchers and industries for being able to optimise the plant economics such as product efficiency, product quality and process safety in the wake of increasing global competitions. The success of RTO depends much on the quality of model being used in the optimisation. The present study was carried out to explore the use of artificial neural network (ANN) to improve the quality of the model being used in the modified two step (MTS) technique. The MTS is a real-time optimising control algorithm of the modifier adaptation scheme which is used to determine the optimum steady-state control set-points. The proposed new version of MTS technique will be using process model based on ANN. A laboratory scale process of a two continuous stirred tank heat exchanger in series (2CSTHEs) is used as a case study. The multilayer feed forward ANN architecture 4-10-6 with linear function was used to model the 2CSTHEs and then integrates into the MTS technique, the resulted algorithm will be known as Iterative Neural Network Modified Two Step (INNMTS) technique. Simulation studies were conducted to test the performance of the INNMTS technique on the 2CSTHEs process. The results show that the overall value for the coefficient of determination(R<sup>2</sup>)is equal to one, which indicates adequacy of the model proposed for the prediction of the behavior of 2CSTHEs system. When NN model of 2CSTHEs is applied to the INNMTS technique, the model-plant mismatch is greatly reduced to almost zero, which indicates by significant reduction in the number of iterations to 5which requires by INNMTS compared to 16 iterations by the MTS technique to converge to optimal real solution.</p><p>Chemical Engineering Research Bulletin 19(2017) 129-138</p>

Interaction of real and virtual [formula omitted] pairs in [formula omitted] decays

The pp¯ invariant mass spectra of the processes J/ψ→pp¯ω, J/ψ→pp¯ρ, and J/ψ→pp¯γ close to the pp¯ threshold are calculated by means of the NN¯ optical potential. The potential model for NN¯ interaction in the S01 state is proposed. The parameters of the model are obtained by fitting the cross section of NN¯ scattering together with the pp¯ invariant mass spectra of the J/ψ decays. Good agreement with the available experimental data is achieved. Using our potential and the Green's function approach we also describe the peak in the η′π+π− invariant mass spectrum in the decay J/ψ→γη′π+π− in the energy region near the NN¯ threshold.