Neural Network Learning Methods Research Articles

Unified and evolved approach based on neural network and deep learning methods for intrusion detection

Currently, network security has become a major concern for all entities around the world. Attackers employ various methods to disrupt services, which requires new methods to stop them all in one way. Moreover, these intrusions can evolve and overcome security measures and devices, which pushes to use new evolving methods able to accompany the evolution of these threats, to block them. In our paper, we propose a new approach for intrusion detection, founded on neural network (NN) and deep learning (DL) methods. This approach is planned to not only identify threats, but also to develop a long-term memory of them, in order to detect new ones resembling these memorized attacks, and simultaneously, to provide a single way to stop all kinds of intrusions. To test our model, we have chosen the most recently employed methods in literature, NN and DL algorithms: feedforward neural network (FNN), convolutional neural network (CNN), and long short-term memory (LSTM), then we have applied them on network security layer-knowledge discovery in databases (NSL KDD) intrusions dataset. The results of experiments were impressive for all the algorithms, with maximum performances noted by LSTM, which affirms the efficacy of our proposed method for intrusion detection.

RNEP: Random Node Entropy Pairing for Efficient Decentralized Training with Non-IID Local Data

The proliferation of edge devices and advancements in Internet of Things technology have created a vast array of distributed data sources, necessitating machine learning models that can operate closer to the point of data generation. Traditional centralized machine learning approaches often struggle with real-time big data applications, such as climate prediction and traffic simulation, owing to high communication costs. In this study, we introduce random node entropy pairing, a novel distributed learning method for artificial neural networks tailored to distributed computing environments. This method reduces communication overhead and addresses data imbalance by enabling nodes to exchange only the weights of their local models with randomly selected peers during each communication round, rather than sharing the entire dataset. Our findings indicate that this approach significantly reduces communication costs while maintaining accuracy, even when learning from non-IID local data. Furthermore, we explored additional learning strategies that enhance system accuracy by leveraging the characteristics of this method. The results demonstrate that random node entropy pairing is an effective and efficient solution for distributed learning in environments where communication costs and data distribution present significant challenges.

Wind Field Reconstruction Method Using Incomplete Wind Data Based on Vision Mamba Decoder Network

Accurate meteorological information is crucial for the safety of civil aviation flights. Complete wind field information is particularly helpful for planning flight routes. To address the challenge of accurately reconstructing wind fields, this paper introduces a deep learning neural network method based on the Vision Mamba Decoder. The goal of the method is to reconstruct the original complete wind field from incomplete wind data distributed along air routes. This paper proposes improvements to the Vision Mamba model to fit our mission, showing that the developed model can accurately reconstruct the complete wind field. The experimental results demonstrate a mean absolute error (MAE) of wind speed of approximately 1.83 m/s, a mean relative error (MRE) of around 7.87%, an R-square value of about 0.92, and an MAE of wind direction of 5.78 degrees.

Use of machine learning methods in predicting the main components of essential oils: Laurus nobilis L.

Laurus nobilis L. is an export plant that contributes to the industry with its medicinal and aromatic properties, as well as the use of its leaves or fruits in the food, pharmaceutical, and cosmetic industries. This study aims to predict the first three main components of bay laurel essential oil with different machine learning methods as a result of GC-MS analysis, taking into account a wide range of parameters used before and after the analysis of the essential oil in pharmacognostic studies. In this study, the K-Means algorithm was used to detect unknown relationships (chemical or physical properties) between the data obtained as a result of the analysis, and Artificial Neural Network (ANN) and Decision Tree machine learning methods were used to perform prediction operations. Based on this, considering the R2 values obtained from ANN analyses, R2 values of 0.9992 for testing, 1 for training, 1 for validation value, and 0.99992 for all were obtained. The ANN model produced results close to the real values with an accuracy rate of 98.97% in predicting the three main components. The Decision Tree algorithm achieved classification with a 96.23% accuracy rate in predicting the three main components. K-Means clustering was performed with an accuracy rate of 97.83% and significant relationships between the features of the Laurus nobilis dataset were detected. This comprehensive study serves as a guide for machine learning research on other essential oils.

Refining Euclidean Obfuscatory Nodes Helps: A Joint-Space Graph Learning Method for Graph Neural Networks.

Many graph neural networks (GNNs) are inapplicable when the graph structure representing the node relations is unavailable. Recent studies have shown that this problem can be effectively solved by jointly learning the graph structure and the parameters of GNNs. However, most of these methods learn graphs by using either a Euclidean or hyperbolic metric, which means that the space curvature is assumed to be either constant zero or constant negative. Graph embedding spaces usually have nonconstant curvatures, and thus, such an assumption may produce some obfuscatory nodes, which are improperly embedded and close to multiple categories. In this article, we propose a joint-space graph learning (JSGL) method for GNNs. JSGL learns a graph based on Euclidean embeddings and identifies Euclidean obfuscatory nodes. Then, the graph topology near the identified obfuscatory nodes is refined in hyperbolic space. We also present a theoretical justification of our method for identifying obfuscatory nodes and conduct a series of experiments to test the performance of JSGL. The results show that JSGL outperforms many baseline methods. To obtain more insights, we analyze potential reasons for this superior performance.

Adjusted SpikeProp algorithm for recurrent spiking neural networks with LIF neurons

A problem related to the development of a supervised learning method for recurrent spiking neural networks is addressed in the paper. The widely used Leaky-Integrate-and-Fire model has been adopted as a spike neuron model. The proposed method is based on a known SpikeProp algorithm. In detail, the developed method enables gradient descent learning of recurrent or multi-layer feedforward spiking neural networks. The research included an extended verification study for the classical XOR classification problem. In addition, the developed learning method has been used to provide a spiking neural black-box model of fast processes occurring in a pressurised water nuclear reactor. The obtained simulation results demonstrate satisfactory effectiveness of the proposed approach.

Predicting Weld Quality in Duplex Stainless Steel Butt Joints During Laser Beam Welding: A Hybrid DNN-HEVA Approach

Duplex stainless steel (DSS) welding is critical for producing structures and components in a variety of industries. Because traditional optimization approaches cannot manage the complexity of welding, additional solutions must be investigated. This study addresses the need for a systematic exploration of the welding parameters affecting the quality and performance of DSS butt joints, specifically employing fiber laser welding. This study focuses on butt joints in UNS S32304 and 304L using a fiber laser. Welding process parameters were systematically varied, including laser power, scanning speed, beam diameter, and focal position. To efficiently examine the parameters in the design space, the response surface methodology (RSM) with Box–Behnken design (BBD) was used. The combination of the specific parameter values used in the 7th run (Laser Power: 1600 W, Scanning Speed: 2000 mm/min, Beam Diameter: 1.5 [Formula: see text]m, Focal Position: 30 mm) led to better yield point stress (YPS), maximum tensile stress (MTS), break stress (BS) and reduction in area (RA). ANOVA and lack of fit analysis confirmed the significance of the experiment. In particular, the selected welding parameters significantly influenced all responses. A hybrid machine learning method of deep neural network (DNN) and human eye vision algorithm (HEVA) was used for predicting weld quality during laser beam welding (LBW). The hybrid DNN-HEVA consistently outperformed other optimization methods (RSM-BBD, FNN, CNN, and RNN) across key welding quality parameters. High [Formula: see text]2 values (0.9922, 0.9962, 0.9983, and 0.9907) indicated a strong correlation between predicted and actual values for all calculated responses. Lower RMSE, MSE, and MAE values were also highlighted in the precision of predicted values to the experimental data.

FairScene: Learning unbiased object interactions for indoor scene synthesis

In this paper, we propose an unbiased graph neural network learning method called FairScene for indoor scene synthesis. Conventional methods directly apply graphical models to represent the correlation of objects for subsequent furniture insertion. However, due to the object category imbalance in dataset collection and complex object entanglement with implicit confounders, these methods usually generate significantly biased scenes. Moreover, the performance of these methods varies greatly for different indoor scenes. To address this, we propose a framework named FairScene which can fully exploit unbiased object interactions through causal reasoning, so that fair scene synthesis is achieved by calibrating the long-tailed category distribution and mitigating the confounder effects. Specifically, we remove the long-tailed object priors subtract the counterfactual prediction obtained from default input, and intervene in the input feature by cutting off the causal link to confounders based on the causal graph. Extensive experiments on the 3D-FRONT dataset show that our proposed method outperforms the state-of-the-art indoor scene generation methods and enhances vanilla models on a wide variety of vision tasks including scene completion and object recognition.

Optimizing Uterine Synchronization Analysis in Pregnancy and Labor Through Window Selection and Node Optimization

1) Introduction: Preterm labor (PL) has globally become the leading cause of death in children under the age of 5 years. One of the most significant keys to preventing preterm labor is its early detection. 2) Objectives: The primary objectives of this study are to address the problem of PL by providing a new approach by analyzing the electrohysterographic (EHG) signals, which are recorded on the mother's abdomen during labor and pregnancy. 3) Methods: The EHG signal reflects the electrical activity that induces the mechanical contraction of the myometrium. Because EHGs are known to be non-stationary signals, and because we anticipate connectivity to alter during contraction (due to electrical diffusion and the mechanotransduction process), we applied the windowing approach on real signals to identify the best windows and the best nodes with the most significant data to be used for classification. The suggested pipeline includes: i) dividing the 16 EHG signals that are recorded from the abdomen of pregnant women in N windows; ii) apply the connectivity matrices on each window; iii) apply the Graph theory-based measures on the connectivity matrices on each window; iv) apply the consensus Matrix on each window in order to retrieve the best windows and the best nodes. Following that, several neural network and machine learning methods are applied to the best windows and best nodes to categorize pregnancy and labor contractions, based on the different input parameters (connectivity method alone, connectivity method plus graph parameters, best nodes, all nodes, best windows, all windows). 4) Results: Results showed that the best nodes are nodes 8, 9, 10, 11, and 12; while the best windows are 2, 4, and 5. The classification results obtained by using only these best nodes are better than when using the whole nodes. The results are always better when using the full burst, whatever the chosen nodes. 5) Conclusion: The windowing approach proved to be an innovative technique that can improve the differentiation between labor and pregnancy EHG signals.

CardioDPi: An explainable deep-learning model for identifying cardiotoxic chemicals targeting hERG, Cav1.2, and Nav1.5 channels

The cardiotoxic effects of various pollutants have been a growing concern in environmental and material science. These effects encompass arrhythmias, myocardial injury, cardiac insufficiency, and pericardial inflammation. Compounds such as organic solvents and air pollutants disrupt the potassium, sodium, and calcium ion channels cardiac cell membranes, leading to the dysregulation of cardiac function. However, current cardiotoxicity models have disadvantages of incomplete data, ion channels, interpretability issues, and inability of toxic structure visualization. Herein, an interpretable deep-learning model known as CardioDPi was developed, which is capable of discriminating cardiotoxicity induced by the human Ether-à-go-go-related gene (hERG) channel, sodium channel (Na_v1.5), and calcium channel (Ca_v1.5) blockade. External validation yielded promising area under the ROC curve (AUC) values of 0.89, 0.89, and 0.94 for the hERG, Na_v1.5, and Ca_v1.5 channels, respectively. The CardioDPi can be freely accessed on the web server CardioDPipredictor (http://cardiodpi.sapredictor.cn/). Furthermore, the structural characteristics of cardiotoxic compounds were analyzed and structural alerts (SAs) can be extracted using the user-friendly CardioDPi-SAdetector web service (http://cardiosa.sapredictor.cn/). CardioDPi is a valuable tool for identifying cardiotoxic chemicals that are environmental and health risks. Moreover, the SA system provides essential insights for mode-of-action studies concerning cardiotoxic compounds.

Fast constrained optimization for cloth simulation parameters from static drapes

AbstractWe present a cloth simulation parameter estimation method that integrates the flexibility of global optimization with the speed of neural networks. While global optimization allows for varied designs in objective functions and specifying the range of optimization variables, it requires thousands of objective function evaluations. Each evaluation, which involves a cloth simulation, is computationally demanding and impractical time‐wise. On the other hand, neural network learning methods offer quick estimation results but face challenges such as the need for data collection, re‐training when input data formats change, and difficulties in setting constraints on variable ranges. Our proposed method addresses these issues by replacing the simulation process, typically necessary for objective function evaluations in global optimization, with a neural network for inference. We demonstrate that, once an estimation model is trained, optimization for various objective functions becomes straightforward. Moreover, we illustrate that it is possible to achieve optimization results that reflect the intentions of expert users through visualization of a wide optimization space and the use of range constraints.

Survey on graph neural network-based community detection and its applications

Detecting communities within a network is a critical component of network analysis. The process involves identifying clusters of nodes that exhibit greater similarity to each other compared to other nodes in the network. In the context of Complex networks (CN), community detection becomes even more important as these clusters provide relevant information of interest. Traditional mathematical and clustering methods have limitations in terms of data visualization and high-dimensional information extraction. To address these challenges, graph neural network learning methods have gained popularity in community detection, as they are capable of handling complex structures and multi-dimensional data. Developing a framework for community detection in complex networks using graph neural network learning is a challenging and ongoing research objective. Therefore, it is essential for researchers to conduct a thorough review of community detection techniques that utilize cutting-edge graph neural network learning methods [102], in order to analyze and construct effective detection models. This paper provides a brief overview of graph neural network learning methods based on community detection methods and summarizes datasets, evaluation metrics, applications, and challenges of community detection in complex networks.

Detection of diseases in rice leaf using convolutional neural network with transfer learning based on ResNeXt

Rice is the most cultivated crop in the agricultural field and is a major food source for more than 60% of the population in India. Occurrence of disease in rice leaves, majorly affects the rice quality, production and also the farmers’ income. Nowadays, new variety of diseases in rice leaves are identified and detected periodically throughout the world. Manual monitoring and detection of plant diseases proves to be time consuming for the farmer and also a costly affair for using chemicals in the disease treatment. In this paper, a deep learning method of convolutional neural networks (CNN) with a transfer learning technique is proposed for the detection of a variety of diseases in rice leaves. This method uses the ResNeXt network model for classifying the images of disease-affected plants. The proposed model’s performance is evaluated using accuracy, precision, recall, F1-score and specificity. The experimental results of ResNeXt model measured for accuracy, precision, recall, F1-score, and specificity, are respectively 99.22%, 92.87%, 91.97%, 90.95%, and 99.05%, which proves greater accuracy improvement than the existing methods of SG2-ADA, YOLOV5, InceptionResNetV2 and Raspberry Pi.

A sub-grid scale model for Burgers turbulence based on the artificial neural network method

The present study proposes a sub-grid scale model for the one-dimensional Burgers turbulence based on the neural network and deep learning method. The filtered data of the direct numerical simulation is used to establish the training data set, the validation data set, and the test data set. The artificial neural network (ANN) method and Back Propagation method are employed to train parameters in the ANN. The developed ANN is applied to construct the sub-grid scale model for the large eddy simulation of the Burgers turbulence in the one-dimensional space. The proposed model well predicts the time correlation and the space correlation of the Burgers turbulence.

Effect of calcium content on the performance of CPVC cable protective tube

The influence mechanism of different calcium content on the mechanical properties of cable protection pipe was studied, and the feasibility of feedback verification on the using performance of cable protection pipe through neural network learning method was also studied. Methods Shore hardness tester, universal electronic testing machine and densitometer were used to test the performance of cable protection tubes with different calcium content, and the quantitative relationship curve between cable protection tubes and performance was obtained. The database of hardness, density and composition was established through BP neural network, and the feasibility of feedback quality prediction was verified through test data. Results The hardness and density of the cable protective tube can be effectively improved by increasing the calcium element in the cable protective tube. Through the feedback prediction of performance by BP neural network, its qualification rate can be accurately predicted. Through this study, the content of calcium element can be adjusted appropriately according to the use conditions to make its performance meet the use conditions. Through Shore hardness tester and spectrometer, the quality of cable protection tube can be determined, and the nondestructive testing effect can be achieved.

Identification and Classification of Coix seed Storage Years Based on Hyperspectral Imaging Technology Combined with Deep Learning

Developing a fast and non-destructive methodology to identify the storage years of Coix seed is important in safeguarding consumer well-being. This study employed the utilization of hyperspectral imaging (HSI) in conjunction with conventional machine learning techniques such as support vector machines (SVM), k-nearest neighbors (KNN), random forest (RF), extreme gradient boosting (XGBoost), as well as the deep learning method of residual neural network (ResNet), to establish identification models for Coix seed samples from different storage years. Under the fusion-based modeling approach, the model’s classification accuracy surpasses that of visible to near infrared (VNIR) and short-wave infrared (SWIR) spectral modeling individually. The classification accuracy of the ResNet model and SVM exceeds that of other conventional machine learning models (KNN, RF, and XGBoost). Redundant variables were further diminished through competitive adaptive reweighted sampling feature wavelength screening, which had less impact on the model’s accuracy. Upon validating the model’s performance using an external validation set, the ResNet model yielded more satisfactory outcomes, exhibiting recognition accuracy exceeding 85%. In conclusion, the comprehensive results demonstrate that the integration of deep learning with HSI techniques effectively distinguishes Coix seed samples from different storage years.

Expert System for Fourier Transform Infrared Spectra Recognition Based on a Convolutional Neural Network With Multiclass Classification.

Fourier transform infrared spectroscopy (FT-IR) is a widely used spectroscopic method for routine analysis of substances and compounds. Spectral interpretation of spectra is a labor-intensive process that provides important information about functional groups or bonds present in compounds and complex substances. In this paper, based on deep learning methods of convolutional neural networks, models were developed to determine the presence of 17 classes of functional groups or 72 classes of coupling oscillations in the FT-IR spectra. Using web scanning, the spectra of 14 361 FT-IR spectra of organic molecules were obtained. Several different variants of model architectures with different sizes of feature maps have been tested. Based on the Shapley additive explanations (SHAP) and gradient-weighted class activation mapping (GradCAM) methods, visualization tools have been developed for visualizing and highlighting the areas of absorption bands manifestation for corresponding functional groups or bonds in the spectrum. To determine 17 and 72 classes, the F1-weighted metric, which is the harmonic mean of the class' precision and class' recall weighted by class' fraction, reached 93 and 88%, respectively, when using data on the position of absorption maxima in the spectrum as an additional source layer. The resulting model can be used to facilitate the routine analysis of spectra for all areas such as organic chemistry, materials science, and biology, as well as to facilitate the preparation of the obtained experimental data for publication.

Multi-objective optimization analysis of combined heating and cooling transcritical CO2 system integrated with mechanical subcooling utilizing hydrocarbon mixture based on machine learning

To meet the simultaneous needs of high temperature disinfection and freezing in the field of food processing, a new concept of combined heating and cooling transcritical CO2 system integrated with dedicated mechanical subcooling utilizing hydrocarbon mixture is proposed. The system performance in terms of thermodynamics, economy and environment is studied and compared with the baseline combined heating and cooling transcritical CO2 system and four traditional combined heating and cooling solutions, considering the influence of temperature glide and the heat transfer deterioration. The new proposed system is further optimized by using the machine learning method of artificial neural network and non-dominated sorting genetic algorithm. Multi-objective optimization is conducted considering the objective function of energy efficiency, initial capital cost and life cycle carbon emissions of the new system, to obtain the optimum components and concentration ratio of the hydrocarbon mixture. The results indicate the thermodynamic performance and environmental benefits of subcooling subsystem with hydrocarbon mixture are better than those of the pure system. In contrast to that using pure R290 and R601, the coefficient of performance is enhanced by 8.20 % and 8.13 % and the life cycle carbon emission is reduced by 8.54 % and 9.31 %, respectively, when R290/R601 (50/50) is used. However, the initial capital cost is 9.25 % and 10.23 % higher than that of pure R290 and R601, respectively. Finally, the hydrocarbon mixture corresponding to the optimal design point is R1270/R601a (53/47), the corresponding discharge pressure is 12.86 MPa, and the subcooling degree is 37.50 °C. This study can provide a theoretical reference for the application of CO2 refrigeration and heat pump technology.

Hybrid ARMA-GARCH-Neural Networks for intraday strategy exploration in high-frequency trading

The frequency of armed conflicts increased during the last 20 years. The problems of the emergence of military disputes, not only concern social parameters, but also economic and financial dimensions. This study examines the potential impact of global geopolitical events on the stock market prices of the Dow Jones U.S. Aerospace & Defense Index and Foreign Exchange (FOREX) markets movements. We analyse whether defence stocks and exchange rate perform similarly during military incidents or geopolitical crises. We built an Autoregressive Moving Average Model with a Generalized Autoregressive Conditional Heteroskedasticity process (ARMA-GARCH) with the machine learning methods of Neural Networks, Deep Recurrent Convolutional Neural Networks, Deep Neural Decision Trees, Quantum Neural Networks, and Quantum Recurrent Neural Networks, aimed at detecting intraday patterns for forecasting defence stock market and FOREX markets disturbances in a market microstructure framework. The empirical results provide preliminary findings on the foreseeability of market disturbances and small differences are observed before and during geopolitical events. Additionally, we confirm the effectiveness of the hybrid model ARMA-GARCH with the machine learning approaches, being ARMA-GARCH-Quantum Recurrent Neural Network the technique that achieves the best accuracy results. Our work has a large potential impact on investment market agents and portfolio managers, as shocks from geopolitical events could provide a new methodology to support the decision-making process for trading in High-Frequency Trading.

CLASSIFICATION OF POTATO LEAF DISEASES USING CONVOLUTIONAL NEURAL NETWORK

Potatoes are an agricultural product that has the fourth highest content of wheat flour after corn, wheat, and rice. Although potatoes play a critical role in agriculture, this crop is susceptible to various diseases and pests. There are several potato leaf diseases that are not yet known to farmers. Dry spot potato leaf disease (late blight) and late blight. If not treated, this disease on potato leaves will spread to the stem and reduce crop yields, causing crop failure. By using technology in the form of digital image processing, this problem can be overcome. This research proposes an appropriate method for detecting disease in potato leaves. Classification will be carried out in three classes, namel, Early Blight, Healthy and Late Blight using the Deep Learning method of Convolutional Neural Network (CNN). The data used comes from an online dataset via the kaggle.com page with the file name Potato Disease Leaf Dataset (PLD) totaling 3251 training datasets which are then divided into training, testing, and validation. The processes carried out are image pre-processing, image augmentation, then image processing using a Convolutional Neural Network (CNN). In the classification process using the CNN method with RMSprop optimizer, the accuracy was 97.53% with a loss value of 0.1096.