Forward Propagation Research Articles

WOA-DNN for Intelligent Intrusion Detection and Classification in MANET Services

Mobile ad-hoc networks (MANET) are garnering a lot of attention because of their potential to provide low-cost solutions to real-world communications. MANETs are more vulnerable to security threats. Changes in nodes, bandwidth limits, and centralized control and management are some of the characteristics. IDS (Intrusion Detection System) are the aid for detection, determination, and identification of illegal system activity such as use, copying, modification, and destruction of data. To address the identified issues, academics have begun to concentrate on building IDS-based machine learning algorithms. Deep learning is a type of machine learning that can produce exceptional outcomes. This study proposes that WOA-DNN be used to detect and classify incursions in MANET (Whale Optimized Deep Neural Network Model) WOA (Whale Optimization Algorithm) and DNN (Deep Neural Network) are used to optimize the preprocessed data to construct a system for classifying and predicting unanticipated cyber-attacks that are both effective and efficient. As a result, secure data transport to other nodes is provided, preventing intruder attacks. The invaders are found using the (Machine Learning) ML-IDS and WOA-DNN methods. The data is reduced in dimensionality using Principal Component Analysis (PCA), which improves the accuracy of the outputs. A classifier is used in forward propagation to predict whether a result is normal or malicious. To compare the traditional and proposed models’ effectiveness, the accuracy of classification, detection of the attack rate, precision rate, and F-Measure, Recall are utilized. The proposed WOA-DNN model has higher assessment metrics and a 99.1% accuracy rate. WOA-DNN also has a greater assault detection rate than others, resulting in fewer false alarms. The classification accuracy of the proposed WOA-DNN model is 99.1%.

CoordiNet: Constrained Dynamics Learning for State Coordination Over Graph

A neural architecture to efficiently coordinate the transitions of state variables (states) over a graph is proposed. We consider the coordination of the time evolution of the state variables associated with the nodes on a graph. The states are associated with physical attributes of agents, e.g., the speed and/or location of vehicles. Efficient coordination then corresponds to the optimization of dynamics (how fast do vehicles travel) that may be subject to constraints (travel must be collision-free). The aim of this paper is (i) the formulation of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">learnable constrained dynamics</i> which governs the transition of states under constraints over a graph and (ii) to show its industrial application. Firstly, we formulate continuous ordinary differential equations (ODEs), namely forward propagation of state transitions over a graph and backward propagation to optimize model parameters for the learnable constrained dynamics. Discretization of these continuous ODEs results in the neural architecture <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CoordiNet</i> . Secondly, as an application of CoordiNet, we address a traffic coordination problem by learning constrained dynamics such that vehicles can travel as fast as possible without collisions. Simulation experiments of traffic coordination confirm that our method maximizes the vehicles' speed states while avoiding collisions.

Data-driven probabilistic quantification and assessment of the prediction error model in damage detection applications

Advances in computer hardware and sensor technologies have led to a surge in the use of data-driven modeling and machine learning for structural engineering applications, with Structural Health Monitoring (SHM) being one of them. Despite considerable interest, it remains a research topic due to the difficulty in accurately quantifying aleatoric and epistemic uncertainty in SHM systems. Sources of uncertainty are related to operational and environmental variability, as well as measurement noise and the model prediction error associated with the data used to train damage identification algorithms. In this work, the authors aim to explicitly quantify the statistical structure of model prediction error and assess its influence on the detection performance of strain-based SHM architectures under the existence of aleatoric variability. A structural beam, subjected to probabilistic static loading is used as the reference structure and strain measurements as the damage-sensitive features. Model prediction error is quantified explicitly using robust statistical tools through available laboratory observations and synthetic (Finite Element) data. Monte Carlo simulations enabled the forward propagation of uncertainty to the feature space to generate training data for three binary detectors (Likelihood Ratio Test, Quadratic Discriminant Analysis and Mahalanobis Distance), based on statistical pattern recognition. Detection performance was compared between the explicitly quantified prediction model error and the commonly assumed white Gaussian noise model, showcasing the influence of systematic error (bias) and correlation on the robustness of an SHM system using real-world data.

Wind Turbine Prediction using Deep Learning and Long Short Term Memory (LSTM)

Accurate forecasting is essential for the long-term success of adding wind energy to the national power system. In this study, we look at forecasting wind turbine using a LSTM deep learning model. To forecast potential outcomes for a time series, it is sufficient to initially obtain pertinent details from past data. While many methods struggle with understanding the long-term dependencies encoded in data sets, LSTM options, an instance of the strategy in deep learning, show potential for efficiently overcoming this challenge. An overview of LSTM's architecture and forward propagation method is provided initially. LSTM network is applied to the wind turbine prediction dataset. This dataset has 9 features and 6575 records. There are four performance matrices used to test the model. The four matrices are mean squared error (MSE), mean absolute error (MAE), mean absolute percentage error (MAPE), and root mean squared error (RMSE). MAPE obtained the least error.

An On-Chip Fully Connected Neural Network Training Hardware Accelerator Based on Brain Float Point and Sparsity Awareness

In recent years, deep neural networks (DNNs) have brought revolutionary progress in various fields with the advent of technology. It is widely used in image pre-processing, image enhancement technology, face recognition, voice recognition, and other applications, gradually replacing traditional algorithms. It shows that the rise of neural networks has led to the reform of artificial intelligence. Since neural network algorithms are computationally intensive, they require GPUs or accelerated hardware for real-time computation. However, the high cost and high power consumption of GPUs result in low energy efficiency. It recently led to much research on accelerated digital circuit hardware design for deep neural networks. In this paper, we propose an efficient and flexible neural network training processor for fully connected layers. Our proposed training processor features low power consumption, high throughput, and high energy efficiency. It uses the sparsity of neuron activations to reduce the number of memory accesses and memory space to achieve an efficient training accelerator. The proposed processor uses a novel reconfigurable computing architecture to maintain high performance when operating Forward Propagation and Backward Propagation. The processor is implemented in Xilinx Zynq UltraSacle+MPSoC ZCU104 FPGA, with an operating frequency of 200MHz and power consumption of 6.444W, and can achieve 102.43 GOPS.

FM-Unet: Biomedical image segmentation based on feedback mechanism Unet.

With the development of deep learning, medical image segmentation technology has made significant progress in the field of computer vision. The Unet is a pioneering work, and many researchers have conducted further research based on this architecture. However, we found that most of these architectures are improvements in the backward propagation and integration of the network, and few changes are made to the forward propagation and information integration of the network. Therefore, we propose a feedback mechanism Unet (FM-Unet) model, which adds feedback paths to the encoder and decoder paths of the network, respectively, to help the network fuse the information of the next step in the current encoder and decoder. The problem of encoder information loss and decoder information shortage can be well solved. The proposed model has more moderate network parameters, and the simultaneous multi-node information fusion can alleviate the gradient disappearance. We have conducted experiments on two public datasets, and the results show that FM-Unet achieves satisfactory results.

Self-Parameter Distillation Dehazing.

In this paper, we propose a novel dehazing method based on self-distillation. In contrast to conventional knowledge distillation approaches that transfer large models (teacher networks) to small models (student networks), we introduce a single knowledge distillation network that transfers network parameters to itself for dehazing. In the early stages, the proposed network transfers scene content (identity) information to the next stage of itself using haze-free data. However, in the later stages, the network transfers haze information to itself using haze data, enabling the accurate dehazing of input images using scene information from the early stages. In a single network, parameters are seamlessly updated from extracting global scene features to dehazing the scene. During the training, forward propagation acts as a teacher network, whereas backward propagation acts as a student network. The experimental results demonstrate that the proposed method considerably outperforms other state-of-the-art dehazing methods.

Generation of collisionless electrostatic shock waves in interaction between strong intense laser and near-critical-density plasma

Weak and strong collisionless electrostatic shock wave (CESW) generated in the interaction between strong intense laser and near-critical-density plasma are studied by the one-dimensional particle-in-cell simulation in this work. And the effects of the ranges of plasma density profiles, non-relativistic and relativistic laser intensities on the generation of CESWs are also investigated. The non-relativistic weakly driven laser generates the weak CESW in the interaction between the laser and near-critical-density plasma. The electron spectra show double-temperature distribution because the non-relativistic driven laser cannot heat the electrons sufficiently. The low-temperature electrons have an important influence on the generation of weak CESW, and they can also cause the protons to be accelerated and reflected from the CESWs. The spectra of the weak CESW protons show a continuously distributed profile. When the range of plasma density up-ramp is large, the process can be observed that the post-soliton structure evolves into the ion acoustic wave and further into the weak collisionless electrostatic shock wave. When the driven laser intensity is relativistic, the electrons are heated sufficiently to a single relativistic temperature. The effect of the range of plasma density profile on the generation of CESW is further analyzed and it is found that 1) when the range of plasma density up-ramp is large, the potential barrier of ion acoustic wave is shielded by the hot electrons; 2) when the range of plasma density up-ramp is small, the effective distance (i.e. the Debye length) of accelerating field is larger and the endurance time is longer than when the range of plasma density up-ramp is large. This makes the ion acoustic wave structure more stable in its forward propagation process. When the difference in velocity between the ion acoustic wave accelerating protons and the target normal sheath accelerating protons satisfies the proton reflection condition of CESW, the ion acoustic wave further evolves into the strong CESW, the monoenergetic protons generated at the same time.

COMBINED DATA AND DEEP LEARNING MODEL UNCERTAINTIES: AN APPLICATION TO THE MEASUREMENT OF SOLID FUEL REGRESSION RATE

In complex physical process characterization, such as the measurement of the regression rate for solid hybrid rocket fuels, where both the observation data and the model used have uncertainties originating from multiple sources, combining these in a systematic way for quantities of interest (QoI) remains a challenge. In this paper, we present a forward propagation uncertainty quantification (UQ) process to produce a probabilistic distribution for the observed regression rate r. We characterized two input data uncertainty sources from the experiment (the distortion from the camera &lt;i&gt;U&lt;/i&gt;&lt;sub&gt;c&lt;/sub&gt; and the non-zero-angle fuel placement &lt;i&gt;U&lt;/i&gt;&lt;sub&gt;Y&lt;/sub&gt;), the prediction and model form uncertainty from the deep neural network (&lt;i&gt;U&lt;/i&gt;&lt;sub&gt;m&lt;/sub&gt;), as well as the variability from the manually segmented images used for training it (&lt;i&gt;U&lt;/i&gt;&lt;sub&gt;s&lt;/sub&gt;). We conducted seven case studies on combinations of these uncertainty sources with the model form uncertainty. The main contribution of this paper is the investigation and inclusion of the experimental image data uncertainties involved, and how to include them in a workflow when the QoI is the result of multiple sequential processes.

Efficient Adaptive Monte Carlo Uncertainty Forecasting for High Dimensional Nonlinear Dynamic Systems

The problem of uncertainty forecasting for complex dynamical systems in the framework of particle methods can be effectively addressed through the solution methodology known as adaptive Monte Carlo (AMC). Monte Carlo (MC) methods involve discretizing the initial probability density function (pdf) followed by forward propagation of particles through system dynamics to obtain an approximate particle representation of the evolved state uncertainty. While simple to implement, MC faces questions surrounding transient statistical consistency and rate of convergence. AMC (adopted here) addresses these issues on-the-fly using defined bounds on estimation accuracy alongside ensemble enrichment routines. This paper presents several improvements in the ensemble enrichment module of AMC. The AMC platform is re-engineered to include the novel implementation of a parallel global stochastic optimization routine in conjunction with additional module enhancements that work together towards the goal of efficient forecasting. Moreover, the efficacy of algorithms utilized within every submodule of AMC are detailed and improved upon under the framework of arithmetic minimization and parallelization. Each submodule is profiled to determine computational bottlenecks, optimized or replaced with more efficient methods such as simulated annealing (SA), and parallelized ultimately leading to a clock time reduction of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$200 - 400\%$ </tex-math></inline-formula> for benchmark entry descent and landing, Lorenz-96, and Lotka-Volterra models.

Phase-preserving theory and its linearization approximation for forward scattering field of scalar acoustic wave equation

The conventional wave-equation linearization methods, such as the first-order Born or Rytov approximation, always implicitly imply a weak-scattering assumption, making it valid only for weak perturbation models. To extend the wave-equation linearization theory to strong perturbation models, we consider a scenario that the reference model is smooth within the scale of the incident wave length, and propose a phase-preserving method which can predict the phase perturbation of forward scattering wave field. First, we introduce the WKBJ approximation to the scattered- and incident wave fields so that the integral of the unknown solution (i.e. the scattered field) in the nonlinear Ricatti integral equation can be replaced by the integral of scattering-angle and model perturbation, yielding an explicit expression of the scattered field. Theoretical derivation shows that the proposed phase-preserving method can accurately predict the phase-perturbation of forward scattered wave field regardless of the strength of velocity perturbations for one-dimensional wave propagation problem. To apply the phase-preserving approximation to the inverse problem, we further consider a scenario of small-angle forward propagation. In this case, the phase-preserving approximation can be linearized by neglecting the influence of scattering angles, leading to a linear relation between the scattered field and the model perturbation, which we refer to as the generalized Rytov approximation. Numerical experiments demonstrate that the generalized Rytov approximation can predict the phase perturbation of the scattered field with higher accuracy for small-angle forward propagation, and is suitable for strong model perturbations. The generalized Rytov approximation extends the validity and the scope of application of the traditional Rytov approximation. In specific application fields such as the seismic traveltime tomography or medical ultrasonic transmission imaging, a new traveltime/phase sensitivity kernel can be derived by replacing the conventional Rytov approximation with the proposed method, which can increase the inversion accuracy and speed up the convergence.

DL-Net: Sparsity Prior Learning for Grasp Pattern Recognition

The purpose of grasp pattern recognition is to determine the grasp type for an object to be grasped, which can be applied to prosthetic hand control and ease the burden of amputees. To enhance the performance of grasp pattern recognition, we propose a network DL-Net inspired by dictionary learning. Our method includes two parts: 1) forward propagation for sparsity representation learning and 2) backward propagation for dictionary learning, which utilizes the sparsity prior effectively and learns a discriminative dictionary with stronger expressive ability from a mass of training data. The experiment was performed on two household object datasets, the RGB-D Object dataset, and the Hit-GPRec dataset. The experimental results illustrate that the DL-Net performs better than traditional deep learning methods in grasp pattern recognition.

Featurization of Ultrafast Expansion and Geometrical Properties of Heterogeneous Colliding Plasmas

Numerous fields of research and industry have undergone revolutionary change because of the unique characteristics of ultrashort laser pulses. Moreover, the ultrafast imaging sensors, such as ICCD technique, can help to understand the ionization features and expansion properties of colliding laser-induced plasma (CLPP) and related stagnation layer (S.L.) geometry. In this work, the effort will be focused on CLPP experiments from two seeds of heterogeneous elements. The research’s goal is to analyse the geometrical development of the colliding plasma, the temporal evolution of plume composition features and its associated characteristics. The expansion velocity and forward propagation range (FPR) of the stagnation layer in a nanosecond scale—both of which have been discovered. The ultrafast imaging results give the sight and explain the possibilities of extant technologies that can help to re-engineer the plasma characteristics for the next generation of lithography applications or new selective physical concepts.

Modeling wastewater treatment plant (WWTP) performance using artificial neural networks: Case of Adana (Seyhan)

In this study, performance estimation of biological wastewater treatment plants (WWTP) was made by applying Artificial Neural Network (ANN) techniques. As material, 355-day data from Adana Metropolitan Municipality Seyhan wastewater treatment plant for 2021 were used. Of the data used, 240 were evaluated as training data and 115 as test data. In the establishment of the ANN model, the daily chemical oxygen demand (COD), daily water flow (Qw) and daily suspended solids (SS) parameters at the entrance of the WWTP were used as input parameters. The daily biological oxygen demand (BOD) parameter was determined as the output parameter. In the study, feed forward back propagation ANN model (FFBPANN) was used to estimate the daily BOD amounts at the entrance of the WWTP. In the statistical analysis, the correlation (R2) values of the input parameters with BOD were found to be 0.906 for COD, 0.294 for Qw and 0.605 for SS. The R2 value was determined as 0.891, the MAE value was 10.32% and the RMSE value was 722.21 in the network structures where the best results were obtained for the test and training data (in the 4-4-1 ANN model). As a result of the study, it was concluded that the ANN model was successful in estimating the BODs of the WWTPs in obtaining reliable and realistic results, and that effective analyzes with the simulation of their nonlinear behavior could be used as a good performance evaluation tool in terms of reducing operating costs.

Yapay Sinir Ağları (YSA) ve ARIMA Modelleri ile Türkiye’de Aylık Sıfır km Otomobil Satış Adetlerinin Tahmin Edilmesi

Çalışmada, yapay zekâ temelli tahmin modellerinden Yapay Sinir Ağları (YSA) ve Zaman Serileri Analizi modellerinden Arima ile Türkiye’deki aylık sıfır km otomobil satış adetleri tahmin edilmiştir. Bağımlı değişken aylık sıfır km otomobil satışlarıdır. Bağımsız değişkenler ise aylık otomobil ihracatı (USD), aylık otomobil ithalatı (USD), aylık Amerikan Dolar kuru (TL), aylık Türkiye ihracatı (USD) ve aylık Türkiye ithalatı (USD)’dir. Türkiye İstatistik Kurumu (TÜİK), Türkiye Cumhuriyet Merkez Bankasından (TCMB) elde edilen aylık veriler (Ocak 2002-Aralık 2020, 228 ay-19 yıl) kullanılarak, Ocak 2021 ile Mart 2022 arasında gerçekleşen 15 aylık sıfır km otomobil satış adetleri tahmin edilmiştir. YSA modelinin performansı Arima modeli ile karşılaştırılmış, tahminlerin farkları ve sonuçlar yorumlanmıştır. Çalışmada Arima’nın YSA’ya kıyasla daha iyi sonuç verdiği tespit edilmiştir.

Training Autoencoders Using Relative Entropy Constraints

Autoencoders are widely used for dimensionality reduction and feature extraction. The backpropagation algorithm for training the parameters of the autoencoder model suffers from problems such as slow convergence. Therefore, researchers propose forward propagation algorithms. However, the existing forward propagation algorithms do not consider the characteristics of the data itself. This paper proposes an autoencoder forward training algorithm based on relative entropy constraints, called relative entropy autoencoder (REAE). When solving the feature map parameters, REAE imposes different constraints on the average activation value of the hidden layer outputs obtained by the feature map for different data sets. In the experimental section, different forward propagation algorithms are compared by applying the features extracted from the autoencoder to an image classification task. The experimental results on three image classification datasets show that the classification performance of the classification model constructed by REAE is better than that of the classification model constructed by other forward propagation algorithms.

Learning Data-Driven Propagation Mechanism for Graph Neural Network

A graph is a relational data structure suitable for representing non-Euclidean structured data. In recent years, graph neural networks (GNN) and their subsequent variants, which utilize deep neural networks to complete graph analysis and representation, have shown excellent performance in various application fields. However, the propagation mechanism of existing methods relies on hand-designed GNN layer connection architecture, which is prone to information redundancy and over-smoothing problems. To alleviate this problem, we propose a data-driven propagation mechanism to adaptively propagate information between layers. Specifically, we construct a bi-level optimization objective and use the gradient descent algorithm to learn the forward propagation architecture, which improves the efficiency of learning different layer combinations in multilayer networks. The experimental results of the model on seven benchmark datasets demonstrate the effectiveness of the proposed method. Furthermore, combining this data-driven propagation mechanism with models, such as Graph Attention Networks, can consistently improve the performance of these models.

Interface mode between gyroelectric and hyperbolic media

We report the prediction of a type of electromagnetic surface wave that propagates at the interface between a gyroelectric material and a hyperbolic medium. By solving Maxwell’s equations, the existence conditions of this surface mode are discussed as determined by the medium parameters, working frequency, and direction of the principal axes of the hyperbolic medium. As one would expect, gyrotropy gives non-reciprocal features to these waves, such as the asymmetry between forward and backward propagations. We show that the field distribution of the new wave resembles the Zenneck wave, albeit in the present case, material losses are not required. Finally, we analyze a realistic configuration that supports the surface modes here predicted and allows for their excitation.

Road Pothole Detection Using Unmanned Aerial Vehicle Imagery and Deep Learning Technique

Potholes are considered the main factor for road defects, which leads to road status deterioration, which, consequently will lead to increased road accidents. The first step in road maintenance is to inspect the road surface and then accurately detect potholes. However, manually identifying them is costly and time-consuming. In this study, unmanned aerial vehicle (UAV) imagery was used to create orthophotos of the roads and, using deep learning methods, potholes were detected. The used deep learning method in this study is the "you only look once" (YOLO) algorithm. YOLO is one of the "deep learning-based approaches" to detecting objects and is a single-stage network which requires only one forward propagation across the neural network and focuses on the entire image. The fourth version of YOLO is YOLOV4, which has two different architectures (YOLOv4 and YOLOv4-tiny). Two roads were chosen as the study areas, and to generate the orthophotos of the roads, UAV was used to acquire images. To train both methods in the process of detecting potholes using deep learning, 5300 images were used, 90% used for training and 10% applied for testing. The two used architectures were trained for 6000 iterations. Both methods were evaluated based on the average loss, mean average precision (mAP), and training and testing time. The results showed that the (mAP) values for YOLOv4 and YOLOv4-tiny were 91. 2% and 85.7%, respectively. At the end of the 6000 iterations, the average loss for YOLOv4 is 0.30% and for YOLOv4-tiny is 0.34%. In the training process, YOLOv4 needs 29 seconds for each iteration, while YOLOv4-tiny requires only 8 seconds. In the test process, YOLOv4-tiny is faster at detecting potholes than YOLOv4. The approaches were tested on orthophotos created by processing UAV photos. When comparing the detection of both architectures with visual detection, the results showed that YOLOv4 was able to detect most of the potholes on roads, but YOLOv4-tiny detected a lower number of potholes.

Comparison of process-based and statistical approaches for simulation and projections of rainfed crop yields

Accurate and comprehensive modelling aimed at investigating the impact of climate change on rainfed crop yields is of great importance due to the interconnected issues of water scarcity and food security. Because the process-based and statistical approaches to simulating crop yields are different in nature, a comparison between them is needed. This study investigates the accuracy of crop yield simulations in the historical period as well as future projections using two modelling approaches: 1) a process-based approach employing the Soil and Water Assessment Tool+ (SWAT+) model, and 2) a statistical approach employing a data-driven model, Feed Forward Back Propagation Neural Network (FFBPNN) over a medium-sized catchment in north-western Poland. The application of two potential evapotranspiration methods (Penman-Monteith and Hargreaves) in SWAT+ permitted calibration (2004–2011) and validation (2012–2019) of runoff and yields of winter wheat and spring barley. Different combinations of climatic parameters with a drought index based on Joint Deficit Index were applied to simulate and project rainfed crop yields (winter wheat, barley, potato, rye, rapeseed, sugar beets, cereals, maize for grain, maize for green forage, pulses) with FFBPNN. The results reveal that adding the new drought index helped increase the FFBPNN performance. This approach showed that future yields of the studied crops would slightly increase under RCP8.5 by 2060. Winter wheat and spring barley projections from SWAT+ showed very small changes using both the Penman-Monteith and Hargreaves method. Policy-wise, the results should be of interest to climate change adaptation practitioners and food security experts. Future studies should aim at more thorough investigation of the role of the downscaling technique and extreme events, as well as the effect of elevated CO2 on future crop yields.