Two-layer Feed-forward Neural Network Research Articles

Resource optimization of MIMO using neural network for 5G communication

The principal objective of this work is to optimize the resources of massive MIMO techniques by an application of a neural network algorithm in order to reduce the manufacturing cost of 5G technology within the wireless peripheries of tropical regions. For plenty of annual precipitation in the tropical regions, the radio signals of these areas get attenuated in communicating any message i.e. signal fading occurs. Various diversity techniques have been used for diminishing signal fading in these tropical areas. MIMO technique is a spatial diversity technique used to minimize signal fading. In this paper, 1 × 1 to 256 × 256 MIMO techniques have been designed with the help of MATLAB and Xilinx system generator, and an 8 × 8 MIMO has been implemented in Xilinx Kintex7 FPGA. A CMOS logic technique has been incorporated to optimize the resources of different MIMO techniques, and for this purpose a two-layer feed forward neural network algorithm has been adopted. A comparison of the theoretical values obtained from the design by using both Xilinx system generator and Simulink with the values derived from the design by using CMOS logic gates, has been displayed in this paper. For neural network optimization, three types of training algorithms have been applied.

Approximation properties of Gaussian-binary restricted Boltzmann machines and Gaussian-binary deep belief networks

Despite the successful use of Gaussian-binary restricted Boltzmann machines (GB-RBMs) and Gaussian-binary deep belief networks (GB-DBNs), little is known about their theoretical approximation capabilities to represent distributions of continuous random variables. In this paper, we address the expressive properties of GB-RBMs and GB-DBNs, contributing theoretical insights to the optimal number of hidden variables. We first treat the GB-RBM’s unnormalized log-likelihood as a sum of a special two-layer feedforward neural network and a negative quadratic term. Then, a series of simulation results are established, which can be used to relate GB-RBMs to general two-layer feedforward neural networks whose expressive properties are much better understood. On this basis, we show that a two-layer ReLU network with all weights in the second layer being 1, along with a negative quadratic term, can approximate all continuous functions. In addition, we provide qualified lower bounds for the number of hidden variables of GB-RBMs required to approximate distributions whose log-likelihood are given by some classes of smooth functions. Moreover, we further study the universal approximation of GB-DBNs with two hidden layers by providing a sufficient number of hidden variables O(ɛ−2) that are guaranteed to approximate any given strictly positive continuous distribution within a given error ɛ. Finally, numerical experiments are carried out to verify some of the proposed theoretical results.

Neural network simulation for selective dissociation of SF6 molecules by single-mode and multi-mode carbon dioxide lasers

Selective dissociation of sulfur hexafluoride molecules was simulated using single-mode and multi-mode carbon dioxide lasers using a two-layer feedforward neural network with 4 and 2nodes in the first and second layers, respectively. Existing experimental results (Sajad et al., 2004 [1]) were used to train and verify neural networks. Simulation was performed using MATLAB software neural network toolbox. Using the trained neural network, the results of non-existent physical conditions in the previous experimental results were predicted and presented.

Accuracy Analysis of Data Fraud Detection for Company Transactions Using Two-Layered Feed Forward Neural Network Approach Compared with Random Forest

Aim: To find accuracy of data fraud detection for company transactions using two-layered feed forward Neural Network approaches compared with Random Forest. Materials and methods: Number of groups is 2. Group 1 - Two-layered feed forward Neural Network, and Group 2 - Random Forest. Results: The accuracy of the innovative feed forward Neural Network method is 83.4% and Random Forest is 85.5%. The accuracy of the Random Forest algorithm appears to be a slight increase. Conclusion: Random Forest algorithm finds the fraud detection appearance to be better accuracy than the feed forward Neural Network.

Empowering saving energy at home through serious games on thermostat interfaces

The residential Heating Ventilation and Air-Conditioning (HVAC) system use around 3/5 of the total energy consumption. Connected thermostats optimize the HVAC operation; however, householders have personality traits that lead into behavioral and usability problems toward the thermostat’s interface usage. Thus, a serious game applied in the thermostat interface can balance entertainment and education. Therefore, thermostat interfaces must address strategies that reduce energy without losing thermal comfort. This paper proposed an interactive interface type and a predicted interface type based on an HVAC strategy and a Natural Ventilation strategy. These strategies measured the impact of adaptive thermal comfort, energy consumption, and costs. Hence, twelve energy models located in California (Concord, Riverside, Los Angeles, and San Diego) were simulated using EnergyPlus™ through LadybugTools. The first interactive interface included Serious Game elements, so the householder interacted with the date, location, and setpoint. The second interface predicted the energy consumption and thermal comfort during winter and summer in Concord by a two-layer feed-forward Artificial Neural Network structure. The proposed structure decreases the energy consumption by at least 62% without losing thermal comfort.

In-situ prediction of the spatial surface roughness profile during slot milling

Quality inspection is traditionally considered non-productive. That is why the manufacturing industries aim to decrease inspection times to a bare minimum without sacrificing part quality. Alongside the implementation of the Industry 4.0 paradigm, data-driven in-situ quality control is a potential enabler for minimizing inspection times. In that, the surface roughness parameter prediction is the subject of a large body of research, but studies on the spatial surface roughness profile prediction are limited. This research contributes to this field by using vibration signals and physics-informed machine learning models for the in-situ prediction of the surface roughness profile. A tri-axial accelerometer mounted on the machine tool spindle is used to capture the vibrations during a slot milling process. For one tool revolution during a stable cut, the observed acceleration in the three axes and the surface roughness profile are periodic. A model is constructed to establish the correlation between the input signals and the spatial surface roughness profile by utilizing a physics-based model of the tool trajectory together with a two-layer feed-forward neural network. Furthermore, the feature engineering of denoised velocities and displacements derived by the numerical integration of the acceleration signals improves the prediction performance with overfitting. The results show a good correlation between the spatial surface roughness and the accelerometer signals.

A computational model of familiarity detection for natural pictures, abstract images, and random patterns: Combination of deep learning and anti-Hebbian training

We present a neural network model for familiarity recognition of different types of images in the perirhinal cortex (the FaRe model). The model is designed as a two-stage system. At the first stage, the parameters of an image are extracted by a pretrained deep learning convolutional neural network. At the second stage, a two-layer feed forward neural network with anti-Hebbian learning is used to make the decision about the familiarity of the image. FaRe model simulations demonstrate high capacity of familiarity recognition memory for natural pictures and low capacity for both abstract images and random patterns. These findings are in agreement with psychological experiments.

Intelligent numerical computing paradigm for heat transfer effects in a Bodewadt flow

The current research article is intended to examine the nonlinear input-output fitting with two-layer feed forward neural network for numerical treatment of heat transfer effects in Bodewadt flow over a permeable disk. Estimation parameters in Bodewadt flow model include wall suction parameter (1.7 ≤ A ≤ 6.7), magnetic field (0.1 ≤ M ≤ 1.0), Prandtl number (0.2 ≤ Pr ≤ 5.0) and variable viscosity (2.0 ≤ θe ≤ 20.0). The influences of magnetic field, Joule heating, internal heat generation/absorption, wall suction, viscous dissipation along with variable fluid properties are also contemplated as well. The energy equation in Bodewadt flow for permeable disk governing heat transfer and fluid motion are transformed into self-similar dimensionless differential equation by using Von-Karman variables. The temperature and velocity of the fluid about the disk by taking various values of the physical parameter are solved by Adams Bashforth method to determine the reference computational results of Bodewadt flow model. The values of skin friction co-efficient and Nusselt number are also calculated and physically interpreted for the assorted parameters. Further, the obtained experimental dataset of the system is used to authenticate the artificial neural network modeling with optimization of Levenberg-Marquardt backpropagation. Fitting data precision is examined on mean squared error based cost function for the system and the outputs of intelligent networks are demonstrated using performance parameter, error histograms, regression and fitting plots. Least mean square error trained at decreasing gradient with optimized weights having strong correlation R = 1 between target and network output and a consistent convergence further certified the worth of methodology.

Fault-Tolerant Control of a Nonlinear Uncertain System: A Neural Network-Based Passive Approaches and Comparative Study with State-of-the-Art Control Approaches

This article suggests passive methods for designing Fault-Tolerant Control (FTC) for nonlinear uncertain systems with actuator and leak faults. To anticipate the Fault-Tolerant Control (FTC) action to overcome the actuator and leak faults, two-layer Feed-Forward Back-Propagation Neural Network (FFBPNN) and two-layer Cascade Forward Neural Network (CFNN) have been used, it will also tolerate external process additive disturbances. We employ the passive approach for fault-tolerant control using Proportional Integral Derivative (PID) control methodology to create a fault-tolerant controller without a fault detection mechanism. Further, we use the four residue signal features (i.e., mean, variance, skewness and normalize data of residue signal) to train the neural network in this study to tackle the issue originating from having less faults and uncertainty from residue signal. To show the efficacy of the suggested approach, simulations are run. The measurement of the residue signal was done using a healthy and a faulty uncertain non-linear system model. A comparison of findings utilizing a state of-the-art control methodology provided in (Dutta et al., 2014) was also presented to validate the proposed FTC methodology.

Probabilistic flutter analysis of a long-span bridge in typhoon-prone regions considering climate change and structural deterioration

In this paper, a long-term probabilistic flutter analysis of long-span bridges, with consideration of the time-variant probability density function (PDF) of annual maximum wind speed caused by climate change and the deterioration effects of dynamic properties obtained by field monitoring data, is investigated. Estimated uncertainty, which occurs in probabilistic flutter analysis, is quantified by the generalized density evolution equation (GDEE). A suspension bridge, with the center span of 1650 m, is chosen as application example. Long-term deterioration and inter-seasonal varying characteristics of modal frequencies and damping ratios are discussed. An implicit formulation among radius to maximum winds Rmax, central pressure deficit Δp, latitude ψ and sea surface temperature SST is set up by training a two-layer feed-forward artificial neural network (ANN) with historical records, and then the full-track typhoon simulation is conducted based on Vickery’s empirical model. Lastly, long-term probabilistic flutter analysis is conducted in conjunction with three prospective climate change scenarios, RCP2.6, RCP4.5 and RCP8.5, showing that the likelihood of annual flutter failure will increase greatly mainly due to higher annual maximum wind speed in a warming climate.

Using Two-Layered Feed-Forward Neural Networks to Model Blood Uric Acid Among Diabetic Patients

Introduction: The end product of purine metabolism in humans is uric acid (UA). Although uric acid can function as either an antioxidant or an oxidant depending on the surrounding environment, the chemical environment can also impact uric acid.The uric acid level in the serum can predict the development of diabetic nephropathy in type 1 diabetes.
 Objective: This study aims to determine factors that are perhaps having an association with acid uric.
 Method: Variables selection is basedon clinical importance. The most significant variable will be assigned and analyzed using Artificial Neural Network (ANN) through multilayer feed-forward and contour plot.
 Results: Through the architecture of MLFF with two hidden layers, it was found that Creatinine level, Urea level, Systolic Blood Pressure reading, Waist circumference reading, Gender play an essential role toward uric level with an accuracy of 97.7% and the predicted mean squared error (MSE.net) is 0.005. The combination of the selected variable showing the highest significance in predicting the level of uric acid.
 Conclusion: These findings offer useful future management action plans for patients with diabetes.By controlling these four variables can improve the level of health among diabetic patients.
 Bangladesh Journal of Medical Science Vol.20(4) 2021 p.741-747

A novel machine learning-based approach for prediction of nitrogen content in hydrochar from hydrothermal carbonization of sewage sludge

In this work, 138 datapoints, including elemental composition and ultimate analysis of various types of sewage sludge, and the hydrothermal carbonization reaction conditions, are used to develop a prediction model for the nitrogen content of the hydrochar. The results suggested that a two-layer feedforward neural network with five (05) neurons in the hidden layer can accurately predict the nitrogen content of the hydrochar based on the reaction temperature and the contents of nitrogen, carbon, volatiles and fixed carbon in the feedstock. Over 100 runs, the R2 and RMSE are in [87.547–99.097%] and [0.243–1.431] wt.% (db), respectively. Moreover, a statistical and regression analysis revealed that the sewage sludge-N is the main contributor to the hydrochar-N. Mostly, 40–70% of sewage sludge-N goes to hydrochar-N. The results are consistent with previous experimental reports, and this model can be used to predict the sewage sludge-derived hydrochar-N.

Investigations on the relationship among the porosity, permeability and pore throat size of transition zone samples in carbonate reservoirs using multiple regression analysis, artificial neural network and adaptive neuro-fuzzy interface system

Finding an accurate method for estimating permeability aside from well logs has been a difficult task for many years. The most commonly used methods targeted towards regression technique to understand the correlation between pore throat radii, porosity and permeability are Winland and Pittman equation approaches. While these methods are very common among petrophysicists, they do not give a good prediction in certain cases. Consequently, this paper investigates the relationship among porosity, permeability, and pore throat radii using three methods such as multiple regression analysis, artificial neural network (ANN), and adaptive neuro-fuzzy inference system (ANFIS) for application in transition zone permeability modeling. Firstly, a comprehensive mercury injection capillary pressure (MICP) test was conducted using 228 transition zone carbonate core samples from a field located in the Middle-East region. Multiple regression analysis was later performed to estimate the permeability using pore throat and porosity measurement. For the ANN, a two-layer feed-forward neural network with sigmoid hidden neurons and a linear output neuron was used. The technique involves training, validation, and testing of input/output data. However, for the ANFIS method, a hybrid optimization consisting of least-square and backpropagation gradient descent methods with a subtractive clustering technique was used. The ANFIS combines both the artificial neural network and fuzzy logic inference system (FIS) for the training, validation, and testing of input/output data. The results show that the best correlation for the multiple regression technique is achieved for pore throat radii with 35% mercury saturation (R35). However, for both the ANN and ANFIS techniques, pore throat radii with 55% mercury saturation (R55) gives the best result. Both the ANN and ANFIS are later found to be more effective and efficient and thus recommended as compared with the multiple regression technique commonly used in the industry.

Abstract 16866: Artificial Neural Networks as Prediction Tools: Predicting Permanent Pacemaker Implantation (PPMI) in Patients Undergoing Transcatheter Aortic Valve Replacement (TAVR) Utilizing a Shallow Feed-Forward Neural Network

Introduction: Despite increasing TAVR volumes after a series of recent favorable clinical trials, adverse outcomes remain frequent, including new or worsened conduction disease requiring PPMI, life-threatening bleeding, paravalvular leak, and stroke. PPMI carries a reported incidence ranging from 8.8-14.6% and is of particular concern given the increased risk of mortality and rehospitalization. New techniques in signal processing may inform novel statistical approaches to better predict PPMI from a set of clinical variables. Artificial neural networks (ANN) comprise a family of algorithms that utilize non-linear activation functions to enable improved prediction of PPMI in TAVR patients. Objective: To examine the predictive utility of a feed-forward neural network in classifying PPM implantation in patients undergoing TAVR. Methods: Pre and post-operative data from a single institution were collected for all patients undergoing TAVR without prior pacemaker implantation from January 2016 to December 2019. Data was imported into Matlab, partitioned into training, validation, and test sets, and processed in a two-layer feed-forward neural network with sigmoid hidden and softmax output neurons. Performance data included confusion matrices and receiver operating characteristic (ROC) curves. Results: The total sample size for the cohort was 513 patients with a PPMI incidence of 8.6%. The training set contained 40 variables and 359 patients, while the validation and test sets contained 77 patients each. The final optimized model showed cross-entropy of 0.25 with 6 iterations and an area under ROC curve of 0.73. Overall model accuracy was 92.7% in the validation set and 88.3% in the test set. Conclusions: In summary, we show that feed-forward neural networks can be useful in processing multiple interdependent variables to aid clinical prediction. Our network demonstrated modest discriminatory ability in predicting the need for PPMI after TAVR.

Air compressor load forecasting using artificial neural network

Air compressor systems are responsible for approximately 10% of the electricity consumed in United States and European Union industry. As many researches have proven the effectiveness of using Artificial Neural Network in air compressor performance prediction, there is still a need to forecast the air compressor electrical load profile. The objective of this study is to predict compressed air systems' electrical load profile, which is valuable to industry practitioners as well as software providers in developing better practice and tools for load management and look-ahead scheduling programs. Two artificial neural networks, Two-Layer Feed-Forward Neural Network and Long Short-Term Memory were used to predict an air compressors electrical load. Compressors with three different control mechanisms are evaluated with a total number of 11,874 observations. The forecasts were validated using out-of-sample datasets with 5-fold cross-validation. Models produced average coefficient of determination values from 0.24 to 0.94, average root-mean-square errors from 0.05 kW - 5.83 kW, and mean absolute scaled errors from 0.20 to 1.33. The results indicate that both artificial neural networks yield good results for compressors using variable speed drive (average R2 = 0.8 and no naïve forecasting), only the long short-term memory model gives acceptable results for compressors using on/off control (average R2 = 0.82 and no naïve forecasting), and no satisfactory results are obtained for load/unload type air compressors (models constituting naïve forecasting).

Impact investigation of road characteristics on the accident rate based on the neural network modelling

The use of neural networks promotes flexible non-linear modelling. Also, it is the perspective direction. The article investigates the influence of the main road characteristics on the accident rate. Twelve variables are selected as the main road characteristics. The result of modelling is the two-layer feed-forward neural network with hidden sigmoid nodes and linear output nodes. The operability of the model is estimated by calculating the index of regression by training, validation and testing. The results are acceptable for the intended purpose. The resulting model can be used for assessing and forecasting the safety rate on the two-lane roads with the intensity of 120 to 340 cars/hour. These include a significant part of the streets of CIS countries and other states. Besides, the neural network model can become the basis for the development of software products for assessing the accident rate based on road characteristics or can be implemented in existing software.

Efficiency-House Optimization to Widen the Operation Range of the Double-Suction Centrifugal Pump

Most pumping machineries have a problem of obtaining a higher efficiency over a wide range of operating conditions. To solve that problem, an optimization strategy has been designed to widen the high-efficiency range of the double-suction centrifugal pump at design (Qd) and nondesign flow conditions. An orthogonal experimental scheme is therefore designed with the impeller hub and shroud angles as the decision variables. Then, the “efficiency-house” theory is introduced to convert the multiple objectives into a single optimization target. A two-layer feedforward artificial neural network (ANN) and the Kriging model were combine based on a hybrid approximate model and solved with swarm intelligence for global best parameters that would maximize the pump efficiency. The pump performance is predicted using three-dimensional Reynolds-averaged Navier–Stokes equations which is validated by the experimental test. With ANN, Kriging, and a hybrid approximate model, an optimization strategy is built to widen the high-efficiency range of the double-suction centrifugal pump at overload conditions by 1.63%, 1.95%, and 4.94% for flow conditions 0.8Qd, 1.0Qd, and 1.2Qd, respectively. A higher fitting accuracy is achieved for the hybrid approximation model compared with the single approximation model. A complete optimization platform based on efficiency-house and the hybrid approximation model is built to optimize the model double-suction centrifugal pump, and the results are satisfactory.

Credit risk evaluation model with textual features from loan descriptions for P2P lending

Abstract Previous studies on the peer-to-peer (P2P) lending market have shown that borrowers’ default status is related to textual factors derived from the loan descriptions in loan applications. However, textual loan descriptions have not been fully explored in terms of their possible utility in credit risk evaluation models. In this study, we propose a new approach to construct a credit risk assessment model for the P2P lending market. This approach first utilizes a Transformer encoder to extract the textual features from the loan description, and then combines them with the hard features derived from the loan application; together, they comprise the final features of a loan. Finally the combined features are fed into a two-layer feed-forward neural network to predict the loan’s default probability. We perform empirical studies on two data sets of real transactions: LendingClub loan data from the American market and Renrendai loan data from the Chinese market. The results show that the model considering the textual loan description outperforms that which does not in terms of loan default prediction. Furthermore, the model is based on the Transformer encoder achieving the best performance under the AUC and G-mean metrics.

Neural-Network-Based Root Mean Delay Spread Model for Ubiquitous Indoor Internet-of-Things Scenarios

Massive robust communication demands among machines and humans are required in ubiquitous Internet-of-Things (IoT) applications. To design the appropriate communication system, the knowledge of the propagation characteristics for various IoTs scenarios is necessary. In this article, a measurement-based neural-network-based root-mean-square (RMS) delay spread model for ubiquitous indoor IoTs scenarios is presented. The proposed model is a two-layer feedforward neural network plus a random variable, characterizing the average RMS delay spread and uncertain shadowing effect, respectively. The neural network consists of five inputs, including transmitting/receiving antennas (Tx/Rx) separation, frequency, antenna height, environment, and line-of-sight/non-line-of-sight (LOS/NLOS) propagation condition, seven hidden layer neurons, and one output layer neuron. Compared with different configurations of the neural network, the hyperbolic tangent sigmoid functions and the Levenberg–Marquardt backpropagation algorithm are selected as neurons’ activation functions and training method, respectively. Additionally, the random variable is found to follow the normal distribution using the maximum-likelihood estimation. Finally, the novel model is experimentally validated to be accurate, general, and extensible compared with the conventional normally distributed RMS delay spread model. This model is well applicable to the design and planning of the ubiquitous communication links for future IoTs scenarios.

Modeling and optimizing the removal of cadmium by Sinapis alba L. from contaminated soil via Response Surface Methodology and Artificial Neural Networks during assisted phytoremediation with sewage sludge

The study was aimed to model and optimize the removal of cadmium from contaminated post-industrial soil via Sinapis alba L. by comparing two modeling approaches: Response Surface Methodology (RSM) and Artificial Neural Networks (ANN). The experimental design was done using the Box–Behnken Design method. In the RSM model, the quadratic model was shown to predict the closest results in comparison to our experimental data. For ANN approach, a two-layer Feed-Forward Back-Propagation Neural Network model was designed. The results showed that sewage sludge supplementation increased the efficiency of the Sinapis alba plant in removing Cd from the soil. After 28 days of exposure, the removal rate varied from 10.96% without any supplementation to 65.9% after supplementation with the highest possible (law allowed) dose of sewage sludge. The comparison proved that the prediction capability of the ANN model was much higher than that of the RSM model (adjusted R-square: 0.98, standard error of the Cd prediction removal: 0.85 ± 0.02). Thus, the ANN model could be used for the prediction of heavy metal removal during assisted phytoremediation with sewage sludge. Moreover, such approach could also be used to determinate the dose of sewage sludge that will ensure highest process efficiency.