Two-layer Feed-forward Neural Network Research Articles

Credit risk evaluation model with textual features from loan descriptions for P2P lending

Abstract Previous studies on the peer-to-peer (P2P) lending market have shown that borrowers’ default status is related to textual factors derived from the loan descriptions in loan applications. However, textual loan descriptions have not been fully explored in terms of their possible utility in credit risk evaluation models. In this study, we propose a new approach to construct a credit risk assessment model for the P2P lending market. This approach first utilizes a Transformer encoder to extract the textual features from the loan description, and then combines them with the hard features derived from the loan application; together, they comprise the final features of a loan. Finally the combined features are fed into a two-layer feed-forward neural network to predict the loan’s default probability. We perform empirical studies on two data sets of real transactions: LendingClub loan data from the American market and Renrendai loan data from the Chinese market. The results show that the model considering the textual loan description outperforms that which does not in terms of loan default prediction. Furthermore, the model is based on the Transformer encoder achieving the best performance under the AUC and G-mean metrics.

Neural-Network-Based Root Mean Delay Spread Model for Ubiquitous Indoor Internet-of-Things Scenarios

Massive robust communication demands among machines and humans are required in ubiquitous Internet-of-Things (IoT) applications. To design the appropriate communication system, the knowledge of the propagation characteristics for various IoTs scenarios is necessary. In this article, a measurement-based neural-network-based root-mean-square (RMS) delay spread model for ubiquitous indoor IoTs scenarios is presented. The proposed model is a two-layer feedforward neural network plus a random variable, characterizing the average RMS delay spread and uncertain shadowing effect, respectively. The neural network consists of five inputs, including transmitting/receiving antennas (Tx/Rx) separation, frequency, antenna height, environment, and line-of-sight/non-line-of-sight (LOS/NLOS) propagation condition, seven hidden layer neurons, and one output layer neuron. Compared with different configurations of the neural network, the hyperbolic tangent sigmoid functions and the Levenberg–Marquardt backpropagation algorithm are selected as neurons’ activation functions and training method, respectively. Additionally, the random variable is found to follow the normal distribution using the maximum-likelihood estimation. Finally, the novel model is experimentally validated to be accurate, general, and extensible compared with the conventional normally distributed RMS delay spread model. This model is well applicable to the design and planning of the ubiquitous communication links for future IoTs scenarios.

Modeling and optimizing the removal of cadmium by Sinapis alba L. from contaminated soil via Response Surface Methodology and Artificial Neural Networks during assisted phytoremediation with sewage sludge

The study was aimed to model and optimize the removal of cadmium from contaminated post-industrial soil via Sinapis alba L. by comparing two modeling approaches: Response Surface Methodology (RSM) and Artificial Neural Networks (ANN). The experimental design was done using the Box–Behnken Design method. In the RSM model, the quadratic model was shown to predict the closest results in comparison to our experimental data. For ANN approach, a two-layer Feed-Forward Back-Propagation Neural Network model was designed. The results showed that sewage sludge supplementation increased the efficiency of the Sinapis alba plant in removing Cd from the soil. After 28 days of exposure, the removal rate varied from 10.96% without any supplementation to 65.9% after supplementation with the highest possible (law allowed) dose of sewage sludge. The comparison proved that the prediction capability of the ANN model was much higher than that of the RSM model (adjusted R-square: 0.98, standard error of the Cd prediction removal: 0.85 ± 0.02). Thus, the ANN model could be used for the prediction of heavy metal removal during assisted phytoremediation with sewage sludge. Moreover, such approach could also be used to determinate the dose of sewage sludge that will ensure highest process efficiency.

Determination of Heavy Metal Soil Contaminants Based on Photoacoustic Spectroscopy

Heavy metal (HM) contamination in soil is a threat to human health and environmental safety. This paper presents an infrared photoacoustic spectroscopic non-destructive testing for HM evaluation by a robust customized photoacoustic spectrometric system. Cadmium (Cd) was selected as the target contamination which was blended in soil samples. A two-layer feed-forward artificial neural network (ANN) model was developed for HM concentration estimation. The results show that the standard normal variate and continuum removal methods are the best preprocessing algorithms regarding the maximal correction coefficient (R2 > 0.90) and the minimal root mean square error criteria. The correction coefficient analysis shows a better prediction performance (R2 > 0.95) for a HM concentration estimation with the two preprocessing methods. In conclusion, the prediction accuracy can be significantly improved by implementing specific preprocessing and training methods. The ANN model-based infrared photoacoustic spectroscopic method has a future potential for the featureless HM contamination estimation in soil.

Comparison of Neural Networks and Regression Analysis to Predict In-process Straightness in CNC Turning

The objective of this research is to predict the in-process straightness of aluminum (Al 6063) and carbon steel (S45C) by monitoring the in-process cutting forces during CNC turning. The Fast Fourier Transform (FFT) is adopted to prove the relation between cutting force and straightness in frequency domain, which appear the same frequency. The cutting force ratio is proposed and normalized to predict the in-process straightness regardless of the cutting conditions. Firstly, the straightness is calculated by employing the two-layer feed-forward neural networks. The Levenberg-Marquardt backpropagation algorithm is utilized to train the system. Secondly, the multiple regression analysis has been applied to model the in-process prediction of straightness and the cutting force ratio under various cutting conditions with the use of least square method at 95% confidence level. Finally, the experimentally obtained results from the neural networks are compared with the ones obtained from the developed straightness model. It had been proved that the in-process straightness can be well predicted under various cutting conditions by using the trained neural networks.

Greedy search based data-driven algorithm of centralized thermoelectric generation system under non-uniform temperature distribution

The generation efficiency of thermoelectric generation system is relatively low, thus how maximize its power production is of great importance. This paper designs a novel greedy search based data-driven method for centralized thermoelectric generation system to achieve maximum power point tracking under non-uniform temperature distribution. In order to effectively distinguish the local maximum power points and the global maximum power point under non-uniform temperature distribution, greedy search based data-driven employs a two-layer feed-forward neural network to accurately fit the curve between the power output and the controllable variable based on the real-time updated operation data. Based on the approximation curve, a greedy search is designed to efficiently approach the global maximum power point from a shrinking search space. Cases studies such as start-up test, step variation of temperature, stochastic temperature change, and analyse of sensitivity, are implemented to prove the effectiveness and superiority of the proposed algorithm. Simulation results verify that the proposed method can generate the highest energy under non-uniform temperature distribution condition, e.g., 391.34%, 115.71%, 110.92%, and 109.43% to that of perturb and observe, particle swarm optimization, whale optimization algorithm, and grey wolf optimizer in the stochastic temperature change. Lastly, the implementation feasibility of the proposed method is demonstrated by the hardware-in-the-loop experiment based on dSpace platform.

Neural Networks Based Prediction Model for Vessel Track Control

The problem of neural networks implementation for the construction of a predictive model for vessel track control was studied. It has been shown that the vessel track control problem may be considered as an approximation task, and neural networks may be implemented as universal approximating tools. The general structure of the prediction model, based on neural networks, has been developed. The model consists of several two-layered feedforward neural networks, which architectures satisfy the conditions of universal approximation properties. The analysis of the functions of the different neural networks in the prediction model has been performed. The network predicting WGS-84 geodetic latitude as a part of the predictive model has been constructed, trained and validated by using MATLAB software. The validation results show the good prediction precision of the net.

Computing interface curvature from volume fractions: A machine learning approach

The volume of fluid (VOF) method is widely used to simulate the flow of immiscible fluids. It uses a discrete and sharp volume fractions field to represent the fluid-fluid interface on a Eulerian grid. The most challenging part of the VOF method is the accurate computation of the local interface curvature which is essential for evaluation of the surface tension force at the interface. In this paper, a machine learning approach is used to develop a model which predicts the local interface curvature from neighbouring volume fractions. A novel data generation methodology is devised which generates well-balanced randomized data sets comprising of spherical interface patches of different configurations/orientations. A two-layer feed-forward neural network with different network parameters is trained on these data sets and the developed models are tested for different shapes i.e. ellipsoid, 3D wave and Gaussian. The best model is selected on the basis of specific criteria and subsequently compared with conventional curvature computation methods (convolution and height function) to check the nature and grid convergence of the model. The model is also coupled with a multiphase flow solver to evaluate its performance using standard test cases: i) stationary bubble, ii) oscillating bubble and iii) rising bubble under gravity. Our results demonstrate that machine learning is a feasible approach for fairly accurate curvature computation. It easily outperforms the convolution method and even matches the accuracy of the height function method for some test cases.

Image-based Classification of Tumor Type and Growth Rate using Machine Learning: a preclinical study

Medical images such as magnetic resonance (MR) imaging provide valuable information for cancer detection, diagnosis, and prognosis. In addition to the anatomical information these images provide, machine learning can identify texture features from these images to further personalize treatment. This study aims to evaluate the use of texture features derived from T1-weighted post contrast scans to classify different types of brain tumors and predict tumor growth rate in a preclinical mouse model. To optimize prediction models this study uses varying gray-level co-occurrence matrix (GLCM) sizes, tumor region selection and different machine learning models. Using a random forest classification model with a GLCM of size 512 resulted in 92%, 91%, and 92% specificity, and 89%, 85%, and 73% sensitivity for GL261 (mouse glioma), U87 (human glioma) and Daoy (human medulloblastoma), respectively. A tenfold cross-validation of the classifier resulted in 84% accuracy when using the entire tumor volume for feature extraction and 74% accuracy for the central tumor region. A two-layer feedforward neural network using the same features is able to predict tumor growth with 16% mean squared error. Broadly applicable, these predictive models can use standard medical images to classify tumor type and predict tumor growth, with model performance, varying as a function of GLCM size, tumor region, and tumor type.

Modelling and Optimizing Pyrene Removal from the Soil by Phytoremediation using Response Surface Methodology, Artificial Neural Networks, and Genetic Algorithm

This study aimed to model and optimize pyrene removal from the soil contaminated by sorghum bicolor plant using Response Surface Methodology (RSM) and Artificial Neural Network (ANN) with Genetic Algorithm (GA) approach. Here, the effects of indole acetic acid (IAA) and pseudomonas aeruginosa bacteria on increasing pyrene removal efficiency by phytoremediation process was studied. The experimental design was done using the Box-Behnken Design (BBD) technique. In the RSM model, the non-linear second-order model was in good agreement with the laboratory results. A two-layer Feed-Forward Back-Propagation Neural Network (FFBPNN) model was designed. Various training algorithms were evaluated and the Levenberg Marquardt (LM) algorithm was selected as the best one. Existence of eight neurons in the hidden layer leads to the highest R and lowest MSE and MAE. The results of the GA determined the optimum performance conditions. The results showed that using indole acetic acid and pseudomonas bacteria increased the efficiency of the sorghum plant in removing pyrene from the soil. The comparison obviously indicated that the prediction capability of the ANN model was much better than that of the RSM model.

Wavelet Entropy-Based Inter-subject Associative Cortical Source Localization for Sensorimotor BCI.

We propose event-related cortical sources estimation from subject-independent electroencephalography (EEG) recordings for motor imagery brain computer interface (BCI). By using wavelet-based maximum entropy on the mean (wMEM), task-specific EEG channels are selected to predict right hand and right foot sensorimotor tasks, employing common spatial pattern (CSP) and regularized common spatial pattern (RCSP). EEG from five healthy individuals (Dataset IVa, BCI Competition III) were evaluated by a cross-subject paradigm. Prediction performance was evaluated via a two-layer feed-forward neural network, where the classifier was trained and tested by data from two subjects independently. On average, the overall mean prediction accuracies obtained using all 118 channels are (55.98±6.53) and (71.20±5.32) in cases of CSP and RCSP, respectively, which are slightly lower than the accuracies obtained using only the selected channels, i.e., (58.95±6.90) and (71.41±6.65), respectively. The highest mean prediction accuracy achieved for a specific subject pair by using selected EEG channels was on average (90.36±5.59) and outperformed that achieved by using all available channels (86.07 ± 10.71). Spatially projected cortical sources approximated using wMEM may be useful for capturing inter-subject associative sensorimotor brain dynamics and pave the way toward an enhanced subject-independent BCI.

Neural Network Modelling to Characterize Steel Continuous Casting Process Parameters and Prediction of Casting Defects

The current work outlines application of a data-driven multilayer perceptron-based artificial neural network (ANN) model to characterize the influence of melt compositions, tundish temperature, tundish superheat, casting speed and mould oscillation frequency on the important processing parameters such as mould powder consumption rate, oscillation mark depth and metallurgical length during continuous casting process. A two-layer feedforward back-propagation neural network model has been developed for predicting the probability of occurrence of defect in the cast product. The network training architecture has been optimized using a gradient-based algorithm, namely the back-propagation algorithm. The neural network predictions are found to be in good agreement with regard to oscillation mark depth, mould powder consumption rate, metallurgical length and probability of occurrence of defect using data obtained from an operating Indian steel plant (Rashtriya Ispat Nigam Limited, Visakhapatnam). The ANN model prediction has been validated successfully with multiple linear regression analysis carried out on each data set.

High Degree of Freedom Hand Pose Tracking Using Limited Strain Sensing and Optical Training

The ability to track human operators' hand usage when working in production plants and factories is critically important for developing realistic digital factory simulators as well as manufacturing process control. We propose a proof-of-concept instrumented glove with only a few strain gage sensors and a microcontroller that continuously tracks and records the hand configuration during actual use. At the heart of our approach is a trainable system that can predict the fourteen joint angles in the hand using only a small set of strain sensors. First, ten strain gages are placed at various joints in the hand to optimize the sensor layout using the English letters in the American Sign Language (ASL) as a benchmark for assessment. Next, the best sensor configurations for three through ten strain gages are computed using a support vector machine (SVM) classifier. Following the layout optimization, our approach learns a mapping between the sensor readouts to the actual joint angles optically captured using a Leap Motion system. Five regression methods including linear, quadratic, and neural regression are then used to train the mapping between the strain gage data and the corresponding joint angles. The final proposed model involves four strain gages mapped to the fourteen joint angles using a two-layer feed-forward neural network (NN).

Abnormal Behavior Recognition Based on Transfer Learning

In order to identify abnormal behavior in image samples, a transfer learning recognition model based on Inception-V3 neural network is proposed in this paper. The whole process consists of two parts: 1) Inception-V3 neural network is used to extract the features of image samples.2) The abnormal behavior recognition model is established to classify the obtained features. In this paper, three-layer feedforward neural network algorithm is used to compare with other common machine learning classification models (k-nearest neighbor algorithm, random forest, LightGBM, SVM, two-layer feedforward neural network).The results show that the performance of anomalous behavior recognition based on three-layer feedforward neural network is better than other classification models.

THERMAL CONDUCTIVITY ENHANCEMENT OF AQUEOUS IONIC LIQUID AND NANOPARTICLE SUSPENSION

Abstract In the present study, the thermal conductivities of ionic liquid + water in the concentrations 0.1-0.6 %w/w and temperatures (302.5-337.9 K) are determined. The thermal conductivity of aqueous 1-butyl-3-methylimidazolium bromide declined with increasing concentration and temperature. A quadratic model is developed for predicting the thermal conductivity of aqueous ionic liquid using response surface methodology with R2 = 0.99. A two-layered feed forward back propagation neural network 2-10-1 is also modeled with mean square error, root means square and absolute average error percentage of 0.005, 0.071 and 2.684, respectively. The thermal conductivity of nanofluid (γ-Al2O3/water) is estimated and compared with the ionic liquid solution at the same concentration and temperature range. Thermal conductivity enhancement ratios of aqueous ionic liquids are found to be more than nanoparticle suspensions.

Artificial Neural Networks Approach for a Multi-Objective Cavitation Optimization Design in a Double-Suction Centrifugal Pump

Double-suction centrifugal pumps are widely used in industrial and agricultural applications since their flow rate is twice that of single-suction pumps with the same impeller diameter. They usually run for longer, which makes them susceptible to cavitation, putting the downstream components at risk. A fast approach to predicting the Net Positive Suction Head required was applied to perform a multi-objective optimization on the double-suction centrifugal pump. An L32 (84) orthogonal array was designed to evaluate 8 geometrical parameters at 4 levels each. A two-layer feedforward neural network and genetic algorithm was applied to solve the multi-objective problem into pareto solutions. The results were validated by numerical simulation and compared to the original design. The suction performance was improved by 7.26%, 3.9%, 4.5% and 3.8% at flow conditions 0.6Qd, 0.8Qd, 1.0Qd and 1.2Qd respectively. The efficiency increased by 1.53% 1.0Qd and 1.1% at 0.8Qd. The streamline on the blade surface was improved and the vapor volume fraction of the optimized impeller was much smaller than that of the original impeller. This study established a fast approach to cavitation optimization and a parametric database for both hub and shroud blade angles for double suction centrifugal pump optimization design.

A distortion-agnostic video quality metric based on multi-scale spatio-temporal structural information

Considering the high sensitivity of human visual perception towards the structural information, a general purpose no-reference video quality assessment (NR-VQA) metric based on local binary patterns (LBP) is proposed. In the proposed metric, 2D LBP method is extended to third dimension using LBP-TOP (LBP from Three Orthogonal Planes) for extracting the spatio-temporal structural information. Further, spatio-temporal structural information is evaluated for multi-resolution local pixel windows to incorporate the human vision system (HVS) characteristic of multi-scale perception. Since LBP only encodes the sign information of local pixel differences, and ignores the degree of variations, a second set of feature based on the magnitude information of local pixel differences is extracted. The magnitude information based LBP-TOP feature set is accumulated on the sign information based LBP-TOP feature set such that the resultant feature set has more spatio-temporal structural information with same feature vector length. To emulate the HVS mechanism of quality feature mapping to a scalar quantity, a simple two-layer feed-forward artificial neural network (ANN) of single hidden layer with four neurons is proposed. The performance of the proposed metric is validated on LIVE, CSIQ, and IVP video quality datasets and consistent performance is observed. Further, the performance of proposed metric is compared with the existing state-of-the-art NR-VQA and other IQA/VQA metrics and have exhibited better performance than the others. The proposed metric is highly consistent in performance with low computational complexity and thus suitable for real-time applications.

Artificial neural network based classification of lung nodules in CT images using intensity, shape and texture features

The cancer of lung has been one of the major threats to human life for decades in developed and developing countries. The Computer Aided Detection CAD could be a powerful tool for initial lung nodule detection and preventing the deaths caused by the lung tumor. In this paper, an advanced technique for lung-nodule detection by using a hybrid feature set and artificial neural network is proposed. Initially, the lung volume is segmented from the input Computed Tomography image using optimal thresholding which is followed by image enhancement using with multi scale dot augmentation filtering. Next, lung nodule candidates have been detected from enhanced image and certain features are extracted. The set feature consists of the texture features, shape 2D and 3D and intensity. Finally, lung nodule’s classification is attained using two-layer feed forward neural network. The Lung Image Database Consortium dataset has been used to evaluate the novel system which achieved a sensitivity of 95.5% with only 5.72 FP per scan.

Cat Swarm Optimization applied to alcohol use disorder identification

(Aim) Alcohol use disorder may put health at risk and cause serious health problems. It is of increasing importance to identify alcohol use disorder as early as possible. (Method) This study proposed a computer-vision based technique. The dataset was scanned by magnetic resonance imaging in China participating hospitals. Afterwards, we combined wavelet entropy, two-layer feedforward neural network, and cat swarm optimization (CSO). The CSO mimics the behavior of cat and is composed of two modes: seeking mode and tracing mode. (Results) The results showed that our method achieves a sensitivity of 91.84 ± 1.66%, a specificity of 92.40 ± 1.22%, and an accuracy of 92.13 ± 0.70%. Using grid searching approach, we found the classification performance is the best, when decomposition level is assigned with 2 and the mixture ratio is assigned with a value of 0.8. (Conclusion) The CSO is superior to four bioinspired algorithms: genetic algorithm, immune genetic algorithm, particle swarm optimization, and chaotic self-adaptive particle swarm optimization. In addition, our proposed alcoholism identification system is superior to four state-of-the-art alcoholism detection approaches.

Prediction of rupture risk in anterior communicating artery aneurysms with a feed-forward artificial neural network.

Anterior communicating artery (ACOM) aneurysms are the most common intracranial aneurysms, and predicting their rupture risk is challenging. We aimed to predict this risk using a two-layer feed-forward artificial neural network (ANN). 594 ACOM aneurysms, 54 unruptured and 540 ruptured, were reviewed. A two-layer feed-forward ANN was designed for ACOM aneurysm rupture-risk analysis. To improve ANN efficiency, an adaptive synthetic (ADASYN) sampling approach was applied to generate more synthetic data for unruptured aneurysms. Seventeen parameters (13 morphological parameters of ACOM aneurysm measured from these patients' CT angiography (CTA) images, two demographic factors, and hypertension and smoking histories) were adopted as ANN input. Age, vessel size, aneurysm height, perpendicular height, aneurysm neck size, aspect ratio, size ratio, aneurysm angle, vessel angle, aneurysm projection, A1 segment configuration, aneurysm lobulations and hypertension were significantly different between the ruptured and unruptured groups. Areas under the ROC curve for training, validating, testing and overall data sets were 0.953, 0.937, 0.928 and 0.950, respectively. Overall prediction accuracy for raw 594 samples was 94.8 %. This ANN presents good performance and offers a valuable tool for prediction of rupture risk in ACOM aneurysms, which may facilitate management of unruptured ACOM aneurysms. • A feed-forward ANN was designed for the prediction of rupture risk in ACOM aneurysms. • Two demographic parameters, 13 morphological aneurysm parameters, and hypertension/smoking history were acquired. • An ADASYN sampling approach was used to improve ANN quality. • Overall prediction accuracy of 94.8 % for the raw samples was achieved.