Three-layer Multilayer Perceptron Research Articles

An Expressway ETC Missing Data Restoration Model Considering Multi-Attribute Features

Electronic toll collection (ETC) data mining has become one of the hotspots in the research of intelligent expressway extension applications. Ensuring the integrity of ETC data stands as a critical measure in upholding data quality. ETC data are typical structured data, and although deep learning holds great potential in the ETC data restoration field, its applications in structured data are still in the early stages. To address these issues, we propose an expressway ETC missing transaction data restoration model considering multi-attribute features (MAF). Initially, we employ an entity embedding neural network (EENN) to automatically learn the representation of categorical features in multi-dimensional space, subsequently obtaining embedding vectors from networks that have been adequately trained. Then, we use long short-term memory (LSTM) neural networks to extract the changing patterns of vehicle speeds across several continuous sections. Ultimately, we merge the processed features with other features as input, using a three-layer multilayer perceptron (MLP) to complete the ETC data restoration. To validate the effectiveness of the proposed method, we conducted extensive tests using real ETC datasets and compared it with methods commonly used for structured data restoration. The experimental results demonstrate that the proposed method significantly outperforms others in restoration accuracy on two different datasets. Specifically, our sample data size reached around 400,000 entries. Compared to the currently best method, our method improved the restoration accuracy by 19.06% on non-holiday ETC datasets. The MAE and RMSE values reached optimal levels of 12.394 and 23.815, respectively. The fitting degree of the model to the dataset also reached its peak (R2 = 0.993). Meanwhile, the restoration stability of our method on holiday datasets increased by 5.82%. An ablation experiment showed that the EENN and LSTM modules contributed 7.60% and 9% to the restoration accuracy, as well as 4.68% and 7.29% to the restoration stability. This study indicates that the proposed method not only significantly improves the quality of ETC data but also meets the timeliness requirements of big data mining analysis.

Neural Modelling in the Study of the Relationship between Herd Structure, Amount of Manure and Slurry Produced, and Location of Herds in Poland

In the presented study, data regarding the size and structure of cattle herds in voivodeships in Poland in 2019 were analysed and modelled using artificial neural networks (ANNs). The neural modelling approach was employed to identify the relationship between herd structure, biogas production from manure and slurry, and the geographical location of herds by voivodeship. The voivodeships were categorised into four groups based on their location within Poland: central, southern, eastern, and western. In each of the analysed groups, a three-layer MLP (multilayer perceptron) with a single hidden layer was found to be the optimal network structure. A sensitivity analysis of the generated models for herd structure and location within the eastern group of voivodeships revealed significant contributions from dairy cows, heifers (both 6–12 and 12–18 months old), calves, and bulls aged 12–24 months. For the western voivodeships, the analysis indicated that only dairy cows and herd location made significant contributions. The optimal models exhibited similar values of RMS errors for the training, testing, and validation datasets. The model characterising biogas production from manure in southern voivodeships demonstrated the smallest RMS error, while the model for biogas from manure in the eastern region, as well as the model for slurry in central parts of Poland, yielded the highest RMS errors. The generated ANN models exhibited a high level of accuracy, with a fitting quality of approximately 99% for correctly predicting values. Comparable results were obtained for both manure and slurry in terms of biogas production across all location groups.

Development of a hybrid neural network model for mine detection by using ultrawideband radar data

The object of the study is the architecture of a hybrid neural network for mine recognition using ultra-wideband radar data. The work solves the problem of filtering reflected signals with interference and recognizing mines detected by ultra-wideband (UWB) radar. A hybrid neural network model in combination with the Adam learning algorithm is proposed. Filtering of reflected signals from mines is carried out using an MLP (multilayer perceptron) filter, which selects low-amplitude parts of signals that carry information about a hidden mine from the entire reflected signal. Mine recognition is carried out by a Hilbert block and an oscillatory neural network, which are included in the structure of a hybrid neural network. The peculiarity of the obtained results, which allowed to solve the investigated problem, is the transformation of the signal frequency by the Hilbert block and the recognition of mines by the oscillatory neural network in the resonant mode. The three-layer MLP filter effectively filters out the unwanted component in the total signal reflected from the subsurface object, as the MSE (Mean Squared Error) of the MLP filter is 1·10-5. If the frequency of the Hilbert signal is equal to the natural frequency of oscillations of neurons then the recognition of signals with a small amplitude from subsurface objects is carried out by an oscillatory neural network based on the resonant amplitude, which is indicated by a small value of cross-entropy. The proposed model of a hybrid neural network provides amplification of useful signals due to resonance and has higher performance compared to existing models of artificial neural networks. The practical significance of the obtained results lies in their application in the field of automated neural network technologies for detection and recognition of subsurface objects of various nature based on reflected radar signals with an amplitude at the noise level

Spin–Orbit Torque-Driven Memristor in L10 FePt Systems with Nanoscale-Thick Layers for Neuromorphic Computing

In this study, a memristor driven by spin–orbit torque (SOT) is realized in the nanoscale thickness L10 FePt systems with high perpendicular magnetization anisotropy (PMA). Due to the domain nucleation and expansion driven by current pulses, multilevel Hall resistance states can be continuously tuned by current density, where the memristive states are retained by the domain wall pinning effects. The properties of multilevel resistance states for samples with different structures are associated with the magnitude of field-like torque, and the larger efficiency of field-like torque enhances multiple resistance characteristics. Furthermore, the stable memristive behavior is obtained in the FePt/MgO/NiFe heterostructure. Finally, a three-layer multilayer perceptron (MLP) neural network is built to perform the MNIST handwritten digit recognition task based on the device’s memristive behaviors, and the accuracy of weight update can reach up to 88.55%. These results pave the way for the application of L10 FePt nanomaterials in neuromorphic computing.

Deployment of the Microeconomic Consumer Theory in the Artificial Neural Networks Modelling: Case of Organic Food Consumption

Organic food consumption has become a significant trend in consumer behaviour, determined by various motives, among which the price does not play a major role, thus reflecting the Lancaster approach to the microeconomic consumer theory. Additionally, artificial neural networks (ANNs) have proven to have significant potential in providing accurate and efficient models for predicting consumer behaviour. Considering these two trends, this study aims to deploy the Lancaster approach in the emerging area of artificial intelligence. The paper aims to develop the ANN-based predictive model to investigate the relationship between organic food consumption, demographic characteristics, and health awareness attitudes. Survey research has been conducted on a sample of Croatian inhabitants, and ANN models have been used to assess the importance of various determinants for organic food consumption. A Three-layer Multilayer Perceptron Neural Networks (MLPNN) structure has been constructed and validated to select the optimal number of neurons and transfer functions. One layer is used as the first input, while the other two are hidden layers (the first covers the radially symmetrical input, sigmoid function; the second covers the output, softmax function). Three versions of the testing, training, and holdout data structures were used to develop ANNs. The highest accuracy was achieved with a 7-2-1 partition. The best ANN model was determined as the model that was showing the smallest percent of incorrect predictions in the holdout stage, the second-lowest cross-entropy error, the correct classification rate, and the area under the ROC curve. The research results show that the availability of healthy food shops and consumer awareness of these shops strongly impacts organic food consumption. Using the ANN methodology, this analysis confirmed the validity of the Lancaster approach, stating that the characteristics or attributes of goods are defined by the consumer and not by the product itself.

A High-Performance Customer Churn Prediction System Based on Self-Attention

Customer churn prediction is a challenging domain of research that contributes to customer retention strategy. The predictive performance of existing machine learning models, which are often adopted by churn communities, appear to be at a bottleneck, partly due to models' poor feature extraction capability. Therefore, a novel algorithm, a hybrid neural network with self-attention enhancement (HNNSAE), is proposed in this paper to improve the efficiency of feature screening and feature extraction, consequently improving the model's predictive performance. This model consists of three main blocks. The first block is the entity embedding layer, which is employed to process the categorical variables transformed into 0-1 code. The second block is the feature extractor, which extracts the significant features through the multi-head self-attention mechanism. In addition, to improve the feature extraction effect, we stack the residual connection neural network on multi-head self-attention modules. The third block is a classifier, which is a three-layer multilayer perceptron. This work conducts experiments on publicly available dataset related to commercial bank customers. The result demonstrates that HNNSAE significantly outperforms the other Individual Machine Learning (IML), Ensemble Machine Learning (EML), and Deep Learning (DL) methods tested in this paper. Furthermore, we compare the performance of the feature extractor proposed in this paper with that of other three feature extractors and find that the method proposed in this paper significantly outperforms other methods. In addition, four hypotheses about model prediction performance and overfitting risk are tested on the publicly available dataset.

Development of a Machine Learning Classifier for Brain Tumors Diagnosis Based on DNA Methylation Profile.

Background: More than 150 types of brain tumors have been documented. Accurate diagnosis is important for making appropriate therapeutic decisions in treating the diseases. The goal of this study is to develop a DNA methylation profile-based classifier to accurately identify various kinds of brain tumors. Methods: Thirteen datasets of DNA methylation profiles were downloaded from the Gene Expression Omnibus (GEO) database, of which GSE90496 and GSE109379 were used as the training set and the validation set, respectively, and the remaining 11 sets were used as the independent test set. The random forest algorithm was used to select the CpG sites based on the importance of the features and a multilayer perceptron (MLP) model was trained to classify the samples. Deconvolution with the debCAM package was used to explore the cellular composition difference among tumors. Results: From training datasets with 2,801 samples, 396,568 CpG sites were retained after preprocessing, of which 767 were selected as the modeling features. A three-layer MLP model was developed, which consists of 1,320 nodes in the hidden layer, to predict the histological types of brain tumors. The prediction accuracy is 99.2, 87.0, and 96.58%, respectively, on the training, validation and test sets. The results of deconvolution analysis showed that the cell proportions of different tumor subtypes were different, and it is approximately enough to distinguish different tumor entities. Conclusion: We developed a classifier that is robust for the classification of central nervous system tumors, and tried to analyze the reasons for the classification performance.

A Temporally Smoothed MLP Regression Scheme for Continuous Knee/Ankle Angles Estimation by Using Multi-Channel sEMG

With coexisting-cooperative-cognitive robots gradually appearing in daily life, an instinct and efficient human-robot interaction (HRI) is becoming more and more challenging and necessary. Surface electromyography (sEMG) signals, as one of mainstream manners of the interactions, are employed to predict human intentions. In this paper, to provide natural assistance for standing up and sitting down, sEMG signals acquired from active muscles of one’s lower limb are utilized to predict continuous movements. A temporally smoothed multilayer perceptron (MLP) regression scheme is proposed for continuous knee/ankle angles estimation by multi-channel sEMG signals. After correlation analyses of sEMG signals and movements, a series of linear and nonlinear regression models are trained to decode human intentions from pre-processed sEMG. Furthermore, to remove out local fluctuations of direct mappings, temporally smoothed techniques are further implemented as post-processings. In the experiments of standing up and sitting down, extensive results of ten healthy subjects show that a three-layer MLP with the Savitzky-Golay filter achieves the best performance on the mean squared error (MSE, testing: 59.58) and the $R^{2}$ score ( $R^{2}$ , testing: 0.948). The proposed regression scheme is compared with other methods and is also verified by measurements of a high-precision visual motion capture system.

An Application of ANN Trained by ABC Algorithm for Classification of Wheat Grains

Artificial Neural Networks (ANNs) have emerged as an important tool for classification problem. This paper presents an application of ANN model trained by artificial bee colony (ABC) optimization algorithm for classification the wheat grains into bread and durum. ABC algorithm is used to optimize the weights and biases of three-layer multilayer perceptron (MLP) based ANN. The classification is carried out through data of wheat grains (#200) acquired using image-processing techniques (IPTs). The data set includes five grain’s geometric parameters: length, width, area, perimeter and fullness. The ANN-ABC model input with the geometric parameters are trained through 170 wheat grain data and their accuracies are tested via 30 data. The ANN-ABC model numerically calculate the outputs with mean absolute error (MAE) of 0.0034 and classify the grains with accuracy of 100% for the testing process. The results of ANN-ABC model are compared with other ANN models trained by 4 different learning algorithms. These results point out that the ANN trained by ABC optimization algorithm can be successfully applied to classification of wheat grains.

Artificial Neural Network (ANN) Modeling for Prediction of Pesticide Wastewater Degradation by FeGAC/H2O2Process

The study examined artificial neural network (ANN) modeling for the prediction of chlorpyrifos, cypermethrin and chlorothalonil pesticides degradation by the FeGAC/H2O2process. The operating condition was the optimum condition from a series of experiments. Under these conditions; FeGAC 5 g/L, H2O2concentration 100 mg/L, pH 3 and 60 min reaction time, the COD removal obtained was 96.19%. The ANN model was developed using a three-layer multilayer perceptron (MLP) neural network to predict pesticide degradation in terms of COD removal. The configuration of the model with the smallest mean square error (MSE) of 0.000046 contained 5 inputs, 9 hidden and, 1 output neuron. The Levenberg–Marquardt backpropagation training algorithm was used for training the network, while tangent sigmoid and linear transfer functions were used at the hidden and output neurons, respectively. The predicted results were in close agreement with the experimental results with correlation coefficient (R2) of 0.9994 i.e. 99.94% showing a close agreement to the actual experimental results. The sensitivity analysis showed that FeGAC dose had the highest influence with relative importance of 25.33%. The results show how robust the ANN model could be in the prediction of the behavior of the FeGAC/H2O2process.

Automatic identification of ramie and cotton fibers based on iodine blue reaction, Part II: Automatic identification of ramie and cotton using color, texture, shape, and stripes

A new method of automatically identifying ramie and cotton fibers using analysis of shape, color, texture, and surface stripes is here introduced. The shape, color, texture, and stripe features of the fibers were extracted by transformation from red, green, blue images to hue saturation intensity images and then to grayscale and binary images, by segmentation of the fiber from background, by edge detection of the outline of fibers, and by stripes on the surface of the fiber. Eighteen characteristic parameters suitable for identification were selected according to their probability distribution curves. A three-layer multilayer perceptron artificial-neural-network-based prediction system is here presented as a means of distinguishing cotton fibers from ramie fibers. The system training was carried out using a back propagation algorithm. The proposed system was tested on more than 2000 cotton and more than 2000 ramie fibers. The experimental results showed that the overall tolerance for false identification of cotton or ramie fiber was under 5%.

Prediction of outcome of construction dispute claims using multilayer perceptron neural network model

The occurrence of disputes in Indian construction contracts results in damaging the relationship between the parties apart from the time and cost overruns. However, if the parties to a dispute can predict the outcome of the dispute with some certainty, they are more likely to settle the matter out of court resulting in the avoidance of expenses and aggravation associated with adjudication. Dispute resolution process is mainly based upon the facts about the case like conditions of the contracts; actual situations on site; documents presented during arbitrational proceedings, etc., which are termed as ‘intrinsic factors’ in this research. These facts and evidences being intrinsic to the cases have been explored by researchers to develop dispute resolution mechanisms. This study focuses on determining the intrinsic factors for construction disputes related to claims raised due to variation from 72 arbitration awards through Case Study approach and furthermore statistically proving their importance in arbitral decision making by seeking professional cognizance through a questionnaire survey. It also further asserts the feasibility of the multilayer perceptron neural network approach based on the intrinsic factors existing in the construction dispute case for predicting the outcome of a dispute. Data from 204 variation claims from the awards is employed for developing the model. A three-layer multilayer perceptron neural network was appropriate in building this model, which has been trained, validated, and tested. The tool so developed would result in dispute avoidance, to some extent, and would reduce the pressure on the Indian judiciary.

Spatial Disaggregation of Areal Rainfall Using Two Different Artificial Neural Networks Models

The objective of this study is to develop artificial neural network (ANN) models, including multilayer perceptron (MLP) and Kohonen self-organizing feature map (KSOFM), for spatial disaggregation of areal rainfall in the Wi-stream catchment, an International Hydrological Program (IHP) representative catchment, in South Korea. A three-layer MLP model, using three training algorithms, was used to estimate areal rainfall. The Levenberg–Marquardt training algorithm was found to be more sensitive to the number of hidden nodes than were the conjugate gradient and quickprop training algorithms using the MLP model. Results showed that the networks structures of 11-5-1 (conjugate gradient and quickprop) and 11-3-1 (Levenberg-Marquardt) were the best for estimating areal rainfall using the MLP model. The networks structures of 1-5-11 (conjugate gradient and quickprop) and 1-3-11 (Levenberg–Marquardt), which are the inverse networks for estimating areal rainfall using the best MLP model, were identified for spatial disaggregation of areal rainfall using the MLP model. The KSOFM model was compared with the MLP model for spatial disaggregation of areal rainfall. The MLP and KSOFM models could disaggregate areal rainfall into individual point rainfall with spatial concepts.

Nitrate and Sulfate Estimations in Water Sources Using a Planar Electromagnetic Sensor Array and Artificial Neural Network Method

The primary advantages of planar electromagnetic sensors can be listed as low cost, convenient, suitable for in situ measurement systems, rapid reaction, and highly durable. In this paper, the outputs of a planar electromagnetic sensors array were observed and analyzed after testing it with different types of water samples at different concentrations. The output parameters were derived to decompose by wavelet transform. The energy and mean features of decomposed signals were extracted and used as inputs for an artificial neural network (ANN) model. The analysis model was targeted to classify the amount of nitrate and sulfate contamination in water. Nitrates and sulfate samples in the form of KNO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> and K <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> SO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sub> , each having different concentrations between 5 and 114 mg dissolved in 1 L of distilled water, were used. Furthermore, the analysis model was tested with seven sets of mixed KNO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> and K <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> SO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sub> water samples. A three-layer multilayer perceptron is used as a classifier. It is understood from the results that the model can detect the presence of nitrate and sulfate added in distilled water and is capable of distinguishing the concentration level in the presence of other types of contamination with a root mean square error (RMSE) of 0.0132. The validity of the ANN model was verified by removing the ANN model in estimating the water contamination, where the RMSE rose to 0.0977. The system and approach presented in this paper have the potential to be used as a useful low-cost tool for water source monitoring.

Modeling neural circuits in Parkinson's disease.

Parkinson's disease (PD) is caused by abnormal neural activity of the basal ganglia which are connected to the cerebral cortex in the brain surface through complex neural circuits. For a better understanding of the pathophysiological mechanisms of PD, it is important to identify the underlying PD neural circuits, and to pinpoint the precise nature of the crucial aberrations in these circuits. In this paper, the general architecture of a hybrid Multilayer Perceptron (MLP) network for modeling the neural circuits in PD is presented. The main idea of the proposed approach is to divide the parkinsonian neural circuitry system into three discrete subsystems: the external stimuli subsystem, the life-threatening events subsystem, and the basal ganglia subsystem. The proposed model, which includes the key roles of brain neural circuit in PD, is based on both feed-back and feed-forward neural networks. Specifically, a three-layer MLP neural network with feedback in the second layer was designed. The feedback in the second layer of this model simulates the dopamine modulatory effect of compacta on striatum.

Artificial Neural Network to Predict Leaf Population Chlorophyll Content from Cotton Plant Images

Leaf population chlorophyll content in a population of crops, if obtained in a timely manner, served as a key indicator for growth management and diseases diagnosis. In this paper, a three-layer multilayer perceptron (MLP) artificial neural network (ANN) based prediction system was presented for predicting the leaf population chlorophyll content from the cotton plant images. As the training of this prediction system relied heavily on how well those leaf green pixels were separated from background noises in cotton plant images, a global thresholding algorithm and an omnidirectional scan noise filtering coupled with the hue histogram statistic method were designed for leaf green pixel extraction. With the obtained leaf green pixels, the system training was carried out by applying a back propagation algorithm. The proposed system was tested to predict the chlorophyll content from the cotton plant images. The results using the proposed system were in sound agreement with those obtained by the destructive method. The average prediction relative error for the chlorophyll density (μg cm −2) in the 17 testing images was 8.41%.

Incorporating statistical and neural network approaches for student course satisfaction analysis and prediction

Students’ perception on course satisfaction through student surveys has become more influential in institutional operations because their experience in study may affect not only the prospective student’s decision in choosing the institution for their tertiary education, but also the retention of existing students. Student course satisfaction is a multivariate nonlinear problem. Neural network (NN) techniques have been successfully applied to approximating nonlinear functions in many disciplines, but there has been little information available in applying NN to the modelling of student course satisfaction. In this paper, based on the student survey results collected from 43 courses in 11 semesters from 2002 to 2007, statistical analysis and NN techniques are incorporated for establishing some dynamic models for analysing and predicting student course satisfaction. The factors identified from this process also allow new strategies to be drawn for improving course satisfaction in the future. This study shows that both the number of students (NS) enrolled to a course and the high distinction (HD) rate in final grading are the two most influential factors to student course satisfaction. The three-layer multilayer perceptron (MLP) models outperform the linear regressions in predicting student course satisfaction, with the best outcome being achieved by combining both NS and HD as the input to the networks.

Classification of lamb carcass using machine vision: Comparison of statistical and neural network analyses

In this study, the ability of artificial neural network (ANN) models to predict the lamb carcass grades using features extracted from lamb chop images was compared with multivariate statistical model (discriminant function analysis (DFA)) with respect to the classification accuracy. Twelve geometric features were extracted from each of the acquired lamb chop images. In addition, 136 texture features (90 co-occurrence, 10 run length and 36 grey-level difference histogram) were also extracted from the acquired images. Four sets of reduced features comprising six geometric, eight co-occurrence texture, four run length texture and four grey-level difference histogram features were generated based on the results of dimensionality reduction. The four sets of reduced features, individually and in different combinations, were utilised for classification using ANN and DFA. Several network configurations were tested and the classification accuracy of 96.9% was achieved from the three-layer multi-layer perceptron (MLP) network. Its performance was 12% better than that from the DFA. Geometric features play a very important role in classification. Co-occurrence features also play an equally important role in classification.

Gradual wear monitoring of turning inserts using wavelet analysis of ultrasound waves

Discrete wavelet transforms of ultrasound waves is used to measure the gradual wear of carbide inserts during turning operations. Ultrasound waves, propagating at a nominal frequency of 10 MHz, were pulsed into the cutting tools towards the cutting edge at a burst frequency of 10 KHz. The reflected waves off the mark, nose and flank surfaces were digitized at a sampling rate of 100 MHz. Daubechies Quadrature Mirror Filter pair was used to decompose ultrasound signals into frequency packets using a tree structure. Normalized signals in each level of decomposition were used to search for a neural network architecture that correlates the ultrasound measurements to the wear level on the tool. A three-layer Multi-Layer Perceptron architecture yielded the best correlation (95.9%) using the wave packets from the fourth level of decomposition with frequencies 3.75–4.375 and 5.625–6.875 MHz.

Tool Chatter Monitoring in Turning Operations Using Wavelet Analysis of Ultrasound Waves

This paper presents a new method for tool chatter monitoring using the wavelet analysis of ultrasound waves. Ultrasound waves are pulsed through the cutting tool towards the nose and are reflected back off the cutting edge. Fluctuating states of contact and non-contact between the tool insert and the workpiece, which are generated as a result of tool chatter, affect the amount of the transmitted ultrasound energy into the workpiece material and, in turn, the amount of the reflected energy. The change in the energy of the echo signals can be related directly to the severity and frequency of tool chatter. Wavelet packet analysis was used to filter the ultrasound signals. A three-layer multilayer perceptron (MLP) artificial neural network (ANN) was used to correlate the response of the ultrasound sensor to the accelerometer measurement of tool chatter. The main advantage of the ultrasound sensor is its ability to monitor other parameters such as the first contact of the tool and workpiece tool chipping and flank gradual tool wear. Experimental results show that the severity of tool chatter can be successfully monitored using the proposed ultrasound system. The system response to various frequency levels of tool chatter was investigated, however, the measurement of the chatter frequency is beyond the system capability at the current time.