Multilayer Perceptron Approach Research Articles

An efficient attention-driven deep neural network approach for continuous estimation of knee joint kinematics via sEMG signals during running

The smooth interaction and coordination between lower-limb amputees and their prosthetics is crucial when performing complex tasks such as running. To address this, simultaneous and proportional control (SPC) based on continuous estimation of joint angles through surface electromyogram (sEMG) signals offers a more seamless interaction between users and their prosthetic limbs than conventional pattern recognition-based control, which relies on recognizing pre-defined movement classes from specific sEMG patterns using classification algorithms. This study proposes a deep learning-based model (AM-BiLSTM) that integrates the attention mechanism (AM) and bidirectional long short-term memory (BiLSTM) network to provide an accurate and robust SPC for estimating knee joint angle during running from a minimal number of sEMG sensors. The sEMG signals of four muscles recorded from 14 subjects during treadmill running at various speeds were utilized by the AM-BiLSTM model to decode knee joint kinematics. To comprehensively investigate the generalizability of the proposed model, it was tested in intra-subject and speed, intra-subject and inter-speed, and inter-subject and speed scenarios and compared with BiLSTM, standard LSTM, and multi-layer perceptron (MLP) approaches. Normalized root-mean-square error and correlation coefficient were used as performance metrics. According to nonparametric statistical tests, the proposed AM-BiLSTM model significantly outperformed the BiLSTM network as well as classical techniques including LSTM and MLP networks in all three experiments (p-value < 0.05) and achieved state-of-the-art performance. Based on our findings, the AM-BiLSTM model may facilitate the deployment of intuitive user-driven deep learning-based control schemes for a wide variety of miniaturized robotic lower-limb devices, including myoelectric prostheses.

A Model for Robot Arm Pattern Identification using K-Means Clustering and Multi-Layer Perceptron

Predictive maintenance of industrial machines is one of the challenging applications in Industry 4.0. This paper presents a comprehensive methodology to identify robot arm (SCARA) movement patterns to detect the mechanical aging of the robot, which is determined by the abnormal movement of the robot arm. The dataset used is two robot arm movements that go from point A to B and then back to point A. Accelerometer data is used to measure the signal of SCARA actions, mainly focus on the non-linear movement. The identification of the movement pattern of the robot arm is made by combining k-means and multilayer perceptron. The proposed approach first extracts valuable features as characteristics of the two datasets from the time domain statistical value parameters. K-means clustering technique is initiated to label the training dataset. In this phase, the elbow curve is used to determine the number of clusters in the dataset, which is 2 clusters. Moreover, the assumption is used to determine which cluster is labeled as a normal and abnormal movement. Hence, a multilayer perceptron approach is proposed to predict the testing dataset. The proposed multilayer perceptron model yields an accuracy of 94.14%, whereas its cross-validation yields an accuracy of 96.12%.

MLP (multi-layer perceptron) and RBF (radial basis function) neural network approach for estimating and optimizing 6-gingerol content in Zingiber officinale Rosc. in different agro-climatic conditions

The principal bioactive phenolic component of ginger (Zingiber officinale Rosc.) rhizome is 6-gingerol, which is known to have several sensory and therapeutic properties. Due to its antimicrobial, anti-biofilm, anti-cancer, and anti-inflammatory properties, a sizable global market for ginger has been generated. The variation in yield of 6-gingerol is observed with changes in environmental, agronomic, and genetic conditions. The current study aims to examine how environmental and climatic factors affect 6-gingerol content using an artificial neural network (ANN) model and predict a new location to produce 6-gingerol. ANN-based models; MLP (multilayer perceptron) and RBF (radial basis function) with the desirable learning algorithms, transfer functions, number of neurons, and hidden layers were designed. The dataset of 60 unique Zingiber officinale rhizomes obtained from ten agro-climatic zones of Odisha was included in this study. The content of 6-gingerol was ascertained using a high-performance thin-layer chromatography (HPTLC) technique. The experimental dataset was randomly divided into 70 % for training, 15 % for testing, and 15 % for validation to carry out the optimization. In 60 accessions from various locations of Odisha, the concentration of 6-gingerol in ginger rhizomes ranged from 0.203 % to 0.813 % on a wet weight basis. It was revealed that the ANN model with the MLP algorithm was superior to the RBF algorithm. The findings showed that an ANN-MLP architecture with a single hidden layer of 11 neurons and a 16–11–1 topology could accurately predict the content of 6-gingerol, with an R2 (coefficient of determination) of 0.99. Altitude, pH, maximum temperature, and phosphorus were identified as the factors that influenced the model's prediction accuracy, which was 95 %. The content of 6-gingerol was enhanced from 0.363 % to 0.408 % by regulating the present model's sensitive parameters. The current study concluded that the developed ANN model efficiently predicts the best region for optimum 6-gingerol yield.

Using Deep Learning to Map Ionospheric Total Electron Content over Brazil

The low-latitude ionosphere has an active behavior causing the total electron content (TEC) to vary spatially and temporally very dynamically. The solar activity and the geomagnetic field have a strong influence over the spatiotemporal distribution of TEC. These facts make it a challenge to attempt modeling the ionization response. Single frequency GNSS users are particularly vulnerable due to these ionospheric variations that cause degradation of positioning performance. Motivated by recent applications of machine learning, temporal series of TEC available in map formats were employed to build an independent TEC estimator model for low-latitude environments. A TEC dataset was applied along with geophysical indices of solar flux and magnetic activity to train a feedforward artificial neural network based on a multilayer perceptron (MLP) approach. The forecast for the next 24 h was made relying on TEC maps over the Brazilian region using data collected on the previous 5 days. The performance of this approach was evaluated and compared with real data. The accuracy of the model was evaluated taking into account seasonality, spatial coverage and dependence on solar flux and geomagnetic activity indices. The results of the analysis show that the developed model has a superior capacity describing the TEC behavior across Brazil, when compared to global ionosphere maps and the NeQuick G model. TEC predictions were applied in single point positioning. The achieved errors were 27% and 33% lower when compared to the results obtained using the NeQuick G and global ionosphere maps, respectively, showing success in estimating TEC with small recent datasets using MLP.

Assessment and Prediction of the Water Quality Index for the Groundwater of the Ghiss-Nekkor (Al Hoceima, Northeastern Morocco)

Water quality index (WQI) is the primary method applied to characterize water quality in the world. The current study employed the statistical analysis and multilayer perceptron (MLP) approaches for predicting groundwater quality in the Ghiss-Nekkor aquifer, northeast of Al Hoceima, Morocco. Fifty sampled groundwater were identified and analyzed for major anions and cations throughout May 2019. Several physicochemical parameters of all the samples were identified in this investigation, such as TDS, pH, EC, Na, K, Ca, Mg, HCO3, NO3, Br, SO4, and Cl. The entropy-weighted groundwater quality index (EWQI) was calculated from these parameters. The WQI procedure determined the suitability of groundwater for consumption. The WQI value varied from 90.98 to 337.28. The EC, TDS, WQI, and Cl− spatial distribution showed that EC and Cl− are associated with poor groundwater quality. A single sample (W16) represented unsuitable water for drinking purposes and offered a WQI value of 337.28, indicating poor drinking quality due to seawater intrusion, overexploitation, and harsh weather conditions. The majority of the values obtained for the parameters exceeded the recommended limit of the World Health Organization (WHO)’s guidelines for consumption. The findings show that using parameters is a straightforward method for predicting water quality indexes with sufficient and suitable precision. The MLP model shows good predictive performances in terms of the coefficient of determination R2, mean absolute error (MAE), and root-mean-square error (RMSE) with values of 0.9885, 5.8031, and 4.7211, respectively. The ANN approach was applied to develop a model that can accurately predict WQI utilizing mineralization, TH, NO3, and NO2 as inputs. The MAE for the model’s performance was calculated to be 4.72. A Bland–Altman test was used to validate that the model is suitable. Following the test, it was determined that the model is appropriate for predicting WQI, with an error of just 0.1%.

Predicting adverse drug effects: A heterogeneous graph convolution network with a multi-layer perceptron approach.

We apply a heterogeneous graph convolution network (GCN) combined with a multi-layer perceptron (MLP) denoted by GCNMLP to explore the potential side effects of drugs. Here the SIDER, OFFSIDERS, and FAERS are used as the datasets. We integrate the drug information with similar characteristics from the datasets of known drugs and side effect networks. The heterogeneous graph networks explore the potential side effects of drugs by inferring the relationship between similar drugs and related side effects. This novel in silico method will shorten the time spent in uncovering the unseen side effects within routine drug prescriptions while highlighting the relevance of exploring drug mechanisms from well-documented drugs. In our experiments, we inquire about the drugs Vancomycin, Amlodipine, Cisplatin, and Glimepiride from a trained model, where the parameters are acquired from the dataset SIDER after training. Our results show that the performance of the GCNMLP on these three datasets is superior to the non-negative matrix factorization method (NMF) and some well-known machine learning methods with respect to various evaluation scales. Moreover, new side effects of drugs can be obtained using the GCNMLP.

A comparative study on the most effective machine learning model for blast loading prediction: From GBDT to Transformer

In this paper, we present a rigorous comparative study to assess and identify the most effective machine learning model for blast loading prediction. Blast loads are known to produce catastrophic effects including structural collapse and personnel fatality. Accurate and efficient prediction of these extreme loads using empirical methods and numerical solvers remains a challenging problem. Machine learning provides a promising alternative solution, which has been increasingly used in various engineering applications. However, there is seldom any analysis or justification of the selection of machine learning method that would lead to the best performance for such applications. For example, most existing machine learning-based approaches for blast loading prediction utilise the classic multi-layer perceptron (MLP) network with no justifications of their suitability and efficiency nor attempts of leveraging other state-of-the-art neural network architectures. In this study, four well-known machine learning models, including one ensemble tree method and three neural networks of different types, are investigated to demonstrate the effectiveness of different machine learning methods for blast loading prediction. It is showcased using BLEVE (boiling liquid expanding vapour explosion) pressure prediction that the Transformer model achieves the best performance, reaching a relative error of 3.5% and R2 0.997 that outperforms the existing MLP approach (relative error 6.0%, R2 0.985) with a clear margin. This study shows that the Transformer network is an effective tool for prediction of blast loading from BLEVE as well as other explosion sources.

Privacy-Preserving and Real-Time Detection of Vehicular Congestion Using Multilayer Perceptron Approach for Internet of Vehicles

Detection of vehicular congestion has a significant effect on the transportation sector. Several research approaches have been proposed to improve accuracy and efficiency with the advent of the Internet of Vehicles (IoV). Nonetheless, the privacy issue has attracted little attention on congestion detection. In this paper, we put forward a privacy-preserving and efficient scheme for vehicular congestion detection with both horizontal and vertical partitioned vehicular sensing data using Multilayer Perceptron (MLP) approach. Particularly, this work considers that different vehicle manufacturers would choose their encryption systems, which poses a challenge on how to collaboratively train an MLP model on these heterogeneous encrypted sensing data. To address this challenge, we utilize proxy re-encryption and additive homomorphic encryption schemes for preserving the confidentiality of raw sensing data and learning models while maintaining secure computation efficiency. Formally security analysis is provided to demonstrate that our proposed scheme is able to guarantee the privacy of the MLP learning process. Finally, we implement our scheme and evaluate it with real datasets. Experimental results show the effectiveness and feasibility of our MLP learning under multiple encryption systems for vehicular congestion detection.

Comparative Evaluation of the Multilayer Perceptron Approach with Conventional ARIMA in Modeling and Prediction of COVID-19 Daily Death Cases.

COVID-19 continues to pose a dangerous global health threat, as cases grow rapidly and deaths increase day by day. This increasing phenomenon does not only affect economic policy but also international policy around the world. In this paper, Pakistan daily death cases of COVID-19, from February 25, 2020, to March 23, 2022, have been modeled using the long-established autoregressive-integrated moving average (ARIMA) model and the machine learning multilayer perceptron (MLP) model. The most befitting model is selected based on the root mean square error (RMSE), mean square error (MSE), and mean absolute error (MAE). Values of the key performance indicator (KPI) showed that the MLP model outperformed the ARIMA model. The MLP model with 20 hidden layers, which emerged as the overall most apt model, was used to predict future daily COVID-19 deaths in Pakistan to enable policymakers and health professionals to put in place systematic measures to reduce death cases. We encourage the Government of Pakistan to intensify its vaccination campaign and encourage everyone to get vaccinated.

A Comparative Study of Machine Learning Techniques for Android Malware Detection

The rapid growth and wide availability of Android applications in recent years has resulted in a spike in the number of sophisticated harmful applications targeting Android users. Because of the popularity and amount of open-sourced supported features of Android OS, cyber attackers prefer to target Android-based devices over other smartphones. Malicious programs endanger user privacy and device integrity. To address this issue, the authors investigated machine learning algorithms for detecting malware in Android in this study. They employed a static analysis approach, collecting permissions from each application's APK and then generating feature vectors based on the extracted permissions. Finally, they trained several machine learning algorithms to create classification models that can distinguish between benign and malicious applications. According to experimental findings, random forest and multi-layer perceptron approaches, which have accuracy levels of 95.4% and 95.1%, respectively, have the best classification performance.

Optimization for the minimum fuel consumption problem of a hybrid electric vehicle using mixed-integer linear programming

In this study, a mixed-integer linear programming (MILP) formulation for obtaining the operating points of hybrid electric vehicle (HEV) powertrains is proposed. This study focuses on linearizing the nonlinear terms, such as an engine fuel consumption map, or bilinear terms represented by the energy of the motor. To address this problem, a conventional piecewise linear (PWL) method and a multi-layer perceptron (MLP) regression approach are adopted. Although the optimal solution cannot be determined using a PWL approximation for the fuel consumption map, it can be obtained using an MLP regression. Furthermore, the PWL method achieves better results than the MLP approach in terms of the accuracy of its bilinear approximation. Obtaining the optimal solution using MILP helps in acquiring a Lagrange multiplication of the design variables by solving the dual problem, which allows an efficient design revision strategy to be obtained.

Machine learning analysis and prediction of N2, N2O, and O2 adsorption on activated carbon and carbon molecular sieve.

This research focuses on predicting the adsorbed amount of N2, O2, and N2O on carbon molecular sieve and activated carbon using the artificial neural network (ANN) approach. Experimental isotherm data (data set 1242) on adsorbent type, gas type, temperature, and pressure of the process adsorption were used as input datasets for network investigation utilizing the Sips and dual-site Langmuir isotherm models. The network's output has been used to assess the quantity of gas adsorbed. The Gaussian algorithm was applied as a single 98-neuron hidden layer from a radial based functions (RBF) approach, and the Bayesian regularization (BR) algorithm was used as a two-layer network deep learning from a multi-layer perceptron (MLP) approach utilizing 20 neurons. The MLP and RBF networks would have the best mean square error (MSE) after 98 and 100 epochs, respectively, validating efficiencies of 0.00008 and 0.00033, while the square of the coefficient of correlations (R2) was 0.9996 and 0.9993, respectively. The ANN weight matrix generated can accurately predict the adsorption process behavior of different carbon-based adsorbents under various process conditions for air separation and N2O adsorption. The results of this study have the potential to assist a wide range of process industries.

Modeling and optimization of vegetable oil biodiesel production with heterogeneous nano catalytic process: Multi-layer perceptron, decision regression tree, and K-Nearest Neighbor methods

Biodiesels are the renewable diesel fuels prepared from natural sources. As the production cost of biodiesels stands for the major problem for commercialization therefore in this work the Machine Learning (ML) approaches were used to simulate and optimize the biodiesel production process. Modeling biodiesel production with ML is possible without an in-depth understanding of biological systems. Here, a novel approach based on the K-Nearest Neighbor (KNN) regression, Decision Regression Tree (DT), and Multi-layer perceptron (MLP), have been suggested to predict biodiesel production yield (%) from Soybean oil through transesterification process as a function of molar ratio of methanol to oil and catalyst loading (wt.%). The performance of models was compared, and all models showed high accuracy and R2 value higher than 0.9. MLP, DT, and KNN models represented a high performance with a RMSE (root mean square error) of 4.9460E−01, 1.8596E+00, and 8.0422E−01, respectively. Although, all models were accurate for predicting the production process of biodiesel, the MLP model was found to be superior to other models in terms of its accuracy. The optimization of biodiesel production yield by MLP approach demonstrated 83.88% production yield using 10.67 molar ratio of methanol to oil and 3.45 wt% of catalyst loading.

Modeling the performance of cold plasma in CO2 splitting using artificial neural networks

Using dielectric barrier discharge reactor (DBD) to convert CO2 has attracted considerable attention, recently. The primary challenge with its industrial use, however, is the eligibility and effectiveness of this technology in CO2 conversion, as well as its cost of energy. In this research use has been made of Artificial Neural Network to investigate the effective factors on a DBD reactor, a unique modern instrument for CO2 conversion. A multilayer perceptron approach of feed-forward back-propagation (BP) has been utilized to increase both the energy and CO2 conversion efficiency (outputs) by modeling the effective factors, such as chamber size, gas flow rate, and plasma-generator power (inputs). The findings revealed that an artificial neural network can be used to explain the eligibility and efficiency. Despite the network’s complexity in terms of input and output parameters, the predicted and actual results were found to be in good agreement. The results showed that multilayer perceptron with structure 3-6-2 was the most suitable (MSE = 0.62 and R2 &amp;gt; 0.99). As a result, the artificial neural network can be utilized as a practical and effective tool in predicting the efficiency of energy and carbon dioxide conversion in a DBD reactor.

A development of a graph-based ensemble machine learning model for skin sensitization hazard and potency assessment.

Many defined approaches (DAs) for skin sensitization assessment based on the adverse outcome pathway (AOP) have been developed to replace animal testing because the European Union has banned animal testing for cosmetic ingredients. Several DAs have demonstrated that machine learning models are beneficial. In this study, we have developed an ensemble prediction model utilizing the graph convolutional network (GCN) and machine learning approach to assess skin sensitization. The model integrates in silico parameters and data from alternatives to animal testing of well-defined AOP to improve DA predictivity. Multiple ensemble models were created using the probability produced by the GCN with six physicochemical properties, direct peptide reactivity assay, KeratinoSens™, and human cell line activation test (h-CLAT), using a multilayer perceptron approach. Models were evaluated by predicting the testing set's human hazard class and three potency classes (strong, weak, and non-sensitizer). When the GCN feature was used, 11 models out of 16 candidates showed the same or improved accuracy in the testing set. The ensemble model with the feature set of GCN, KeratinoSens™, and h-CLAT produced the best results with an accuracy of 88% for assessing human hazards. The best three-class potency model was created with the feature set of GCN and all three assays, resulting in 64% accuracy. These results from the ensemble approach indicate that the addition of the GCN feature could provide an improved predictivity of skin sensitization hazard and potency assessment.

A Fuzzy MLP Approach for Nonlinear System Identification

In case of decision making problems, identification of nonlinear systems is an important issue. Identification of nonlinear systems using a multilayer perceptron (MLP) trained with back propagation becomes much complex with an increase in number of input data, number of layers, number of nodes, and number of iterations in computation. In this paper, an attempt has been made to use fuzzy MLP and its learning algorithm for identification of nonlinear system. The fuzzy MLP and its training algorithm which allows to accelerate a process of training, which exceeds in comparison with classical MLP is proposed. Results show a sharp reduction in search for optimal parameters of a neuro fuzzy model as compared to the classical MLP. A training performance comparison has been carried out between MLP and the proposed fuzzy-MLP model. The time and space complexities of the algorithms have been analyzed. It is observed that the number of epochs has sharply reduced and performance increased compared with classical MLP.

Multilayer perceptron models for the estimation of the attained level of education in the Italian Permanent Census

In the Italian Permanent Census, estimates of the attained level of education are derived by the integration of administrative data, 2011 census data, and sample survey data. The result of the integration procedure is the prediction of the attained level of education (ALE) for each single resident. Due to the complexity and heterogeneity of the available information, traditional statistical methods require the construction of different imputation models for different subpopulations, with a considerable effort in terms of human intervention. We study the use of a multilayer perceptron (MLP) model to make the process more automatic, i.e., less costly in terms of human resources, and possibly more accurate in terms of estimates. The MLP model is applied to Istat data referred to an Italian administrative region (Lombardia) in 2018, and the results are compared with those obtained using the official procedure. The study shows that the MLP approach is indeed less demanding in terms of human work needed for data preparation and modeling, yet it leads to estimates characterized by the same level of accuracy as the ones provided by the official procedure.

Combined Multi-Atlas and Multi-Layer Perception for Alzheimer's Disease Classification.

Alzheimer's disease (AD) is a progressive and irreversible neurodegenerative disease. To distinguish the stage of the disease, AD classification technology challenge has been proposed in Pattern Recognition and Computer Vision 2021 (PRCV 2021) which provides the gray volume and average cortical thickness data extracted in multiple atlases from magnetic resonance imaging (MRI). Traditional methods either train with convolutional neural network (CNN) by MRI data to adapt the spatial features of images or train with recurrent neural network (RNN) by temporal features to predict the next stage. However, the morphological features from the challenge have been extracted into discrete values. We present a multi-atlases multi-layer perceptron (MAMLP) approach to deal with the relationship between morphological features and the stage of the disease. The model consists of multiple multi-layer perceptron (MLP) modules, and morphological features extracted from different atlases will be classified by different MLP modules. The final vote of all classification results obtains the predicted disease stage. Firstly, to preserve the diversity of brain features, the most representative atlases are chosen from groups of similar atlases, and one atlas is selected in each group. Secondly, each atlas is fed into one MLP to fetch the score of the classification. Thirdly, to obtain more stable results, scores from different atlases are combined to vote the result of the classification. Based on this approach, we rank 10th among 373 teams in the challenge. The results of the experiment indicate as follows: (1) Group selection of atlas reduces the number of features required without reducing the accuracy of the model; (2) The MLP architecture achieves better performance than CNN and RNN networks in morphological features; and (3) Compared with other networks, the combination of multiple MLP networks has faster convergence of about 40% and makes the classification more stable.

Analysis of the coexistence of gaming and viewing activities in Twitch users and their relationship with pathological gaming: a multilayer perceptron approach

The enormous expansion of the video game sector, driven by the emergence of live video game streaming platforms and the professionalisation of this hobby through e-sports, has spurred interest in research on the relationships with potential adverse effects derived from cumulative use. This study explores the co-occurrence of the consumption and viewing of video games, based on an analysis of the motivations for using these services, the perceived positive uses, and the gamer profile. To that end, a multilayer perceptron artificial neural network is developed and tested on a sample of 970 video game users. The results show that the variables with a significant influence on pathological gaming are the motivation of a sense of belonging to the different platforms, as well as the positive uses relating to making friends and the possibility of making this hobby a profession. Furthermore, the individual effects of each of the variables have been estimated. The results indicate that the social component linked to the positive perception of making new friends and the self-perceived level as a gamer have been identified as possible predictors, when it comes to a clinical assessment of the adverse effects. Conversely, the variables age and following specific streamers are found to play a role in reducing potential negative effects.

Deep learning based dual encoder retrieval model for citation recommendation

Citation recommendation recommends relevant documents to users based on their inputs and other information. Many traditional citation recommendation models use keywords to describe item attributes and ignore the semantics of sequences, which cause the relevance of the search results unsatisfactory. This paper proposes a deep-learning-based dual encoder retrieval (DER) model, which combines a text representation technique and a sentence pair matching approach, to improve the performance of citation recommendation. First, an input query and paper titles from publication databases are encoded to semantic vectors separately by two deep-learning-based encoders. Second, the semantic vector of the input query is matched with vectors that representing papers in the published databases by the multilayer perceptron approach to compute similarity scores. Finally, a list of documents, which are sorted in descending order of similarity scores, is generated. To validate the effectiveness of the proposed approach, it is compared with five baselines using a citation dataset. The results show that the proposed model achieves the best performance in terms of accuracy, recall, F1-measure, and AUC. In addition, we compare the DER (Glove) model with Google Scholar using a small example of twenty articles. The DER (Glove) model outperformed Google Scholar in seven recommendations, and tied in ten recommendations.