Multilayer Perceptron Learning Research Articles

Application of Machine Learning Models for Malware Classification With Real and Synthetic Datasets

Stacking of multiple Machine Learning (ML) classifiers have gained popularity in addressing anomalous data classification along with Deep Learning (DL) algorithms. This study compares traditional ML classifiers, multi-layer stacking ML classifiers, and DL classifiers using an open-source malware dataset-containing equal numbers of benign and malware samples. The results on the realistic dataset indicate that the DL classifier, utilizing a Bidirectional Long Short-Term Memory (BiLSTM) model, outperformed the stacked classifiers with Logistic Regression (LR) and Support Vector Machine (SVM) as Meta learners by 36.78% and 39.69%, respectively, in terms of classification accuracy and performance. The research work was extended to study the impact of Generative Adversarial Network (GAN) based synthetic dataset of relatively smaller size on deep learning models. It was observed that the Deep Learning Multi-Layer Perceptron (DLMLP) Model had relatively superior performance as compared to complex deep learning models like Long Short-Term Memory LSTM and BiLSTM

Editorial for the Special Issue: "Novel Solutions and Novel Approaches in Operational Research"

Abstract This special issue of Business Systems Research (SI of the BSR) is being co-published by the Slovenian Society INFORMATIKA – Section for Operational Research (SSI -SOR). It focuses on recent advances in Operations Research and Management Science (OR / MS), with a particular emphasis on linking OR / MS with other areas of quantitative and qualitative methods in the context of a multidisciplinary framework. The ten papers that were chosen for this Special Issue of the BSR present advancements and new techniques (methodology) in the field of Operations Research (OR), as well as their application in a variety of fields, including risk management, mathematical programming, game theory, gravity, spatial analysis, logistics, circular economy, continuous improvement, sustainability, e-commerce, forecasting, Gaussian processes, linear regression, multi-layer perceptron, and machine learning.

Predicting non-suicidal self-injury among Chinese adolescents: The application of ten algorithms of machine learning

Background and aimsHigh non-suicidal self-injury (NSSI) prevalence among adolescents is a global health issue. However, current prediction models for adolescent NSSI rely on a limited set of algorithms, resulting in biased predictions. Therefore, the aim of this study is to develop multiple machine learning models to enhance prediction accuracy and mitigate biases among Chinese adolescents. MethodsA total of 4487 junior and senior high school students in China were recruited. Multiple algorithms were included, such as logistic regression, decision tree, support vector machine, Naive Bayes, multi-layer perceptron, K-nearest neighbors, and ensemble learning algorithm like random forest, bagging, AdaBoost, and stacking to build predictive models. Data processing techniques, including standardization and the synthetic minority oversampling technique, were employed to optimize the predictive model. The model was trained on 70 % of the data, reserving 30 % for testing. ResultsThe ten prediction models achieved a good performance, with area under the receiver operating characteristic curve (AUC) scores above 0.700 in the test set. The stacking and random forest models achieved AUC scores of 0.904 and 0.898, respectively. The prediction performance of the Naive Bayes model was relatively poor. The top five important variables were resilience, bully, suicidal ideation, internet addiction, and depression. ConclusionsThe ensemble machine learning algorithm showed promising results predicting NSSI among adolescents. Such algorithms should be recommended for future NSSI research to enhance predictive accuracy. Identification of important features in NSSI prediction can help develop screening protocols and lay a foundation for clinical diagnosis and intervention in adolescent populations.

Formation of Bachelor`s Skills in the Field of Machine Learning and Intelligent Data Analysis

The article presents the content of the educational disciplines “Workshop on Python Programming” and “Fundamentals of Artificial Intelligence”, included in the module “Data Analysis. Machine learning. Artificial Intelligence”, implemented in Ural universities. Modern teachers note that when preparing the next generation of teachers, the priority should be their acquisition of experience in using artificial intelligence tools in teaching. The objectives of mastering the module are developing students’ skills in the field of neural network mathematical modeling, developing knowledge about basic data analysis algorithms, development of skills in visualization and interpretation of data to solve applied problems using object-oriented programming technologies and artificial neural network technologies. As part of project activities, students gain experience in data mining by completing research projects (for example, a project on comparative assessment of the effectiveness of multilayer perceptron learning algorithms) and applied projects (for example, on the development of a neural network system for predicting the success of schoolchildren’s research work; on the development of a neural network system for predicting student attendance training sessions). At USPU, software systems have been developed in Python that implement the presented neural network prognostic systems. Educational and methodological materials for the disciplines of the module include video lectures and presentations for them, laboratory work (including those based on GoogleColab notebooks), datasets, test sets, theory questions and practical tasks for exams and tests. The effectiveness of the developed teaching methodology is confirmed by the results of a pedagogical experiment. Keywords: neural network technologies, data mining, predictive systems, machine learning, content of the academic discipline

Decoupled graph knowledge distillation: A general logits-based method for learning MLPs on graphs

While Graph Neural Networks (GNNs) have demonstrated their effectiveness in processing non-Euclidean structured data, the neighborhood fetching of GNNs is time-consuming and computationally intensive, making them difficult to deploy in low-latency industrial applications. To address the issue, a feasible solution is graph knowledge distillation (KD), which can learn high-performance student Multi-layer Perceptrons (MLPs) to replace GNNs by mimicking the superior output of teacher GNNs. However, state-of-the-art graph knowledge distillation methods are mainly based on distilling deep features from intermediate hidden layers, this leads to the significance of logit layer distillation being greatly overlooked. To provide a novel viewpoint for studying logits-based KD methods, we introduce the idea of decoupling into graph knowledge distillation. Specifically, we first reformulate the classical graph knowledge distillation loss into two parts, i.e., the target class graph distillation (TCGD) loss and the non-target class graph distillation (NCGD) loss. Next, we decouple the negative correlation between GNN’s prediction confidence and NCGD loss, as well as eliminate the fixed weight between TCGD and NCGD. We named this logits-based method Decoupled Graph Knowledge Distillation (DGKD). It can flexibly adjust the weights of TCGD and NCGD for different data samples, thereby improving the prediction accuracy of the student MLP. Extensive experiments conducted on public benchmark datasets show the effectiveness of our method. Additionally, DGKD can be incorporated into any existing graph knowledge distillation framework as a plug-and-play loss function, further improving distillation performance. The code is available at https://github.com/xsk160/DGKD.

Optimization of Early Detection of Tuberculosis Through Machine Learning Using Clinical Data: Development and Evaluation of Multilayer Perceptron and Extreme Learning Machine for Tuberculosis Classification

This research takes an innovative step in the fight against Tuberculosis (TB), one of Indonesia's prominent public health challenges, by developing and evaluating Multilayer Perceptron (MLP) and Extreme Learning Machine (ELM) models in machine learning for early detection of TB using clinical data. The main result of this study was the discovery that the MLP model, when applied without the Synthetic Minority Over-sampling Technique (SMOTE), achieved an impressive accuracy of 95.00%, signaling significant progress in TB early detection efforts. This discovery not only highlights the great potential of applying machine learning technology in improving the accuracy of TB diagnosis but also paves the way for the possible application of advanced technology in the health sector to deal with infectious diseases. This research illustrates how machine learning technology can be integrated into clinical practice to effectively detect TB cases at an early stage, thus enabling faster and more precise treatment, which can ultimately reduce the spread of the disease. This is particularly important given TB's significant impact on public health, especially in developing countries. The results also open up opportunities for further research into the application of machine learning techniques to other infectious diseases, promising a paradigm shift in the way we detect and manage various health conditions.

RQ-OSPTrans: A Semantic Classification Method Based on Transformer That Combines Overall Semantic Perception and “Repeated Questioning” Learning Mechanism

The pre-trained language model based on Transformers possesses exceptional general text-understanding capabilities, empowering it to adeptly manage a variety of tasks. However, the topic classification ability of the pre-trained language model will be seriously affected in the face of long colloquial texts, expressions with similar semantics but completely different expressions, and text errors caused by partial speech recognition. We propose a long-text topic classification method called RQ-OSPTrans to effectively address these challenges. To this end, two parallel learning modules are proposed to learn long texts, namely, the repeat question module and the overall semantic perception module. The overall semantic perception module will conduct average pooling on the semantic embeddings produced by BERT, in addition to multi-layer perceptron learning. The repeat question module will learn the text-embedding matrix, extracting detailed clues for classification based on words as fundamental elements. Comprehensive experiments demonstrate that RQ-OSPTrans can achieve a generalization performance of 98.5% on the Chinese dataset THUCNews. Moreover, RQ-OSPTrans can achieve state-of-the-art performance on the arXiv-10 dataset (84.4%) and has a comparable performance with other state-of-the-art pre-trained models on the AG’s News dataset. Finally, the results indicate that our method exhibits a superior performance compared with the baseline methods on small-scale domain-specific datasets by validating RQ-OSPTrans on a specific task scenario by using our custom-built dataset CCIPC.

On the additive artificial intelligence-based discovery of nanoparticle neurodegenerative disease drug delivery systems.

Neurodegenerative diseases are characterized by slowly progressing neuronal cell death. Conventional drug treatment strategies often fail because of poor solubility, low bioavailability, and the inability of the drugs to effectively cross the blood-brain barrier. Therefore, the development of new neurodegenerative disease drugs (NDDs) requires immediate attention. Nanoparticle (NP) systems are of increasing interest for transporting NDDs to the central nervous system. However, discovering effective nanoparticle neuronal disease drug delivery systems (N2D3Ss) is challenging because of the vast number of combinations of NP and NDD compounds, as well as the various assays involved. Artificial intelligence/machine learning (AI/ML) algorithms have the potential to accelerate this process by predicting the most promising NDD and NP candidates for assaying. Nevertheless, the relatively limited amount of reported data on N2D3S activity compared to assayed NDDs makes AI/ML analysis challenging. In this work, the IFPTML technique, which combines information fusion (IF), perturbation theory (PT), and machine learning (ML), was employed to address this challenge. Initially, we conducted the fusion into a unified dataset comprising 4403 NDD assays from ChEMBL and 260 NP cytotoxicity assays from journal articles. Through a resampling process, three new working datasets were generated, each containing 500,000 cases. We utilized linear discriminant analysis (LDA) along with artificial neural network (ANN) algorithms, such as multilayer perceptron (MLP) and deep learning networks (DLN), to construct linear and non-linear IFPTML models. The IFPTML-LDA models exhibited sensitivity (Sn) and specificity (Sp) values in the range of 70% to 73% (>375,000 training cases) and 70% to 80% (>125,000 validation cases), respectively. In contrast, the IFPTML-MLP and IFPTML-DLN achieved Sn and Sp values in the range of 85% to 86% for both training and validation series. Additionally, IFPTML-ANN models showed an area under the receiver operating curve (AUROC) of approximately 0.93 to 0.95. These results indicate that the IFPTML models could serve as valuable tools in the design of drug delivery systems for neurosciences.

Urban flood susceptibility mapping using remote sensing, social sensing and an ensemble machine learning model

Flood susceptibility mapping is crucial for urban disaster management. However, the heterogeneity of urban land use and the complexity of terrain pose challenges to the accuracy and generalizability of flood models. In this paper, we propose a framework named the EMF model for urban flood mapping. Specifically, the social sensing and remote sensing were used for flooding information collection. The XGBoost, Support Vector Classifier (SVC), Multilayer Perceptron (MLP), and Multimodal Deep Learning (MDL), were used as the predictive models, and their results were ensembled using a Random Forest model to produce the final outcome. Results show that the EMF model outperforms the standalone models, with an accuracy of 0.942 on the training set and 0.940 on the testing set. The accuracy of the five models is ranked as EMF > MDL > XGBoost > SVC > MLP. The flood map indicates that flooding has a greater impact on the outskirts and suburbs of the city compared to its central urban areas. Farmland is the most affected land type, making up 54.8 % of the flooded area. Overall, the proposed framework enables rapid and accurate identification of flood-prone areas, providing technical support for managers in formulating effective flood prevention strategies.

Drug Solubility Prediction: A Comparative Analysis of GNN, MLP, and Traditional Machine Learning Algorithms

The effective development and design of pharmaceuticals hold fundamental importance in the fields of medicine and the pharmaceutical industry. In this process, the accurate prediction of drug molecule solubility is a critical factor influencing the bioavailability, pharmacokinetics, and toxicity of drugs. Traditionally, mathematical equations based on chemical and physical properties have been used for drug solubility prediction. However, in recent years, with the advancement of artificial intelligence and machine learning techniques, new approaches have been developed in this field. This study evaluated different modeling approaches consisting of Graph Neural Networks (GNN), Multilayer Perceptron (MLP), and traditional Machine Learning (ML) algorithms. The Random Forest (RF) model stands out as the optimal performer, manifesting superior efficacy through the attainment of minimal error rates. It attains a Root Mean Square Error (RMSE) value of 1.2145, a Mean Absolute Error (MAE) value of 0.9221, and an R-squared (R2) value of 0.6575. In contrast, GNN model displays comparatively suboptimal performance, as evidenced by an RMSE value of 1.8389, an MAE value of 1.4684, and an R2 value of 0.2147. These values suggest that the predictions of this model contain higher errors compared to other models, and its explanatory power is lower. These findings highlight the performance differences among different modeling approaches in drug solubility prediction. The RF model is shown to be more effective than other methods, while the GNN model performs less effectively. This information provides valuable insights into which model should be preferred in pharmaceutical design and development processes.

Research on predicting hematoma expansion in spontaneous intracerebral hemorrhage based on deep features of the VGG-19 network.

To construct a clinical noncontrastive computed tomography (NCCT) deep learning joint model for predicting early hematoma expansion (HE) after cerebral hemorrhage (sICH) and evaluate its predictive performance. All 254 patients with primary cerebral hemorrhage from January 2017 to December 2022 in the General Hospital of the Western Theater Command were included. According to the criteria of hematoma enlargement exceeding 33% or the volume exceeding 6ml, the patients were divided into the HE group and the hematoma non-enlargement (NHE) group. Multiple models and the 10-fold cross-validation method were used to screen the most valuable features and model the probability of predicting HE. The area under the curve (AUC) was used to analyze the prediction efficiency of each model for HE. They were randomly divided into a training set of 204 cases in an 8:2 ratio and 50 cases of the test set. The clinical imaging deep feature joint model (22 features) predicted the area under the curve of HE as follows: clinical Navie Bayes model AUC 0.779, traditional radiology logistic regression (LR) model AUC 0.818, deep learning LR model AUC 0.873, and clinical NCCT deep learning multilayer perceptron model AUC 0.921. The combined clinical imaging deep learning model has a high predictive effect for early HE in sICH patients, which is helpful for clinical individualized assessment of the risk of early HE in sICH patients.

Comparative Analysis of Linear Models and Artificial Neural Networks for Sugar Price Prediction

Sugar is an important commodity that is used beyond the food industry. It can be produced from sugarcane and sugar beet, depending on the region. Prices worldwide differ due to high volatility, making it difficult to estimate their forecast. Thus, the present work aims to predict the prices of kilograms of sugar from four databases: the European Union, the United States, Brazil, and the world. To achieve this, linear methods from the Box and Jenkins family were employed, together with classic and new approaches of artificial neural networks: the feedforward Multilayer Perceptron and extreme learning machines, and the recurrent proposals Elman Network, Jordan Network, and Echo State Networks considering two reservoir designs. As performance metrics, the MAE and MSE were addressed. The results indicated that the neural models were more accurate than linear ones. In addition, the MLP and the Elman networks stood out as the winners.

Real Estate Price Prediction: A Comprehensive Review

Abstract: The document below outlines the execution of a price prediction project for housing and real estate markets. A multitude of algorithms have been employed to enhance the accuracy of predictions. Various researchers have undertaken this project and implemented algorithms such as hedonic regression, artificial neural networks, AdaBoost, and J48 tree, which are deemed as the optimal models for price prediction. These models serve as the foundation, and with the aid of sophisticated data mining tools, algorithms like random forest, gradient boosted trees, multilayer perceptron, and ensemble learning models are utilized to achieve a higher rate of prediction accuracy. The results and evaluation of these models, facilitated by machine learning and advanced data mining tools like Weka and Rapid Miner, significantly impact price prediction.

Real Estate Price Prediction: A Comprehensive Review

Abstract: The document below outlines the execution of a price prediction project for housing and real estate markets. A multitude of algorithms have been employed to enhance the accuracy of predictions. Various researchers have undertaken this project and implemented algorithms such as hedonic regression, artificial neural networks, AdaBoost, and J48 tree, which are deemed as the optimal models for price prediction. These models serve as the foundation, and with the aid of sophisticated data mining tools, algorithms like random forest, gradient boosted trees, multilayer perceptron, and ensemble learning models are utilized to achieve a higher rate of prediction accuracy. The results and evaluation of these models, facilitated by machine learning and advanced data mining tools like Weka and Rapid Miner, significantly impact price prediction

Predicting Cancer Risk from Genome Data: A Multilayer Perceptron Approach

This paper proposes a deep learning method to predict cancer risk from gene symbols using a multilayer perceptron (MLP) feed forward neural network. The paper uses a data set of gene symbols and their corresponding cancer risk labels, obtained from a DNA microarray analysis. The paper then builds and compares different machine learning models, such as logistic regression, linear discriminant analysis, quadratic discriminant analysis, decision tree classifier, gaussian nb, ada boost classifier, gaussian process classifier, support vector machine, and random forest. Deep learning MLP model is built, tuned and optimized for hyperparameters which improves the accuracy significantly by 9.09% compared to the best machine learning model. The paper evaluates the performance of the MLP model on the data set using accuracy, precision, recall, and F1-score metrics. This paper contributes to the field of machine learning and bioinformatics by providing a novel and effective way to predict cancer risk from gene symbols.

Forecasting High Speed Diesel Demand in India with Econometric and Machine Learning Methods

According to International Energy Agency (IEA), India is expected to surpass China by 2024 to become the second largest consumer of oil in the world followed by the United States. High-Speed Diesel (HSD) has the biggest share in the total petroleum products consumed in India accounting for around 38% of the total consumption. Considering the volatile global oil market and an oil import dependency ratio of more than 80% during the last four years, the probability of supply disruptions is high in the Indian context. As any uncertainty about the supply of diesel can affect the smooth functioning of the economy and may create inflationary pressures. Accurate forecasting of HSD demand will be essential for appropriate supply management arrangements. Artificial Neural Networks (ANN) with Multi-Layer Perceptron (MLP) and extreme learning machines is used for forecasting diesel demand in this study. Demand forecasting has been carried out using monthly HSD demand data drawn from the “Indiastat” database for the period 1991-2022. Comparison of ANN with traditional forecasting methods of Autoregressive Integrated Moving Average(ARIMA)and Exponential Smoothing has also been undertaken in this study. This study identifies the deep learning technique of ANN with MLP as the best diesel demand forecasting technique.

Subspace learning machine (SLM): Methodology and performance evaluation

Inspired by the feedforward multilayer perceptron (FF-MLP), decision tree (DT) and extreme learning machine (ELM), a new classification model, called the subspace learning machine (SLM), is proposed in this work. SLM first identifies a discriminant subspace, S0, by examining the discriminant power of each input feature. Then, it uses probabilistic projections of features in S0 to yield 1D subspaces and finds the optimal partition for each of them. This is equivalent to partitioning S0 with hyperplanes. A criterion is developed to choose the best q partitions that yield 2q partitioned subspaces among them. We assign S0 to the root node of a decision tree and the intersections of 2q subspaces to its child nodes of depth one. The partitioning process is recursively applied at each child node to build an SLM tree. When the samples at a child node are sufficiently pure, the partitioning process stops and each leaf node makes a prediction. The idea can be generalized to regression, leading to the subspace learning regressor (SLR). Furthermore, ensembles of SLM/SLR trees can yield a stronger predictor. Extensive experiments are conducted for performance benchmarking among SLM/SLR trees, ensembles and classical classifiers/regressors.

Transfer learning for forecasting hourly indoor air temperatures of buildings with electrochromic glass

AbstractThis study aimed to employ transfer learning with a fully connected feed‐forward neural network for forecasting the indoor air temperatures of adaptive buildings with electrochromic (EC) glass. This study predicted indoor air temperatures for an intermediate season requiring heating and cooling. Forecasting indoor air temperature can help control the EC glass to avoid overheating the interiors. The forecasting times for the predictions varied from 1 to 5 h between early morning and noon, which is when the interior is often overheated. The pretrained model was created using multilayer perceptron learning with the simulation data of a source building in Tokyo and transfer learning with feature‐based extraction models that used datasets from the simulation of target buildings in Tokyo and Fukuoka. Further, the effects of facade orientation were investigated. The root mean squared error (RMSE) of the pretrained model varied from 0.027 to 0.935 when predicting the indoor air temperatures from 1 to 5 h. The RMSE of the transfer learning models using the pretrained model with the same and different orientations varied from 0.022 to 1.205 and from 0.9301 to 2.566. This study demonstrated that utilizing predicted indoor air temperatures to control EC glass can help protect against overheating.

Construction Cost Estimation in Data-Poor Areas Using Grasshopper Optimization Algorithm-Guided Multi-Layer Perceptron and Transfer Learning

Construction Cost Estimation in Data-Poor Areas Using Grasshopper Optimization Algorithm-Guided Multi-Layer Perceptron and Transfer Learning

An effective networks intrusion detection approach based on hybrid Harris Hawks and multi-layer perceptron

This paper proposes an Intrusion Detection System (IDS) employing the Harris Hawks Optimization algorithm (HHO) to optimize Multilayer Perceptron learning by optimizing bias and weight parameters. HHO-MLP aims to select optimal parameters in its learning process to minimize intrusion detection errors in networks. HHO-MLP has been implemented using EvoloPy NN framework, an open-source Python tool specialized for training MLPs using evolutionary algorithms. For purposes of comparing the HHO model against other evolutionary methodologies currently available, specificity and sensitivity measures, accuracy measures, and mse and rmse measures have been calculated using KDD datasets. Experiments have demonstrated the HHO MLP method is effective at identifying malicious patterns. HHO-MLP has been tested against evolutionary algorithms like Butterfly Optimization Algorithm (BOA), Grasshopper Optimization Algorithms (GOA), and Black Widow Optimizations (BOW), with validation by Random Forest (RF), XGBoost. HHO-MLP showed superior performance by attaining top scores with accuracy rate of 93.17%, sensitivity level of 89.25%, and specificity percentage of 95.41%.