Multilayer Perceptron Neural Network Research Articles

A dictionary learning based unsupervised neural network for single image compressed sensing

In the field of Compressed Sensing (CS), the sparse representation of signals and the advancement of reconstruction algorithms are two critical challenges. However, conventional CS algorithms often fail to sufficiently exploit the structured sparsity present in images and suffer from poor reconstruction quality. Most deep learning-based CS methods are typically trained on large-scale datasets. Obtaining a sufficient number of training sets is challenging in many practical applications and there may be no training sets available at all in some cases. In this paper, a novel deep Dictionary Learning (DL) based unsupervised neural network for single image CS (dubbed DL-CSNet) is proposed. It is an effective trainless neural network that consists of three components and their corresponding loss functions: 1) a DL layer that consists of multi-layer perceptron (MLP) and convolution neural networks (CNN) for latent sparse features extraction with the L1-norm sparsity loss function; 2) an image smoothing layer with the Total Variation (TV) like image smoothing loss function; and 3) a CS acquisition layer for image compression, with the Mean Square Error (MSE) loss function between the original image compression and the reconstructed image compression. In particular, the proposed DL-CSNet is a lightweight and fast model that does not require datasets for training and exhibits a fast convergence speed, making it suitable for deployment in resource-constrained environments. Experiments have demonstrated that the proposed DL-CSNet achieves superior performance compared to traditional CS methods and other unsupervised state-of-the-art deep learning-based CS methods.

Engineering Sequestration-Based Biomolecular Classifiers with Shared Resources.

Constructing molecular classifiers that enable cells to recognize linear and nonlinear input patterns would expand the biocomputational capabilities of engineered cells, thereby unlocking their potential in diagnostics and therapeutic applications. While several biomolecular classifier schemes have been designed, the effects of biological constraints such as resource limitation and competitive binding on the function of those classifiers have been left unexplored. Here, we first demonstrate the design of a sigma factor-based perceptron as a molecular classifier working based on the principles of molecular sequestration between the sigma factor and its antisigma molecule. We then investigate how the output of the biomolecular perceptron, i.e., its response pattern or decision boundary, is affected by the competitive binding of sigma factors to a pool of shared and limited resources of core RNA polymerase. Finally, we reveal the influence of sharing limited resources on multilayer perceptron neural networks and outline design principles that enable the construction of nonlinear classifiers using sigma-based biomolecular neural networks in the presence of competitive resource-sharing effects.

LEVERAGING MACHINE LEARNING FOR IMPROVED SPAM DETECTION IN ONLINE NETWORKS

This paper proposes an advanced methodology of spam detection by including N-gram tf.idf feature selection and a deep multi-layer perceptron neural network, with further improvement through the modified distribution-based balancing algorithm. Considering high-dimensional data and class imbalance problems, the proposed method proved to outperform the state-of-the-art methods on benchmark datasets, including Enron, SpamAssassin, SMS spam collection, and social networking data. It also makes up an important enhancement in the classification of spam, as it captures complex features that reduce false positives and false negatives. These results show that combining deep learning with improved feature extraction and balancing techniques provides a very robust approach for spam detection.

Enhancing Fake News Detection with Word Embedding: A Machine Learning and Deep Learning Approach

The widespread dissemination of fake news on social media has necessitated the development of more sophisticated detection methods to maintain information integrity. This research systematically investigates the effectiveness of different word embedding techniques—TF-IDF, Word2Vec, and FastText—when applied to a variety of machine learning (ML) and deep learning (DL) models for fake news detection. Leveraging the TruthSeeker dataset, which includes a diverse set of labeled news articles and social media posts spanning over a decade, we evaluated the performance of classifiers such as Support Vector Machines (SVMs), Multilayer Perceptrons (MLPs), and Convolutional Neural Networks (CNNs). Our analysis demonstrates that SVMs using TF-IDF embeddings and CNNs employing TF-IDF embeddings achieve the highest overall performance in terms of accuracy, precision, recall, and F1 score. These results suggest that TF-IDF, with its capacity to highlight discriminative features in text, enhances the performance of models like SVMs, which are adept at handling sparse data representations. Additionally, CNNs benefit from TF-IDF by effectively capturing localized features and patterns within the textual data. In contrast, while Word2Vec and FastText embeddings capture semantic and syntactic nuances, they introduce complexities that may not always benefit traditional ML models like MLPs or SVMs, which could explain their relatively lower performance in some cases. This study emphasizes the importance of selecting appropriate embedding techniques based on the model architecture to maximize fake news detection performance. Future research should consider integrating contextual embeddings and exploring hybrid model architectures to further enhance detection capabilities. These findings contribute to the ongoing development of advanced computational tools for combating misinformation.

Group-feature (Sensor) selection with controlled redundancy using neural networks

In this work, we present a novel embedded feature selection method based on a Multi-layer Perceptron (MLP) network and generalize it for group-feature or sensor selection problems, which can control the level of redundancy among the selected features or groups and it is computationally more efficient than the existing ones in the literature. Additionally, we have generalized the group lasso penalty for feature selection to encompass a mechanism for selecting valuable groups of features while simultaneously maintaining control over redundancy. We establish the monotonicity and convergence of the proposed algorithm, with a smoothed version of the penalty terms, under suitable assumptions. The effectiveness of the proposed method for both feature selection and group feature selection is validated through experimental results on various benchmark datasets. The performance of the proposed methods is compared with some state-of-the-art methods.

Improved Monitoring and Classification of Engine Oil Condition through Two Machine Learning Techniques

<div>This study explores the effectiveness of two machine learning models, namely multilayer perceptron neural networks (MLP-NN) and adaptive neuro-fuzzy inference systems (ANFIS), in advancing maintenance management based on engine oil analysis. Data obtained from a Mercedes Benz 2628 diesel engine were utilized to both train and assess the MLP-NN and ANFIS models. Six indices—Fe, Pb, Al, Cr, Si, and PQ—were employed as inputs to predict and classify engine conditions. Remarkably, both models exhibited high accuracy, achieving an average precision of 94%. While the radial basis function (RBF) model, as presented in a referenced article, surpassed ANFIS, this comparison underscored the transformative potential of artificial intelligence (AI) tools in the realm of maintenance management. Serving as a proof-of-concept for AI applications in maintenance management, this study encourages industry stakeholders to explore analogous methodologies.</div> <section> <h2>Highlights</h2> <div> <ul> <li> <div>Two machine learning models, multilayer perceptron neural networks (MLP-NN) and adaptive neuro-fuzzy inference systems (ANFIS), were employed to predict and classify the performance condition of diesel engines.</div> </li> <li> <div>Among various training algorithms, Levenberg–Marquardt and the Bayesian regularization demonstrated superior classification accuracy, achieving a 95%–96% range.</div> </li> <li> <div>To assess the generalizability of MLP-NN and ANFIS, the training set size was varied from 90% to 10%.</div> </li> <li> <div>The ANFIS model exhibited greater stability than MLP-NN, with a 50% higher performance.</div> </li> </ul> </div> </section> <section> <h2>Graphical Abstract</h2> <div> <img/> </div> </section>

A Soft Voting Ensemble Model for Hotel Revenue Prediction

In recent years, the hotel industry has faced unprecedented opportunities and challenges due to the increasing demand for travel and business trips. This growth not only presents significant opportunities but also brings challenges to resource management and price setting. Accurate hotel revenue prediction is crucial for the hotel industry as it influences pricing strategies and resource allocation. However, traditional hotel revenue prediction models fail to capture the diversity and complexity of hotel revenue data, resulting in inefficient and inaccurate predictions. Then, with the development of the ensemble learning, its application to hotel revenue prediction has emerged as an influential research direction. This study proposes a soft voting ensemble model for hotel revenue prediction, which includes six base models: Convolutional Neural Network, K-nearest Neighbors, Linear Regression, Long Short-term Memory, Multi-layer Perceptron, and Recurrent Neural Network. Firstly, the hyper-parameters of the base models are optimized with Bayesian optimization. Subsequently, a soft voting ensemble method is used to aggregate the predictions of each base model. Finally, experimental results on the hotel revenue dataset demonstrate that the soft voting ensemble model outperforms base models across six key performance metrics, providing hotel managers with more accurate revenue prediction tools to aid in scientific management decisions and resource allocation strategies. This study confirms the effectiveness of the soft voting ensemble model in enhancing the accuracy of hotel revenue forecasts, demonstrating its significant potential for application in strategic planning within the modern hotel industry.

Machine Learning Discovers Invariants of Braids and Flat Braids

We use machine learning to classify examples of braids (or flat braids) as trivial or non-trivial. Our machine learning takes the form of supervised learning, specifically multilayer perceptron neural networks. When they achieve good results in classification, we are able to interpret their structure as mathematical conjectures and then prove these conjectures as theorems. As a result, we find new invariants of braids and prove several theorems related to them. This work evolves from our experiments exploring how different types of AI cope with untangling braids with 3 strands, this is why we concentrate mostly on braids with 3 strands.

An Artificial Neural Network Prediction Model of Depressive Symptoms among Women with Abnormal Papanicolaou Smear Results before and after Diagnostic Procedures.

(1) Background: Cervical screening and additional diagnostic procedures often lead to depression. This research aimed to develop a prediction model for depression in women who received an abnormal Papanicolaou screening test, prior to and following the diagnostic procedures. (2) Methods: The study included women who had a positive Papanicolaou screening test (N = 172) and attended the Clinical Center of Kragujevac in Serbia for additional diagnostic procedures (colposcopy/biopsy/endocervical curettage). Women filled out a sociodemographic survey and the Center for Epidemiologic Studies Depression questionnaire (CES-D scale) before and after diagnostic procedures. A prediction model was built with multilayer perceptron neural networks. (3) Results: A correlation-based filter method of feature selection indicated four variables that correlated with depression both prior to and following the diagnostic procedures-anxiety, depression, worry, and concern about health consequences. In addition, the use of sedatives and a history of both induced and spontaneous abortion correlated with pre-diagnostic depression. Important attributes for predicting post-diagnostic depression were scores for the domains 'Tension/discomfort' and 'Embarrassment' and depression in personal medical history. The accuracy of the pre-diagnostic procedures model was 70.6%, and the area under the receiver operating characteristic curve (AUROC) was 0.668. The model for post-diagnostic depression prediction showed an accuracy of 70.6%, and an AUROC = 0.836. (4) Conclusions: This study helps provide means to predict the occurrence of depression in women with an abnormal Papanicolaou screening result prior to and following diagnostic procedures, which can aid healthcare professionals in successfully providing timely psychological support to those women who are referred to further diagnostics.

Enhancing solar energy conversion efficiency: Thermophysical property predicting of MXene/Graphene hybrid nanofluids via bayesian-optimized artificial neural networks

Accurately predicting thermo-physical properties (TPPs) of MXene/graphene-based nanofluids is crucial for photovoltaic/thermal solar systems, driving focused research on developing precise TPP predictive models. This study presents optimized multi-layer perceptron neural network (MLPNN) models, leveraging Bayesian optimization to refine architectural and training hyperparameters, including hidden layers, neurons, activation functions, standardization, and regularization terms. A comparative analysis of Bayesian acquisition functions—the probability of improvement (POI), lower confidence bound (LCB), expected improvement (EI), expected improvement plus (EIP), expected improvement per second plus (EIPSP), and expected improvement per second (EIPS)—demonstrated that the POI-MLPNN achieves the most accurate results, as evidenced by the lowest MAPE of 1.0923 % and exceptional consistency with an R-value of 0.99811. The EI-MLPNN and EIP-MLPNN models recorded the same outputs. The EI/EIP-MLPNN (R = 0.99668) model excels in consistency over LCB-MLPNN (R = 0.99529) and EIPSP-MLPNN (R = 0.99667). The optimized models offer a reliable, cost-efficient alternate for experimental and computational TPP analyses. Leveraging insights from these models enables better control over nanofluid TPPs in solar systems, enhancing energy conversion efficiency.

A robust deep learning attack immune MRAM-based physical unclonable function

The ubiquitous presence of electronic devices demands robust hardware security mechanisms to safeguard sensitive information from threats. This paper presents a physical unclonable function (PUF) circuit based on magnetoresistive random access memory (MRAM). The circuit utilizes inherent characteristics arising from fabrication variations, specifically magnetic tunnel junction (MTJ) cell resistance, to produce corresponding outputs for applied challenges. In contrast to Arbiter PUF, the proposed effectively satisfies the strict avalanche criterion (SAC). Additionally, the grid-like structure of the proposed circuit preserves its resistance against machine learning-based modeling attacks. Various machine learning (ML) attacks employing multilayer perceptron (MLP), linear regression (LR), and support vector machine (SVM) networks are simulated for two-array and four-array architectures. The MLP-attack prediction accuracy was 53.61% for a two-array circuit and 49.87% for a four-array circuit, showcasing robust performance even under the worst-case process variations. In addition, deep learning-based modeling attacks in considerable high dimensions utilizing multiple networks such as convolutional neural network (CNN), recurrent neural network (RNN), MLP, and Larq are used with the accuracy of 50.31%, 50.25%, 50.31%, and 50.31%, respectively. The efficiency of the proposed circuit at the layout level is also investigated for simplified two-array architecture. The simulation results indicate that the proposed circuit offers intra and inter-hamming distance (HD) with a mean of 0.98% and 49.96%, respectively, and a mean diffuseness of 49.09%.

Designing a reliable machine learning system for accurately estimating the ultimate condition of FRP-confined concrete

Precisely forecasting how concrete reinforced with fiber-reinforced polymers (FRP) responds under compression is essential for fine-tuning structural designs, ensuring constructions fulfill safety criteria, avoiding overdesigning, and consequently minimizing material expenses and environmental impact. Therefore, this study explores the viability of gradient boosting regression tree (GBRT), random forest (RF), artificial neural network-multilayer perceptron (ANNMLP) and artificial neural network-radial basis function (ANNRBF) in predicting the compressive behavior of fiber-reinforced polymer (FRP)-confined concrete at ultimate. The accuracy of the proposed machine learning approaches was evaluated by comparing them with several empirical models concerning three different measures, including root mean square errors (RMSE), mean absolute errors (MAE), and determination coefficient (R2). In this study, the evaluations were conducted using a substantial collection of axial compression test data involving 765 circular specimens of FRP-confined concrete assembled from published sources. The results indicate that the proposed GBRT algorithm considerably enhances the performance of machine learning models and empirical approaches for predicting strength ratio of confinement (f′cc/f′co) by an average improvement in RMSE as 17.3%, 0.65%, 66.81%, 46.12%, 46.31%, 46.87% and 69.94% compared to RF, ANNMLP, ANNRBF, and four applied empirical models, respectively. It is also found that the proposed ANNMLP algorithm exhibits notable superiority compared to other models in terms of reducing RMSE values as 9.67%, 11.29%, 75.11%, 68.83%, 73.64%, 69.49% and 83.74% compared to GBRT, RF, ANNRBF and four applied empirical models for predicting strain ratio of confinement (εcc/εco), respectively. The superior performance of the GBRT and ANNMLP compared to other methods in predicting the strength and strain ratio confinements is important in evaluating structural integrity, guaranteeing secure functionality, and streamlining engineering plans for effective utilization of FRP confinement in building projects.

In Situ Modulation of Oxygen Vacancy Concentration in Hf0.5Zr0.5O2−x Thin Films and the Mechanism of Its Impact on Ferroelectricity

Oxygen vacancies play a crucial role in stabilizing the ferroelectric phase in hafnium (Hf) oxide-based thin films and in shaping the evolution of their ferroelectric properties. In this study, we directly manipulated the oxygen vacancy concentration in Hf0.5Zr0.5O2−x (HZO) ferroelectric thin films in situ using oxygen plasma treatment. We scrutinized the variations in the ferroelectric properties of HZO films across different oxygen vacancy concentrations by integrating the findings from ferroelectric performance tests. Additionally, we elucidated the mechanism underlying the influence of oxygen vacancies on the coercive field and polarization properties of HZO ferroelectric films through the first-principles density functional theory (DFT) calculations. Finally, to study the impact of oxygen vacancies on the practical application of HZO ferroelectric synaptic devices, leveraging the plasticity of the ferroelectric polarization, we constructed a multilayer perceptron (MLP) network. We simulated its recognition accuracy and convergence speed under different oxygen vacancy concentrations in the MNIST recognition task.

Intelligent resource allocation in wireless networks: Predictive models for efficient access point management

With the significant increase in mobile users connected to the wireless network, coupled with the escalating energy consumption and the risk of network saturation, the search for resource management has become paramount. Managing several access points throughout a whole region is hugely relevant in this context. Moreover, a wireless network must keep its Service Level Agreement, regardless of the number of connected users. With that in mind, in this work, we propose four prediction models that allow one to predict the number of connected users on a wireless network. Once the number of users has been predicted, the network resources can be properly allocated, minimizing the number of active access points. We investigate the use of Particle Swarm Optimization and Genetic Algorithms to hyper-parameterize a Multilayer Perceptron neural network and a Decision Tree. We evaluate our proposal using a campus-based wireless network dataset with more than 20,000 connected users. As a result, our model can considerably improve network performance by intelligently allocating the number of access points, thereby addressing concerns related to energy consumption and network saturation. The results have shown an average accuracy of 95.18%, managing to save network resources effectively.

Carbon dioxide storage and cumulative oil production predictions in unconventional reservoirs applying optimized machine-learning models

To achieve carbon dioxide (CO2) storage through enhanced oil recovery, accurate forecasting of CO2 subsurface storage and cumulative oil production is essential. This study develops hybrid predictive models for the determination of CO2 storage mass and cumulative oil production in unconventional reservoirs. It does so with two multi-layer perceptron neural networks (MLPNN) and a least-squares support vector machine (LSSVM), hybridized with grey wolf optimization (GWO) and/or particle swarm optimization (PSO). Large, simulated datasets were divided into training (70%) and testing (30%) groups, with normalization applied to both groups. Mahalanobis distance identifies/eliminates outliers in the training subset only. A non-dominated sorting genetic algorithm (NSGA-II) combined with LSSVM selected seven influential features from the nine available input parameters: reservoir depth, porosity, permeability, thickness, bottom-hole pressure, area, CO2 injection rate, residual oil saturation to gas flooding, and residual oil saturation to water flooding. Predictive models were developed and tested, with performance evaluated with an overfitting index (OFI), scoring analysis, and partial dependence plots (PDP), during training and independent testing to enhance model focus and effectiveness. The LSSVM-GWO model generated the lowest root mean square error (RMSE) values (0.4052 MMT for CO2 storage and 9.7392 MMbbl for cumulative oil production) in the training group. That trained model also exhibited excellent generalization and minimal overfitting when applied to the testing group (RMSE of 0.6224 MMT for CO2 storage and 12.5143 MMbbl for cumulative oil production). PDP analysis revealed that the input features “area” and “porosity” had the most influence on the LSSVM-GWO model's prediction performance. This paper presents a new hybrid modeling approach that achieves accurate forecasting of CO2 subsurface storage and cumulative oil production. It also establishes a new standard for such forecasting, which can lead to the development of more effective and sustainable solutions for oil recovery.

Forecasting of sub-surface flow velocity through a temporarily open/close estuary under closed condition using machine learning

TOCE or temporarily open/closed estuary is a complex estuary system created by interaction of longshore transport and river outflows. Under the closed condition when an estuarine sandbar is formed, river flow is impeded creating backwater rise that could result in floods. At present, there are no hydrological models that could cater for river flows under closed estuary conditions. This paper proposes the application of a machine learning (ML) approach to model flow velocity in an estuarine sandbar which is crucial for flow estimations. A TOCE located at Mengabang Telipot, Terengganu, Malaysia was chosen for model development where river water and groundwater levels (inside the sandbar) were measured and tidal levels from an existing station were used. Average flow velocities were calculated using Darcy’s equation to represent observed data. Multilayer perceptron (MLP) neural networks were trained by the Levenberg-Marquardt algorithm and developed concerning the parameters above to predict groundwater average velocity. In this study, the optimal model was a neural network with four neurons in the hidden layer and the degree of influence of each parameter was determined by the results of the sensitivity analysis, where the R2 was obtained to be about 97–99% when predicted data was compared to observed data. Hence, the results indicated that neural networks were able to predict groundwater mean velocity quite well. The developed model was further verified using data from another TOCE from a different time. The results were very good having a R2 of 0.990. Hence, it was concluded that the ML approach was able to model flow in a TOCE and has great potential application in such an environment seeing that it is simpler than complex physics-based models.

Machine learning approaches for assessing stability in acid-crude oil emulsions: application to mitigate formation damage

The stability of acid-crude oil emulsion poses manifold issues in the oil industry. Experimentally evaluating this phenomenon may be costly and time-consuming. In contrast, machine learning models have proven effective in predicting and evaluating various phenomena. This research is the first of its kind to assess the stability of acid-crude oil emulsion, employing various classification machine learning models. For this purpose, a data set consisting of 249 experimental data points belonging to 11 different crude oil samples was collected. Three tree-based models, namely decision tree (DT), random forest (RF), and categorical boosting (CatBoost), as well as three artificial neural network models, namely radial basis function (RBF), multi-layer perceptron (MLP) and convolutional neural network (CNN), were developed based on the properties of crude oil, acid, and protective additive. The CatBoost model obtained the highest accuracy with 0.9687, followed closely by the CNN model with 0.9673. In addition, confusion matrix findings showed the superiority of the CatBoost model. Finally, by applying the SHapley Additive exPlanations (SHAP) method to analyze the impact of input parameters, it was found that the crude oil viscosity has the most significant effect on the model's output with the mean absolute SHAP value of 0.88.

Artificial intelligence in land use prediction modeling: a review

<p><span lang="EN-US">This study aims to review methods of artificial intelligence (AI) in land use modelling. Data were extracted from journals in the Scopus and Google Scholar databases using the preferred reporting items for systematic reviews and meta-analyses (PRISMA) method. The review demonstrates that modelling land use predictions is a complex matter that involves land use maps and driving forces. AI technology can support land use forecasting by interpreting land use data, analyzing drivers, and modeling. However, AI has limitations in terms of broad contextual understanding and algorithmic errors. To anticipate this, it is necessary to select the appropriate image resolution and interpretation method in accordance with digital data segmentation. It is also recommended to use spatial regression methods to determine the driving forces that affect land use. Hybrid models such as multilayer perceptron neural network Markov chain (MLPNN-MC), random forest algorithm (RFA), and cellular automata (CA)-Markov chain (MC) are recommended for modelling. The selection of a model should be based on the data's characteristics and tested for accuracy. The use of AI for land use prediction modelling is expected to provide accurate predictions that can be used as a basis for land use policy.</span></p>

A new approach to estimate total nitrogen concentration in a seasonal lake based on multi-source data methodology

Nitrogen, a key limiter in lake eutrophication, presents serious threats to both human health and ecological balance. Despite its non-optically active nature, this study introduces an advanced retrieval approach for total nitrogen, utilizing a synthesis of multi-source data and sophisticated machine learning algorithms to markedly boost estimation precision. This innovative method integrates environmental variables, such as water temperature, depth, and flow rate with spectral reflectance, significantly enhancing the predictive accuracy of our machine learning models with high stability. The models tested, including Support Vector Machine (SVM), Random Forest (RF), Extreme Gradient Boosting (XGB), Multilayer Perceptron (MLP), and Convolutional Neural Networks (CNN), with XGB outperforming others by achieving robust metrics: an R2 of 0.78, a Mean Absolute Error (MAE) of 0.21 mg/L, and a Mean Absolute Percentage Error (MAPE) of 16.04 %. Applying the optimized XGB model, we documented fluctuations in nitrogen concentrations within Poyang Lake across different hydrological phases in 2021, revealing the lowest nitrogen levels during the flood season and the highest in low water periods, with high concentrations at the inlets of the North Branch of the Ganjiang River and the Raohe River estuaries. Monte Carlo simulations reveal that the model is not much sensitive to input feature errors, validating its stability. The approach proposed in this study may help more precise total nitrogen retrieval in other similar lake waters.

Prediction Model using a Multi-Layer Perceptron Neural Network for Military Plastic Explosive PE4 Blast Performance in a Significant Effect of Tropicalisation

Over the years, plastic explosive PE4 has been imported from the United Kingdom and is fully utilised in military activities and drills. This study is to discover a ratio of the plastic explosive PE4 explosion performance in tropical conditions and develop a multilayer perceptron neural network model for the prediction of plastic explosive in Malaysia. Several environmental tests have been performed in Kem Kongkoi, Jelebu, Negeri Sembilan. Six parameters were considered, including environmental temperature, distance measurement, explosive material, variety of shapes, weight and ignition point. In this paper, the Bayesian Regularization, Levenberg-Marquardt algorithm and Scaled Conjugate Gradient model were developed. Every model was tested with 24 datasets to discover the root mean square error and regression performance. The Bayesian Regularization model provides the best prediction model as it has a mean square error of 0.0005 and a regression performance value that is close to 1 at 0.9992.