Multilayer Perceptron Research Articles

Analyzing the dynamics between crude oil spot prices and futures prices by maturity terms: Deep learning approaches to futures-based forecasting

Predicting crude oil prices is critical due to their significant impact on the global economy. While numerous studies have focused on time series forecasting and machine learning methods to predict crude oil spot prices, the use of futures prices segmented by the maturity terms as predictors is comparatively underexplored. This study explored a deep learning approach to investigate using futures prices across different maturities to predict spot prices. Specifically, this paper analyzes the predictive power of one-, two-, three-, six-, and 12-month crude oil futures contracts. This study employs multiple deep learning algorithms, including Multilayer Perceptron (MLP), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN), and Temporal Convolutional Neural Network (TCN), to forecast crude oil spot prices. This research investigates the performance of these machine learning models when exploring the relationship between futures and spot prices of crude oil. In addition, this research incorporates extensive hyperparameter tuning to enhance the interpretability of the machine learning models when forecasting spot prices using futures prices, thereby contributing to the field of Explainable Artificial Intelligence (XAI) with an optimal set of hyperparameters. In summary, this research systematically shows results demonstrating predictive power in terms of XAI between spot prices and futures prices with different maturities, and machine learning algorithms.

Plant lncRNA-miRNA Interaction Prediction Based on Counterfactual Heterogeneous Graph Attention Network.

Identifying interactions between long non-coding RNAs (lncRNAs) and microRNAs (miRNAs) provides a new perspective for understanding regulatory relationships in plant life processes. Recently, computational methods based on graph neural networks (GNNs) have been widely employed to predict lncRNA-miRNA interactions (LMIs), which compensate for the inadequacy of biological experiments. However, the low-semantic and noise of graph limit the performance of existing GNN-based methods. In this paper, we develop a novel Counterfactual Heterogeneous Graph Attention Network (CFHAN) to improve the robustness to against the noise and the prediction of plant LMIs. Firstly, we construct a real-world based lncRNA-miRNA (L-M) heterogeneous network. Secondly, CFHAN utilizes the node-level attention, the semantic-level attention, and the counterfactual links to enhance the node embeddings learning. Finally, these embeddings are used as inputs for Multilayer Perceptron (MLP) to predict the interactions between lncRNAs and miRNAs. Evaluating our method on a benchmark dataset of plant LMIs, CFHAN outperforms five state-of-the-art methods, and achieves an average AUC and average ACC of 0.9953 and 0.9733, respectively. This demonstrates CFHAN's ability to predict plant LMIs and exhibits promising cross-species prediction ability, offering valuable insights for experimental LMI researches.

Exploring the inverse line-source scattering problem in dielectric cylinders with deep neural networks

Abstract This study presents a novel approach utilizing deep neural networks to
address the inverse line-source scattering problem in dielectric cylinders. By employing
Multi-layer Perceptron models, we intend to identify the number, positions, and
strengths of hidden internal sources. This is performed by using single-frequency
phased data, from limited measurements of real electric and real magnetic surface fields.
Training data are generated by solving corresponding direct problems, using an exact
solution representation. Through extended numerical experiments, we demonstrate
the efficiency of our approach, including scenarios involving noise, reduced sample
sizes, and fewer measurements. Additionally, we examine the empirical scaling laws
governing model performance and conduct a local analysis to explore how our neural
networks handle the inherent ill-posedness of the considered inverse problems.

Thermal Runaway Diagnosis of Lithium-Ion Cells Using Data-Driven Method

Fault diagnosis is crucial to guarantee safe operation and extend the operating time while preventing the thermal runaway of the lithium-ion battery. This study presents a data-driven thermal runaway diagnosis framework where Bayesian optimization techniques are applied to optimize the hyperparameter of various machine learning techniques. We use different machine learning models such as support vector machine, naive Bayes, decision tree ensemble, and multi-layer perceptron to estimate a high likelihood of causes of thermal runaway by using the experimental measurements of open-source battery failure data. We analyze different evaluation metrics, including the prediction accuracy, confusion metrics, and receiver operating characteristic curves of different models. An experimental evaluation shows that the classification accuracy of the decision tree ensemble outperforms that of other models. Furthermore, the decision tree ensemble provides robust prediction accuracy even with the strictly limited dataset.

Effectiveness of Multilayer Perceptron for Indoor Localization in Wi-Fi Enabled IoT Environments

Effectiveness of Multilayer Perceptron for Indoor Localization in Wi-Fi Enabled IoT Environments

Disinformation, Fakes and Propaganda Identifying Methods in Online Messages Based on NLP and Machine Learning Methods

A new method of propaganda analysis is proposed to identify signs and change the dynamics of the behaviour of coordinated groups based on machine learning at the processing disinformation stages. In the course of the work, two models were implemented to recognise propaganda in textual data - at the message level and the phrase level. Within the framework of solving the problem of analysis and recognition of text data, in particular, fake news on the Internet, an important component of NLP technology (natural language processing) is the classification of words in text data. In this context, classification is the assignment or assignment of textual data to one or more predefined categories or classes. For this purpose, the task of binary text classification was solved. Both models are built based on logistic regression, and in the process of data preparation and feature extraction, such methods as vectorisation using TF-IDF vectorisation (Term Frequency – Inverse Document Frequency), the BOW model (Bag-of-Words), POS marking (Part-Of-Speech), word embedding using the Word2Vec two-layer neural network, as well as manual feature extraction methods aimed at identifying specific methods of political propaganda in texts are used. The analogues of the project under development are analysed the subject area (the propaganda used in the media and the basis of its production methods) is studied. The software implementation is carried out in Python, using the seaborn, matplotlib, genism, spacy, NLTK (Natural Language Toolkit), NumPy, pandas, scikit-learn libraries. The model's score for propaganda recognition at the phrase level was obtained: 0.74, and at the message level: 0.99. The implementation of the results will significantly reduce the time required to make the most appropriate decision on the implementation of counter-disinformation measures concerning the identified coordinated groups of disinformation generation, fake news and propaganda. Different classification algorithms for detecting fake news and non-fakes or fakes identification accuracy from Internet resources ana social mass media are used as the decision tree (for non-fakes identification accuracy 0.98 and fakes identification accuracy 0.9903), the k-nearest neighbours (0.83/0.999), the random forest (0.991/0.933), the multilayer perceptron (0.9979/0.9945), the logistic regression (0.9965/0.9988), and the Bayes classifier (0.998/0.913). The logistic regression (0.9965) the multilayer perceptron (0.9979) and the Bayesian classifier (0.998) are more optimal for non-fakes news identification. The logistic regression (0.9988), the multilayer perceptron (0.9945), and k-nearest neighbours (0.999) are more optimal for identifying fake news identification.

Comparison of Machine Learning-Based Predictive Models of the Nutrient Loads Delivered from the Mississippi/Atchafalaya River Basin to the Gulf of Mexico

Predicting nutrient loads is essential to understanding and managing one of the environmental issues faced by the northern Gulf of Mexico hypoxic zone, which poses a severe threat to the Gulf’s healthy ecosystem and economy. The development of hypoxia in the Gulf of Mexico is strongly associated with the eutrophication process initiated by excessive nutrient loads. Due to the complexities in the excessive nutrient loads to the Gulf of Mexico, it is challenging to understand and predict the underlying temporal variation of nutrient loads. The study was aimed at identifying an optimal predictive machine learning model to capture and predict nonlinear behavior of the nutrient loads delivered from the Mississippi/Atchafalaya River Basin (MARB) to the Gulf of Mexico. For this purpose, monthly nutrient loads (N and P) in tons were collected from US Geological Survey (USGS) monitoring station 07373420 from 1980 to 2020. Machine learning models—including autoregressive integrated moving average (ARIMA), gaussian process regression (GPR), single-layer multilayer perceptron (MLP), and a long short-term memory (LSTM) with the single hidden layer—were developed to predict the monthly nutrient loads, and model performances were evaluated by standard assessment metrics—Root Mean Square Error (RMSE) and Correlation Coefficient (R). The residuals of predictive models were examined by the Durbin–Watson statistic. The results showed that MLP and LSTM persistently achieved better accuracy in predicting monthly TN and TP loads compared to GPR and ARIMA. In addition, GPR models achieved slightly better test RMSE score than ARIMA models while their correlation coefficients are much lower than ARIMA models. Moreover, MLP performed slightly better than LSTM in predicting monthly TP loads while LSTM slightly outperformed for TN loads. Furthermore, it was found that the optimizer and number of inputs didn’t show effects on the LSTM performance while they exhibited impacts on MLP outcomes. This study explores the capability of machine learning models to accurately predict nonlinearly fluctuating nutrient loads delivered to the Gulf of Mexico. Further efforts focus on improving the accuracy of forecasting using hybrid models which combine several machine learning models with superior predictive performance for nutrient fluxes throughout the MARB.

Vibration Analysis of a Centrifugal Pump with Healthy and Defective Impellers and Fault Detection Using Multi-Layer Perceptron

Centrifugal pumps (CPs) are widely utilized in many different industries, and their operations are maintained by their reliable performance. CPs’ most common faults can be categorized as mechanical or flow-related faults: the first ones are often associated with damage at the impeller, while the second ones are associated with cavitation. It is possible to use computational algorithms to monitor both failures in CPs. In this study, two different problems in pumps, the defective impeller and cavitation, have been considered. When a CP is working in a faulty condition, it generates vibrations that can be measured using piezoelectric sensors. Collected data can be analyzed to extract time- and frequency-domain data. Interpreting the time-domain data showed that distinguishing the type of defect is not possible. However, indicators like kurtosis, skewness, mean, and variance can be used as input for the multi-layer perceptron (MLP) algorithm to classify pump faults. This study presents a detailed discussion of the vibration-based method outcomes, emphasizing the benefits and drawbacks of the multi-layer perceptron method. The results show that the suggested algorithm can identify the occurrence of different faults and quantify their severity during pump operation in real time.

Accelerating Neural Network Inference in Handwritten Digit Recognition — Comparative Study

In deep learning, one of the popular subclasses of multilayer perceptron is the Convolutional Neural Networks (CNNs), optimized for image and semantic segmentation tasks. With a large number of floating-point operations and relatively less data transfer in the training phase, CNNs are well suited to be handled by parallel architectures. The high accuracy of CNNs comes at the cost of consequential compute and memory demands. In this article, we present a comparative analysis of a pre-trained CNN framework for recognizing a handwritten digit on different processing platforms. The performance of the neural network as the function of the parallel processing platforms offers parametric insights and factors that influence the inference on state-of-the-art Graphics Processing Units (GPUs) and systolic arrays such as Eyeriss and Tensor Processing Unit (TPU). Through inference time analysis, we observed that the systolic arrays can outperform upto 58.7 times better than a Turing architecture powered GPU. We show that while the efficiency with which the available resources are utilized is higher in the existing GPUs (upto 32%) than the TPUs, the efficiency of application-specific systolic arrays can be on par with that of GPUs. We present the results obtained from three different customized systolic array-based platforms which can be adopted by the designers to decide the hardware optimization goal.

Coupling Taguchi experimental designs with deep adaptive learning enhanced AI process models for experimental cost savings in manufacturing process development

The Aluminum alloy AA7075 workpiece material is observed under dry finishing turning operation. This work is an investigation reporting promising potential of deep adaptive learning enhanced artificial intelligence process models for L18 (6133) Taguchi orthogonal array experiments and major cost saving potential in machining process optimization. Six different tool inserts are used as categorical parameter along with three continuous operational parameters i.e., depth of cut, feed rate and cutting speed to study the effect of these parameters on workpiece surface roughness and tool life. The data obtained from special L18 (6133) orthogonal array experimental design in dry finishing turning process is used to train AI models. Multi-layer perceptron based artificial neural networks (MLP-ANNs), support vector machines (SVMs) and decision trees are compared for better understanding ability of low resolution experimental design. The AI models can be used with low resolution experimental design to obtain causal relationships between input and output variables. The best performing operational input ranges are identified for output parameters. AI-response surfaces indicate different tool life behavior for alloy based coated tool inserts and non-alloy based coated tool inserts. The AI-Taguchi hybrid modelling and optimization technique helped in achieving 26% of experimental savings (obtaining causal relation with 26% less number of experiments) compared to conventional Taguchi design combined with two screened factors three levels full factorial experimentation.

NeRSI: Neural Implicit Representations for 5D Seismic Data Interpolation

Due to challenging field operations and resource constraints, seismic data acquisition often requires coping with missing traces. Interpolation algorithms are crucial for reconstructing these missing traces to enable improved subsurface analysis and interpretation. While deep learning has made exciting advances in seismic reconstruction, its focus has predominantly been on 2D and 3D datasets with relatively low rates of missing data. Five-dimensional (5D) seismic data reconstruction entails considering simultaneous sources and receivers deployed in areal arrays to solve the reconstruction problem. The latter offers greater data redundancy, which can be leveraged to enhance interpolation quality. Traditional 5D deep learning interpolation methods rely heavily on synthetic training pairs, posing challenges when applied to real-world data. This necessitates transfer learning techniques, which can be cumbersome. Addressing this, we introduce a self-supervised, coordinate-based deep interpolation algorithm that obviates the need for labeled data. Employing a multi-layer perceptron (MLP) network can effectively encode the continuous seismic wavefield inherent in 5D data. Once trained, the MLP can infer missing trace amplitudes from their coordinates. We contribute to boosting the MLP, enabling it to generate seismic profiles rather than single-point predictions. This enhancement significantly strengthens the model’s performance and efficiency. Moreover, we apply nuclear norm regularization to the output profiles, improving the reconstruction quality. The effectiveness of our algorithm is supported by both synthetic and field data experiments, demonstrating its superior reconstruction capabilities.

Global optimization: a machine learning approach

AbstractMany approaches for addressing global optimization problems typically rely on relaxations of nonlinear constraints over specific mathematical primitives. This is restricting in applications with constraints that are implicit or consist of more general primitives. Trying to address such limitations, Bertsimas and Ozturk (2023) proposed OCTHaGOn as a way of solving very general global optimization problems by approximating the nonlinear constraints using hyperplane-based decision-trees and then using those trees to construct a unified MIO approximation of the original problem. We provide extensions to this approach, by (i) approximating the original problem using other MIO-representable ML models besides decision trees, such as gradient boosted trees, multi layer perceptrons and suport vector machines (ii) proposing adaptive sampling procedures for more accurate ML-based constraint approximations, (iii) utilizing robust optimization to account for the uncertainty of the sample-dependent training of the ML models, (iv) leveraging a family of relaxations to address the infeasibilities of the final MIO approximation. We then test the enhanced framework in 81 global optimization instances. We show improvements in solution feasibility and optimality in the majority of instances. We also compare against BARON, showing improved optimality gaps and solution times in more than 9 instances.

Handwritten Digit Recognition by Deep Learning

This paper evaluates recent advances in handwritten digit recognition models, focusing on strategies developed and deployed in practical applications. The project utilizes both traditional and deep learning approaches, employing architectures such as Convolutional Neural Networks (CNNs). This paper explores the comparative performance of various models, discusses their deployment in real-world scenarios, and highlights future prospects for enhancing handwritten digit recognition technology. Keywords: Handwritten Digit Recognition, Multi-Layer Perceptron (MLP), Convolutional Neural Networks (CNN), AlexNet, Deep Learning for Image Classification, Neural Networks, Image Preprocessing, Data Augmentation, Feature Extraction, Image to Grayscale Conversion, Image Normalization, Training and Validation, Accuracy Metrics, Model Evaluation, Transfer Learning, Classification Algorithms, Real- Time Processing, Computer Vision, Image Recognition.

CRISPR-GEM: A Novel Machine Learning Model for CRISPR Genetic Target Discovery and Evaluation.

CRISPR gene editing strategies are shaping cell therapies through precise and tunable control over gene expression. However, limitations in safely delivering high quantities of CRISPR machinery demand careful target gene selection to achieve reliable therapeutic effects. Informed target gene selection requires a thorough understanding of the involvement of target genes in gene regulatory networks (GRNs) and thus their impact on cell phenotype. Effective decoding of these complex networks has been achieved using machine learning models, but current techniques are limited to single cell types and focus mainly on transcription factors, limiting their applicability to CRISPR strategies. To address this, we present CRISPR-GEM, a multilayer perceptron (MLP) based synthetic GRN constructed to accurately predict the downstream effects of CRISPR gene editing. First, input and output nodes are identified as differentially expressed genes between defined experimental and target cell/tissue types, respectively. Then, MLP training learns regulatory relationships in a black-box approach allowing accurate prediction of output gene expression using only input gene expression. Finally, CRISPR-mimetic perturbations are made to each input gene individually, and the resulting model predictions are compared to those for the target group to score and assess each input gene as a CRISPR candidate. The top scoring genes provided by CRISPR-GEM therefore best modulate experimental group GRNs to motivate transcriptomic shifts toward a target group phenotype. This machine learning model is the first of its kind for predicting optimal CRISPR target genes and serves as a powerful tool for enhanced CRISPR strategies across a range of cell therapies.

Анализ угроз информационной безопасности органов государственной власти с помощью нейронных сетей

<p>Ensuring the information security of public authorities requires the use of specialized tools that take into account various sources of information threats and their constant changes. This study aims to develop a methodology for analyzing these threats using neural networks. The research uses methods such as machine learning and neural network analysis to systematize the data. The authors adapted the Multi-Layer Perceptron (MLP) architecture and configured the hyperparameters of the neural network to achieve their objectives. The neural network was trained using the Python programming language, and its effectiveness was evaluated using metrics such as accuracy, precision, recall, and F1 score. The results of the study included the development of a method for creating a data set that encompasses assessments of threats to the information security of various public authorities and their sources. Additionally, the study evaluated the effectiveness of neural networks in solving classification problems for public authorities. Finally, the study interpreted the results of neural network analysis to determine the resistance of public authorities against information security threats.</p>

Prediction of optical properties of uniaxial hyperbolic nanospheres via artificial neural network

In this study, absorption and scattering multilayer perceptron models are developed and validated to predict the optical spectra of uniaxial hyperbolic nanospheres. The finite difference time domain method is used to generate the dataset of the absorption and scattering optical spectra. The models’ optimal performance is achieved for 5 hidden layers and 80 neurons for the absorption and the same hidden layers with 120 neurons for the scattering. The predictions by the model on the test dataset give a low average mean squared error of 0.000145 for the absorption and 0.00071 for the scattering. We also performed a robustness test by using parameters outside the initial parameters used for the training and the predictions are in good agreement with the actual datasets for both absorption and scattering. This research shows the application of artificial neural networks to predict the optical properties of hyperbolic materials and lays the groundwork for developing more complicated neural network models to predict complex phenomena in hyperbolic metamaterials, which are faster and computationally less expensive than using conventional simulation methods. Hyperbolic metamaterials offer unique optical properties that can be used to design new optical devices.

A multi-stage machine learning approach for stock price prediction: Engineered and derivative indices

In this paper, a machine learning approach is proposed to predict the next day's stock prices. The methodology involves comprehensive data collection and feature generation, followed by predictions utilizing Multi-Layer Perceptron (MLP) networks. We selected 5,283 records of daily historical data, including open prices, close prices, highest prices, lowest prices, and trading volumes from four well-known stocks in the FTSE 100 index. A novel set of engineered and derivative indices is extracted from the original time series to enhance prediction accuracy. Two Multi-Layer Perceptron (MLP) are proposed to predict the next day's stock prices using the engineered discrete and continuous indices. The case study uses the daily historical time series of stock prices between January 1, 2000, and December 31, 2020. The proposed machine learning approach presents suitable applicability and accuracy, respectively.

Computer Modelling of Textures on Images with Human Skin Wound Areas

In this paper, we concentrate on the images of the wounds on the human skin and propose to consider each image as a set of smaller pieces – crops or patches containing different textures. We overview, develop and compare deep learning feature extraction methods to model image crops as 200-dimensional feature vectors using various artificial neural network architectures: convolutional autoencoders, variational convolutional autoencoders, and Siamese convolutional networks trained in the contrastive learning manner. Also, we develop a custom convolutional encoder and decoder, use them in the aforementioned architectures and compare them with the ResNet encoder and decoder alternatives. Finally, we train and evaluate k-nearest neighbors and Multi-Layer Perceptron classifiers on the features extracted with the model above options to discriminate skin, wound, and background image patches. Classification evaluation results on the features, extracted with the Siamese network, show the best test accuracy for all implementations without a significant shift between model versions (accuracy > 93%); variational autoencoders show random results for all options (accuracy around 33%), and convolutional autoencoders reached good results (accuracy > 77%) but with a noticeable difference between the custom and ResNet versions; the latter is better. Custom encoder and decoder implementations are faster and smaller than the ResNet alternatives but may be less stable on larger datasets, which still needs investigation. Possible applications of the feature vectors include an area of interest extraction during wound segmentation or classification and usage as patch embeddings while training vision transformer architectures.

Predicting and Analyzing Electric Bicycle Adoption to Enhance Urban Mobility in Belgrade Using ANN Models

The development of alternative environmentally friendly modes of transportation is becoming an increasingly promising solution in traffic-congested and polluted urban areas. E-bikes, as one of them, are recognized as an ecologically sustainable means of transportation that has significant potential to replace motorized modes of transportation that can improve urban mobility. Relying on artificial intelligence and considering an ecological approach when considering the acceptability of e-bikes by setting a direct question for users influences the development of an innovative way of understanding and evaluating the use of more sustainable modes of transportation. In this regard, this study aims to elucidate the main variables influencing the acceptability of e-bike use using artificial neural network (ANN) models—multilayer perceptron (MLP) and radial basis function (RBF). For training and testing the models, data from a random sample obtained through an online questionnaire, which was answered by 626 residents of Belgrade (Serbia), were used. A multilayer perceptron with nine and seven neurons in two hidden layers, respectively, hyperbolic tangent activation function in the hidden layer and identity function in the output layer, gave better results than the radial basis function model. With an accuracy of 89%, a precision of 83%, a recall of 79%, and an area under the receiver operating characteristic (ROC) curve of 0.927, the multilayer perceptron model recognized the influential variables in predicting acceptability. The results of the model indicate that the mileage traveled, the frequency of motorcycle use, the respondents’ awareness of the pollution in Belgrade, and the age of the respondents have the greatest influence on the acceptability of using e-bikes. In addition to majority acceptability (69.8%), the results obtained by the model can represent a useful basis for decision-makers when defining strategies for the development and application of e-bikes while reducing traffic congestion and environmental pollution in Belgrade.

Remote sensing detection of plastic-mulched farmland using a temporal approach in machine learning: case study in tomato crops.

The increasing use of plastics in rural environments has led to concerns about agricultural plastic waste (APW). However, the plasticulture information gap hinders waste management planning and may lead to plastic residue leakage into the environment with consequent microplastic formation. The location and estimated quantity of the APW are crucial for territorial planning and public policies regarding land use and waste management. Agri-plastic remote detection has attracted increased attention but requires a consensus approach, particularly for mapping plastic-mulched farmlands (PMFs) scattered across vast areas. This article tests whether a streamlined time-series approach minimizes PMF confusion with the background using less processing. Based on the literature, we performed a vast assessment of machine learning techniques and investigated the importance of features in mapping tomato PMF. We evaluated pixel-based and object-based classifications in harmonized Sentinel-2 level-2A images, added plastic indices, and compared six classifiers. The best result showed an overall accuracy of 99.7% through pixel-based using the multilayer perceptron (MLP) classifier. The 3-time series with a 30-day composite exhibited increased accuracy, a decrease in background confusion, and was a viable alternative for overcoming the impact of cloud cover on images at certain times of the year in our study area, which leads to a potentially reliable methodology for APW mapping for future studies. To our knowledge, the presented PMF map is the first for Latin America. This represents a first step toward promoting the circularity of all agricultural plastic in the region, minimizing the impacts of degradation on the environment.