Use Of Multilayer Perceptron Research Articles

Development of a neural network with a layer of trainable activation functions for the second stage of the ensemble classifier with stacking

One of the promising directions for improving the quality of object recognition in images and parallelizing calculations is the use of ensemble classifiers with stacking. A neural network at the second level makes it possible to achieve the resulting quality of classification, which is significantly higher than each of the networks of the first level separately. The classification quality of the entire ensemble classifier with stacking depends on the efficiency of the neural networks at the first stage, their number, and the quality of the classification of the neural network of the second stage. This paper proposes a neural network architecture for the second stage of the ensemble classifier, which combines the approximating properties of traditional neurons and learning activation functions. Gaussian Radial Basis Functions (RBFs) were chosen to implement the learned activation functions, which are summed with the learned weights. The experimental studies showed that when working with the CIFAR-10 data set, the best results are obtained when six RBFs are used. A comparison with the use of multilayer perceptron (MLP) in the second stage showed a reduction in classification errors by 0.45–1.9 % depending on the number of neural networks in the first stage. At the same time, the proposed neural network architecture for the second degree had 1.69–3.7 times less learning coefficients than MLP. This result is explained by the fact that the use of an output layer with ordinary neurons allowed us not to enter into the architecture many learning activation functions for each output signal of the first stage, but to limit ourselves to only one. Since the results were obtained on the CIFAR-10 universal data set, a similar effect could be obtained on a large number of similar practical data sets.

Prediction of the Remaining Useful Life of Lithium-Ion Battery Using Multilayer Perceptron

Cogitating the reliability of the supply and ensuring continuous delivery of power to the loads, especially in the growing demand for Lithium-Ion batteries in electric vehicle applications, prediction of the remaining useful life of Lithium-Ion batteries is crucial for the timely replacement. For prediction of non-linear and chaotic relationship, experience-based approach, physics-based approach and data driven approach are used among which data driven approach is a model free, accurate and reliable approach. Therefore, a driven approach in predicting remaining useful life can be implemented in the battery management system. This research uses a multilayer perceptron to predict the remaining useful life of the battery. The NASA Ames Prognostics Center of Excellence (PCoE) battery dataset is used to test the proposed methodology. The use of multilayer perceptron for remaining life prediction seems promising despite the significant number of jump points, gaps in data and a small quantity of experimental data in the National Aeronautics and Space Administration (NASA) dataset. The predicted result was obtained with 8.52 % mean absolute error and 9.59 % root mean square error. When compared with the predicted results of different literatures, proposed multilayer perceptron with sliding window approach outperforms most of the existing approach. Incorporation of optimization techniques and hybrid algorithm in proposed approach can further enhance the accuracy of the model.

FTMLP: MLP with Feature-Temporal Block for multivariate time series forecasting

Multivariate time series (MTS) forecasting is extensively applied in real-world scenarios. Recent studies introduced the concept of channel independence, which has achieved significant results. However, this approach often misses the intricate interrelations among features and lacks physical interpretation. To address this problem, we propose a new framework, the Feature-Temporal block, designed to extract both temporal and feature information. Each block contains two modules: a feature module using a gating mechanism to understand competitive interactions among features, and a temporal module using learnable filters for frequency domain filtering. This innovative design allows FTMLP to combine information across feature and sequence dimensions, as well as time and frequency domains. The computational efficiency of FTMLP is enhanced due to its exclusive use of Multi-Layer Perceptron (MLP). Comprehensive experiments on several real-world datasets demonstrate that FTMLP achieves state-of-the-art results in terms of overall performance. Code is available at https://github.com/whxlearning/FTMLP.

Strengthening Wireless Network Security: Supervised Machine Learning-Based Intrusion Detection for Enhanced Threat Mitigation

With the wireless networks deemed vital to our laced and unetched world, it has become fundamental to safeguard these networks against the various cyber threats that are on the rise. As a research proposal I show concern about the application of a Supervised Machine Learning-Based Intrusion Detection system, making use of Multilayer Perceptron (MLP), Support Vector Machine (SVM), and Logistic Regression (LR) algorithms. The survey begins with the introduction of an intrusive inter-dataset technology that merges two catalytic datasets into a single efficient model that can address both model security and data sensitivity concerns. The evaluation of several machine learning algorithms within the proposed framework is a core component of the analysis. The accuracy measure as well as others such as recall, precision, and F1 score, are applied to evaluate the algorithms' level in detecting intrusions from different datasets.The investigation shows that the intra-dataset routing noticeably improves the detecting modules' efficiency. Notably, the model of MLP algorithm has enhanced recall which in turn shows the enhancement of the positive instance identification. SVM shows increased precision, which accounts for the improved accuracy since correct names of positive cases are produced more often. LR finds it overall enhance the precisions thus attesting to its efficiency in correct deductions. The research documents the multifaceted nature of IDS, prompting the intervention of the machine learning algorithms with ddoswdelcome intrusion detection system. A study which provides a practical guide for network administrators and security professionals hinting on how to select algorithms to meet the distinct security needs in their enterprises. These findings contribute to the open discourse on wireless network security and serve as a base for future study of this engaging research topic in constantly developing field of intrusion detection.

Enhancing Multi-Layer Perceptron Performance with K-Means Clustering

Machine learning plays a crucial role in identifying patterns within data, with classification being a prominent application. This study investigates the use of Multilayer Perceptron (MLP) classification models and explores preprocessing techniques, particularly K-Means clustering, to enhance model performance. Overfitting, a common challenge in MLP models, is addressed through the application of K-Means clustering to streamline data preparation and improve classification accuracy. The study begins with an overview of overfitting in MLP models, highlighting the significance of mitigating this issue. Various techniques for addressing overfitting are reviewed, including regularization, dropout, early stopping, data augmentation, and ensemble methods. Additionally, the complementary role of K-Means clustering in enhancing model performance is emphasized. Preprocessing using K-Means clustering aims to reduce data complexity and prevent overfitting in MLP models. Three datasets - Iris, Wine, and Breast Cancer Wisconsin - are employed to evaluate the performance of K-Means as a preprocessing technique. Results from cross-validation demonstrate significant improvements in accuracy, precision, recall, and F1 scores when employing K-Means clustering compared to models without preprocessing. The findings highlight the efficacy of K-Means clustering in enhancing the discriminative power of MLP classification models by organizing data into clusters based on similarity. These results have practical implications, underlining the importance of appropriate preprocessing techniques in improving classification performance. Future research could explore additional preprocessing methods and their impact on classification accuracy across diverse datasets, advancing the field of machine learning and its applications

Modeling CO2 Emission in Residential Sector of Three Countries in Southeast of Asia by Applying Intelligent Techniques

Residential sector is one of the energy-consuming districts of countries that causes CO2 emission in large extent. In this regard, this sector must be considered in energy policy making related to the reduction of emission of CO2 and other greenhouse gases. In the present work, CO2 emission related to the residential sector of three countries, including Indonesia, Thailand, and Vietnam in Southeast Asia, are discussed and modeled by employing Group Method of Data Handling (GMDH) and Multilayer Perceptron (MLP) neural networks as powerful intelligent methods. Prior to modeling, data related to the energy consumption of these countries are represented, discussed, and analyzed. Subsequently, to propose a model, electricity, natural gas, coal, and oil products consumptions are applied as inputs, and CO2 emission is considered as the model’s output. The obtained R2 values for the generated models based on MLP and GMDH are 0.9987 and 0.9985, respectively. Furthermore, values of the Average Absolute Relative Deviation (AARD) of the regressions using the mentioned techniques are around 4.56% and 5.53%, respectively. These values reveal significant exactness of the models proposed in this article; however, making use of MLP with the optimal architecture would lead to higher accuracy.

The use of multilayer perceptron and radial basis function: an artificial intelligence model to predict progression of oral cancer.

The use of multilayer perceptron and radial basis function: an artificial intelligence model to predict progression of oral cancer.

Analysis of factors influencing recovery of low permeability and strong heterogeneous gas reservoirs and establishment of prediction model

Jialingjiang Formation is a typical low permeability carbonate reservoir in Sichuan Basin, which is characterized by strong heterogeneity, low matrix permeability and high-water saturation. Based on seven influencing factors, such as permeability, reservoir thickness and reservoir porosity, this paper evaluates and predicts the oil recovery of Jialingjiang Formation gas reservoir by using two neural network models of multilayer perceptor and radial basis function and nonlinear surface fitting method. The results show that: 1) In the neural network prediction model, the correlation coefficient between the prediction result curve of multilayer perceptron and the original recovery curve reaches 0.88, which is higher than the radial basis function (0.81), indicating that the use of multilayer perceptron can better predict the recovery of gas reservoirs. 2) In the nonlinear curved surface fitting prediction model, the two influencing factors with the greatest linear correlation with the recovery factor, reservoir thickness and gas recovery rate, are selected to fit the prediction model, and the prediction model is obtained. According to the prediction model, the recovery factor is positively correlated with the reservoir thickness and gas recovery rate on the whole. The model can be used to estimate gas recovery from two factors: reservoir thickness and gas recovery rate.

The Use of Multilayer Perceptron (MLP) to Reduce Delamination during Drilling into Melamine Faced Chipboard

Drilling into melamine-faced-wood-based panels is one of the most common processes in modern furniture manufacturing. Delamination is usually the main and the most troublesome quality defect in this case. A lot of scientific studies draw the conclusion that the progress of tool wearing during the cutting of wood-based materials is the key problem. Therefore, tool condition monitoring and the replacement of worn tools at the right time is the most useful and common (in the industrial practice) way to reduce delamination. However, the automation of this process is still a problem due to various issues. There is yet no commercial (even prototypical) offer for the furniture industry in this regard. For this reason, it is considered advisable to try to use the multilayer perceptron (MLP) algorithm to automatically identify a drill’s condition during drilling in a laminated chipboard. It has been established that, for practical purposes, it is important to distinguish between the three different classes of tool conditions, which can be conventionally described as “Green” (keep working), “Red” (implicitly stop and replace) and “Yellow” (warning signal—stop and replace if you want to avoid deterioration in cutting quality). To register the signals generated in the cutting zone and those constituting the basis for the identification of the tool condition in the “on-line” mode, the following elements were used: contact sensor of acoustic emission, accelerometer for vibration, two-component force gauge and a microphone. The classification effects (with an overall accuracy above 70%) were ultimately fairly decent but slightly worse than those of the classification algorithms tested earlier (i.e., “nearest neighbors” or “support vector machine” algorithms). The most troublesome, however, is the fact that serious errors (mistakes between “Green” and “Red” classes) were occasionally noted (for about 1% of the analyzed cases).

Predicting vehicle parking space availability using multilayer perceptron neural network

In this study, we have investigated potential use of Multilayer Perceptron (MLP) to predict parking space availability for use within Field Programmable Gate Array (FPGA) accelerated embedded devices. While previous studies have explored the use of MLP for classification problem in FPGA, very little studies concentrated on the potential use of MLP in regression problem, especially in parking space forecasting. Therefore we formulated five Multi-Layer Perceptron (MLP) models with varying hidden units to perform single-step prediction to forecast parking space availability within the next 15 minutes based on the previous one-hour parking occupancy. The proposed models were trained on the historical data of Kuala Lumpur Convention Center dataset and evaluated against baseline ARIMA models. The results have shown that our proposed MLP model performed relatively well against baseline model with the root mean square error between (RMSE) 78.25 to 78.41 and mean absolute error (MAE) between 37.02 to 39.17.

Addressing management practices of private forests by remote sensing and open data: A tentative procedure

The qualitative and quantitative knowledge of forestry stands is a fundamental requirement for their management and exploitation planning. Traditional field measurements, based on plots, are certainly accurate, but time-consuming and expensive; moreover, it cannot provide a wall-to-wall estimate of measures over large areas. Even though they are essential for calibrate and validate models and respectively results, remote sensing could be a useful support for forest planners when trying to describe wide areas. Optical images from medium-high resolution satellite missions are widely employed due to their accessibility, affordability and readiness to use. Within this framework, in this study an approach based on open-data from the Copernicus Sentinel-2 mission and Regional low density LiDAR data was developed in order to assess private forest wood resource in the Susa Valley (Piemonte Region, NW - Italy). The proposed methodology aims to support private forest management by the local forest consortium for basal area and wood volume estimation of forest stands. Specifically, the process was based on the jointly use of Multi-layer Perceptron (MLP) artificial neural network (ANN), trained and validated with respect to ground data from 285 surveyed plot. Furtherly, to refined wood volume estimates correcting factors accounting for slope and local stand fertility were applied. Estimates of volume stands were generated separately for conifers and broadleaves. Results prove that the adoption of correcting factors improved volume estimates by the MLP and ANN about 2% (relative mean absolute error) and 13% in conifer and broadleaf stands respectively.

PUConv: Upsampling convolutional network for point cloud semantic segmentation

Due to the issue of disorder, it is difficult to directly utilise a 2D convolutional neural networks to process 3D point clouds. Recently, PointNet can directly use 3D point sets as the input of convolutional neural networks and complete the processing of point clouds with multi-layer perceptron (MLP) and symmetric functions. However, the use of MLP to compute the weight function ignores the problem of non-uniformity sampling caused by the density of point set data. To address the above problem, based on the PointNet++ structure, a kernel density estimation based method is proposed to calculate the density level of the local point sets region under the optimal bandwidth selection principle, and the density re-weighting of the weight function is developed to better fit the structure of local point clouds. In addition, the authors utilise the upsampling convolution operation to avoid duplicate storages and calculations, making the point cloud reconstruction more efficient. The experiments carried out the semantic segmentation on both the synthetic data and the real indoor scenes show that the proposed method is capable of obtaining promising semantic segmentation results.

Optimal Deep Learning based Data Classification Model for Type-2 Diabetes Mellitus Diagnosis and Prediction System

In recent days, deep learning models become a significant research area because of its applicability in diverse domains. In this paper, we employ an optimal deep neural network (DNN) based model for classifying diabetes disease. The DNN is employed for diagnosing the patient diseases effectively with better performance. To further improve the classifier efficiency, multilayer perceptron (MLP) is employed to remove the misclassified instance in the dataset. Then, the processed data is again provided as input to the DNN based classification model. The use of MLP significantly helps to remove the misclassified instances. The presented optimal data classification model is experimented on the PIMA Indians Diabetes dataset which holds the medical details of 768 patients under the presence of 8 attributes for every record. The obtained simulation results verified the superior nature of the presented model over the compared methods.

Integrating MLP With Algorithm with AHP Modification For Car Evaluation

Analytic Hierarchy Process is a well-known technique in decision making to handle the complexity of multicriteria problems based on hierarchical criteria and sub-criteria structures, evaluate alternative problems and rankings, and choose the best alternative. Decision results in the form of the best alternative are obtained based on the priority weight that each attribute has. The weight search process is carried out by forming a pairwise comparison matrix. But the problems that often occur when doing pairwise comparisons, the AHP method cannot provide a consistent comparison value, especially for problems with a large number of criteria and alternatives. Inconsistent pairwise comparisons will result in the weight of the criteria compared cannot describe the actual conditions. To avoid the problem of inconsistency in the AHP method can be done by minimizing the formation of pairwise comparison matrix. This study discusses how the search process weighted on the problem of Multiple Attribute Decision Making (MADM), especially in AHP techniques using a multilayer perceptron network in determining alternative selection. The results of the study prove that the use of multilayer perceptron can provide better results and save time because the creation of a pairwise comparison matrix is smaller.

Forecasting Movie Box Office Profitability

This study intends to estimate the profit of a movie through the construction of a predictive model that uses several Data Mining techniques, namely neural networks, regression and decision trees. The model will allow obtaining the prediction of box office revenue. Three different dependent variable approaches were used (interval, categorical and binary) aiming to study the difference and predictive influence that each one has on the results. Two metrics were used to determine the most accurate predictions: the misclassification error for the categorical models and the average squared error for the continuous one. In this study, the best predictive results were obtained through the use of multi-layer perceptron. Regarding the representative distinction between the dependent variable, the multiclass model presents a much higher error rate comparing to the remaining ones, which is explained with the increase of the number of classes to predict.

A Novel Feature Selection Method for the Dynamic Security Assessment of Power Systems Based on Multi-Layer Perceptrons

In this study, the effect of feature selection methods on the performance of multi-layer perceptrons used for the dynamic security assessment of electric power systems is investigated. The existence of many measurable parameters (features) characterizing the power system security status complicates the use of multi-layer perceptron both in terms of prediction accuracy and training time. In this paper, the dynamic security of a power system subject to a number of critical contingencies is assessed as the critical clearing time of any credible fault is predicted by a multi-layer perceptron. In addition to the study of two different feature selection methods, which are Minimum Redundancy Maximum Relevance (mRMR), and Regressional ReliefF (RReliefF), a novel multi-layer perceptron based feature selection method is proposed to be applied in the prediction of security indices. The performance of the feature selection methods on the dynamic security assessment is investigated on a 16-generator, 68-bus test system.

Synergy of modeling processes in the area of soft and hard modeling

High complexity of production processes results in more frequent use of computer systems for their modeling and simulation. Process modeling helps to find optimal solution, verify some assumptions before implementation and eliminate errors. In practice, modeling of production processes concerns two areas: hard modeling (based on differential equations of mathematical physics) and soft (based on existing data). In the paper the possibility of synergistic connection of these two approaches was indicated: it means hard modeling support based on the tools used in soft modeling. It aims at significant reducing the time in order to obtain final results with the use of hard modeling. Some test were carried out in the Calibrate module of NovaFlow&Solid (NF&S) simulation system in the frame of thermal analysis (ATAS-cup). The authors tested output values forecasting in NF&S system (solidification time) on the basis of variable parameters of the thermal model (heat conduction, specific heat, density). Collected data was used as an input to prepare soft model with the use of MLP (Multi-Layer Perceptron) neural network regression model. The approach described above enable to reduce the time of production process modeling with use of hard modeling and should encourage production companies to use it.

Intelligent Sensor Based Bayesian Neural Network for Combined Parameters and States Estimation of a Brushed DC Motor

The objective of this paper is to develop an Artificial Neural Network (ANN) model to estimate simultaneously, parameters and state of a brushed DC machine. The proposed ANN estimator is novel in the sense that his estimates simultaneously temperature, speed and rotor resistance based only on the measurement of the voltage and current inputs. Many types of ANN estimators have been designed by a lot of researchers during the last two decades. Each type is designed for a specific application. The thermal behavior of the motor is very slow, which leads to large amounts of data sets. The standard ANN use often Multi-Layer Perceptron (MLP) with Levenberg-Marquardt Backpropagation (LMBP), among the limits of LMBP in the case of large number of data, so the use of MLP based on LMBP is no longer valid in our case. As solution, we propose the use of Cascade-Forward Neural Network (CFNN) based Bayesian Regulation backpropagation (BRBP). To test our estimator robustness a random white-Gaussian noise has been added to the sets. The proposed estimator is in our viewpoint accurate and robust.

Technoeconomic Projections with Artificial Neural Networks using an Ensemble of Sparsely-Sampled Bootstrapped Data

Aims: The present study refers to developing an artificial neural network (ANN) that can be designed exclusively for ex ante forecasting in technoeconomic contexts using an ensemble set of sparse and insufficient sampled-data availed ex post. Study Design: In general, the samples in a data set of technoeconomic structures would largely be limited in number due to sparse-sampling; also, availability of number of such sets is mostly inadequate for robust training of an ANN so as to obtain realistic inferences subsequently in the prediction phase. Hence, a sparsity-recovery strategy is advocated via a cardinality enhancement procedure (through Nyquist sampling) performed on the sparse data set in order to augment the number of samples in its sampled-data space. Further, the concept of statistical bootstrapping technique of resampling is invoked and applied on the cardinality-improved subset so as to obtain an enhanced number of data sets. This ensemble of data set is then adopted to facilitate robust training of the test Original Research Article British Journal of Economics, Management & Trade, 4(2): 228-263, 2014 229 ANN. Place and Duration of Study: The studies were conducted (2012-2013) at: Department of Computer and Electrical Engineering and Computer Science, College of Engineering & Computer Science, Florida Atlantic University, Boca Raton, Florida 33431, USA. Methodology: The study governs technoeconomic ex ante projections pertinent to a wind-power generation business complex elucidated via ANN-based forecasting. Relevant test ANN is designed to accommodate training with an ensemble of sampled set available ex post but, in limited numbers. The associated scarcity is recovered by artificially enhancing the data space to an adequate extent via Nyquist sampling and bootstrapping techniques. Further, the test ANN designed corresponds to a multilayer perceptron (MLP) supporting backpropagation of the perceived error at the output with respect to a supervisory value. It accommodates the bootstrapped data space at its input relevant to technoeconomic details on a practical wind-power system performance reported in the literature. The training and prediction exercises on the test ANN corresponds to optimally elucidating output predictions in the context of the technoeconomics framework of the power generation considered. Results: Using the test ANN trained with bootstrap-enhanced, scarcity-recovered sparse data on wind-power generation statistics and associated plant economics, reliable inference (in the prediction phase) is achieved on the system performance. That is, the ANN output obtained depicts forecast projections on the productivity of electric power generation in the ex ante regime. Simulation studies thereof and results obtained demonstrate the efficacy of the method proposed, bootstrapping algorithm developed and the use of MLP in the technoeconomic contexts.

Online handwritten signature verification using neural network classifier based on principal component analysis.

One of the main difficulties in designing online signature verification (OSV) system is to find the most distinctive features with high discriminating capabilities for the verification, particularly, with regard to the high variability which is inherent in genuine handwritten signatures, coupled with the possibility of skilled forgeries having close resemblance to the original counterparts. In this paper, we proposed a systematic approach to online signature verification through the use of multilayer perceptron (MLP) on a subset of principal component analysis (PCA) features. The proposed approach illustrates a feature selection technique on the usually discarded information from PCA computation, which can be significant in attaining reduced error rates. The experiment is performed using 4000 signature samples from SIGMA database, which yielded a false acceptance rate (FAR) of 7.4% and a false rejection rate (FRR) of 6.4%.