Multilayer Perceptron Training Research Articles

Identify main-sequence binaries from the Chinese Space Station Telescope Survey with machine learning

ABSTRACT The statistical properties of double main sequence (MS) binaries are very important for binary evolution and binary population synthesis. To obtain these properties, we need to identify these MS binaries. In this paper, we have developed a method to differentiate single MS stars from double MS binaries from the Chinese Space Station Telescope (CSST) Survey with machine learning. This method is reliable and efficient to identify binaries with mass ratios between 0.20 and 0.80, which is independent of the mass ratio distribution. But the number of binaries identified with this method is not a good approximation to the number of binaries in the original sample due to the low detection efficiency of binaries with mass ratios smaller than 0.20 or larger than 0.80. Therefore, we have improved this point by using the detection efficiencies of our method and an empirical mass ratio distribution and then can infer the binary fraction in the sample. Once the CSST data are available, we can identify MS binaries with our trained multi-layer perceptron model and derive the binary fraction of the sample.

A simulation-driven supervised learning framework to estimate brain microstructure using diffusion MRI

We propose a framework to train supervised learning models on synthetic data to estimate brain microstructure parameters using diffusion magnetic resonance imaging (dMRI). Although further validation is necessary, the proposed framework aims to seamlessly incorporate realistic simulations into dMRI microstructure estimation.Synthetic data were generated from over 1,000 neuron meshes converted from digital neuronal reconstructions and linked to their neuroanatomical parameters (such as soma volume and neurite length) using an optimized diffusion MRI simulator that produces intracellular dMRI signals from the solution of the Bloch–Torrey partial differential equation. By combining random subsets of simulated neuron signals with a free diffusion compartment signal, we constructed a synthetic dataset containing dMRI signals and 40 tissue microstructure parameters of 1.45 million artificial brain voxels.To implement supervised learning models we chose multilayer perceptrons (MLPs) and trained them on a subset of the synthetic dataset to estimate some microstructure parameters, namely, the volume fractions of soma, neurites, and the free diffusion compartment, as well as the area fractions of soma and neurites. The trained MLPs perform satisfactorily on the synthetic test sets and give promising in-vivo parameter maps on the MGH Connectome Diffusion Microstructure Dataset (CDMD). Most importantly, the estimated volume fractions showed low dependence on the diffusion time, the diffusion time independence of the estimated parameters being a desired property of quantitative microstructure imaging.The synthetic dataset we generated will be valuable for the validation of models that map between the dMRI signals and microstructure parameters. The surface meshes and microstructures parameters of the aforementioned neurons have been made publicly available.

Neural-symbolic BDI-Agent as a Multi-Context System: A case study with negotiating agent

Intelligent systems are already being deployed in several fields (e.g., medicine, education, automation, or legal), providing relevant tools to assist in our daily tasks. It is claimed that the next step of AI is the integration of connectionist and symbolic methods. Connectionist methods embody knowledge by assigning numerical conductivities or weights to the connections inside a network of nodes. Symbolic methods work by carrying on a sequence of logic-like reasoning steps over a set of symbols consisting of language-like representations. Combining the strengths of connectionist and symbolic methods can allow the development of robust intelligent systems. We propose a neural-symbolic BDI-agent based Multi-Context Systems (MCS) model to integrate these two methods. Neural-symbolic area explores the effective integration of connectionist and symbolic methods, more precisely, learning and reasoning. MCSs allow the representation of information exchange among heterogeneous sources. BDI-agents offer robust and flexible behavior, rapid and modular development, intelligibility, and verifiability. In our work, MCSs represent the symbolic method and Neural Networks the connectionist method. We present a case study of how the proposed agent can be implemented using the Sigon framework. Sigon facilitates the development of MCS agents using a programming language-like paradigm. In this case study, we employed a trained MultiLayer Perceptron (MLP) neural network, enabling us to mitigate the necessity of modeling some hand-crafted rules in symbolic systems. We performed two experiments, in which the main goal was to analyze the impact of using a neural network during the agent’s decision-making. In the first experiment, we compare the proposed agent with 136 negotiating agents participating in the Automated Negotiating Agents Competition (ANAC) available on the GENIUS framework. In the second experiment, we compare our work with one similar to ours. The results show that our agent can adapt and achieve high utility functions for different situations. However, it is necessary to consider the computational cost of using the NN’s output in every reasoning cycle. The contributions of this paper are: (i) - integration of connectionist methods as part of the agent’s decision-making, enabling the development of agents in a modular and flexible fashion; (ii) - custom sensors to handle different data types; (iii) - practical implementation of the proposed integration method; and (iv) - experiments of the proposed integration method, focusing on mitigating the necessity of hard-coded and hand-crafted rules.

A New Parallel Cuckoo Flower Search Algorithm for Training Multi-Layer Perceptron

This paper introduces a parallel meta-heuristic algorithm called Cuckoo Flower Search (CFS). This algorithm combines the Flower Pollination Algorithm (FPA) and Cuckoo Search (CS) to train Multi-Layer Perceptron (MLP) models. The algorithm is evaluated on standard benchmark problems and its competitiveness is demonstrated against other state-of-the-art algorithms. Multiple datasets are utilized to assess the performance of CFS for MLP training. The experimental results are compared with various algorithms such as Genetic Algorithm (GA), Grey Wolf Optimization (GWO), Particle Swarm Optimization (PSO), Evolutionary Search (ES), Ant Colony Optimization (ACO), and Population-based Incremental Learning (PBIL). Statistical tests are conducted to validate the superiority of the CFS algorithm in finding global optimum solutions. The results indicate that CFS achieves significantly better outcomes with a higher convergence rate when compared to the other algorithms tested. This highlights the effectiveness of CFS in solving MLP optimization problems and its potential as a competitive algorithm in the field.

A multi-agent optimization algorithm and its application to training multilayer perceptron models

A multi-agent optimization algorithm and its application to training multilayer perceptron models

Data-driven and uncertainty-aware robust airstrip surface estimation

The performances of aircraft braking control systems are strongly influenced by the tire friction force experienced during the braking phase. The availability of an accurate estimate of the current airstrip characteristics is a recognized issue for developing optimized braking control schemes. The study presented in this paper is focused on the robust online estimation of the airstrip characteristics from sensory data usually available on an aircraft. In order to capture the nonlinear dependency of the current best slip on sequential slip-friction measurements acquired during the braking maneuver, multilayer perceptron (MLP) approximators have been proposed. The MLP training is based on a synthetic data set derived from a widely used tire–road friction model. In order to achieve robust predictions, MLP architectures based on the drop-out mechanism have been applied not only in the offline training phase but also during the braking. This allowed to online compute a confidence interval measure for best friction estimate that has been exploited to refine the estimation via Kalman Filtering. Open loop and closed loop simulation studies in 15 representative airstrip scenarios (with multiple surface transitions) have been performed to evaluate the performance of the proposed robust estimation method in terms of estimation error, aircraft braking distance, and time, together with a quantitative comparison with a state-of-the-art benchmark approach.

Multiclass Apple Varieties Classification Using Machine Learning with Histogram of Oriented Gradient and Color Moments

It is critically necessary to maximize the efficiency of agricultural methods while concurrently reducing the cost of production. Varieties, types, and fruit classification grades are crucial to fruit production. High expenditure, inconsistent subjectivity, and tedious labor characterize traditional and manual varieties classification. This study developed machine learning (ML) models to classify ten apple varieties, extracting the histogram of oriented gradient (HOG) and color moments from RGB apple images. Support vector machine (SVM), random forest classifier (RFC), multilayer perceptron (MLP), and K-nearest neighbor (KNN) classification models were trained with 10-fold stratified cross-validation (Skfold) by using the textural and color features, and a GridSearch was implemented to fine-tune the hyperparameters. The trained models, SVM, RFC, MLP, and KNN were tested with separate test data and performed well, having an accuracy of 98.17%, 96.67%, 98.62%, and 91.28%, respectively. Having the top results, the MLP and SVM models demonstrated the potential of applying HOG and color moments to train ML models for classifying apple varieties. This study suggests conducting further research to thoroughly examine additional image features and determine the impact of combining features and utilizing different classifiers.

Remaining Useful Life Prediction of Aircraft Turbofan Engine Based on Random Forest Feature Selection and Multi-Layer Perceptron

The accurate prediction of the remaining useful life (RUL) of aircraft engines is crucial for improving engine safety and reducing maintenance costs. To tackle the complex issues of nonlinearity, high dimensionality, and difficult-to-model degradation processes in aircraft engine monitoring parameters, a new method for predicting the RUL of aircraft engines based on the random forest algorithm and a Bayes-optimized multilayer perceptron (MLP) was proposed here. First, the random forest algorithm was used to evaluate the importance of historical monitoring parameters of the engine, selecting the key features that significantly impact the engine’s lifetime operation cycle. Then, the single exponent smoothing (SES) algorithm was introduced for smoothing the extracted features to reduce the interference of original noise. Next, an MLP-based RUL prediction model was established using a neural network. The Bayes’ online parameter updating formula was used to solve the objective function and return the optimal parameters of the MLP training model and the minimum value of the evaluation index RMSE. Finally, the probability density function of the predicted RUL value of the aircraft engine was calculated to obtain the RUL prediction results.The effectiveness of the proposed method was verified and analyzed using the C-MAPSS dataset for turbofan engines. Experimental results show that, compared with several other methods, the RMSE of the proposed method in the FD001 test set decreases by 6.1%, demonstrating that the method can effectively improve the accuracy of RUL prediction for aircraft engines.

Noise estimation using an artificial neural network in the urban area of Jaen, Cajamarca

Jaen is a city in constant urban growth which generates an increase in vehicular traffic and active noise pollution. The research presents the development of an artificial neural network (ANN) to estimate the noise produced by vehicular traffic in the urban area of the city. Consequently, information was collected from two investigations coded as T1 and T2, for which a matrix of 10 variables was elaborated with 210 and 273 data respectively. Random random sampling was performed to divide the data matrix into 80% (training) and 20% (validation). Weka software and the multi-layer perceptron (MLP) training algorithm were used to model the ANN. An ANN for T1 with 6-19-1 architecture and an ANN for T2 with 6-15-1 architecture were obtained. The performance of the ANNs was evaluated using the correlation coefficient (R), coefficient of determination (R2) and root mean square error (RMSE). The results show that the MLP networks are able to estimate the sound pressure level with values of R=0.9927, R2=0.9854 and RMSE=0.7313 for T1, R=0.9989, R2=0.9978, and RMSE=0.1515 for T2.

Multiobjective bilevel programming model for multilayer perceptron neural networks

The architecture of multilayer perceptron (MLP) neural networks dictates the network's performance. However, aiming at the specific classification problems, suitable architectures of MLPs must be determined beforehand. More often, the design decisions rely on a trial and error learning and the experience knowledge. To automatically design an MLP's network architecture and optimize network parameters, a multiobjective bilevel programming model is built. In this model, a multiobjective optimization problem is constructed in the upper level for obtaining a set of Pareto optimal architectures of the MLPs, considering network complexity, training error rate, and validation error rate, while a single-objective optimization problem is established in the lower level to search for the optimum network parameters for a given network architecture. For dealing with this model efficiently, a novel multiobjective hierarchical learning algorithm (MOHLA) is proposed, in which an integer-coding NSGA-II is developed as the upper-level optimizer for a set of Pareto optimal network structures of the MLPs, while a non-iterative method is regarded as the lower-level solver for the MLPs' connection parameters. After a set of trained MLPs is obtained finally by using MOHLA, a selective ensemble strategy is adopted for improving identification accuracy. Three types of multiobjective bilevel programming models are investigated and compared in the experiments. Moreover, the proposed MOHLA is compared with several state-of-the-art learning approaches on various classification problems. The experimental results confirm that MOHLA performs well.

Fortified Cuckoo Search Algorithm on training multi-layer perceptron for solving classification problems

Fortified Cuckoo Search Algorithm on training multi-layer perceptron for solving classification problems

A competitive learning-based Grey wolf Optimizer for engineering problems and its application to multi-layer perceptron training.

This article presents a competitive learning-based Grey Wolf Optimizer (Clb-GWO) formulated through the introduction of competitive learning strategies to achieve a better trade-off between exploration and exploitation while promoting population diversity through the design of difference vectors. The proposed method integrates population sub-division into majority groups and minority groups with a dual search system arranged in a selective complementary manner. The proposed Clb-GWO is tested and validated through the recent CEC2020 and CEC2019 benchmarking suites followed by the optimal training of multi-layer perceptron's (MLPs) with five classification datasets and three function approximation datasets. Clb-GWO is compared against the standard version of GWO, five of its latest variants and two modern meta-heuristics. The benchmarking results and the MLP training results demonstrate the robustness of Clb-GWO. The proposed method performed competitively compared to all its competitors with statistically significant performance for the benchmarking tests. The performance of Clb-GWO the classification datasets and the function approximation datasets was excellent with lower error rates and least standard deviation rates.

Training of Classification Models via Federated Learning and Homomorphic Encryption.

With the rise of social networks and the introduction of data protection laws, companies are training machine learning models using data generated locally by their users or customers in various types of devices. The data may include sensitive information such as family information, medical records, personal habits, or financial records that, if leaked, can generate problems. For this reason, this paper aims to introduce a protocol for training Multi-Layer Perceptron (MLP) neural networks via combining federated learning and homomorphic encryption, where the data are distributed in multiple clients, and the data privacy is preserved. This proposal was validated by running several simulations using a dataset for a multi-class classification problem, different MLP neural network architectures, and different numbers of participating clients. The results are shown for several metrics in the local and federated settings, and a comparative analysis is carried out. Additionally, the privacy guarantees of the proposal are formally analyzed under a set of defined assumptions, and the added value of the proposed protocol is identified compared with previous works in the same area of knowledge.

ICE: Implicit Coordinate Encoder for Multiple Image Neural Representation.

In recent years, implicit neural representations (INR) have shown their great potential to solve many computer graphics and computer vision problems. With this technique, signals such as 2D images or 3D shapes can be fit by training multi-layer perceptrons (MLP) on continuous functions, providing many advantages over conventional discrete representations. Despite being considered a promising approach to 2D image encoding and compression, the application of INR to image collections remains a challenge, since the number of parameters needed rapidly grow with the number of images. In this paper, we propose a fully implicit approach to INR which drastically reduces the size of MLP models in multiple image representation tasks. We introduce the concept of implicit coordinate encoder (ICE) and show it can be used to scale INR with the image number; specifically, by learning a common feature space between images. Furthermore, we show that our method is valid not only for image collections but also for large (gigapixel) images by applying a "divide-and-conquer" strategy. We propose an auto-encoder deep neural network architecture, with a single ICE (encoder) and multiple MLP (decoders), which are jointly trained following a multi-task learning strategy. We demonstrate the benefits coming from ICE when it is implemented as a one-dimensional convolutional encoder, including a better performance of the downstream MLP models with an order of magnitude fewer parameters. Our method is the first one to make use of convolutional blocks in INR networks, unlike the conventional approach of using MLP architectures only. We show the benefits of ICE in two experimental scenarios: a collection of twenty-four small ( 768×512 ) images (Kodak dataset), and a single large ( 3072×3072 ) image (dwarf planet Pluto), achieving better quality than previous fully-implicit methods, using up to 50% fewer parameters.

Diagnosing Faults in Power Transformers With Variational Autoencoder, Genetic Programming, and Neural Network

This work presents a new approach for the diagnosis of incipient faults in power transformers by considering dissolved gas analysis (DGA). A multilayer perceptron (MLP) neural network was trained to diagnose the type of transformer fault. For training and testing of the classifier, data were used from in-service transformers obtained from the IEC TC 10 database and other data obtained from the literature. To address the imbalance of the data from the database adopted and thus improve the generalization power of the classifier, a data augmentation technique based on a variational autoencoder neural network was used. For the selection and extraction of characteristics from the inputs to the classifier, a technique based on genetic programming (GP) is proposed, which allows the creation of a new n-dimensional space of characteristics, providing a greater ability to increase interclass distances and intraclass compaction. For the performance analysis of the proposed classifier, comparisons were made using the classification results obtained through the IEC 60599 conventional fault diagnosis method and other trained MLPs without the use of data augmentation and the proposed characteristics extractor. The results obtained demonstrate the applicability of the proposed methodology for fault diagnosis, with the proposed system obtaining an accuracy of 95.18% in the test basis, which is higher than the results achieved by the other methods used to perform a comparison and analysis of results.

Embarrassingly Parallel Independent Training of Multi-Layer Perceptrons with Heterogeneous Architectures

In this paper we propose a procedure to enable the training of several independent Multilayer Perceptron Neural Networks with a different number of neurons and activation functions in parallel (ParallelMLPs) by exploring the principle of locality and parallelization capabilities of modern CPUs and GPUs. The core idea of this technique is to represent several sub-networks as a single large network and use a Modified Matrix Multiplication that replaces an ordinal matrix multiplication with two simple matrix operations that allow separate and independent paths for gradient flowing. We have assessed our algorithm in simulated datasets varying the number of samples, features and batches using 10,000 different models as well as in the MNIST dataset. We achieved a training speedup from 1 to 4 orders of magnitude if compared to the sequential approach. The code is available online.

Comparison of template-based and multilayer perceptron-based approach for automatic question generation system

Intelligent tutoring systems are computer-assisted learning systems with adaption to students using artificial intelligence tools. An intelligent tutoring system can drastically improve education efficiency as it provides solutions to many issues that now plague the educational industry. One important component in education is questioning learners to assess and reinforce learning. This research compares two approaches for automatic question generation, a template-based question generation strategy and the phrase-Level automatic question generation system utilizing the Multilayer perceptron model. A template-based technique is a baseline for automatic question generation that uses templates taken from the training set to generate questions by filling certain templates with specific topic items. We utilize question-answer sentence composition datasets and manually constructed datasets for our experiments and comparison of the Multilayer perceptron training model. We used both human and automatic evaluation metrics to assess the efficiency of our suggested methods. Regarding automatic metrics, we selected the BLEU-n gram and ROUGE-N methodologies. The evaluation results demonstrate that the phrase-level Multilayer perceptron-based strategy dominates the template-based approach and has a promising score in both ROUGE automatic and human evaluation metrics.

Comparing the performance of multi-layer perceptron training on electrical and optical network-on-chips

Comparing the performance of multi-layer perceptron training on electrical and optical network-on-chips

Feature Selection and Training Multilayer Perceptron Neural Networks Using Grasshopper Optimization Algorithm for Design Optimal Classifier of Big Data Sonar

The complexity and high dimensions of big data sonar, as well as the unavoidable presence of unwanted signals such as noise, clutter, and reverberation in the environment of sonar propagation, have made the classification of big data sonar one of the most interesting and applicable topics for active researchers in this field. This paper proposes the use of the Grasshopper Optimization Algorithm (GOA) to train Multilayer Perceptron Artificial Neural Network (MLP-NN) and also to select optimal features in big data sonar (called GMLP-GOA). GMLP-GOA hybrid classifier first extracts the features of experimental sonar data using MFCC. Then, the most optimal features are selected using GOA. In the last step, MLP-NN trained with GOA is used to classify big data sonar. To evaluate the performance of GMLP-GOA, this classifier is compared with MLP-GOA, MLP-GWO, MLP-PSO, MLP-ACO, and MLP-GSA classifiers in terms of classification rate, convergence rate, local optimization avoidance power, and processing time. The results indicated that GMLP-GOA achieved a classification rate of 98.12% in a processing time of 3.14 s.

Sonar Data Classification Using Neural Network Trained by Hybrid Dragonfly and Chimp Optimization Algorithms

This paper proposes a hybrid Dragonfly Algorithm (DA) for training Multi-Layer Perceptron Neural Network (MLP NN) to design the classifier for solving complicated problems and distinguishing the real target from liars’ targets in sonar applications. Due to improve the cost computation and reducing the waste of time, a modified low-cost DA is designed for evaluation. To assess the accuracy of the technique, some well-known meta-heuristic trainers include Chimp Optimization Algorithm, Gravitational Search Algorithm (GSA), DA, and Particle Swarm Optimization (PSO) compared to show the accuracy of similar algorithms. DA and ChoA algorithms have remarkable features and a hybrid algorithm of them is proposed. The proposed classifier has acceptable performance, and two standard benchmark datasets are used to evaluate performance. The results show that the modified hybrid DA-ChoA has 15% less time-consuming and 4% better performance rather than the original dragonfly method.