Implementation Of Multilayer Perceptron Research Articles

Efficient Implementation of Multilayer Perceptrons: Reducing Execution Time and Memory Consumption

A technique is presented that reduces the required memory of neural networks through improving weight storage. In contrast to traditional methods, which have an exponential memory overhead with the increase in network size, the proposed method stores only the number of connections between neurons. The proposed method is evaluated on feedforward networks and demonstrates memory saving capabilities of up to almost 80% while also being more efficient, especially with larger architectures.

Secure CAPTCHA by Genetic Algorithm (GA) and Multi-Layer Perceptron (MLP)

To achieve an acceptable level of security on the web, the Completely Automatic Public Turing test to tell Computer and Human Apart (CAPTCHA) was introduced as a tool to prevent bots from doing destructive actions such as downloading or signing up. Smartphones have small screens, and, therefore, using the common CAPTCHA methods (e.g., text CAPTCHAs) in these devices raises usability issues. To introduce a reliable, secure, and usable CAPTCHA that is suitable for smartphones, this paper introduces a hand gesture recognition CAPTCHA based on applying genetic algorithm (GA) principles on Multi-Layer Perceptron (MLP). The proposed method improves the performance of MLP-based hand gesture recognition. It has been trained and evaluated on 2201 videos of the IPN Hand dataset, and MSE and RMSE benchmarks report index values of 0.0018 and 0.0424, respectively. A comparison with the related works shows a minimum of 1.79% fewer errors, and experiments produced a sensitivity of 93.42% and accuracy of 92.27–10.25% and 6.65% improvement compared to the MLP implementation. The range of the supported hand gestures can be a limit for the application of this research as a limited range may result in a vulnerable CAPTCHA. Also, the processes of training and testing require significant computational resources. In the future, we will optimize the method to run with high reliability in various illumination conditions and skin color and tone. The next development plan is to use augmented reality and create unpredictable random patterns to enhance the security of the method.

Implementation of multilayer perceptron (MLP) and radial basis function (RBF) neural networks for predicting Shatavarin IV content in Asparagus racemosus accessions.

Implementation of multilayer perceptron (MLP) and radial basis function (RBF) neural networks for predicting Shatavarin IV content in Asparagus racemosus accessions.

Exploring the Potential of Machine Learning for Modeling Growth Dynamics in an Uneven-Aged Forest at the Level of Diameter Classes: A Comparative Analysis of Two Modeling Approaches

Growth models of uneven-aged forests on the diameter class level can support silvicultural decision making. Machine learning brings added value to the modeling of dynamics at the stand or individual tree level based on data from permanent plots. The objective of this study is to explore the potential of machine learning for modeling growth dynamics in uneven-aged forests at the diameter class level based on inventory data from practice. Two main modeling approaches are conducted and compared: (i) fine-tuned linear models differentiated per diameter class, (ii) an artificial neural network (multilayer perceptron) trained on all diameter classes. The models are trained on the inventory data of the Canton of Neuchâtel (Switzerland), which are area-wide data without individual tree-level growth monitoring. Both approaches produce convincing results for predicting future diameter distributions. The linear models perform better at the individual diameter class level with test R2 typically between 50% and 70% for predicting increments in the numbers of stems at the diameter class level. From a methodological perspective, the multilayer perceptron implementation is much simpler than the fine-tuning of linear models. The linear models developed in this study achieve sufficient performance for practical decision support.

Establishment of optimized in vitro disinfection protocol of Pistacia vera L. explants mediated a computational approach: multilayer perceptron–multi−objective genetic algorithm

BackgroundContamination−free culture is a prerequisite for the success of in vitro − based plant biotechnology. Aseptic initiation is an extremely strenuous stride, particularly in woody species. Meanwhile, over−sterilization is potentially detrimental to plant tissue. The recent rise of machine learning algorithms in plant tissue culture proposes an advanced interpretive tool for the combinational effect of influential factors for such in vitro − based steps.ResultsA multilayer perceptron (MLP) model of artificial neural network (ANN) was implemented with four inputs, three sterilizing chemicals at various concentrations and the immersion time, and two outputs, disinfection efficiency (DE) and negative disinfection effect (NDE), intending to assess twenty−seven disinfection procedures of Pistacia vera L. seeds. Mercury chloride (HgCl2; 0.05–0.2%; 5–15 min) appears the most effective with 100% DE, then hydrogen peroxide (H2O2; 5.25–12.25%; 10–30 min) with 66–100% DE, followed by 27–77% DE for sodium hypochlorite (NaOCl; 0.54–1.26% w/v; 10–30 min). Concurrently, NDE was detected, including chlorosis, hard embryo germination, embryo deformation, and browning tissue, namely, a low repercussion with NaOCl (0–14%), a moderate impact with H2O2 (6–46%), and pronounced damage with HgCl2 (22–100%). Developed ANN showed R values of 0.9658, 0.9653, 0.8937, and 0.9454 for training, validation, testing, and all sets, respectively, which revealed the uprightness of the model. Subsequently, the model was linked to multi−objective genetic algorithm (MOGA) which proposed an optimized combination of 0.56% NaOCl, 12.23% H2O2, and 0.068% HgCl2 for 5.022 min. The validation assay reflects the high utility and accuracy of the model with maximum DE (100%) and lower phytotoxicity (7.1%).ConclusionIn one more case, machine learning algorithms emphasized their ability to resolve commonly encountered problems. The current successful implementation of MLP–MOGA inspires its application for more complicated plant tissue culture processes.

Study of Using Machine Learning for Level 1 Trigger Decision in JUNO Experiment

A study on the use of a machine learning algorithm for the level 1 trigger decision in the JUNO experiment ispresented. JUNO is a medium baseline neutrino experiment in construction in China, with the main goal of determining the neutrino mass hierarchy. A large liquid scintillator (LS)volume will detect the electron antineutrinos issued from nuclear reactors. The LS detector is instrumented by around 20000 large photomultiplier tubes. The hit information from each PMT will be collected into a center trigger unit for the level 1 trigger decision. The current trigger algorithm used to select a neutrino signal event is based on a fast vertex reconstruction. We propose to study an alternative level 1 (L1) trigger in order to achieve a similar performance as the vertex fitting trigger but with less logic resources by using firmware implemented machine learning model at the L1 trigger level. We treat the trigger decision as a classification problem and train a Multi-Layer Perceptron (MLP)model to distinguish the signal events with an energy higher than a certain threshold from noise events. We use JUNO software to generate datasets which include 100K physics events with noise and 100K pure noise events coming from PMT dark noise.For events with energy higher than 100 keV, the L1 trigger based on the converged MLP model can achieve an efficiency higher than 99%. After the training performed on simulations,we successfully implemented the trained model into a Kintex 7FPGA. We present the technical details of the neural network development and training, as well as its implementation in the hardware with the FPGA programming. Finally the performance of the L1 trigger MLP implementation is discussed.

Automatic Sleep Stage Detection: A Study on the Influence of Various PSG Input Signals.

Automatic sleep stage detection can be performed using a variety of input signals from a polysomnographic (PSG) recording. In this study, we investigate the effect of different input signals on the performance of feature-based automatic sleep stage classification algorithms with both a Random Forest (RF) and Multilayer Perceptron (MLP) classifier. Combinations of the EEG (electroencephalographic) signal and ECG (electrocardiographic), EMG (electromyographic) and respiratory signals as input are investigated as input with respect to using single channel and multi-channel EEG as input. The Physionet "You Snooze, You Win" dataset is used for the study. The RF classifier consistently outperforms our MLP implementation in all cases and is positively affected by specific signal combinations. The overall classification performance using a single channel EEG is high (an accuracy, precision and recall of 86.91 %, 89.52%, 86.91% respectively) using RF. The results are comparable to the performance obtained using six EEG channels as input. Adding respiratory signals to the inputs processed by RF increases the N2 stage detection performance with 20%, while adding the EMG signal improves the accuracy of the REM stage detection with 5%. Our analysis shows that adding specific signals as input to RF improves the accuracy of specific sleep stages and increases the overall performance. Using a combination of EEG and respiratory signals we achieved an accuracy of 93% for the RF classifier.

Efficient FPGA Implementation of Multilayer Perceptron for Real-Time Human Activity Classification

The smartphone-based human activity recognition (HAR) systems are not capable to deliver high-end performance for challenging applications. We propose a dedicated hardware-based HAR system for smart military wearables, which uses a multilayer perceptron (MLP) algorithm to perform activity classification. To achieve the flexible and efficient hardware design, the inherent MLP architecture with parallel computation is implemented on FPGA. The system performance has been evaluated using the UCI human activity dataset with 7767 feature samples of 20 subjects. The three combinations of a dataset are trained, validated, and tested on ten different MLP models with distinct topologies. The MLP design with the 7-6-5 topology is finalized from the classification accuracy and cross entropy performance. The five versions of the final MLP design (7-6-5) with different data precision are implemented on FPGA. The analysis shows that the MLP designed with 16-bit fixed-point data precision is the most efficient MLP implementation in the context of classification accuracy, resource utilization, and power consumption. The proposed MLP design requires only 270 ns for classification and consumes 120 mW of power. The recognition accuracy and hardware results performance achieved are better than many of the recently reported works.

Analog programmable neuron and case study on VLSI implementation of Multi-Layer Perceptron (MLP)

In this work, a programmable neuron is proposed to approximate the following activation functions: sigmoid, hyperbolic tangent and linear. In other words, the neural network designer can use several control bits to choose the type of activation function without any physical change. The proposed neuron which is simulated in a 0.18 μm CMOS technology, shows a good approximation with maximum error from the ideal hyperbolic tangent function and the ideal sigmoid function by 29.33% and 7.4%, respectively. In order to evaluate the functionality of the neuron, it is applied in two Multi-Layer Perceptron (MLP) neural networks. The first one which is trained to implement XOR gate, is capable of processing signals in the frequency range from 2.5 mHz to 50 MHz. The accuracy of the network is more than 99.9%. The second one is a pattern recognition neural network. Comparison with previous work reveals that there is 48% decrease in the network's power consumption. Moreover, the proposed neuron has been applied in the fully connected (FC) layers of a convolutional neural network (CNN) and an experiment has been conducted on a benchmark dataset MNIST.

Implementation of multilayer perceptron (MLP) and radial basis function (RBF) neural networks to predict solution gas-oil ratio of crude oil systems

Abstract Exact determination of pressure-volume-temperature (PVT) properties of the reservoir oils is necessary for reservoir calculations, reservoir performance prediction, and the design of optimal production conditions. The objective of this study is to develop intelligent and reliable models based on multilayer perceptron (MLP) and radial basis function (RBF) neural networks for estimating the solution gas–oil ratio as a function of bubble point pressure, reservoir temperature, oil gravity (API), and gas specific gravity. These models were developed and tested using a total of 710 experimental data sets representing the samples of crude oil from various geographical locations around the world. Performance of the developed MLP and RBF models were evaluated and investigated against a number of well-known empirical correlations using statistical and graphical error analyses. The results indicated that the proposed models outperform the considered empirical correlations, providing a strong agreement between predicted and experimental values, However, the developed RBF exhibited higher accuracy and efficiency compared to the proposed MLP model.

Adiabatic superconducting artificial neural network: Basic cells

We consider adiabatic superconducting cells operating as an artificial neuron and synapse of a multilayer perceptron (MLP). Their compact circuits contain just one and two Josephson junctions, respectively. While the signal is represented as magnetic flux, the proposed cells are inherently nonlinear and close-to-linear magnetic flux transformers. The neuron is capable of providing the one-shot calculation of sigmoid and hyperbolic tangent activation functions most commonly used in MLP. The synapse features both positive and negative signal transfer coefficients in the range ∼(−0.5,0.5). We briefly discuss implementation issues and further steps toward the multilayer adiabatic superconducting artificial neural network, which promises to be a compact and the most energy-efficient implementation of MLP.

Towards an integrated evolutionary strategy and artificial neural network computational tool for designing photonic coupler devices

Photonics has been widely explored in computing and communications, mainly to rationalize the relationship between device size minimization and data processing/transmission maximization. Generally driven by optimization and modeling techniques, the design of photonic devices is often performed by bio-inspired algorithms integrated to electromagnetic solvers, which have achieved advances but is still time-consuming. As an alternative to a costly finite element method (FEM) solver, a multilayer perceptron (MLP) neural network is proposed for computing power coupling efficiency of photonic couplers, originally designed through an integrated evolutionary strategy (ES) and FEM routine. We address the ES-FEM design of two efficient couplers, present the MLP implementation and the MLP training and testing over the routine generated datasets, and measure MLP and FEM runtime. MLP suitably predicted the power coupling efficiency of a variety of unknown couplers on tests. The measured runtime showed MLP is ∼105 faster than FEM. In conclusion, MLP is a potential tool to be integrated to ES on the design of such photonic couplers.

Model Multi Layer Perceptron untuk Indoor Positioning System Berbasis Wi-Fi

Indoor positioning system issue is an open problem that still needs some improvements. This research explores the utilization of multilayer perceptron in determining someone’s position inside a building or a room, which generally known as Indoor Positioning System. The research was conducted in some steps: dataset normalization, multilayer perceptron implementation, training process of multilayer perceptron, evaluation, and analysis. The training process has been conducted many times to find the best parameters that produce the best accuracy rate. The experiment produces 79,16% as the highest accuracy rate. Compared to previous research, this result is comparably lower and needs some parameters tweaking or changing the neural networks architectures.

Application of Machine Learning for Dragline Failure Prediction

Overburden stripping in open cast coal mines is extensively carried out by walking draglines. Draglines’ unavailability and unexpected failures result in delayed productions and increased maintenance and operating costs. Therefore, achieving high availability of draglines plays a crucial role for increasing economic feasibility of mining projects. Applications of methodologies which can forecast the failure type of dragline based on the available failure data not only help to reduce the maintenance and operating costs but also increase the availability and the production rate. In this study, Machine Learning approaches have been applied for data which has been gathered from an operating coal mine in Turkey. The study methodology consists of three algorithms as: i) implementation of K-Nearest Neighbors, ii) implementation of Multi-Layer Perceptron, and iii) implementation of Radial Basis Function. The algorithms have been utilized for predicting the draglines’ failure types. In this sense, the input data, which are mean time-to-failure, and the output data, failure types, have been fed to the algorithms. The regression analysis of methodologies have been compared and showed the K- Nearest Neighbors has a higher rate of regression which is around 70 percent. Thus, the K-Nearest Neighbor algorithm can be applied in order to preventive components replacement which causes to minimized preventive and corrective cost parameters. The accurate prediction of failure type, indeed, causes to optimized number of inspections. The novelty of this study is application of machine learning approaches in draglines’ reliability subject for first time.

Design and Implementation of Multilayer Perceptron with On-chip Learning in Virtex-E

Due to advancements in technology, many integrated circuits are fabricated to develop an artificial system that could perform “intelligent” tasks similar to those performed by the human brain. Many of them use off-chip learning method either by analog hardware or massively by parallel computers. This proposed work is about a trainable neural chip using Field Programmable Gate Array (FPGA) as this helps in learning capability by exploiting the inherent parallelism of neural network. By this fast prototyping is possible for real-time applications, such as speech recognition, speech synthesis, image processing, pattern recognition and classification. In this work on-chip learning method is designed for standard benchmark XOR problem using back propagation based multilayer perceptron and is implemented in VIRTEX-E FPGA using VHDL. The design works at 5.332MHz and the total gate count is 4, 73,237.

IP Core Based Hardware Implementation of Multi-Layer Perceptrons on FPGAs: A Parallel Approach

There’re many models derived from the famous bio-inspired artificial neural network (ANN). Among them, multi-layer perceptron (MLP) is widely used as a universal function approximator. With the development of EDA and recent research work, we are able to use rapid and convenient method to generate hardware implementation of MLP on FPGAs through pre-designed IP cores. In the mean time, we focus on achieving the inherent parallelism of neural networks. In this paper, we firstly propose the hardware architecture of modular IP cores. Then, a parallel MLP is devised as an example. At last, some conclusions are made.

Comparing Sigmoid Transfer Functions for Neural Network Multistep Ahead Streamflow Forecasting

One of the main problems of neural networks is the lack of consensus on how to best implement them. This work targets the question of the transfer function selection—a vital part of neural network providing nonlinear mapping potential. Three nonlinear transfer functions bounded by 1 and 1 are selected for testing, based on a literature review: the Elliott sigmoid, the bipolar sigmoid, and the tangent sigmoid. They are used to design multilayer perceptron neural networks for multistep ahead streamflow forecasting over five diverse watersheds and lead times from 1 to 5 days. All multilayer perceptrons have shown a good performance on the account of the four selected criteria, which confirms that the selected multilayer perceptron implementation procedure was adequate, namely, the data set length, the Kohonen network clustering method to create the training and testing sets, and the Levenberg-Marquardt back-propagation training procedure with Bayesian regularization. Specifically, results endorsed the tangent sigmoid as the most pertinent transfer function for streamflow forecasting, over the bipolar logistic and Elliott sigmoids, but the latter requires less computing time and as such may be a valuable option for operational hydrology. Also, results averaged over five lead times confirmed the universal approximation theorem that a linear transfer function is suitable for the output layer—a nonlinear transfer function in the output layer failed to improve performance values. DOI: 10.1061/ASCEHE.1943-5584.0000188 CE Database subject headings: Neural networks; Streamflow; Forecasting; Transfer functions; Computation. Author keywords: Neural networks; Streamflow forecasting; Transfer function; Performance evaluation; Computing time.

FPGA Implementation of a Multilayer Perceptron (MLP) Network

In this paper, we suggest a method for designing and implementing of multilayer Perceptron (MLP) neural network based on backpropagation (PB) learning algorithm. The method is described using very high speed integrated circuit hardware description language (VHDL), that used in developing the designs of a very large scale integration (VLSI). Firstly artificial neuron with sigmoid activation function has been designed and implemented which is considered as a basic unit of MLP. The MLP network is trained by BP algorithms, in the Matlab environment in order to obtain the ideal parameters of the network. Then hardware implementation of MLP on FPGAs, of types Spartan 3E and Virtex4 is achieved by using integer format and floating point format respectively . A comparison is done between the two arithmetic formats of MLP implementations on FPGAs. Keyword: MLP neural networks , floating-point (FLP) arithmetic, FPGA, VHDL.

System on programmable chip implementation of neural network-based isolated digit recognition system

In this article, design and implementation of an isolated digit recognition system on a NiosII soft-core processor with a CycloneII FPGA is considered. Linear predictive coefficients and Mel frequency cepstral coefficients (MFCC) are used as features and their recognition performances are compared. A multilayer perceptron (MLP) is used for classification and a self-organised feature map is employed for dimensionality reduction of features. Various MLP architectures are studied in this article and it is found that a recognition accuracy of 100% is obtained with the least computational complexity using a single-layer MLP with 10 hidden nodes for the MFCC feature. Recognition performance and convergence time for training is compared for the ‘modified’ back propagation (BP) algorithm and the BP algorithm. It is observed that the modified BP is 2.82 times faster than BP with 4% increase in recognition accuracy. A hardware accelerator is proposed for the NiosII processor for the implementation of MLP and it increases the recognition speed by a factor of 278. The technique proposed in this article is also applicable for other soft-core processors such as Microblaze and Picoblaze.

FPGA 상에서 은닉층 뉴런에 최적화된 MLP의 설계 방법

일반적으로 신경망은 비선형성 문제를 해결하기 위해서 소프트웨어로 많이 구현되었지만, 영상처리 및 패턴인식과 같은 실시간 처리가 요구되는 응용에서는 빠른 처리가 가능한 하드웨어로 구현되고 있다. 다양한 종류의 신경망 중에서 다층 신경망(MLP: multi-layer perceptron)의 하드웨어 설계는 빠른 처리속도와 적은 면적 그리고 구현의 용이성으로 고정소수점 연산을 많이 사용하였다. 하지만 고정소수점 연산을 사용하는 하드웨어 설계는 높은 정확도의 부동소수점 연산을 많이 사용하는 소프트웨어 MLP를 쉽게 적용할 수 없는 문제점을 가진다. 본 논문에서는 높은 정확도와 높은 유연성을 가지는 부동소수점 연산을 사용하면서도 은닉층 뉴런수를 주기(cycle)로 빠르게 수행하는 MLP의 완전 파이프라이닝(fully-pipelining) 설계방법을 제안한다. MLP는 주어진 문제에 의해서 자연스럽게 입력층과 출력층의 구조가 결정되지만, 은닉층 구조는 사용자에 의해서 결정된다. 그러므로 제안된 설계방법은 많은 반복수행이 요구되는 영상처리 및 패턴인식 등의 분야에서 은닉층 뉴런수를 최적화 하여 쉽게 성능 향상을 이룰 수 있다. Neural Networks(NNs) are applied for solving a wide variety of nonlinear problems in several areas, such as image processing, pattern recognition etc. Although NN can be simulated by using software, many potential NN applications required real-time processing. Thus they need to be implemented as hardware. The hardware implementation of multi-layer perceptrons(MLPs) in several kind of NNs usually uses a fixed-point arithmetic due to a simple logic operation and a shorter processing time compared to the floating-point arithmetic. However, the fixed-point arithmetic-based MLP has a drawback which is not able to apply the MLP software that use floating-point arithmetic. We propose a design method for MLPs which has the floating-point arithmetic-based fully-pipelining architecture. It has a processing speed that is proportional to the number of the hidden nodes. The number of input and output nodes of MLPs are generally constrained by given problems, but the number of hidden nodes can be optimized by user experiences. Thus our design method is using optimized number of hidden nodes in order to improve the processing speed, especially in field of a repeated processing such as image processing, pattern recognition, etc.