Multilayer Neural Network Architecture Research Articles

Deep reinforcement learning for solving the single container loading problem

In this article, a new deep Reinforcement Learning (RL) model is proposed to solve the Single Container Loading Problem (SCLP) as well as the SCLP with full support. For that purpose, a multilayer neural network architecture is used. Computational experiments, conducted on benchmark instances with homogeneous boxes, revealed that the proposed model yields fairly good results compared to those found with state-of-the-art heuristics. Experiments have also shown good generalization capability of the proposed deep RL model to deal with both homogeneous and heterogeneous classes of instances. Nevertheless, this learning-based optimization approach still has an optimality gap compared to the well-designed heuristics of the operations research literature. Finally, the benefit of training the model under different levels of variability has been analysed. Results revealed that, for better performance, and if a company does not face high demand volatility, it is recommended to train the model under a low level of variability.

Image-based malicious Internet content filtering method for child protection

Children and teenagers are among Internet users and they encounter harmful data in the global network. Young users often become the potential victims of pornographic images. Avoiding pornographic images harmful to the child audience is an important research task in the field of detection, computer vision and multimedia. Malicious content can be prevented using various methods. Current paper presents a ChildNet model that filters harmful image content. The pixels of the digital images are used as a data source for recognition of nudity in the images. For each class, a multi-layer deep neural network architecture with five convolution blocks is developed to study the color patterns of undesirable image pixels. The developed neural network consists of 21 layers; the size of the filters is specified as (3 × 3). The filter's size is reduced to increase the accuracy of pixel recognition. The efficiency of the proposed method is tested on real datasets for evaluation purposes and the superior results are obtained from the proposed method in comparison with classical CNN.

Multilabel Text Classification Using Multilayer DGAT

Text classification is one of the most fundamental tasks in text data analysis. Generally, textual data are extensive in scale and complex in the inherent relationship, leading to low accuracy in traditional classification models. This paper proposes a multi-label classification model based on multi-layer neural network architecture. We first construct a dual-attention mechanism graph neural network (named DGAT for short) to fuse the typological and informational features of the target node and the connected nodes. Secondly, we also build a multi-layer network architecture with multiple DGATs to expand the range of neighborhood nodes participating in the feature fusion to meet the needs of classifying different datasets. To ensure the learning ability of the model, we also use the residual network to solve the problem of error rise and gradient descent caused by multi-layer network architecture. Finally, we conducted a large number of experiments on five benchmark datasets. The results show that the accuracy of the proposed model is significantly better than that of the traditional models, and that there is a noticeable improvement when compared with other deep learning methods.

Benchmarking a Multi-Layer Approach and Neural Network Architectures for Defect Detection in Pbf-Lb/M

Benchmarking a Multi-Layer Approach and Neural Network Architectures for Defect Detection in Pbf-Lb/M

Hysteresis Modeling in Iron-Dominated Magnets Based on a Multi-Layered NARX Neural Network Approach.

A full-fledged neural network modeling, based on a Multi-layered Nonlinear Autoregressive Exogenous Neural Network (NARX) architecture, is proposed for quasi-static and dynamic hysteresis loops, one of the most challenging topics for computational magnetism. This modeling approach overcomes drawbacks in attaining better than percent-level accuracy of classical and recent approaches for accelerator magnets, that combine hybridization of standard hysteretic models and neural network architectures. By means of an incremental procedure, different Deep Neural Network Architectures are selected, fine-tuned and tested in order to predict magnetic hysteresis in the context of electromagnets. Tests and results show that the proposed NARX architecture best fits the measured magnetic field behavior of a reference quadrupole at CERN. In particular, the proposed modeling framework leads to a percent error below 0.02% for the magnetic field prediction, thus outperforming state of the art approaches and paving a very promising way for future real time applications.

Enhanced brain tumor classification using an optimized multi-layered convolutional neural network architecture

Enhanced brain tumor classification using an optimized multi-layered convolutional neural network architecture

Identification of the appropriate architecture of multilayer feed-forward neural network for estimation of NPPs parameters using the GA in combination with the LM and the BR learning algorithms

In this study, accurate estimation of nuclear power plant (NPP) parameters is done using the new and simple technique. The proposed technique using the genetic algorithm (GA) in combination with the Bayesian regularization (BR) and Levenberg- Marquardt (LM) learning algorithms identifies the appropriate architecture for estimation of the target parameters. In the first step, the input patterns features are selected using the features selection (FS) technique. In the second step, the appropriate number of hidden neurons and hidden layers are investigated to provide a more efficient initial population of the architectures. In the third step, the estimation of the target parameter is done using different architectures of multilayer feed-forward neural network (MFFN) with LM learning algorithm in which the maximum number of hidden neurons and the maximum number of hidden layers have been limited. In the fourth step, the proposed technique using the GA in combination with the BR learning algorithm is proposed to determine the more appropriate number and the more appropriate distribution of hidden neurons. To study the performance of the proposed technique, Bushehr nuclear power plant (BNPP) transients are examined. The different important transients/parameters are estimated. The results of the estimations by the identified architecture in comparison with the other appropriate architectures show superiority of the proposed technique. Therefore, the proposed technique can be used reliably for accurate estimation of the important parameters and can be used as a support tool by the operators in confront with transients.

A SECURE AND MANAGED CLOUD STORAGE SYSTEM USING ENCRYPTION WITH MACHINE LEARNING APPROACH

Cloud storage platform is a promising architecture provided by the Cloud Service. However, the use of these services raises many doubts and concerns about the security, confidentiality, reliability and integrity of users' data and information. Because the cloud is based on a per-user payment model, it will take longer to retrieve the required document, which raises the financial burden, and hence affects the satisfaction level of cloud users. This is the major point, where presented research comes to play. The proposed work takes advantage of multi-layered neural network architecture for secure cloud storage system along with the involvement of encryption and similarity approaches such as Cosine Similarity algorithm while encryption check engrosses AES and DSA approaches. Simulation analysis offers a secure cloud platform using cloudsim simulator. Experimentation was performed against 700 text documents to evaluate the proposed work in terms of precision, recall, f-score and accuracy with an average accuracy of 92.48%. Simulation results had demonstrated that the designed algorithm proved to offer data storage in a cloud computing environment with high-end security. In future, the authors aim to involve some deep learning approaches to improve the text mining capabilities using cloud storage without challenging data security.

FPGA-based real-time epileptic seizure classification using Artificial Neural Network

Epilepsy is a neurological disorder characterised by unusual brain activity widely known as seizure affecting 4-7% of the world's population. The diagnosis of this disorder is currently based on analysis of the electroencephalography (EEG) signals in the time-frequency domain. The analysis is performed applying various algorithms that yield high performance, however the challenge of effective real-time epilepsy diagnosis persists.To address this, we have developed a Field Programmable Gate Array (FPGA) based solution for the classification of generalized and focal epileptic seizure types using a feed-forward multi-layer neural network architecture (MLP ANN).The neural network algorithm is trained, validated and tested on 822 captured signals from Temple University Hospital Seizure Detection Corpus (TUH EEG Corpus) database. Inputs into the system were five main features obtained from EEG signals by time-frequency analysis followed by Continuous Wavelet Transform (CWT) and subsequent statistical analysis. Out of the total number of samples, 583 (70 %) of them were utilised during the system development in MATLAB and TensorFlow and 239 (30 %) samples were further used for subsequent testing of the model performance on the FPGA. Subsequently, the adequate parameters of the ANN model were determined by using k-Fold Cross-Validation. Finally, the best performing ANN model in terms of average validation data accuracy achieved during cross-validation was implemented on the FPGA for real-time seizure classification. The digital ANN solution was coded in Very High-Speed Integrated Circuit Hardware Description Language (VHDL) and tested on the FPGA using 30 % reaming data.The results of this research demonstrate that epilepsy diagnosis with quite high accuracy (95.14 %) can be achieved with (5-12-3) MLP ANN implemented on FPGA. Also, the results show the steps towards appropriate implementation of ANN on the FPGA. These results can be utilised as the basis for the design of an application-specific integrated circuit (ASIC) allowing large serial production.

Training a Multi-Layer Photonic Spiking Neural Network With Modified Supervised Learning Algorithm Based on Photonic STDP

We propose a framework for hardware architecture and learning algorithm co-design of multi-layer photonic spiking neural network (SNN). The vertical-cavity surface-emitting laser with an embedded saturable absorber (VCSEL-SA) which contains two polarization-resolved modes is employed as a spiking neuron. The connection between two identical polarization modes is considered as the excitatory synapse, whereas the connection between two orthogonal polarization modes is regarded as the inhibitory synapse. The physical model of the photonic spiking neuron is derived based on the combination of spin-flip model and Yamada model. The photonic spike timing dependent plasticity (STDP) is applied to design a hardware-friendly biologically plausible supervised learning algorithm for a multi-layer photonic SNN. Thanks to the polarization mode competition effect in the VCSEL-SA, the proposed neuromorphic network is capable of solving the classical XOR problem. The effect of physical parameters of photonic neuron on the training convergence is also considered. We further extend the multi-layer photonic SNN to realize other logic learning tasks. To the best of our knowledge, such a modified supervised learning algorithm dedicated for a multi-layer photonic SNN has not yet been reported, which is interesting for spiking learning of neuromorphic photonics.

Model Structures and Identification for Fully Embedded Thrusters: 360-Degrees-Steerable Steering-Grid and Four-Channel Thrusters

The European Watertruck + project introduced a new fleet of self-propelled inland cargo barges to the European waters, in order to induce more sustainable freight transport in the European hinterland. An augmentation of the automation level of this fleet could further advance their competitiveness and potentially pave the way for unmanned inland cargo vessels. The motion control of such a vessel forms a key component in this envisaged automation chain and benefits from the knowledge of the capabilities of the propulsion system, which here envelops a 360-degrees-steerable steering-grid thruster in conjunction with a 360-degrees-steerable four-channel thruster. Therefore, this study details the mechanical design of both thrusters and lists their experimental towing-tank data. Furthermore, two different modelling methods are offered, one theoretically based and one using a multilayer neural network. A model structure comparison, based on a bias-variance trade-off, verifies the adequacy of the theoretical model which got expended with an angle-dependent thrust deduction coefficient. In addition, several multilayer feedforward neural network architectures exemplify their inherent capability to model the complex, nonlinear, flow phenomena inside the thrusters. These identified model structures can additionally improve thrust allocation algorithms and offer better plant models to study more advanced control strategies.

Development of an artificial neural network model to simulate the growth of microalga Chlorella vulgaris incorporating the effect of micronutrients

Artificial neural network (ANN) models can be trained to simulate the dynamic behavior of biological systems. In the present study, an ANN model was developed upon multilayer perceptron neural network architecture with 23-20-1 configuration to predict the cell concentration of microalga Chlorella vulgaris at a given time. Irradiance level, photoperiod, temperature, air flow rate, CO2 percentage of the air stream, initial cell concentration, cultivation time and the nutrient concentrations of the media were considered as the input variables of the model. Resilient backpropagation learning algorithm was used to train the model by means of 484 experimental data belonging to four studies. Bias and accuracy factors of the developed model fall into the range of 0.95–1.11 indicating the model has an excellent prediction ability. Parity plot showed a good agreement between the predicted and experimental values with R2 = 0.98. Relative importance of the inputs was evaluated using Garson’s algorithm. The results of the study indicated that CO2 supply had the highest impact on the growth of C. vulgaris within the selected range of input parameters. Among macronutrients and micronutrients, highest influence was demonstrated by nitrogen and copper respectively.

Malignant melanoma detection using multi layer preceptron with visually imperceptible features and PCA components from MED-NODE dataset

In this paper, a scheme is worked out for classification of images belonging to malignant melanoma and nevus class by multi layer neural network architecture with different trainings and cost functions. Total 1,875 shape, colour and texture features are extracted from 170 images from MED-NODE dataset. With the total 1,875 features an accuracy of 82.05% is achieved. Feature ranking algorithm ReliefF is used for ranking these features. MLP is run with varying number of best ranked features. With 10 best features an accuracy of 83.33%, sensitivity of 86.77% and specificity of 72.78% are achieved with 3 fold cross-validation. Effect of pre-processing the features with principal component analysis is explored and found that the optimal number of principal components is 25, which yields a maximum accuracy of 87.18% which is much higher than the previously reported accuracy level with this dataset.

Malignant melanoma detection using multi layer preceptron with visually imperceptible features and PCA components from MED-NODE dataset

In this paper, a scheme is worked out for classification of images belonging to malignant melanoma and nevus class by multi layer neural network architecture with different trainings and cost functions. Total 1,875 shape, colour and texture features are extracted from 170 images from MED-NODE dataset. With the total 1,875 features an accuracy of 82.05% is achieved. Feature ranking algorithm ReliefF is used for ranking these features. MLP is run with varying number of best ranked features. With 10 best features an accuracy of 83.33%, sensitivity of 86.77% and specificity of 72.78% are achieved with 3 fold cross-validation. Effect of pre-processing the features with principal component analysis is explored and found that the optimal number of principal components is 25, which yields a maximum accuracy of 87.18% which is much higher than the previously reported accuracy level with this dataset.

Nelder–Mead Optimization of Elastic Metamaterials via Machine-Learning-Aided Surrogate Modeling

One of the fundamental challenges of structural optimization of elastic metamaterials (EMMs) with complex geometry lies within the high consumption of computational power associated with finite element analysis (FEA) simulations, which often render the numerical solution an ineffective and costly attempt. Additionally, due to the inherent mesh dependence of the FEA method, minuscule geometry features, which often arise during optimization, demand very fine elements, resulting in enormously high time consumption, especially when repetitive solutions are needed for objective function validations. In this study, we developed a surrogate modeling algorithm to reduce computational time in structural optimization of EMMs. The surrogate model was constructed based on a multilayer feedforward artificial neural network (ANN) architecture, trained with machine learning techniques and optimized through a genetic algorithm (GA) program to improve its accuracy in predicting the location and width of the primary elastic band gap. With the trained and optimized ANN model at the core, a Nelder–Mead (NM) algorithm was established and its performance inspected in comparison to the traditional FEA method. The ANN-NM model shows remarkable accuracy in predicting the band gap width and impressive reduction in the time consumption.

Machine learning load prediction methods for home energy management systems based on human behavior patterns recognition

Load forecasting can enhance the reliability and efficiency of operations in a home energy management system (HEMS). The rise of big data with machine learning in recent years makes it a potential solution. This paper proposes two new energy load forecasting methods, enhancing the traditional sequence to sequence long short-term memory (S2S-LSTM) model. Method 1 integrates S2S-LSTM with human behavior patterns recognition, implemented and compared by 3 types of algorithms: density based spatial clustering of applications with noise (DBSCAN), K-means and Pearson correlation coefficient (PCC). Among them, PCC is proven to be better than the others and suitable for a large number of residential customers. Method 2 further improves Method 1's performance with a modified multi-layer Neural Network architecture, which is constituted by fully-connected, dropout and stable improved softmax layers. It optimizes the process of supervised learning in LSTM and improves the stability and accuracy of the prediction model. The performances of both proposed methods are evaluated on a dataset of 8-week electricity consumptions from 2337 residential customers.

A Two-Stage Approach to Few-Shot Learning for Image Recognition.

This paper proposes a multi-layer neural network structure for few-shot image recognition of novel categories. The proposed multi-layer neural network architecture encodes transferable knowledge extracted from a large annotated dataset of base categories. This architecture is then applied to novel categories containing only a few samples. The transfer of knowledge is carried out at the feature-extraction and the classification levels distributed across the two training stages. In the first-training stage, we introduce the relative feature to capture the structure of the data as well as obtain a low-dimensional discriminative space. Secondly, we account for the variable variance of different categories by using a network to predict the variance of each class. Classification is then performed by computing the Mahalanobis distance to the mean-class representation in contrast to previous approaches that used the Euclidean distance. In the second-training stage, a category-agnostic mapping is learned from the mean-sample representation to its corresponding class-prototype representation. This is because the mean-sample representation may not accurately represent the novel category prototype. Finally, we evaluate the proposed network structure on four standard few-shot image recognition datasets, where our proposed few-shot learning system produces competitive performance compared to previous work. We also extensively studied and analyzed the contribution of each component of our proposed framework.

Robust human pose estimation from distorted wide-angle images through iterative search of transformation parameters

Tracking human motion from video sequences is a well-known video surveillance technique, and many commercially available motion capture devices can now recognize human poses using a depth camera. However, depth camera systems are complicated and have limited optical fields of view. To overcome this problem, it is necessary to develop techniques for recognizing human motion in wide-angle images. In this study, we devised a method for tracking human motion that is robust to wide-angle image distortion. To do so, we developed a new multilayered convolutional neural network architecture for estimating the locations of human body parts in images along with associated transformation parameters that can be applied to a distorted wide-angle image on a frame-by-frame basis. The proposed method was applied to distorted wide-angle images, and its robustness was demonstrated via a quantitative evaluation of human joint prediction and a comparative analysis with a commercially available depth camera-based motion capture system.

Fuzzy modeling network type 2 and principal component analysis for the diagnosis of diabetic retinopathy

Diabetic retinopathy is a disease caused by vascular complications of diabetes mellitus. The more increasing number of people with diabetes mellitus every year, then indirectly the chances of someone’s eye experiencing diabetic retinopathy disorders are also increasing. Fundus photos are one way to detect diabetes mellitus in the retina of the eye. The stages used in the detection process of diabetic retinopathy include the stage of pre-processing fundus images namely grayscale and histogram equalization processes, the stage of reducing image size using Principal Components Analysis (PCA) and the stage of diabetic retinopathy detection on fundus images using fuzzy modeling network type 2. Fuzzy modeling network type 2 is a method using multilayer neural network architecture with backpropagation learning and fuzzy systems for its rules. The results of the validation system test show that the process of detecting diabetic retinopathy using the type 2 fuzzy modeling network algorithm is obtained the accuracy of 80%.

Feature engineering using homogenization theory with multiscale perturbation analysis for supervised model-based learning of physical clogging condition in seepage filters

Aquifer recharge and recovery systems (ARRS), which can broadly be analysed as seepage depth filters, in natural or engineered aquifers are gaining attention worldwide. Engineering predictions of their complex physical clogging behavior, however, continue to be challenging which has hindered the predictive maintenance of these systems for energy and materials savings. To address this problem statement, we leverage the homogenization theory with the multiscale perturbation analysis as the feature engineering step to reduce the complexity of the physical clogging behavior in ARRS. The analytical approach systematically derives a unique homogenized representation which quantifies the clogging condition at the macroscale. A series of physical parameters are identified from the derived homogenized representation to build a pre-processed input layer into our own multi-layered neural network (NN) architecture for predictive analysis. Measured data extracted from the literature is then used to train and verify the NN model. The trained model yields an average error deviation of 20% between the model's predictions and the respective measurements for an optimized set of hyperparameters tested. We then discuss quantitatively how the model can be adhered to predict the timing for a concerned ARRS to reach its breakthrough stage for a range of operational conditions. Finally, we also demonstrate how the homogenized representation can be useful to determine an arbitrary filter's critical reaction rate and diffusion coefficient responsible for its breakthrough stage.