Multilayer Neural Network Architecture Research Articles

RNN-Based Time Series Analysis for Wind Turbine Energy Forecasting

One significant source of renewable energy is wind power, which has the potential to generate sustainable energy. However, wind turbines have many challenges, such as high initial investment costs, the dynamic nature of wind speed, and the challenge of locating wind-efficient energy regions. Wind power predicting is crucial for effective planning of wind power generation, optimization of power generation, grid integration, and security of supply. Therefore, highly accurate forecasts ensure the efficient and sustainable operation of the wind energy sector and contribute to energy security, economic stability, and environmental sustainability. This study proposes a deep learning (DL) approach based on recurrent neural networks (RNNs) for long-term wind power forecasting utilizing climatic data. The input data that forms the basis of this study is obtained directly from a wind turbine system operating under real-world conditions. The proposed model in this study is based on a multilayer back-propagation neural network (RNN) architecture specifically designed to effectively handle complex data sets and time-dependent series. The architecture of the model is built on an RNN consisting of four separate layers, each with 50 hidden neurons, carefully structured to increase its capacity to capture complex features. To improve the robustness of the model and avoid overlearning, each RNN layer is followed by a dropout (regularizing) layer that randomly deactivates 20% of the neurons to enhance the generalization ability of the network. To finalize the prediction capability of the model, a linear function was chosen in the last layer to directly match the actual values. Evaluating the model performance metrics, the proposed architecture achieved a prediction accuracy of 91% R2 on the test dataset. The findings indicate that proposed method based on multilayer RNN can successfully capture the relationships between the sequential data of the wind turbine.

PEB-DDI: A Task-Specific Dual-View Substructural Learning Framework for Drug-Drug Interaction Prediction.

Adverse drug-drug interactions (DDIs) pose potential risks in polypharmacy due to unknown physicochemical incompatibilities between co-administered drugs. Recent studies have utilized multi-layer graph neural network architectures to model hierarchical molecular substructures of drugs, achieving excellent DDI prediction performance. While extant substructural frameworks effectively encode interactions from atom-level features, they overlook valuable chemical bond representations within molecular graphs. More critically, given the multifaceted nature of DDI prediction tasks involving both known and novel drug combinations, previous methods lack tailored strategies to address these distinct scenarios. The resulting lack of adaptability impedes further improvements to model performance. To tackle these challenges, we propose PEB-DDI, a DDI prediction learning framework with enhanced substructure extraction. First, the information of chemical bonds is integrated and synchronously updated with the atomic nodes. Then, different dual-view strategies are selected based on whether novel drugs are present in the prediction task. Particularly, we constructed Molecular fingerprint-Molecular graph view for transduc- tive task, and Bipartite graph-Molecular graph view for inductive task. Rigorous evaluations on benchmark datasets underscore PEB-DDI's superior performance. Notably, on DrugBank, it achieves an outstanding accuracy rate of 98.18% when predicting previously unknown interactions among approved drugs. Even when faced with novel drugs, PEB-DDI consistently exhibits outstanding generalization capabilities with an accuracy rate of 88.06%, attributing to the proper migrating of molecular basic structure learning.

Comparison of multilayer perceptron neural network architecture in photovoltaic plants fault classification

The goal of this study is to assess the application of Multilayer Perceptron Artificial Neural Networks in fault classification within photovoltaic panels, focusing on key characteristics such as the number of neurons and layers, activation functions, training techniques, and the resulting accuracy. The study employs a comparative analysis approach, examining various characteristics and hyperparameters applied to Multilayer Perceptron Artificial Neural Networks for fault classification in photovoltaic panels. The research methodology involves reviewing publications from the past decade to gather data on these characteristics and their impact on fault analysis in photovoltaic generation systems. This study contributes to the originality of the field by providing a comprehensive comparison of various parameters and techniques used in Multilayer Perceptron Artificial Neural Networks for fault classification in photovoltaic panels. The findings offer valuable insights for researchers and practitioners in the renewable energy sector, aiding in the development of more efficient and reliable fault diagnosis systems for photovoltaic generation.

Predicting gene expression divergence between single-copy orthologs in two species.

Predicting gene expression divergence is integral to understanding the emergence of new biological functions and associated traits. Whereas several sophisticated methods have been developed for this task, their applications are either limited to duplicate genes or require expression data from more than two species. Thus, here we present PiXi, the first machine learning framework for predicting gene expression divergence between single-copy orthologs in two species. PiXi models gene expression evolution as an Ornstein-Uhlenbeck process, and overlays this model with multi-layer neural network, random forest, and support vector machine architectures for making predictions. It outputs the predicted class "conserved" or "diverged" for each pair of orthologs, as well as their predicted expression optima in the two species. We show that PiXi has high power and accuracy in predicting gene expression divergence between single-copy orthologs, as well as high accuracy and precision in estimating their expression optima in the two species, across a wide range of evolutionary scenarios, with the globally best performance achieved by a multi-layer neural network. Moreover, application of our best performing PiXi predictor to empirical gene expression data from single-copy orthologs residing at different loci in two species of Drosophila reveals that approximately 23% underwent expression divergence after positional relocation. Further analysis shows that several of these "diverged" genes are involved in the electron transport chain of the mitochondrial membrane, suggesting that new chromatin environments may impact energy production in Drosophila. Thus, by providing a toolkit for predicting gene expression divergence between single-copy orthologs in two species, PiXi can shed light on the origins of novel phenotypes across diverse biological processes and study systems.

PO-04-212 BRUGADA EKG CLASSIFICATION WITH SELF-SUPERVISED VICREG PERTAINING: A NOVEL ARTIFICIAL INTELLIGENCE ARCHITECTURE FOR RARE ARRHYTHMIA

Brugada Syndrome (BrS) is a rare inherited arrhythmia syndrome with an estimated prevalence of 2-5 per 10,000. Despite its low prevalence, it is responsible for an estimated 4% to 12% of all sudden deaths and in patients with structurally normal hearts. Because of the resulting limited number of available BrS EKG examples, Convoluted Neural Networks (CNNs), which require a large labeled dataset for training, are limited in their ability to automate the detection of BrS. To develop an Artificial Intelligence (AI) model for classifying BrS in the surface EKG by applying a novel self-supervised pretraining architecture on unrelated, unlabeled EKGs, compensating for the limited number of BrS examples. We pretrained a convolutional neural network on randomly selected 3,500,000 EKGs from adult NYU patients utilizing VICReg (Variance-Invariance-Covariance Regularization), a novel regularization architecture for self-supervised training allowing the model to learn embedding (vectors) for each EKG that are informative and unique (Figure 1). We then fine tuned the original model on a set containing 200 adult NYU patients with a MUSE (GE, USA) diagnosis of BrS not included in the pretraining phase, and evaluated the fine-tuned model for binary classification of Brugada. We used a modification of a multilayered convolutional neural network architecture similar to ConvNext, with a Softmax activation function for classification. Of the 200 EKGs with a MUSE label of BrS, 114 (57%) were confirmed as BrS by an expert Electrophysiologist and blended with a cohort of randomly selected 200 EKGs with non – BrS label. The combined cohort was divided into 80% training (with 20% validation) and 20% test sub-sets. The CNN classified BrS with an AUC of 0.953 (sensitivity 0.87, specificity 0.87), outperforming all previously described supervised learning models trained on larger datasets. We developed a novel deep learning model using VICReg architecture for self-supervised pretraining. We demonstrate the classification of BrS EKGs with high performance achieved with a remarkably small labeled dataset. Supervised pretraining with VICReg may be a useful architecture for improving model classification performance on rare conditions with limited availability of labeled data.

Scalable serial hardware architecture of multilayer perceptron neural network for automatic wheezing detection

This paper proposes a serial hardware architecture of a multilayer perceptron (MLP) neural network for real-time wheezing detection in respiratory sounds. As an established classification tool, the MLP has proven its ability to identify complex patterns within respiratory sounds. The proposed fully serial architecture uses a single calculation unit, independently of the number of neurons in the MLP network. It is also a fully scalable architecture that permits to implement MLP networks, of any size, easily and efficiently without modifying the design or wiring. The proposed serial architecture has been implemented on a low-cost and power-efficient field programmable gate array (FPGA) chip using a high-level programming tool. The respiratory sounds classification rates are evaluated in terms of sensitivity, specificity, performance, and accuracy. The proposed serial architecture reaches the same classification performances as the parallel one, but it presents the main advantage of using much fewer hardware resources.

Human activity recognition with self-attention

In this paper, a self-attention based neural network architecture to address human activity recognition is proposed. The dataset used was collected using smartphone. The contribution of this paper is using a multi-layer multi-head self-attention neural network architecture for human activity recognition and compared to two strong baseline architectures, which are convolutional neural network (CNN) and long-short term network (LSTM). The dropout rate, positional encoding and scaling factor are also been investigated to find the best model. The results show that proposed model achieves a test accuracy of 91.75%, which is a comparable result when compared to both the baseline models.

Tiny object detection model based on competitive multi-layer neural network (TOD-CMLNN)

Tiny Object Detection (TOD) is a fundamental and difficult task in computer vision. Current state-of-the-art detectors like RCNN, Fast RCNN, Faster RCNN, SSD, and YOLO can't find small objects using single-stage or multi-stage methods. With the exponential growth of deep learning, several researchers have drawn attention to advances in tiny object detection approaches. This study proposes a TOD-CMLNN (Tiny Object Detection Competitive Multi-Layer Neural Network) architecture with three sub components first competitive multi-layer network, second TOD auxiliary and third multi-level continue features aggregation for accurately detecting small objects. Competitive learning for object detection is the basis of the proposed architecture. Comparison study with existing RCNN, Fast RCNN, Faster RCNN, SSD and YOLO shows significant improvement in the results. TOD-CMLNN receives 72.46 % accuracy in terms of mAP which is impressive as compared to state-of-the-art detectors.

High-dimensional multi-objective optimization algorithm for combustion chamber of aero-engine based on artificial neural network-multi-objective particle swarm optimization

This paper considers optimizing the performance of high-temperature combustion chamber of an aero-engine based on a concentric and hierarchical model. First, sample data for the design variables are obtained based on Latin hypercube sampling method, and a one-dimensional program is used to obtain the true values of combustion efficiency and total pressure loss corresponding to each group of variables. The obtained data are then pre-processed to establish a dataset. Second, a multi-layer artificial neural network (ANN) architecture is designed and a surrogate model of the combustion-related performance of the combustor is established using a data-driven method. The results of global sensitivity analysis based on variance show that ratio of fuel flow to air flow (fuel–air ratio) and the total inlet pressure are the most important factors influencing the two objective functions. Finally, we optimize the multi-objective combustion-related performance of the surrogate model by applying the particle swarm optimization algorithm to it. The results of experiments show that the ANN-based model could accurately predict the efficiency of combustion and total pressure loss of the chamber, yielding root mean-squared errors of 0.0107 and 0.3032%, respectively. It also had better generalization ability than the cubic polynomial surrogate model. Compared with the cubic polynomial model, it generated an optimal Pareto solution set as prediction that had higher values in both objective functions. The proposed model might require better data that can be obtained using intelligent sampling methods so that deeper neural networks can be designed to reduce error and improve its optimization design.

Artificial intelligence applied in neoantigen identification facilitates personalized cancer immunotherapy.

The field of cancer neoantigen investigation has developed swiftly in the past decade. Predicting novel and true neoantigens derived from large multi-omics data became difficult but critical challenges. The rise of Artificial Intelligence (AI) or Machine Learning (ML) in biomedicine application has brought benefits to strengthen the current computational pipeline for neoantigen prediction. ML algorithms offer powerful tools to recognize the multidimensional nature of the omics data and therefore extract the key neoantigen features enabling a successful discovery of new neoantigens. The present review aims to outline the significant technology progress of machine learning approaches, especially the newly deep learning tools and pipelines, that were recently applied in neoantigen prediction. In this review article, we summarize the current state-of-the-art tools developed to predict neoantigens. The standard workflow includes calling genetic variants in paired tumor and blood samples, and rating the binding affinity between mutated peptide, MHC (I and II) and T cell receptor (TCR), followed by characterizing the immunogenicity of tumor epitopes. More specifically, we highlight the outstanding feature extraction tools and multi-layer neural network architectures in typical ML models. It is noted that more integrated neoantigen-predicting pipelines are constructed with hybrid or combined ML algorithms instead of conventional machine learning models. In addition, the trends and challenges in further optimizing and integrating the existing pipelines are discussed.

Data Augmentation of Micrographs and Prediction of Impact Toughness for Cast Austenitic Steel by Machine Learning

The Material Genome Initiative has been driven by high-throughput calculations, experiments, characterizations, and machine learning, which has accelerated the efficiency of the discovery of novel materials. However, the precise quantification of the material microstructure features and the construction of microstructure–property models are still challenging in optimizing the performance of materials. In this study, we proposed a new model based on machine learning to enhance the power of the data augmentation of the micrographs and construct a microstructure–property linkage for cast austenitic steels. The developed model consists of two modules: the data augmentation module and microstructure–property linkage module. The data augmentation module used a multi-layer convolution neural network architecture with diverse size filter to extract the microstructure features from irregular micrographs and generate new augmented microstructure images. The microstructure–property linkage module used a modified VGG model to establish the relationship between the microstructure and material property. Taking cast austenitic stainless steels after solution treating in different temperatures as an example, the results showed that the prediction accuracy of the developed machine learning model had been improved. The coefficient R2 of the model was 0.965, and the medians were only ±2 J different with the measured impact toughness.

An Optimized Multi-layer Spiking Neural Network implementation in FPGA Without Multipliers

This paper presents an expansion and evaluation of the hardware architecture for the Optimized Deep Event-driven Spiking Neural Network Architecture (ODESA). ODESA is a state-of-the-art, event-driven multi-layer Spiking Neural Network (SNN) architecture that offers an end-to-end, online, and local supervised training method. In previous work, ODESA was successfully implemented on Field-Programmable Gate Array (FPGA) hardware, showcasing its effectiveness in resource-constrained hardware environments. Building upon the previous implementation, this research focuses on optimizing the ODESA network hardware by introducing a novel approach. Specifically, we propose substituting the dot product multipliers in the Neurons with a low-cost shift-register design. This optimization strategy significantly reduces the hardware resources required for implementing a neuron, thereby enabling more complex SNNs to be accommodated within a single FPGA. Additionally, this optimization results in a reduction in power consumption, further enhancing the practicality and efficiency of the hardware implementation. To evaluate the effectiveness of the proposed optimization, extensive experiments and measurements were conducted. The results demonstrate the successful reduction in hardware resource utilization while maintaining the network's functionality and performance. Moreover, the power consumption reduction contributes to the overall energy efficiency of the hardware implementation.

Multilayer Neural Network Architecture for Predicting Stress/Anxiety in Modern Life: A Novel Approach using Facial Action Units

Multilayer Neural Network Architecture for Predicting Stress/Anxiety in Modern Life: A Novel Approach using Facial Action Units

Benchmarking a multi-layer approach and neural network architectures for defect detection in PBF-LB/M

The substitution of expensive non-destructive material testing by data-based process monitoring is intensively explored in quality assurance for additive manufactured components. Machine learning show promising results for defect detection but require conceptual adaption to layer wise manufacturing and line scanning patterns in laser powder bed fusion. A multi-layer approach to co-register µ-computer tomography measurements with process monitoring data is developed and a workflow for automatic data set generation is implemented. The objective of this research is to benchmark the volumetric multi-layer approach and specifically selected deep learning methods for defect detection. The volumetric approach shows superior results compared to single slice monitoring. All investigated structured neural network topologies deliver similar performance. • Machine learning identifies correlations between µ-CT porosity scans and melt pool monitoring • 3D-multi-layer-based machine learning outperforms single layer approach in defect detection • Saturation of detection performance is observed beyond optimal number of layers • Results hold independently for all investigated neural network topologies

Complementary Memtransistor-Based Multilayer Neural Networks for Online Supervised Learning Through (Anti-)Spike-Timing-Dependent Plasticity.

We propose a complete hardware-based architecture of multilayer neural networks (MNNs), including electronic synapses, neurons, and periphery circuitry to implement supervised learning (SL) algorithm of extended remote supervised method (ReSuMe). In this system, complementary (a pair of n- and p-type) memtransistors (C-MTs) are used as an electrical synapse. By applying the learning rule of spike-timing-dependent plasticity (STDP) to the memtransistor connecting presynaptic neuron to the output one whereas the contrary anti-STDP rule to the other memtransistor connecting presynaptic neuron to the teacher one, extended ReSuMe with multiple layers is realized without the usage of those complicated supervising modules in previous approaches. In this way, both the C-MT-based chip area and power consumption of the learning circuit for weight updating operation are drastically decreased comparing with the conventional single memtransistor (S-MT)-based designs. Two typical benchmarks, the linearly nonseparable benchmark XOR problem and Mixed National Institute of Standards and Technology database (MNIST) recognition have been successfully tackled using the proposed MNN system while impact of the nonideal factors of realistic devices has been evaluated.

An unfeasibility view of neural network learning

We define the notion of a continuously differentiable perfect learning algorithm for multilayer neural network architectures and show that such algorithms do not exist provided that the length of the data set exceeds the number of involved parameters and the activation functions are logistic, tanh or sin.

Double Moving Average Control Chart for Autocorrelated Data

The assumption of normality and independence is necessary for statistical inference of control charts. Misleading results are obtained if the traditional control chart technique is applied on the auto-correlated data. When data is correlated, a time series model is employed to produce an optimum output. The objective is to create a new control chart methodology that takes the autocorrelation of observations into account. Charts of Moving Average, Exponentially Weighted and Cumulative Sum better perform in the existence of autocorrelation data for small and moderate changes. The proposed methodology is highly skilled and receptive to minor, moderate and major changes in the process. Propsed DMA chart increases efficiency of average run length (ARL) chart for moving average (MA) to detect the small to medium magnitude shifts in the mean. The simulation also demonstrates that the DMA chart with spans of w=10 and 15 generally performs well in terms of average run length (ARL) as compared to clasical MA. This research may be extended to a multivariate autocorrelated statistical process control, but it can also be used to recognise and categorise seven categories of traditional control chart patterns, such as Downward, Upward Shift, Normal Trend, Cyclic, Systematic patterns, Increasing and Decresing Trend. In order to identify and categorize a set of subclasses of abnormal patterns, this model (multivariate autocorrelated statistical process control chart) should employ a multilayer feed forward Artificial Neural Network (ANN) architecture controlled by a back-propagation learning rule.

Deep reinforcement learning for solving the single container loading problem

In this article, a new deep Reinforcement Learning (RL) model is proposed to solve the Single Container Loading Problem (SCLP) as well as the SCLP with full support. For that purpose, a multilayer neural network architecture is used. Computational experiments, conducted on benchmark instances with homogeneous boxes, revealed that the proposed model yields fairly good results compared to those found with state-of-the-art heuristics. Experiments have also shown good generalization capability of the proposed deep RL model to deal with both homogeneous and heterogeneous classes of instances. Nevertheless, this learning-based optimization approach still has an optimality gap compared to the well-designed heuristics of the operations research literature. Finally, the benefit of training the model under different levels of variability has been analysed. Results revealed that, for better performance, and if a company does not face high demand volatility, it is recommended to train the model under a low level of variability.

Image-based malicious Internet content filtering method for child protection

Children and teenagers are among Internet users and they encounter harmful data in the global network. Young users often become the potential victims of pornographic images. Avoiding pornographic images harmful to the child audience is an important research task in the field of detection, computer vision and multimedia. Malicious content can be prevented using various methods. Current paper presents a ChildNet model that filters harmful image content. The pixels of the digital images are used as a data source for recognition of nudity in the images. For each class, a multi-layer deep neural network architecture with five convolution blocks is developed to study the color patterns of undesirable image pixels. The developed neural network consists of 21 layers; the size of the filters is specified as (3 × 3). The filter's size is reduced to increase the accuracy of pixel recognition. The efficiency of the proposed method is tested on real datasets for evaluation purposes and the superior results are obtained from the proposed method in comparison with classical CNN.

Multilabel Text Classification Using Multilayer DGAT

Text classification is one of the most fundamental tasks in text data analysis. Generally, textual data are extensive in scale and complex in the inherent relationship, leading to low accuracy in traditional classification models. This paper proposes a multi-label classification model based on multi-layer neural network architecture. We first construct a dual-attention mechanism graph neural network (named DGAT for short) to fuse the typological and informational features of the target node and the connected nodes. Secondly, we also build a multi-layer network architecture with multiple DGATs to expand the range of neighborhood nodes participating in the feature fusion to meet the needs of classifying different datasets. To ensure the learning ability of the model, we also use the residual network to solve the problem of error rise and gradient descent caused by multi-layer network architecture. Finally, we conducted a large number of experiments on five benchmark datasets. The results show that the accuracy of the proposed model is significantly better than that of the traditional models, and that there is a noticeable improvement when compared with other deep learning methods.