Two-layer Feed-forward Network Research Articles

Turbofan engine health status prediction with neural network pattern recognition and automated feature engineering

PurposeThis study aims to present the concept of aircraft turbofan engine health status prediction with artificial neural network (ANN) pattern recognition but augmented with automated features engineering (AFE).Design/methodology/approachThe main concept of engine health status prediction was based on three case studies and a validation process. The first two were performed on the engine health status parameters, namely, performance margin and specific fuel consumption margin. The third one was generated and created for the engine performance and safety data, specifically created for the final test. The final validation of the neural network pattern recognition was the validation of the proposed neural network architecture in comparison to the machine learning classification algorithms. All studies were conducted for ANN, which was a two-layer feedforward network architecture with pattern recognition. All case studies and tests were performed for both simple pattern recognition network and network augmented with automated feature engineering (AFE).FindingsThe greatest achievement of this elaboration is the presentation of how on the basis of the real-life engine operational data, the entire process of engine status prediction might be conducted with the application of the neural network pattern recognition process augmented with AFE.Practical implicationsThis research could be implemented into the engine maintenance strategy and planning. Engine health status prediction based on ANN augmented with AFE is an extremely strong tool in aircraft accident and incident prevention.Originality/valueAlthough turbofan engine health status prediction with ANN is not a novel approach, what is absolutely worth emphasizing is the fact that contrary to other publications this research was based on genuine, real engine performance operational data as well as AFE methodology, which makes the entire research very reliable. This is also the reason the prediction results reflect the effect of the real engine wear and deterioration process.

Classical vs. neural network-based PCA approaches for lossy image compression: Similarities and differences

The paper describes three lossy data compression techniques based on the principal component analysis (PCA), which are compared using the image compression task. The presented approach uses both classical PCA method based on the eigen-decomposition of the image data covariance matrix and two different neural network structures. The first neural structure is a two-layer feed-forward network with supervised learning acting as the so-called autoencoder, and the second one is a single-layered network with an unsupervised learning method commonly known as the generalized Hebbian algorithm. For each compression method mentioned above, the influence of the number of image segments (frames) and the number of eigenvalues/eigenvectors on the compression ratio and the image quality are examined using three different gray-scale test images. The paper also addresses a vital implementation issue regarding selecting appropriate data types to represent the compressed data. The paper’s main conclusion is that the classical PCA method outperforms its neural counterparts, both in terms of the decompressed image quality and the time required to perform the compression procedure. The positive aspect of using neural networks as a tool for PCA-based lossy data compression is that they do not require calculating the correlation matrix explicitly and thus can be used in online data acquisition schemes.

Machine learning-based prediction of abdominal aortic aneurysms for individualized patient care

ObjectiveThe United States Preventative Services Task Force guidelines for screening for abdominal aortic aneurysms (AAA) are broad and exclude many at risk groups. We analyzed a large AAA screening database to examine the utility of a novel machine learning (ML) model for predicting individual risk of AAA. MethodsWe created a ML model to predict the presence of AAAs (>3 cm) from the database of a national nonprofit screening organization (AAAneurysm Outreach). Participants self-reported demographics and comorbidities. The model is a two-layered feed-forward shallow network. The ML model then generated AAA probability based on patient characteristics. We evaluated graphs to determine significant factors, and then compared those graphs with a traditional logistic regression model. ResultsWe analyzed a cohort of 10,033 patients with an AAA prevalence of 2.74%. Consistent with logistic regression analysis, the ML model identified the following predictors of AAA: Caucasian race, male gender, advancing age, and recent or past smoker with recent smoker having a more profound affect (P < .05). Interestingly, the ML model showed body mass index (BMI) was associated with likelihood of AAAs, especially for younger females. The ML model also identified a higher than predicted risk of AAA in several groups, including female nonsmokers with cardiac disease, female diabetics, those with a family history of AAA, and those with hypertension or hyperlipidemia at older ages. An elevated BMI conveyed a higher than expected risk in male smokers and all females. The ML model also identified a complex relationship of both diabetes mellitus and hyperlipidemia with gender. Family history of AAA was a more important risk factor in the ML model for both men and women too. ConclusionsWe successfully developed an ML model based on an AAA screening database that unveils a complex relationship between AAA prevalence and many risk factors, including BMI. The model also highlights the need to expand AAA screening efforts in women. Using ML models in the clinical setting has the potential to deliver precise, individualized screening recommendations.

Hyperspectral imaging-based classification of post-consumer thermoplastics for plastics recycling using artificial neural network

The increasing global output of plastic waste has created a critical need for cost-effective and highly accurate methods of plastic recycling. One of the major challenges in recycling plastics is maintaining the purity of plastic streams, which is essential for secondary plastics to compete with virgin plastics in the market. In this context, this paper presents a novel approach for classifying different polymers in a complex plastic waste stream using a hyperspectral imaging system in the near-infrared (NIR) range (900–1700 nm). The study focuses on the applications of entropy and contrast stretching for image segmentation to identify distinct objects in a given hyperspectral image. The classification process is carried out using a two-layer feedforward network with sigmoid hidden neurons and SoftMax output neurons. The performance of the proposed method was evaluated using a dataset comprising hyperspectral images of unknown plastic waste stream samples, and it was able to achieve an accuracy of about 89.5%. The results of this study demonstrate the potential of using hyperspectral imaging in the NIR range as a cost-effective and highly accurate method for classifying different polymers in plastic waste streams. This approach can be further developed for use in real-world recycling facilities, contributing to the advancement of the plastic waste management field.

Application of neural networks and neuro-fuzzy models in construction scheduling

Construction scheduling is a complex process that involves a large number of variables, making it difficult to develop accurate and efficient schedules. Traditional scheduling techniques rely on manual analysis and intuition, which are prone to errors and often fail to account for all the variables involved. This results in project delays, cost overruns, and poor project performance. Artificial intelligence models have shown promise in improving construction scheduling accuracy by incorporating historical data, site-specific conditions, and other variables that traditional scheduling methods may not consider. In this research study, application of soft-computing techniques to evaluate construction schedule and control of project activities in order to achieve optimal performance in execution of building projects were carried out. Artificial neural network and neuro-fuzzy models were developed using data extracted from a residential two-storey reinforced concrete framed-structure construction schedule and project execution documents. The evaluation of project performance indicators in earned value analysis from 0 to 100% progress at 5% increment with a total of seventeen tasks were carried out using Microsoft Project software and data obtained from the computation were utilized for model development. Using input–output and curve-fitting (nftool) function in MATLAB, a 6-10-1 two-layer feed-forward network with tansig activation-function (AF) for the hidden neurons and linear AF output neurons was generated with Levenberg–Marquardt (Trainlm) training algorithm. Similarly, with the aid of ANFIS toolbox in MATLAB software, the training, testing and validation of the ANFIS model were carried out using hybrid optimization learning algorithm at 100 epochs and the Gaussian-membership-function (gaussmf). Loss-function parameters namely MAE, RMSE and R-values were taken as the performance evaluation criteria of the developed models. The generated statistical results indicates no significant difference between model-results and experimental values with MAE, RMSE, R2 of 1.9815, 2.256 and 99.9% respectively for ANFIS-model and MAE, RMSE, R2 of 2.146, 2.4095 and 99.998% respectively for the ANN-model. The model performance indicated that the ANFIS-model outclassed the ANN-model with their results satisfactory to deal with complex relationships between the model variables to produce accurate target response. The findings from this research study will improve the accuracy of construction scheduling, resulting in improved project performance and reduced costs.

ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training.

We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification. It is a simple residual network that alternates (i) a linear layer in which image patches interact, independently and identically across channels, and (ii) a two-layer feed-forward network in which channels interact independently per patch. When trained with a modern training strategy using heavy data-augmentation and optionally distillation, it attains surprisingly good accuracy/complexity trade-offs on ImageNet. We also train ResMLP models in a self-supervised setup, to further remove priors from employing a labelled dataset. Finally, by adapting our model to machine translation we achieve surprisingly good results. We share pre-trained models and our code based on the Timm library.

Inter-areal transmission of multiple neural signals through frequency-division-multiplexing communication.

Inter-areal information transmission in the brain cortex relates to cognitive functions. Researches used to pay attention to activity pattern transmission, signals gating, or routing in neuronal networks. However, the underlying mechanism of simultaneous transmission of multiple neural signals in the same channel across networks remains unclear. In this work, we construct a two-layer feedforward neuronal network (sender-receiver) with each layer's intrinsic rhythms consisting of slow- (low-frequency) and fast- gamma rhythms (high-frequency), investigating how to realize simultaneous transmission of multiple signals in neuronal systems. With the aid of resonance and frequency analysis, it is shown that low- and high-frequency signals can be transmitted simultaneously in such a feedforward network through frequency division multiplexing (FDM) communication. The transmission performance is related to the local resonance, connectivity, as well as background noise. Moreover, low- and high-frequency signals can also be gated or selected with appropriate adjustments of recurrent connection strength and delay, and background noise. Our model might provide a novel insight into the underlying mechanism of complex signals communication between different cortex areas.

ANN-based performance prediction of electrical discharge machining of Ti-13Nb-13Zr alloys

PurposeThe purpose of this paper is to predict the machining performance of electrical discharge machining of Ti-13Nb-13Zr (TNZ) alloy, a promising biomedical alloy, using artificial neural networks (ANN) models.Design/methodology/approachIn the research, three major performance characteristics, i.e. the material removal rate (MRR), tool wear rate (TWR) and surface roughness (SR), were chosen for the study. The input parameters for machining were the voltage, current, pulse-on time and pulse-off time. For the ANN model, a two-layer feedforward network with sigmoid hidden neurons and linear output neurons were chosen. Levenberg–Marquardt backpropagation algorithm was used to train the neural networks.FindingsThe optimal ANN structure comprises four neurons in input layer, ten neurons in hidden layer and one neuron in the output layer (4–10-1). In predicting MRR, the 60–20-20 data split provides the lowest MSE (0.0021179) and highest R-value for training (0.99976). On the contrary, the 70–15-15 data split results in the best performance in predicting both TWR and SR. The model achieves the lowest MSE and highest R-value for training in predicting TWR as 1.17E-06 and 0.84488, respectively. Increasing the number of hidden neurons of the network further deteriorates the performance. In predicting SR, the authors find the best MSE and R-value as 0.86748 and 0.94024, respectively.Originality/valueThis is a novel approach in performance prediction of electrical discharge machining in terms of new workpiece material (TNZ alloys).

Trans-media resistance investigation of hybrid aerial underwater vehicle base on hydrodynamic experiments and machine learning

In this paper, the trans-media hydrodynamics of a multi-rotor hybrid aerial underwater vehicle (HAUV) is studied base on experiments and machine learning. First, experiments are conducted to obtain the hydrodynamic forces during water exit/entry. Then, based on the experimental data and phenomena, it is evident that the multi-rotor HAUV's total resistance, particularly the impact of the arms, should not be ignored. The overall resistance is primarily affected by the depth and velocity at constant velocity. The total resistance increases as the velocity increases. The maximum total resistance in water exit/entry can be 46%/64% of gravity. Finally, a two-layer feed-forward network with six sigmoid hidden neurons and linear output neurons is used for machine learning training. The training results show that using experimental data and a machine learning method, the trans-media resistance of HAUV can be accurately predicted within a certain range.

Standardizing continuous data classifications in a virtual T-maze using two-layer feedforward networks

There continues to be difficulties when it comes to replication of studies in the field of Psychology. In part, this may be caused by insufficiently standardized analysis methods that may be subject to state dependent variations in performance. In this work, we show how to easily adapt the two-layer feedforward neural network architecture provided by Huang1 to a behavioral classification problem as well as a physiological classification problem which would not be solvable in a standardized way using classical regression or “simple rule” approaches. In addition, we provide an example for a new research paradigm along with this standardized analysis method. This paradigm as well as the analysis method can be adjusted to any necessary modification or applied to other paradigms or research questions. Hence, we wanted to show that two-layer feedforward neural networks can be used to increase standardization as well as replicability and illustrate this with examples based on a virtual T-maze paradigm2–5 including free virtual movement via joystick and advanced physiological data signal processing.

Prediction of phase equilibrium between soybean biodiesel, alcohols and supercritical CO2 using artificial neural networks

In this paper, a non-conventional approach of modeling phase equilibrium was applied. An alternative tool, the artificial neural network (ANN) technique has been used for estimating transition pressures for two ternary systems at high pressures (CO2 + Biodiesel + Methanol) and (CO2 + Biodiesel + Ethanol). Temperatures, molar ratios, and compositions were utilized as input variables at ranges of 303.15 to 343.15 K, 4.30 to 15.62 MPa and 0.4 to 0.99, respectively. The databases taken from the literature were split into training, validating, and testing data. Multiple ANN structures were applied and the model with the lowest mean square error (MSE) was selected. The selected ANN model for the methanol system was a two-layered Feed-Forward Network and achieved a determination coefficient (R2) of 0.99878 and MSE of 0.01612. While the ethanol system was best described by an Elman Network presenting an R2 and MSE of 0.99359 and 0.06078, respectively. Results were then compared with Peng-Robinson models using van der Waals quadratic and Wong-Sandler mixing rules. The results showed a better agreement with experimental data than the thermodynamic models.

Specifications for Modelling of the Phenomenon of Compression of Closed-Cell Aluminium Foams with Neural Networks.

The article presents a novel application of the most up-to-date computational approach, i.e., artificial intelligence, to the problem of the compression of closed-cell aluminium. The objective of the research was to investigate whether the phenomenon can be described by neural networks and to determine the details of the network architecture so that the assumed criteria of accuracy, ability to prognose and repeatability would be complied. The methodology consisted of the following stages: experimental compression of foam specimens, choice of machine learning parameters, implementation of an algorithm for building different structures of artificial neural networks (ANNs), a two-step verification of the quality of built models and finally the choice of the most appropriate ones. The studied ANNs were two-layer feedforward networks with varying neuron numbers in the hidden layer. The following measures of evaluation were assumed: mean square error (MSE), sum of absolute errors (SAE) and mean absolute relative error (MARE). Obtained results show that networks trained with the assumed learning parameters which had 4 to 11 neurons in the hidden layer were appropriate for modelling and prognosing the compression of closed-cell aluminium in the assumed domains; however, they fulfilled accuracy and repeatability conditions differently. The network with six neurons in the hidden layer provided the best accuracy of prognosis at but little robustness. On the other hand, the structure with a complexity of 11 neurons gave a similar high-quality of prognosis at but with a much better robustness indication (80%). The results also allowed the determination of the minimum threshold of the accuracy of prognosis: . In conclusion, the research shows that the phenomenon of the compression of aluminium foam is able to be described by neural networks within the frames of made assumptions and allowed for the determination of detailed specifications of structure and learning parameters for building models with good-quality accuracy and robustness.

A novel out of step relaying algorithm based on wavelet transform and a deep learning machine model

Out-of-step protection of one or a group of synchronous generators is unreliable in a power system which has significant renewable power penetration. In this work, an innovative out-of-step protection algorithm using wavelet transform and deep learning is presented to protect synchronous generators and transmission lines. The specific patterns are generated from both stable and unstable power swing, and three-phase fault using the wavelet transform technique. Data containing 27,008 continuous samples of 48 different features is used to train a two-layer feed-forward network. The proposed algorithm gives an automatic, setting free and highly accurate classification for the three-phase fault, stable power swing, and unstable power swing through pattern recognition within a half cycle. The proposed algorithm uses the Kundur 2-area system and a 29-bus electric network for testing under different swing center locations and levels of renewable power penetration. Hardware-in-the-loop (HIL) tests show the hardware compatibility of the developed out-of-step algorithm. The proposed algorithm is also compared with recently reported algorithms. The comparison and test results on different large-scale systems show that the proposed algorithm is simple, fast, accurate, and HIL tested, and not affected by changes in power system parameters.

Using ANN to Predict the Impact of Communication Factors on the Rework Cost in Construction Projects

The construction sector has a large impact on the environment and available resources. Natural resources and energy consumption occurs not only during the operation of the facility, but also during its construction. In addition, this situation often occurs when work already completed requires rework. In such cases, not only the reuse of resources and energy occurs but also generation of waste. Many studies support the relationship between communication and project efficiency, which is expressed in the cost of rework. At present there is no available tool to quantify the evaluation of this relationship. This study aims to fill this knowledge gap. The article purpose was to create ANNs (artificial neural networks) for assessing and predicting the impact of communication factors on rework costs in construction projects. During the data collection phase, 12 factors that influence communication were identified and assessed. The level of rework costs in 18 construction projects was also calculated. We used ANN, which is a two-layer feedforward network with a sigmoid transfer function in the hidden layer and a linear transfer function in the output layer. The network input layer consists of 12 neurons while the hidden layer consists of 10 neurons and one output neuron. The optimal results of the mean square error and correlation were shown by the Levenberg–Marquardt algorithm. The proposed model can be used by project management as the integration decision support tool aimed at decreasing the number of reworks and reducing energy and resource consumption in construction projects.

A neural network approach to detect winding faults in electrical machine

Abstract In this paper, Neural Network (NN) approach is developed and utilised to detect winding faults in an electrical machine using the samples data of electrical machine in both the healthy and different fault conditions (i.e. shorted-turn fault, phase-to-ground fault and coil-to-coil fault). This is done by interfacing a data acquisition device connected to the machine with a computer in the laboratory. Thereafter, a two-layer feed-forward network with Levenberg–Marquardt back-propagation algorithm is created with the collected input dataset. The NN model developed was tested with both the healthy and the four different fault conditions of the electrical machine. The results from the NN approach was also compared with other results obtained by determining the fault index (FI) of an electrical machine using signal processing approach. The results show that the NN approach can identify each of the electrical machine condition with high accuracy. The percentage accuracy for healthy (normal), shorted-turn, phase-to-ground and coil-to-coil fault conditions are 99, 99.6, 100 and 100% respectively.

Towards the representational power of restricted Boltzmann machines

The restricted Boltzmann machine (RBM), which is a graphical model for binary random variables, has been proven to be a powerful tool in machine learning. However, theoretical foundations for understanding the approximation ability of RBMs are lacking. In this paper, we study the representational power of RBMs, with the focus lying on the sufficient number of hidden units of RBMs required to compute some classes of distributions of interest, with a fixed number of inputs. First, it is constructively shown how RBMs can approximate any distribution that depends on the scalar projection of the inputs onto a given vector up to arbitrary accuracy. Then, for any given distribution, we explore how it can be represented as the form that depends on the scalar projection of the inputs onto some vectors, and then study the properties of these vectors, from which a new proof for the universal approximation theorem of RBMs is deduced. Finally, we investigate the representational efficiency of RBMs by providing a description of all the distributions that can be efficiently computed by RBMs. More specifically, it is shown that a distribution can be computed by a polynomial-size RBM with polynomially bounded parameters, if and only if its mass can be computed by a two-layer feedforward network with threshold/ReLU activation functions, whose size and parameters are polynomially bounded.

What do across-subject analyses really tell us about neural coding?

A key challenge in human neuroscience is to gain information about patterns of neural activity using indirect measures. Multivariate pattern analysis methods testing for generalization of information across subjects have been used to support inferences regarding neural coding. One critical assumption of an important class of such methods is that anatomical normalization is suited to align spatially-structured neural patterns across individual brains. We asked whether anatomical normalization is suited for this purpose. If not, what sources of information are such across-subject cross-validated analyses likely to reveal? To investigate these questions, we implemented two-layered feedforward randomly-connected networks. A key feature of these simulations was a gain-field with a spatial structure shared across networks. To investigate whether total-signal imbalances across conditions—e.g. differences in overall activity—affect the observed pattern of results, we manipulated the energy-profile of images conforming to a pre-specified correlation structure. To investigate whether the level of granularity of the data also influences results, we manipulated the density of connections between network layers. Simulations showed that anatomical normalization is unsuited to align neural representations. Pattern similarity-relationships were explained by the observed total-signal imbalances across conditions. Further, we observed that deceptively complex representational structures emerge from arbitrary analysis choices, such as whether the data are mean-subtracted during preprocessing. These simulations also led to testable predictions regarding the distribution of low-level features in images used in recent fMRI studies that relied on leave-one-subject-out pattern analyses. Image analyses broadly confirmed these predictions. Finally, hyperalignment emerged as a principled alternative to test across-subject generalization of spatially-structured information. We illustrate cases in which hyperalignment proved successful, as well as cases in which it only partially recovered the latent correlation structure in the pattern of responses. Our results highlight the need for robust, high-resolution measurements from individual subjects. We also offer a way forward for across-subject analyses. We suggest ways to inform hyperalignment results with estimates of the strength of the signal associated with each condition. Such information can usefully constrain ensuing inferences regarding latent representational structures as well as population tuning dimensions.

ANN Architecture Specifications for Modelling of Open-Cell Aluminum under Compression

The knowledge on strength properties of porous metals in compression is essential in tailored application design, as well as in elaboration of general material models. In this article, the authors propose specification details of the ANN architecture for adequate modelling of the phenomenon of compressive behaviour of open-cell aluminum. In the presented research, an algorithm was used to build different structures of artificial neural networks (ANNs), which approximated stress-strain relations of an aluminum sponge subjected to compression. Next, the quality of the built approximations was appraised. The mean absolute relative error (MARE), coefficient of determination between outputs and targets R2, root mean square error (RMSE), and mean square error (MSE) were assumed as criterial measures for the assessment of the fitting quality. The studied neural networks (NNs) were two-layer feedforward networks with different numbers of neurons in the hidden layer. A set of experimental stress-strain data from quasistatic uniaxial compression tests of open-cell aluminum of various apparent densities was used as data for training of neural networks. Analysis was performed in two modes: in the first one, all samples were taken for training, and in the second case, one sample was left out during training in order to play the role of external data for testing the trained network later. The taken out samples were maximum and minimum density samples (for extrapolation) and one random from within the density interval. The results showed that good approximation on the engineering level MARE&lt;5% was reached for teaching networks with ≥7 neurons in the hidden layer for the first studied case and with ≥8 neurons for the second. Calculations on external data proved that 8 neurons are enough to actually obtain MARE&lt;10%. Moreover, it was shown that the quality of approximation can be significantly improved to MARE≈7% (tested on external data) if the initial region of the stress-strain relation is modelled by an additional network.

CFD data based neural network functions for predicting hydrodynamic performance of a low-pitch marine cycloidal propeller

Today, various types of propulsion systems are used in different purpose ship types. Marine cycloidal propeller (MCP) is one of these propulsion systems, which has been designed for ships that require high maneuverability. MCP can be considered as an especial type of marine propulsion systems, since it produces the thrust force which is perpendicular to propeller axis of rotation. The magnitude and direction of the thrust force can be adjusted by controlling the pitching angle of the blades, so no separate rudder is needed to manoeuvre the ship. In this study, mathematical functions for predicting the open water hydrodynamic performance of a low-pitch MCP are presented by training a neural network based on computational fluid dynamics (CFD) data. For this purpose, the four nondimensional parameters of blade number (Z), ratio of blade thickness to MCP diameter (t/D), pitch (e) and advance coefficient (λ) are considered as input variables, whereas the hydrodynamic coefficients of thrust (Ks) and torque (Kd) are considered as targets. CFD simulations are performed for different cases of MCP with different combinations of Z, t/D, e and λ. The results showed that a two-layer feedforward network with one hidden layer of sigmoid neurons and at least 4 neurons in the hidden layer can be well trained by CFD data in order to obtain functions with good accuracy in predicting Ks and Kd coefficients of a low-pitch MCP.

Modelling of the mechanical properties of concrete with cement ratio partially replaced by aluminium waste and sawdust ash using artificial neural network

The use of aluminium waste (AW) and sawdust ash (SDA) in concrete was evaluated in this study where the cement ratio was partially replaced by fractions of AW and SDA introduced as a supplementary cementitious material. Artificial neural network (ANN) was adapted as the modelling tool for this study and was developed with a two-layer feed-forward network, hidden neurons with sigmoid activation function and linear output neurons for the simulation of the network. The setting time and concrete compressive strength at varying curing days were predicted using the neural network model with variations of constituents of the cement content consisting of OPC, SDA and AW as the input of the network. Three input and seven output data set were used for the model development using the following algorithms; Data Division: Random, Training: Levenberg–Marquardt and Calculation: MATLAB. The data sets are set aside for validation, training and testing; 70% of the samples are used for training, 15% for validation and 15% are also used for testing. The performance of the networks was evaluated using linear regression, RMSE and R-values. The model performance scored 0.91 and 0.07 for R2 and RMSE, respectively, and performed better than the linear regression model, the results indicate the efficiency, reliability and usefulness of ANN for predicting concrete mechanical properties where AW and SDA are used to replace cement ratio accurately.