Trained Neural Network Model Research Articles

Dense-Structured Network Based Bearing Remaining Useful Life Prediction System

This work is focused on developing an effective method for bearing remaining useful life predictions. The method is useful in accurately predicting the remaining useful life of bearings so that machine damage, production outage, and human accidents caused by unexpected bearing failure can be prevented. This study uses the bearing dataset provided by FEMTO-ST Institute, Besançon, France. This study starts with the exploration of neural networks, based on which the biaxial vibration signals are modeled and analyzed. This paper introduces pre-processing of bearing vibration signals, neural network model training and adjustment of training data. The model is trained by optimizing model parameters and verifying its performance through cross-validation. The proposed model’s superiority is also confirmed through a comparison with other traditional models. In this study, the neural network model is trained with various types of bearing data and can successfully predict the remaining useful life. The algorithm proposed in this study achieves a prediction accuracy of coefficient of determination as high as 0.99.

Применение сверточных нейронных сетей для распознавания диабетической ретинопатии на цифровых изображениях

The aim of the research. Training of a convolutional neural network for recognition of diabetic retinopathy in digital images of the ocular fundus. Material and methods. A set of digital images from the Kaggle open-access digital repository was used. The set included 12,498 scans of the ocular fundus: 4,584 scans included signs of diabetic retinopathy while 7,914 images did not have such signs. The images with signs of diabetic retinopathy were additionally subdivided according to the disease stages. Building of mathematical models for the convolutional neural network was performed in the PyCharm programming environment using the Python programming language. Results. The maximum accuracy achieved during training of the mathematical model of the convolutional neural network for classification into 5 classes (absence of diabetic retinopathy signs and 4 stages in presence of such signs) was only 57.4 % [56.9; 57.9]. Training of the mathematical model of the convolutional neural network for binary classification (presence of retinopathy at any stage or its absence) lead to 80.7 % accuracy [79.9; 81.2] in recognition of diabetic retinopathy on digital images of the eye retina. Conclusion. A trained convolutional neural network model for binary classification provides for a sufficient degree of accuracy in classification and recognition of presence of diabetic retinopathy in digital images of the eye retina. Therefore, this model may be applied for screening of diabetic retinopathy if integrated with medical information systems or relevant medical equipment.

Predictions of \(\alpha \)-decay Half-lives for Neutron-deficient Nuclei with the Aid of Artificial Neural Network

In recent years, artificial neural network (ANN) has been successfully applied in nuclear physics and some other areas of physics. This study begins with the calculations of {\alpha}-decay half-lives for some neutron-deficient nuclei using Coulomb and proximity potential model (CPPM), temperature dependent Coulomb and proximity potential model (CPPMT), Royer empirical formula, new Ren B (NRB) formula, and a trained artificial neural network model (TANN ). By comparison with experimental values, the ANN model is found to give very good descriptions of the half-lives of the neutron-deficient nuclei. Moreover CPPMT is found to perform better than CPPM, indicating the importance of employing temperature-dependent nuclear potential. Furthermore, to predict the {\alpha}-decay half-lives of unmeasured neutron-deficient nuclei, another ANN algorithm is trained to predict the Q {\alpha} values. The results of the Q {\alpha} predictions are compared with the Weizs\"acker-Skyrme-4+RBF (WS4+RBF) formula. The half-lives of unmeasured neutron-deficient nuclei are then predicted using CPPM, CPPMT, Royer, NRB, and TANN , with Q{\alpha} values predicted by ANN as inputs. This study concludes that half-lives of {\alpha}-decay from neutron-deficient nuclei can successfully be predicted using ANN, and this can contribute to the determination of nuclei at the driplines.

Design Optimization of Curved Curtain Wall Based on SPEA II‐GA‐BPNN Algorithm

The optimization of the design of the curved curtain wall to reduce the construction cost can bring about huge economic benefits to the curtain wall project. This paper breaks through the current practice of cost optimization only in the detailed design stage. Based on the characteristics of optimization objectives in different stages, a composite scheme process is proposed, which uses the combination of SPEA II multiobjective algorithm and GA single‐objective algorithm for design optimization. It predicts and evaluates the optimization scheme by BP neural network. The panel meshing scheme is first optimized in the scheme design stage using the SPEA II multiobjective algorithm. The panel meshing scheme is first optimized in the scheme design stage using the SPEA II multiobjective algorithm. The best optimization results are then screened using the GA single‐objective algorithm to optimize the panel type in the detailed design stage. Afterwards, BP neural network training samples are randomly generated in all the optimization schemes, and the BP neural network is trained. The trained BP neural network model is used to predict and select the optimal solution in the solution set. Finally, the initial solution, existing optimization solution, and compound optimization solution are compared and analyzed. The obtained results show that the proposed composite optimization scheme can effectively reduce the cost of panel and keel materials by almost 10%. It can provide new ideas and methods for the design of curved curtain walls.

Direct Retrieval of NO 2 Vertical Columns from UV-Vis (390-495 nm) Spectral Radiances Using a Neural Network

Satellite retrievals of columnar nitrogen dioxide (NO 2 ) are essential for the characterization of nitrogen oxides (NO x ) processes and impacts. The requirements of modeled a priori profiles present an outstanding bottleneck in operational satellite NO 2 retrievals. In this work, we instead use neural network (NN) models trained from over 360,000 radiative transfer (RT) simulations to translate satellite radiances across 390-495 nm to total NO 2 vertical column (NO 2 C). Despite the wide variability of the many input parameters in the RT simulations, only a small number of key variables were found essential to the accurate prediction of NO 2 C, including observing angles, surface reflectivity and altitude, and several key principal component scores of the radiances. In addition to the NO 2 C, the NN training and cross-validation experiments show that the wider retrieval window allows some information about the vertical distribution to be retrieved (e.g., extending the rightmost wavelength from 465 to 495 nm decreases the root-mean-square-error by 0.75%) under high-NO 2 C conditions. Applying to four months of TROPOMI data, the trained NN model shows strong ability to reproduce the NO 2 C observed by the ground-based Pandonia Global Network. The coefficient of determination ( R 2 , 0.75) and normalized mean bias (NMB, -33%) are competitive with the level 2 operational TROPOMI product ( R 2 = 0.77 , NMB = − 29 % ) over clear ( geometric cloud fraction < 0.2 ) and polluted ( N O 2 C ≥ 7.5 × 10 15 molecules/cm 2 ) regions. The NN retrieval approach is ~12 times faster than predictions using high spatial resolution (~3 km) a priori profiles from chemical transport modeling, which is especially attractive to the handling of large volume satellite data.

Assessing the Reidentification Risks Posed by Deep Learning Algorithms Applied to ECG Data

ECG (Electrocardiogram) data analysis is one of the most widely used and important tools in cardiology diagnostics. In recent years the development of advanced deep learning techniques and GPU hardware have made it possible to train neural network models that attain exceptionally high levels of accuracy in complex tasks such as heart disease diagnoses and treatments. We investigate the use of ECGs as biometrics in human identification systems by implementing state-of-the-art deep learning models. We train convolutional neural network models on approximately 81k patients from the US, Germany and China. Currently, this is the largest research project on ECG identification. Our models achieved an overall accuracy of 95.69%. Furthermore, we assessed the accuracy of our ECG identification model for distinct groups of patients with particular heart conditions and combinations of such conditions. For example, we observed that the identification accuracy was the highest (99.7%) for patients with both ST changes and supraventricular tachycardia. We also found that the identification rate was the lowest for patients diagnosed with both atrial fibrillation and complete right bundle branch block (49%). We discuss the implications of these findings regarding the reidentification risks of patients based on ECG data and how seemingly anonymized ECG datasets can cause privacy concerns for the patients.

Real-Time Team Performance and Workload Prediction From Voice Communications

Automatic prediction of team performance and workload plays a crucial role in team selection, training, evaluation, and re-training processes. This study investigated the potential of using voice analysis of team-based communication for predicting team workload (TW) and team performance (TP). Both the TW and TP categories were labeled objectively. Ten teams of three participants were tasked with completing a computer-based command-and-control simulation that required communication of task-specific information to each team member. Recordings of each participant’s voice communications were used to train Convolution Neural Network (CNN) models for each team separately. It was hypothesized that integrating TW and TP information into the prediction process would support the prediction of both TW and TP categories. Two experiments were conducted. In the first experiment, the TP prediction networks were fine-tuned to predict TW, and conversely, the TW prediction networks were fine-tuned to predict TP. In the second experiment, the TP or TW prediction based on the assembly of interconnected TP and TW classifiers was tested. Both experiments confirmed the hypothesis. It was shown that task-related pre-requisite knowledge embedded into the neural network reduced neural network model training time and improved performance without the need to increase the training data size. Predictions based on combined TW and TP classification outcomes - using either separate or interconnected TW or TP classifiers - outperformed the baseline method using a single CNN model trained to predict either TW or TP alone. The classification accuracy was consistent with previously reported cognitive load prediction based on objective measures.

Towards an instant structure-property prediction quality control tool for additive manufactured steel using a crystal plasticity trained deep learning surrogate

The ability to conduct in-situ real-time process-structure-property checks has the potential to overcome process and material uncertainties, which are key obstacles to improved uptake of metal powder bed fusion in industry. Efforts are underway for live process monitoring such as thermal and image-based data gathering for every layer printed. Current crystal plasticity finite element (CPFE) modelling is capable of predicting the associated strength based on a microstructural image and material data but is computationally expensive. This work utilizes a large database of input–output samples from CPFE modelling to develop a trained deep neural network (DNN) model which instantly estimates the output (strength prediction) associated with a given input (microstructure) of multi-phase additive manufactured stainless steels. The DNN model successfully recognizes phase regions and the associated unique crystallographic orientation variations. It also captures differences in macroscopic stress response due to the varying microstructure. However, it is less reliable in terms of fatigue life predictions. The DNN model exhibits high accuracy for the structure–property relationship as a surrogate prediction tool compared to CPFE while significantly reducing the computational cost to just a few seconds.

Untangling the Galaxy. III. Photometric Search for Pre-main-sequence Stars with Deep Learning

Abstract A reliable census of pre-main-sequence stars with known ages is critical to our understanding of early stellar evolution, but historically there has been difficulty in separating such stars from the field. We present a trained neural network model, Sagitta, that relies on Gaia DR2 and 2 Micron All-Sky Survey photometry to identify pre-main-sequence stars and to derive their age estimates. Our model successfully recovers populations and stellar properties associated with known star-forming regions up to five kpc. Furthermore, it allows for a detailed look at the star-forming history of the solar neighborhood, particularly at age ranges to which we were not previously sensitive. In particular, we observe several bubbles in the distribution of stars, the most notable of which is a ring of stars associated with the Local Bubble, which may have common origins with Gould’s Belt.

Engineering Properties and Prediction of Strength of High Performance Fibre Reinforced Concrete using Artificial Neural Networks

ABSTRACT: This paper presents the experimental and numerical studies on high performance fiber concrete (HPFRC) with water-cementitious materials (w/cm) ratios of 0.4- 0.3, steel fiber volume fraction (Vf) varying from 0- 1.5%, polypropylene fiber volume fraction varying from 0- 1% and silica fume replacement at 10% and 15%. Experimental results showed high improvements in 28 day cylinder compressive strength and flex-ural strength of steel fiber reinforced concrete at fiber volume fraction of 1.5%; for polypropylene (PP) FRC improvement in compressive and flexural strengths are marginal and moderate, respectively. Statistical models developed for compressive strength ratios and flexural strength ratios of HPSFRC indicate the prediction ca-pabilities of the models. Due to the complex mix proportions of HPSFRC and the non-linear relationship be-tween the concrete mix proportions and properties, research on HPSFRC has been empirical and no models with reliable predictive capabilities for its behavior have been developed. Based on the large data collected for HPSFRC mixes, a trained artificial neural network (ANN) model which adopts a back propagation algorithm to predict 28-day compressive strength of HPSFRC mixes was employed. This paper describes the comparison of the experimental results obtained for various mixes. Multiple linear regression (MLR) model with R2 = 0.78 was also developed for the prediction of compressive strength of HPSFRC mixes. On validation of the data sets by NNs, the error range is within 2% of the actual values. ANN models give the significant degree of ac-curacy compared to MLR model, and can be easily used to estimate the strength of concrete mixes.

Mathematical Modeling of Lymph Node Drainage Function by Neural Network

The lymph node (LN) represents a key structural component of the lymphatic system network responsible for the fluid balance in tissues and the immune system functioning. Playing an important role in providing the immune defense of the host organism, LNs can also contribute to the progression of pathological processes, e.g., the spreading of cancer cells. To gain a deeper understanding of the transport function of LNs, experimental approaches are used. Mathematical modeling of the fluid transport through the LN represents a complementary tool for studying the LN functioning under broadly varying physiological conditions. We developed an artificial neural network (NN) model to describe the lymph node drainage function. The NN model predicts the flow characteristics through the LN, including the exchange with the blood vascular systems in relation to the boundary and lymphodynamic conditions, such as the afferent lymph flow, Darcy’s law constants and Starling’s equation parameters. The model is formulated as a feedforward NN with one hidden layer. The NN complements the computational physics-based model of a stationary fluid flow through the LN and the fluid transport across the blood vessel system of the LN. The physical model is specified as a system of boundary integral equations (IEs) equivalent to the original partial differential equations (PDEs; Darcy’s Law and Starling’s equation) formulations. The IE model has been used to generate the training dataset for identifying the NN model architecture and parameters. The computation of the output LN drainage function characteristics (the fluid flow parameters and the exchange with blood) with the trained NN model required about 1000-fold less central processing unit (CPU) time than computationally tracing the flow characteristics of interest with the physics-based IE model. The use of the presented computational models will allow for a more realistic description and prediction of the immune cell circulation, cytokine distribution and drug pharmacokinetics in humans under various health and disease states as well as assisting in the development of artificial LN-on-a-chip technologies.

A Practical and Adaptive Approach to Predicting Indoor CO2

COVID-19 has underlined the importance of monitoring indoor air quality (IAQ) to guarantee safe conditions in enclosed environments. Due to its strict correlation with human presence, carbon dioxide (CO2) represents one of the pollutants that most affects environmental health. Therefore, forecasting future indoor CO2 plays a central role in taking preventive measures to keep CO2 level as low as possible. Unlike other research that aims to maximize the prediction accuracy, typically using data collected over many days, in this work we propose a practical approach for predicting indoor CO2 using a limited window of recent environmental data (i.e., temperature; humidity; CO2 of, e.g., a room, office or shop) for training neural network models, without the need for any kind of model pre-training. After just a week of data collection, the error of predictions was around 15 parts per million (ppm), which should enable the system to regulate heating, ventilation and air conditioning (HVAC) systems accurately. After a month of data we reduced the error to about 10 ppm, thereby achieving a high prediction accuracy in a short time from the beginning of the data collection. Once the desired mobile window size is reached, the model can be continuously updated by sliding the window over time, in order to guarantee long-term performance.

Proof of Learning (PoLe): Empowering neural network training with consensus building on blockchains

The advent of neural network (NN) based deep learning, especially the recent development of the automatic design of networks, has brought unprecedented performance gains at heavy computational cost. On the other hand, in order to generate a new consensus block, Proof of Work (PoW) based blockchain systems routinely perform a huge amount of computation that does not achieve practical purposes but to solving a difficult cryptographic hash puzzle problem.In this study, we propose a new consensus mechanism, Proof of Learning (PoLe), which directs the computation spent for block consensus toward optimization of neural networks. In our design, the training and testing data are released to the entire blockchain network and the consensus nodes train NN models on the data, which serves as the proof of learning. As a core component of PoLe, we design a secure mapping layer (SML) to prevent consensus nodes from cheating, which can be straightforwardly implemented as a linear NN layer. When the consensus on the blockchain network is achieved, a new block is appended to the blockchain. We experimentally compare the PoLe protocol with Proof of Work (PoW) and show that PoLe can achieve a more stable block generation rate, which leads to more efficient transaction processing. Experimental evaluation also shows the PoLe can achieve a stable block generation rate without significantly sacrificing training performance.

A convolutional neural network model based reversible data hiding scheme in encrypted images with block-wise Arnold transform

The research in the domain of reversible data hiding (RDH) is recently explored in all aspects due to its applications in cloud computing, forensics, and medical image communication. In this manuscript, we introduce a RDH scheme in encrypted images which can provide a high embedding rate without compromising the bit error rate during the message extraction and image recovery. The proposed scheme follows a block-wise data hiding process. If the processed block size is A×A pixels, and if A=2n+1, then the data hider can hide any number from the set {0, 1, …, (A+2n−1)} in that block. We have introduced an Arnold transform-based data hiding process in which each block will be undergoing a series of scrambling processes based on the bit sequence that we need to embed in the selected block. The message extraction and image restoration are carried out at the receiver side using a trained convolutional neural network (CNN) model.

Energy optimization of internet of things in wireless sensor network models using type‐2 fuzzy neural systems

SummaryIn this paper, type‐2 fuzzy logic design is employed to find the weight values of the radial basis function (RBF) neural network model, and thereby, the trained RBF neural network (RBFNN) model is intended to perform network energy optimization of the cloud‐assisted internet of things in wireless sensor networks (WSNs). RBF neural model comes under the class of feed forward neural network architecture and is a network with better generalization capability. RBFNN employs Gaussian activation function to determine the output of the network and the special feature of this activation function is that it follows a normal probability density function; hence it is a continuous activation function and provides better solutions than the other discrete activation functions. Due to which, in this work RBFNN is employed to determine network energy and to increase the life time of the network by selecting best cluster heads and also the network route. In RBFNN modelling, basically, the weights are initialized in a random manner, and this random initialization of weights at time results in the occurrences of global and local minima. The weights are tuned for their optimal values using type‐2 fuzzy model due to their capability of handling uncertainties, and since this problem of identifying optimal weight values of RBF neural model possesses highest level of uncertainties, type‐2 fuzzy system is applied here. Simulation process is done, and the results prove the effectiveness of the proposed approach in comparison with that of the existing approaches from previous literatures for the energy optimization problem.

Adam Optimization Method Modifications Comparison in the Regression Models Parameters Evaluation Tasks

There is considered a problem of optimization methods comparing for the neural network regression task. Various optimization methods, such as stochastic gradient descent with momentum (SGDM), Adam and its modifications, AdamW and RAdam, were considered. To compare optimization method two regression task were formulated. Both tasks are connected with the preprocessing subtasks in the field of image analysis. The first considered task was the filtering blurred eye images on which confident recognition cannot be achieved. The training samples were generated by Gaussian blurring of the images. The blurring degree was estimated. The test and training sample for the assessment problem was formed on the basis of the BATH and CASSIA eye image databases. The second task was aligning faces in assessment image in face recognition systems. The training samples were generated by rotating face images, and rotation angle was estimated. To solve these tasks the direct estimation of parameters by solving the image regression problem by training neural network models is proposed. The adequate accuracy was acquired with all considered optimization methods for both tasks. Modifications of Adam algorithm show better results than original method. Both AdamW and RAdam methods reduced the error twice in comparison with Adam. The modification of the RAdam algorithm proposed in the work reduced the error by more than 1.5 times in comparison with the model trained by the original algorithm.

State-of-health prediction of lithium-ion battery based on improved gate recurrent unit

State of health (SOH) prediction of lithium-ion batteries is still a very important issue in evaluating the safety and reliability of battery-powered systems. This paper uses the lithiumion battery data in NASA Ames Research Center for research, analyzes the correlation between the relevant feature data, selects the data the most relevant to the capacity through the threshold, and uses it as the input of the neural network. We combine Particle Swarm Optimization (PSO) algorithm and Gate Recurrent Unit (GRU) to form PSO-GRU, and use the PSO-GRU method to find the time step and the number of neurons for the best prediction effect. The experimental results show that, compared with the LSTM method, the PSO-GRU method has higher prediction accuracy and has fewer weight parameters for the neural network training model.

Species reaction rate modelling based on physics-guided machine learning

Deep neural network (DNN) is applied to mean reaction rate modelling. Two DNN structures, species-dependent (SD) and species-independent (SI), are considered.11Code and data to use the trained deep neural network models in this work are available at https://github.com/minamoto-group/PGDNN_RANS_model_Nakazawa_etal. Due to the explicit inclusion of all species variables in the input layer for SD-DNN, this model may consider relationships of different chemical species. However, the prediction can be performed only for simulations with chemical mechanisms considering the same set of species as the one used in training data. SI-DNN circumvents this constraint, and can be used for any set of species appearing in the combustion. For the efficient learning and better prediction performance, two physics-guided loss functions are proposed and employed, which consider mass conservation of the mixture and elemental species in a specific formulation that yields a larger number of constraint conditions. These DNNs are trained and validated using direct numerical simulation (DNS) data of three different turbulent premixed planar flames, and tested using DNS results of a fourth turbulent premixed planar flame and turbulent premixed V-flame to assess the robustness of the present models for an unknown combustion configuration as well as unknown turbulent combustion conditions.

Developing a new deep learning CNN model to detect and classify highway cracks

PurposeThis paper aims to Test the capabilities/accuracies of four deep learning pre trained convolutional neural network (CNN) models to detect and classify types of highway cracks, as well as developing a new CNN model to maximize the accuracy at different learning rates.Design/methodology/approachA sample of 4,663 images of highway cracks were collected and classified into three categories of cracks, namely, “vertical cracks,” “horizontal and vertical cracks” and “diagonal cracks,” subsequently, using “Matlab” to classify the sample to training (70%) and testing (30%) to apply the four deep learning CNN models and compute their accuracies. After that, developing a new deep learning CNN model to maximize the accuracy of detecting and classifying highway cracks and testing the accuracy using three optimization algorithms at different learning rates.FindingsThe accuracies result of the four deep learning pre-trained models are above the averages between top-1 and top-5 and the accuracy of classifying and detecting the samples exceeded the top-5 accuracy for the pre-trained AlexNet model around 3% and by 0.2% for the GoogleNet model. The accurate model here is the GoogleNet model as the accuracy is 89.08% and it is higher than AlexNet by 1.26%. While the computed accuracy for the new created deep learning CNN model exceeded all pre-trained models by achieving 97.62% at a learning rate of 0.001 using Adam’s optimization algorithm.Practical implicationsThe created deep learning CNN model will enable users (e.g. highway agencies) to scan a long highway and detect types of cracks accurately in a very short time compared to traditional approaches.Originality/valueA new deep learning CNN-based highway cracks detection was developed based on testing four pre-trained CNN models and analyze the capabilities of each model to maximize the accuracy of the proposed CNN.

Single-site indoor fingerprint localization based on MIMO-CSI

In order to achieve higher accuracy and lower cost of indoor localization, we propose a positioning method using multiple input and multiple output (MIMO) channel state information (CSI) as a fingerprint. The method can be divided into three stages, feature extraction, offline training and online localization. In the feature extraction, the segmented average and principal component analysis (PCA) are used to reduce the data dimension and decrease system complexity. In the offline training, the deep neural network (NN) model is trained to implement the position classification. In the online localization, the data are input into the trained NN model first, and then its output is further processed by weighted k-nearest neighbor (WKNN) technology to estimate the position. Experimental results show that the proposed method can significantly reduce the positioning error compared to other methods and the average error is 1.39m in a complex indoor environment.