Output Of Neural Network Research Articles

Negative Sequence Current Compensation for Induction Motor Stator Inter-Turn Short Circuit and Off-Diagonal Term in Sequence Impedance Matrix as Fault Indicator

This work proposes an artificial neural network supported method to establish an automatic detection of stator turn fault in induction motor. The sequence current analysis is done for turn fault condition. Various factors influencing the total measured negative sequence current such as unbalanced voltage and inherent asymmetry have been reviewed. To compensate the voltage unbalance and non-idealities in the machine, utilization of measured negative sequence current, impedance, admittance, or semi empirical formula is developed. The output of a well-trained feed forward back propagation neural network classifies the severity of fault level in stator winding. The method of considering the effects of turn faults on inter-turn fault detection improves sensitivity meanwhile reduces the prospect of misdetection.

Mean Field Analysis of Deep Neural Networks

We analyze multilayer neural networks in the asymptotic regime of simultaneously (a) large network sizes and (b) large numbers of stochastic gradient descent training iterations. We rigorously establish the limiting behavior of the multilayer neural network output. The limit procedure is valid for any number of hidden layers, and it naturally also describes the limiting behavior of the training loss. The ideas that we explore are to (a) take the limits of each hidden layer sequentially and (b) characterize the evolution of parameters in terms of their initialization. The limit satisfies a system of deterministic integro-differential equations. The proof uses methods from weak convergence and stochastic analysis. We show that, under suitable assumptions on the activation functions and the behavior for large times, the limit neural network recovers a global minimum (with zero loss for the objective function).

Exploring Uncertainty Measures for Image-caption Embedding-and-retrieval Task

With the significant development of black-box machine learning algorithms, particularly deep neural networks, the practical demand for reliability assessment is rapidly increasing. On the basis of the concept that “Bayesian deep learning knows what it does not know,” the uncertainty of deep neural network outputs has been investigated as a reliability measure for classification and regression tasks. By considering an embedding task as a regression task, several existing studies have quantified the uncertainty of embedded features and improved the retrieval performance of cutting-edge models by model averaging. However, in image-caption embedding-and-retrieval tasks, well-known samples are not always easy to retrieve. This study shows that the existing method has poor performance in reliability assessment and investigates another aspect of image-caption embedding-and-retrieval tasks. We propose posterior uncertainty by considering the retrieval task as a classification task, which can accurately assess the reliability of retrieval results. The consistent performance of the two uncertainty measures is observed with different datasets (MS-COCO and Flickr30k), different deep-learning architectures (dropout and batch normalization), and different similarity functions. To the best of our knowledge, this is the first study to perform a reliability assessment on image-caption embedding-and-retrieval tasks.

Multi-valued neural networks I: a multi-valued associative memory

A new concept of a multi-valued associative memory is introduced, generalizing a similar one in fuzzy neural networks. We expand the results on fuzzy associative memory with thresholds, to the case of a multi-valued one: we introduce the novel concept of such a network without number representation of weights and data, investigate its properties, and give a learning algorithm in the multi-valued case. We discovered conditions under which it is possible to store given pairs of network variable patterns in such a multi-valued associative memory. In the multi-valued neural network, not all variables are numbers, but elements or subsets of a lattice, i.e., they are all only partially ordered. Lattice operations are used to build the network output by inputs. In this paper, the lattice is assumed to be Brouwer and determines the implication used, together with other lattice operations, to determine the neural network output. We give the example of the network use to classify aircraft/spacecraft trajectories.

An ensemble approach based on transformation functions for natural gas price forecasting considering optimal time delays.

Natural gas, known as the cleanest fossil fuel, plays a vital role in the economies of producing and consuming countries. Understanding and tracking the drivers of natural gas prices are of significant interest to the many economic sectors. Hence, accurately forecasting the price is very important not only for providing an effective factor for implementing energy policy but also for playing an extremely significant role in government strategic planning. The purpose of this study is to provide an approach to forecast the natural gas price. First, optimal time delays are identified by a new approach based on the Euclidean Distance between input and target vectors. Then, wavelet decomposition has been implemented to reduce noise. Moreover, fuzzy transform with different membership functions has been used for modeling uncertainty in time series. The wavelet decomposition and fuzzy transform have been integrated into the preprocessing stage. An ensemble method is used for integrating the outputs of various neural networks. The results depict that the proposed preprocessing methods used in this paper cause to improve the accuracy of natural gas price forecasting and consider uncertainty in time series.

CT Cervical Spine Fracture Detection Using a Convolutional Neural Network.

Multidetector CT has emerged as the standard of care imaging technique to evaluate cervical spine trauma. Our aim was to evaluate the performance of a convolutional neural network in the detection of cervical spine fractures on CT. We evaluated C-spine, an FDA-approved convolutional neural network developed by Aidoc to detect cervical spine fractures on CT. A total of 665 examinations were included in our analysis. Ground truth was established by retrospective visualization of a fracture on CT by using all available CT, MR imaging, and convolutional neural network output information. The ĸ coefficients, sensitivity, specificity, and positive and negative predictive values were calculated with 95% CIs comparing diagnostic accuracy and agreement of the convolutional neural network and radiologist ratings, respectively, compared with ground truth. Convolutional neural network accuracy in cervical spine fracture detection was 92% (95% CI, 90%-94%), with 76% (95% CI, 68%-83%) sensitivity and 97% (95% CI, 95%-98%) specificity. The radiologist accuracy was 95% (95% CI, 94%-97%), with 93% (95% CI, 88%-97%) sensitivity and 96% (95% CI, 94%-98%) specificity. Fractures missed by the convolutional neural network and by radiologists were similar by level and location and included fractured anterior osteophytes, transverse processes, and spinous processes, as well as lower cervical spine fractures that are often obscured by CT beam attenuation. The convolutional neural network holds promise at both worklist prioritization and assisting radiologists in cervical spine fracture detection on CT. Understanding the strengths and weaknesses of the convolutional neural network is essential before its successful incorporation into clinical practice. Further refinements in sensitivity will improve convolutional neural network diagnostic utility.

Research on Commodity Recommendation System Based on Deep Learning

The traditional matrix factorization model cannot effectively extract the features of users and items, but the feature information can be extracted well based on the deep learning model. At present, the mainstream recommendation algorithms based on deep learning only make recommendation prediction in the form of the product of neural network output or item features and user features, and cannot fully mine the relationship between users and items. Based on this, this paper proposes a recommendation algorithm based on the combination of text convolution neural network and singular value decomposition (Bias SVD) with biased terms. The text convolution neural network (Text CNN) is used to fully extract the feature information of users and items, and then the singular value decomposition method is used to make recommendations to deeply understand the document context information and further improve the accuracy of recommendation. The algorithm is widely evaluated and analyzed on two real data sets of MovieLens, and the accuracy of recommendation is obviously better than that of ConvMF algorithm and mainstream deep learning recommendation algorithm.

Designing an Intelligent Control Philosophy in Reservoirs of Water Transfer Networks in Supervisory Control and Data Acquisition System Stations

In this paper, a hybrid neural-genetic fuzzy system is proposed to control the flow and height of water in the reservoirs of water transfer networks. These controls will avoid probable water wastes in the reservoirs and pressure drops in water distribution networks. The proposed approach combines the artificial neural network, genetic algorithm, and fuzzy inference system to improve the performance of the supervisory control and data acquisition stations through a new control philosophy for instruments and control valves in the reservoirs of the water transfer networks. First, a multi-core artificial neural network model, including a multi-layer perceptron and radial based function, is proposed to forecast the daily consumption of the water in a reservoir. A genetic algorithm is proposed to optimize the parameters of the artificial neural networks. Then, the online height of water in the reservoir and the output of artificial neural networks are used as inputs of a fuzzy inference system to estimate the flow rate of the reservoir inlet. Finally, the estimated inlet flow is translated into the input valve position using a transform control unit supported by a nonlinear autoregressive exogenous model. The proposed approach is applied in the Tehran water transfer network. The results of this study show that the usage of the proposed approach significantly reduces the deviation of the reservoir height from the desired levels.

A building energy consumption prediction model based on rough set theory and deep learning algorithms

The efficient and accurate prediction of building energy consumption can improve the management of power systems. In this paper, the rough set theory was used to reduce the redundant influencing factors of building energy consumption and find the critical factors of building energy consumption. These key factors were then used as the input of a deep neural network with a “deep” architecture and powerful capabilities in extracting features. Building energy consumption is output of the deep neural network. This study collected data from 100 civil public buildings for rough set reduction, and then collected data from a laboratory building of a university in Dalian for nearly a year to train and test deep neural networks. The test included both the short-term and medium-term predictions of building energy consumption. The prediction results of the deep neural network were compared with that of the back propagation neural network, Elman neural network and fuzzy neural network. The results show that the integrated rough set and deep neural network was the most accurate. The method proposed in this study could provide a practical and accurate solution for building energy consumption prediction.

Neural network surrogate of QuaLiKiz using JET experimental data to populate training space

Within integrated tokamak plasma modeling, turbulent transport codes are typically the computational bottleneck limiting their routine use outside of post-discharge analysis. Neural network (NN) surrogates have been used to accelerate these calculations while retaining the desired accuracy of the physics-based models. This paper extends a previous NN model, known as QLKNN-hyper-10D, by incorporating the impact of impurities, plasma rotation, and magnetic equilibrium effects. This is achieved by adding a light impurity fractional density (nimp,light/ne) and its normalized gradient, the normalized pressure gradient (α), the toroidal Mach number (Mtor), and the normalized toroidal flow velocity gradient. The input space was sampled based on experimental data from the JET tokamak to avoid the curse of dimensionality. The resulting networks, named QLKNN-jetexp-15D, show good agreement with the original QuaLiKiz model, both by comparing individual transport quantity predictions and by comparing its impact within the integrated model, JINTRAC. The profile-averaged RMS of the integrated modeling simulations is <10% for each of the five scenarios tested. This is non-trivial given the potential numerical instabilities present within the highly nonlinear system of equations governing plasma transport, especially considering the novel addition of momentum flux predictions to the model proposed here. An evaluation of all 25 NN output quantities at one radial location takes ∼0.1 ms, 104 times faster than the original QuaLiKiz model. Within the JINTRAC integrated modeling tests performed in this study, using QLKNN-jetexp-15D resulted in a speed increase of only 60–100 as other physics modules outside of turbulent transport become the bottleneck.

Application of Artificial Neural Network to Somatotype Determination

Somatotype characteristics are important for the selection of sporting activities, as well as and the prevalence of several chronic diseases. Nowadays the most common method of somatotyping is the Heath–Carter method, which calculates the somatotype base on 10 anthropometric parameters. Another possibility for evaluation of somatotype gives commonly used bioelectrical impedance analysis), but the accuracy of the proposed formulas is questioned. Therefore, we aimed to investigate the possibility of applying an artificial neural network to achieve the formulas, which allow us to determine the endomorphy and mesomorphy using data on body height and weight and raw bioelectrical impedance analysis data in young women. The endomorphy (Endo), ectomorphy (Ecto), and mesomorphy (Meso) ratings were determined using artificial neural networks and the Heath–Carter method. To identify critical parameters and their degree of impact on the artificial neural network outputs, a sensitivity analysis was performed. The multi-layer perceptron MLP 4-4-1 (input: body mass index (BMI), reactance, resistance, and resting metabolic rate) for the Endo somatotype was proposed (root mean squared error (RMSE) = 0.66, χ2 = 0.66). The MLP 4-4-1 (input: BMI, fat-free mass, resistance, and total body water) for the Meso somatotype was proposed (RMSE = 0.76, χ2 = 0.87). All somatotypes (Endo, Meso and Ecto) can be calculated using MLP 2-4-3 (input: BMI and resistance) with accuracy RMSE = 0.67 and χ2 = 0.51. The bioelectrical impedance analysis and Heath–Carter method compliance was evaluated with the statistical algorithm proposed by Bland and Altman. The artificial neural network-based formulas allow us to determine the endomorphy and mesomorphy in young women’s ratings with high accuracy and agreement with the Heath–Carter method. The results of our study indicate the successful application of artificial neural network-based model in predicting the somatotype of young women. The artificial neural network model can be practically used in bioelectrical impedance analysis devices in the future.

Machine Learning Approach to Decision Making for Insulin Initiation in Japanese Patients With Type 2 Diabetes (JDDM 58): Model Development and Validation Study.

BackgroundApplications of machine learning for the early detection of diseases for which a clear-cut diagnostic gold standard exists have been evaluated. However, little is known about the usefulness of machine learning approaches in the decision-making process for decisions such as insulin initiation by diabetes specialists for which no absolute standards exist in clinical settings.ObjectiveThe objectives of this study were to examine the ability of machine learning models to predict insulin initiation by specialists and whether the machine learning approach could support decision making by general physicians for insulin initiation in patients with type 2 diabetes.MethodsData from patients prescribed hypoglycemic agents from December 2009 to March 2015 were extracted from diabetes specialists’ registries, resulting in a sample size of 4860 patients who had received initial monotherapy with either insulin (n=293) or noninsulin (n=4567). Neural network output was insulin initiation ranging from 0 to 1 with a cutoff of >0.5 for the dichotomous classification. Accuracy, recall, and area under the receiver operating characteristic curve (AUC) were calculated to compare the ability of machine learning models to make decisions regarding insulin initiation to the decision-making ability of logistic regression and general physicians. By comparing the decision-making ability of machine learning and logistic regression to that of general physicians, 7 cases were chosen based on patient information as the gold standard based on the agreement of 8 of the 9 specialists.ResultsThe AUCs, accuracy, and recall of logistic regression were higher than those of machine learning (AUCs of 0.89-0.90 for logistic regression versus 0.67-0.74 for machine learning). When the examination was limited to cases receiving insulin, discrimination by machine learning was similar to that of logistic regression analysis (recall of 0.05-0.68 for logistic regression versus 0.11-0.52 for machine learning). Accuracies of logistic regression, a machine learning model (downsampling ratio of 1:8), and general physicians were 0.80, 0.70, and 0.66, respectively, for 43 randomly selected cases. For the 7 gold standard cases, the accuracies of logistic regression and the machine learning model were 1.00 and 0.86, respectively, with a downsampling ratio of 1:8, which were higher than the accuracy of general physicians (ie, 0.43).ConclusionsAlthough we found no superior performance of machine learning over logistic regression, machine learning had higher accuracy in prediction of insulin initiation than general physicians, defined by diabetes specialists’ choice of the gold standard. Further study is needed before the use of machine learning–based decision support systems for insulin initiation can be incorporated into clinical practice.

Observer-Based Neuro-Adaptive Optimized Control of Strict-Feedback Nonlinear Systems With State Constraints.

This article proposes an adaptive neural network (NN) output feedback optimized control design for a class of strict-feedback nonlinear systems that contain unknown internal dynamics and the states that are immeasurable and constrained within some predefined compact sets. NNs are used to approximate the unknown internal dynamics, and an adaptive NN state observer is developed to estimate the immeasurable states. By constructing a barrier type of optimal cost functions for subsystems and employing an observer and the actor-critic architecture, the virtual and actual optimal controllers are developed under the framework of backstepping technique. In addition to ensuring the boundedness of all closed-loop signals, the proposed strategy can also guarantee that system states are confined within some preselected compact sets all the time. This is achieved by means of barrier Lyapunov functions which have been successfully applied to various kinds of nonlinear systems such as strict-feedback and pure-feedback dynamics. Besides, our developed optimal controller requires less conditions on system dynamics than some existing approaches concerning optimal control. The effectiveness of the proposed optimal control approach is eventually validated by numerical as well as practical examples.

Response surface methodology and artificial neural network modelling for enhancing maturity parameters during vermicomposting of floral waste

In this study, the mixture of floral waste and cattle dung in different proportions was utilised to convert into vermicompost using earthworm Eisenia fetida. In the design of the experiment, the optimum amount of floral waste (1325 g) and cattle dung (500 g) was obtained for vermicompost using central composite design (CCD) and compared with the output of artificial neural network (ANN). The optimum proportions of vermicompost showed pH of 7.10, electrical conductivity of 3.39 mS/cm, total organic carbon of 34.01%, C: N ratio of 13, phosphorous of 5.31 g/kg and potassium of 14.45 g/kg. This vermicompost was enriched with sufficient concentration of nutrients like potassium, sodium, phosphorous, and calcium, which are beneficial for the growth of the plants. The current study was based on comparing response surface methodology (RSM) and ANN for maturity parameters and the value of R2 in both the cases was near 1.

Hidden neural networks for transmembrane protein topology prediction

Hidden Markov Models (HMMs) are amongst the most successful methods for predicting protein features in biological sequence analysis. However, there are biological problems where the Markovian assumption is not sufficient since the sequence context can provide useful information for prediction purposes. Several extensions of HMMs have appeared in the literature in order to overcome their limitations. We apply here a hybrid method that combines HMMs and Neural Networks (NNs), termed Hidden Neural Networks (HNNs), for biological sequence analysis in a straightforward manner. In this framework, the traditional HMM probability parameters are replaced by NN outputs. As a case study, we focus on the topology prediction of for alpha-helical and beta-barrel membrane proteins. The HNNs show performance gains compared to standard HMMs and the respective predictors outperform the top-scoring methods in the field. The implementation of HNNs can be found in the package JUCHMME, downloadable from http://www.compgen.org/tools/juchmme, https://github.com/pbagos/juchmme. The updated PRED-TMBB2 and HMM-TM prediction servers can be accessed at www.compgen.org.

Evaluation of Aerospace Production Site Management and Control Capability based on F-AHP and BP Neural Network

The management and control level of aerospace production site directly affects the production quality of aerospace products. It is one of the important fields of aerospace quality management research to master the management and control ability of aerospace production site efficiently. Based on the analysis of the control scope of aerospace production site, the evaluation index system of space production site control ability is proposed. Through the establishment of Fuzzy-Analytic Hierarchy model, the sample data of process management and control ability evaluation on aerospace production site was collected. BP neural network model was constructed and trained, the output of BP neural network reached the expected accuracy, which meet the requirements of process management and control ability evaluation on aerospace production site. The result shows that the proposed evaluation index system and BP neural network model can not only fully absorb the tacit knowledge of experts, but also reduce the subjective arbitrary defects of artificial weights, which can be used to evaluate of the control ability of aerospace production site efficiently.

Observer-Based Adaptive Neural Network Control for Nonstrict-Feedback Nonlinear Systems With Time Delays

An observer-based adaptive neural network control problem was investigated for a class of nonstrict-feedback nonlinear systems with time delays. A state observer was constructed to estimate unknown variables in nonlinear systems. With the approximation ability of RBF NNs and the backstepping technique, an adaptive neural network output feedback control approach was proposed. The designed controller ensures the semi-global uniform boundedness of all signals in the closed-loop system. Finally, the simulation example shows the effectiveness of the proposed control approach.

Diagnosis Support Model of Cardiomegaly Based on CNN Using ResNet and Explainable Feature Map

Medical expert support systems using medical data significantly contribute to the field of medicine by utilizing intelligent image analysis technology. Such medical systems must be validated and must have a transparent internal structure. Therefore, the technique of analyzing the results of the medical system is an important factor. This study proposes a diagnosis support model of cardiomegaly based on CNN using ResNet and explainable feature map. To configure the model, initially, a cardiomegaly diagnosis model is configured using ResNet, and a chest X-ray data set is used and learned. As the result changes resulting from the input changes of the model configured for model diagnosis support are detected, an explainable feature map for analysis is implemented. The input changes of the model are made as each pixel of the required image is reversed in sequence and the changes of the neural network output layer are saved on the feature map. As a result, the configured analysis method provides information inside the neural network through an accuracy result close to 80% and a visually expressed feature map. In particular, this interpretation method can be extended to a more general form in a medical support system that analyzes patterns.

Explainability in deep reinforcement learning

A large set of the explainable Artificial Intelligence (XAI) literature is emerging on feature relevance techniques to explain a deep neural network (DNN) output or explaining models that ingest image source data. However, assessing how XAI techniques can help understand models beyond classification tasks, e.g. for reinforcement learning (RL), has not been extensively studied. We review recent works in the direction to attain Explainable Reinforcement Learning (XRL), a relatively new subfield of Explainable Artificial Intelligence, intended to be used in general public applications, with diverse audiences, requiring ethical, responsible and trustable algorithms. In critical situations where it is essential to justify and explain the agent’s behaviour, better explainability and interpretability of RL models could help gain scientific insight on the inner workings of what is still considered a black box. We evaluate mainly studies directly linking explainability to RL, and split these into two categories according to the way the explanations are generated: transparent algorithms and post-hoc explainability. We also review the most prominent XAI works from the lenses of how they could potentially enlighten the further deployment of the latest advances in RL, in the demanding present and future of everyday problems.

Robustness Analysis of Neural Networks via Efficient Partitioning With Applications in Control Systems

Neural networks (NNs) are now routinely implemented on systems that must operate in uncertain environments, but the tools for formally analyzing how this uncertainty propagates to NN outputs are not yet commonplace. Computing tight bounds on NN output sets (given an input set) provides a measure of confidence associated with the NN decisions and is essential to deploy NNs on safety-critical systems. Recent works approximate the propagation of sets through nonlinear activations or partition the uncertainty set to provide a guaranteed outer bound on the set of possible NN outputs. However, the bound looseness causes excessive conservatism and/or the computation is too slow for online analysis. This letter unifies propagation and partition approaches to provide a family of robustness analysis algorithms that give tighter bounds than existing works for the same amount of computation time (or reduced computational effort for a desired accuracy level). Moreover, we provide new partitioning techniques that are aware of their current bound estimates and desired boundary shape (e.g., lower bounds, weighted <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{\infty }$ </tex-math></inline-formula> -ball, convex hull), leading to further improvements in the computation-tightness tradeoff. The letter demonstrates the tighter bounds and reduced conservatism of the proposed robustness analysis framework with examples from model-free RL and forward kinematics learning.