Output Of Neural Network Research Articles

An Educated Warm Start for Deep Image Prior-Based Micro CT Reconstruction

Deep image prior (DIP) was recently introduced as an effective unsupervised approach for image restoration tasks. DIP represents the image to be recovered as the output of a deep convolutional neural network, and learns the network's parameters such that the model output matches the corrupted observation. Despite its impressive reconstructive properties, the approach is slow when compared to supervisedly learned, or traditional reconstruction techniques. To address the computational challenge, we bestow DIP with a two-stage learning paradigm: (i) perform a supervised pretraining of the network on a simulated dataset; (ii) fine-tune the network's parameters to adapt to the target reconstruction task. We provide a thorough empirical analysis to shed insights into the impacts of pretraining in the context of image reconstruction. We showcase that pretraining considerably speeds up and stabilizes the subsequent reconstruction task from real-measured 2D and 3D micro computed tomography data of biological specimens. The code and additional experimental materials are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://educateddip.github.io/docs.educated_deep_image_prior/</uri> educateddip.github.io/docs.educated_deep_image_prior/.

Study of an Automatic Picking Method for Multimode Dispersion Curves of Surface Waves Based on an Improved U-Net

Surface wave exploration has been increasingly used in near-surface geophysical investigations. However, the accuracy and efficiency of picking dispersion curves are key to surface wave inversion. Traditional dispersion curve extraction requires manual picking, and the extraction accuracy and efficiency depend on the experience and knowledge of the interpreters. Therefore, developing a fast, high-precision and intelligent dispersion curve extraction method is urgent. This paper improves the structure and output of the U-Net neural network and regards the picking process of dispersion curves as an image classification problem, which quickly and accurately extracts dispersion curves from dispersion energy images. After combining the dispersion energy images of synthetic seismic data with the manually extracted dispersion curve and the theoretical dispersion curve obtained by the Schwab-Knopoff algorithm, the ICM (energy image and dispersion curve extracted manually) training set and the ICS (energy image and dispersion curve calculated by Schwab-Knopoff algorithm) training set are created. The synthetic data tests verify the feasibility of the improved U-Net neural network for automatically picking multimode dispersion curves. The dispersion curve picking results corresponding to two different training sets reveal that the U-Net network model obtained from the ICS training sets exhibits better extraction accuracy. Additionally, we analyze the influence of the sample number of the training set on the dispersion curve picking effect of the improved U-Net and conclude that the improved U-Net network has the advantages of a low training set size requirement and a high dispersion curve extraction accuracy. Finally, the trained network extracts the dispersion curves of two groups of measured surface wave data and obtains plausible extraction results, further proving the proposed method’s effectiveness.

Edge Detection Algorithm of MRI Medical Image Based on Artificial Neural Network

In order to be able to extract useful information from MRI medical images, we need to carry out effective image segmentation of MRI medical image and use appropriate operators to detect the edge of the image. Firstly, this paper adopts a research method based on the characteristics of horizontal, vertical and diagonal differences, and then carries out comparative analysis on different operators from different perspectives such as frequency domain and velocity domain. After analyzing the results, it is concluded that: Canny operator can detect complete, continuous and detailed edges at a faster speed. This paper uses edge detector based on Canny operator as the training output of artificial neural network. By training artificial neural network. The optimized parameters were obtained, and subjective analysis and computational analysis were performed on the quality of edge detection results of MRI medical images based on different methods. The results show that compared with the single Sobel, Canny and other traditional operators, the edge information of MRI medical image obtained by the neural network-based edge detection method is more complete and its processing time is nearly 3 times faster than the traditional edge detection method.

Learning an Accurate State Transition Dynamics Model by Fitting Both a Function and its Derivative

Learning accurate state transition dynamics model in a sample-efficient way is important to predict the future states from the current states and actions of a system both accurately and efficiently in model-based reinforcement learning for many robotic applications. This study proposes a sample-efficient learning approach that can accurately learn a state transition dynamics model by fitting both the predicted next states and their derivatives. The derivatives of the feedforward neural network output (next states) with respect to the inputs (current states and actions) are computed using chain rules. In addition, the effect of the activation functions on the learning derivatives are illustrated via sum of elementary sine functions example and the values are compared with various other activation functions with respect to accuracy. The proposed learning approach exhibits significant improvement in accuracy for both one-step and multi-step prediction cases with a six-degree-of-freedom manipulation robot (UR-10) in both simulation and real environments.

Application of AI techniques for modeling the performance measures in milling of 7075-T6 hybrid aluminum metal matrix composites

The prediction of performance measures is an essential one for manufacturers to increase the service life. This paper deals with the application of Artificial Intelligence (AI) to predict the performance measures like surface roughness, material removal rate, and flank wear during the milling process from the experimental data. The milling experiments were conducted in wet conditions based on the Response Surface Methodology (RSM) design of experiments. The spindle speed, feed rate, and axial depth of cut were considered as process parameters. The experimental data were used to develop the regression model, Mamdani fuzzy inference system, Backpropagation Neural Network (BPNN), and Adaptive Neuro-Fuzzy Inference System (ANFIS) model. The output of regression, fuzzy, neural network, and ANFIS model was compared with the experimental data, and predicted results were found to be in good conformity with the measured data. The prediction capability of the quadratic and Artificial Neural Network (ANN) model was very close to experimentally measured values and the quadratic model had an accuracy of 97.89% for surface roughness, 98.38% for material removal rate (MRR), and 95.72% for flank wear.

Neural networks for clustered and longitudinal data using mixed effects models.

Although most statistical methods for the analysis of longitudinal data have focused on retrospective models of association, new advances in mobile health data have presented opportunities for predicting future health status by leveraging an individual's behavioral history alongside data from similar patients. Methods that incorporate both individual-level and sample-level effects are critical to using these data to its full predictive capacity. Neural networks are powerful tools for prediction, but many assume input observations are independent even when they are clustered or correlated in some way, such as in longitudinal data. Generalized linear mixed models (GLMM) provide a flexible framework for modeling longitudinal data but have poor predictive power particularly when the data are highly nonlinear. We propose a generalized neural network mixed model that replaces the linear fixed effect in a GLMM with the output of a feed-forward neural network. The model simultaneously accounts for the correlation structure and complex nonlinear relationship between input variables and outcomes, and it utilizes the predictive power of neural networks. We apply this approach to predict depression and anxiety levels of schizophrenic patients using longitudinal data collected from passive smartphone sensordata.

Relationship between Head Nurses’ Ethical Leadership and Demographic Characteristics with General Self-Efficacy of ICU Nurses in Educational-Treatment Centers of Hamadan University of Medical Sciences, Hamadan, Iran, 2019

Background and Objective: Ethical leadership has emerged as a new approach in the perspective of leadership and has provided a basis for the promotion of individual and organizational effectiveness by prioritizing ethics in the organization. This study aimed to determine the relationship between head nurses’ ethical leadership and demographic characteristics with the general self-efficacy of nurses working in Intensive Care Units (ICUs). Materials and Methods: This descriptive correlational study was conducted in 2019 on 230 nurses working in ICUs of educational-treatment centers of Hamadan University of Medical Sciences, Hamadan, Iran, who were selected using a stratified proportional sampling method. Data were collected by ethical leadership in nursing and the general self-efficacy scale of Scherer. Data were analyzed using SPSS software (version 16) through descriptive and inferential statistics. Results: There was no statistically significant association between the ethical leadership of head nurses and the general self-efficacy of nurses. The general self-efficacy of formal nurses was higher than contractual (P=0.04) and resident nurses (P=0.02), and the self-efficacy of ICU nurses was higher than those working in burn units (P=0.03). Moreover, general self-efficacy decreased with work experience (P&lt;0.05). Considering the output of the artificial neural network, the normalized significance scores of the independent variables on the dependent variable were obtained at 100, 74.8, 56.2, 52, 47.6, 46.4, 38.1, 37.1, 27, and 13.3, for work experience, pioneering, task orientation, moralism, workplace, age, job status, gender, division of power, and education, respectively. Conclusion: Considering the significant association between some demographic variables and the general self-efficacy of nurses, nursing managers are recommended to consider these variables, especially in retraining, to improve the general self-efficacy of nurses.

Bayesian Multi-modeling of Deep Neural Nets for Probabilistic Crop Yield Prediction

An imperative aspect of agricultural planning is accurate yield prediction. Artificial Intelligence (AI) techniques, such as Deep Learning (DL), have been recognized as effective means for achieving practical solutions to this problem. However, these approaches most often provide deterministic estimates and do not account for the uncertainties involved in model predictions. This study presents a framework that employs the Bayesian Model Averaging (BMA) and a set of Copula functions to integrate the outputs of multiple deep neural networks, including the 3DCNN (3D Convolutional Neural Network) and ConvLSTM (Convolutional Long Short-Term Memory), and provides a probabilistic estimate of soybean crop yield over a hundred counties across three states in the United States. The results of this study show that the proposed approach produces more accurate and reliable soybean crop yield predictions than the 3DCNN and ConvLSTM networks alone while accounting for the models’ uncertainties.

Modeling and optimization of semi-continuous anaerobic co-digestion of activated sludge and wheat straw using Nonlinear Autoregressive Exogenous neural network and seagull algorithm

Anaerobic digestion of bio-wastes has a great potential to substitute fossil fuel consumption and reduce air pollution. In this study, semi-continuous anaerobic co-digestion of waste activated sludge and wheat straw using different mixing ratios under mesophilic conditions has been performed. Furthermore, modeling and optimization of anaerobic co-digestion process have been carried out using the Nonlinear Autoregressive Exogenous (NARX) neural network and Seagull optimization algorithm (SOA). The anaerobic co-digestion improved C/N ratio from 6.64 to 17.85 and enhanced the biogas production by 350% at 2% mixing ratio compared with the mono sludge digestion. Moreover, methane content of biogas ranged from 55% to 65%. The modeling results showed that the simulations of the proposed NARX neural network presented accurate results in predicting the digested sample characteristics and the biogas production. The correlation coefficient R for the data is close to 1 and the results show the stability of the optimum NARX neural network outputs for all predicted values. Moreover, it has high accuracy and effectiveness with minimum average root mean square error (RMSE) of 0.1518564. These findings can support the decision maker and stakeholders in renewable energy sector and sustainable biomass waste management.

On-line precision control of CFETR multipurpose overload robot using deformation model

The China Fusion Engineering Test Reactor (CFETR) multipurpose overload robot (CMOR) is an important part of the CFETR remote handling (RH) system. It is a type of serial robot with a redundant degree of freedom and heavy loads. Due to its long cantilever and heavy load, the deformation is large during operation. However, the deformation of the CMOR in the RH application cannot be ignored. In this paper, an online precision control algorithm using a deformation model is presented. Using this algorithm, the deformations of the CMOR at a limited number of positions are calculated by finite element analysis and used as training datasets. The joint parameters and loads are used as inputs, and the actual position of the CMOR is the output of the Back Propagation Neural Network (BPNN). The deformation model is trained by the BPNN offline, which can calculate the actual position in the workspace. The deformation of the CMOR can be accurately obtained and rapidly compensated to realize online precision control. During the RH operation in the CFETR, the deformation was updated to 2 Hz and the precision control accuracy was up to 5 mm.

Position Estimation at Zero Speed for PMSMs Using Artificial Neural Networks

This paper presents a method for shaft position estimation of a synchronous motor with permanent magnets. Zero speed and very low speed range are considered. The method uses the analysis of high-frequency currents induced by the introduction of additional voltage in the control path in the stationary coordinate system associated with the stator. An artificial neural network estimates the sine and cosine values necessary in the Park’s transformation units. This method can achieve satisfactory accuracy in the case of low asymmetry of inductance in the direct and quadrature axes of the coordinate system associated with the rotor. The TensorFlow/Keras package was used for artificial network calculations and the scikit-learn package for preprocessing. Aggregating the outputs of several artificial neural networks provides an opportunity to reduce the resultant estimation error. The use of as few as four networks has enabled the error to be reduced by approximately 20% compared to a single example network.

Predicting the outputs of finite deep neural networks trained with noisy gradients.

A recent line of works studied wide deep neural networks (DNNs) by approximating them as Gaussian processes (GPs). A DNN trained with gradient flow was shown to map to a GP governed by the neural tangent kernel (NTK), whereas earlier works showed that a DNN with an i.i.d. prior over its weights maps to the so-called neural network Gaussian process (NNGP). Here we consider a DNN training protocol, involving noise, weight decay, and finite width, whose outcome corresponds to a certain non-Gaussian stochastic process. An analytical framework is then introduced to analyze this non-Gaussian process, whose deviation from a GP is controlled by the finite width. Our contribution is threefold: (i) In the infinite width limit, we establish a correspondence between DNNs trained with noisy gradients and the NNGP, not the NTK. (ii) We provide a general analytical form for the finite width correction (FWC) for DNNs with arbitrary activation functions and depth and use it to predict the outputs of empirical finite networks with high accuracy. Analyzing the FWC behavior as a function of n, the training set size, we find that it is negligible for both the very small n regime, and, surprisingly, for the large n regime [where the GP error scales as O(1/n)]. (iii) We flesh out algebraically how these FWCs can improve the performance of finite convolutional neural networks (CNNs) relative to their GP counterparts on image classification tasks.

Explainable activity recognition in videos: Lessons learned

AbstractWe consider the following activity recognition task: given a video, infer the set of activities being performed in the video and assign each frame to an activity. This task can be solved using modern deep learning architectures based on neural networks or conventional classifiers such as linear models and decision trees. While neural networks exhibit superior predictive performance as compared with decision trees and linear models, they are also uninterpretable and less explainable. We address thisaccuracy‐explanability gapusing a novel framework that feeds the output of a deep neural network to an interpretable, tractable probabilistic model called dynamic cutset networks, and performs joint reasoning over the two to answer questions. The neural network helps achieve high accuracy while dynamic cutset networks because of their polytime probabilistic reasoning capabilities make the system more explainable. We demonstrate the efficacy of our approach by using it to build three prototype systems that solve human‐machine tasks having varying levels of difficulty using cooking videos as an accessible domain. We describe high‐level technical details and key lessons learned in our human subjects evaluations of these systems.

Using the modified k-mean algorithm with an improved teaching-learning-based optimization algorithm for feedforward neural network training

In this paper we proposed a novel procedure for training a feedforward neural network. The accuracy of artificial neural network outputs after determining the proper structure for each problem depends on choosing the appropriate method for determining the best weights, which is the appropriate training algorithm. If the training algorithm starts from a good starting point, it is several steps closer to achieving global optimization. In this paper, we present an optimization strategy for selecting the initial population and determining the optimal weights with the aim of minimizing neural network error. Teaching-learning-based optimization (TLBO) is a less parametric algorithm rather than other evolutionary algorithms, so it is easier to implement. We have improved this algorithm to increase efficiency and balance between global and local search. The improved teaching-learning-based optimization (ITLBO) algorithm has added the concept of neighborhood to the basic algorithm, which improves the ability of global search. Using an initial population that includes the best cluster centers after clustering with the modified k-mean algorithm also helps the algorithm to achieve global optimum. The results are promising, close to optimal, and better than other approach which we compared our proposed algorithm with them.

Deep Learning-Based Detection and Diagnosis of Subarachnoid Hemorrhage.

Subarachnoid hemorrhage (SAH) is one of the critical and severe neurological diseases with high morbidity and mortality. Head computed tomography (CT) is among the preferred methods for the diagnosis of SAH, which is confirmed by CT showing high-density shadow in the subarachnoid space. Analysis of these images through a deep learning-based subarachnoid hemorrhage will reduce the approximate rate of misdiagnosis in general and missed diagnosis by clinicians in particular. Deep learning-based detection of subarachnoid hemorrhage mainly includes two tasks, i.e., subarachnoid hemorrhage classification and subarachnoid hemorrhage region segmentation. However, it is difficult to effectively judge reliability of the model and classify bleeding which is based on limited predictive probability of convolutional neural network output. Moreover, deep learning-based bleeding area segmentation requires a large amount of training data to be marked in advance and the large number of network parameters makes the model training unable to reach the optimal. To resolve these problems associated with existing models, Bayesian deep learning and neural network-based hybrid model is presented in this paper to estimate uncertainty and efficiently classify subarachnoid hemorrhage. Uncertainty estimation of the proposed model helps in judging whether the model's prediction is reliable or not. Additionally, it is used to guide clinicians to find the neglected subarachnoid hemorrhage area. In addition, a teacher-student mechanism deep learning model was designed to introduce observational uncertainty estimation for semisupervised learning of subarachnoid hemorrhage. Observation uncertainty estimation detects the uncertain bleeding areas in CT images and then selects areas with high reliability. Finally, it uses these unlabeled data for model training purposes as well.

A data-driven approach to predicting the anisotropic mechanical behaviour of voided single crystals

Machine learning techniques are increasingly used to extract important physical information from a broad range of materials and to identify their processing-structure–property relationships. In this work, a neural network framework is coupled with a crystal plasticity finite element based approach to predict the deformation and ductile failure behaviour of porous FCC single crystals when subjected to multi-axial loading conditions. The work relies on 3D unit cells with a centrally located spherical void to represent the microstructure of the porous single crystal material, and on a crystallographic slip-based crystal plasticity constitutive model to describe its deformation behaviour. Stress–strain data generated by unit cell finite element simulations are relied upon to construct the neural network model. Different strategies for the neural network input and output variables and parameters are first explored so as to optimise performance and accuracy. Both proportional and non-proportional loading conditions resulting from a constant and a varying stress triaxiality during deformation, respectively, are considered. The optimum neural network strategy is shown to successfully predict the behaviour of the porous single crystal under both proportional and non-proportional loading, albeit with the void behaviour at high stress triaxialities better described than that at low triaxialities. The results also reveal that the use of tensorial quantities for both stresses and strains as input and output neural network quantities is more suitable as a form of data representation for multiaxial loading conditions than uniaxial equivalent stress and strain quantities. It was also found that the inclusion of prior knowledge as neural network input quantities in the form of reference stress–strain solutions for the void-free single crystal considerably improves the predictive capabilities of the proposed data-driven approach, even when only a very limited number of training cases was used.

A multi-modal and multi-scale emotion-enhanced inference model based on fuzzy recognition

Only the label corresponding to the maximum value of the fully connected layer is used as the output category when a neural network performs classification tasks. When the maximum value of the fully connected layer is close to the sub-maximum value, the classification obtained by considering only the maximum value and ignoring the sub-maximum value is not completely accurate. To reduce the noise and improve classification accuracy, combining the principles of fuzzy reasoning, this paper integrates all the output results of the fully connected layer with the emotional tendency of the text based on the dictionary to establish a multi-modal fuzzy recognition emotion enhancement model. The provided model considers the enhancement effect of negative words, degree adverbs, exclamation marks, and question marks based on the smallest subtree on the emotion of emotional words, and defines the global emotional membership function of emojis based on the corpus. Through comparing the results of CNN, LSTM, BiLSTM and GRU on Weibo and Douyin, it is shown that the provided model can effectively improve the text emotion recognition when the neural network output result is not clear, especially for long texts.

Artificial Neural Networks for Multiple NEA Rendezvous Missions with Continuous Thrust

The interest for near-Earth asteroids for scientific studies and, in particular, for potentially hazardous asteroids requires the space community to perform multiple-asteroid missions with close-up observations. To this end, multiple near-Earth asteroid rendezvous missions can help reduce the cost of the mission. Given the enormous number of asteroids, this work proposes a method based on artificial neural networks (ANNs) to quickly estimate the transfer time and cost between asteroids using low-thrust propulsion. The neural network output is used in a sequence-search algorithm based on a tree-search method to identify feasible sequences of asteroids to rendezvous. The rendezvous sequences are optimized by solving an optimal control problem for each leg to verify the feasibility of the transfer. The effectiveness of the presented methodology is assessed through sequences of asteroids of interest optimized using two low-thrust propulsion systems, namely solar electric propulsion and solar sailing. The results show that ANNs are able to estimate the duration and cost of low-thrust transfers with high accuracy in a modest computational time.

A novel learning algorithm based on computing the rules’ desired outputs of a TSK fuzzy neural network with non-separable fuzzy rules

In this paper, a novel learning approach to train fuzzy neural networks’ parameters based on calculating the desired outputs of their rules, is proposed. We describe the desired outputs of fuzzy rules as values that make the output error equal to the minimum. To find these desired outputs, a new constrained convex optimization problem is introduced and solved. Afterward, the parameters of fuzzy rules are trained to reduce the error between the current rules’ outputs and the estimated desired ones. Therefore, the proposed learning method avoids direct output error backpropagation, which leads to vanishing gradient and consequently getting stuck in a local optimum. Therefore, the proposed method does not need any sophisticated initialization method. This learning method is successfully utilized to train a new Takagi–Sugeno-Kang (TSK) Fuzzy Neural Network with correlated fuzzy rules. The proposed paradigm, including the proposed TSK correlation-aware architecture along with the learning method, is successfully applied to six real-world time-series predictions, regression problems, and nonlinear system identification. According to the experimental results, the performance of our proposed method outperforms other methods with a more parsimonious structure.

Branching Dueling Q-Network-Based Online Scheduling of a Microgrid With Distributed Energy Storage Systems

This letter investigates a Branching Dueling Q-Network (BDQ) based online operation strategy for a microgrid with distributed battery energy storage systems (BESSs) operating under uncertainties. The developed deep reinforcement learning (DRL) based microgrid online optimization strategy can achieve a linear increase in the number of neural network outputs with the number of distributed BESSs, which overcomes the curse of dimensionality caused by the charge and discharge decisions of multiple BESSs. Numerical simulations validate the effectiveness of the proposed method.