Output Of Neural Network Research Articles

Tunable Neural Networks for Multi-Material Image Formation from Spectral CT Measurements.

Quantitative estimation of multi-material density images is an important goal for Spectral CT imaging. However, material decomposition is a poorly-conditioned nonlinear inverse problem. Maximum-likelihood model-based material decomposition results in very noisy material density image estimates. One increasingly popular strategy for noise reduction is to apply deep neural networks for multi-material image formation. The most common loss function is mean squared error with respect to supervised target images such as ground truth or higher-dose cases. However, we believe that the mean-squared error loss function has several issues for multi-material image formation. In this work, we present a new loss function which includes multiple noise realizations with separate weights on covariance and bias for joint denoising of all material bases. By modulating these weights, it is possible to tune the image quality of neural network output images. To demonstrate our proposed approach, we conducted a simulation of a water/calcium/gadolinium spectral CT imaging scenario using a deep neural network for multi-material image denoising. Our results show that by changing the weights of our proposed loss function, it is possible to control the tradeoff between variance and bias for individual materials as well as the control over the bias coupling between materials.

Frequency domain regularization for iterative adversarial attacks

Adversarial examples have attracted more and more attentions with the prosperity of convolutional neural networks. The transferability of adversarial examples is an important property that makes black-box attacks possible in real-world applications. On the other side, many adversarial defense methods have been proposed to improve the robustness, leading to the requirement for more transferable adversarial examples. Inspired by the regularization term for network parameters at training process, we treat adversarial attacks as training process of inputs and propose regularization constraint for inputs to prevent adversarial examples from overfitting the white-box networks and enhance the transferability. Specifically, we find a universal attribute that the outputs of convolutional neural networks have consistency to the low frequencies of inputs, and based on this, we construct a frequency domain regularization to inputs for iterative attacks. In this way, our method is compatible with existing iterative attack methods and can learn more transferable adversarial examples. Extensive experiments on ImageNet validate the superiority of our method, and compared with several attacks, we achieve attack success rate improvements of 8.0% and 11.5% on average to normal models and defense methods respectively.

Determining the Rheological Parameters of Polymers Using Artificial Neural Networks.

Artificial neural networks have great prospects in solving the problems of predicting the properties of polymers. The purpose of this work was to study the possibility of using artificial neural networks to determine the rheological parameters of polymers from stress relaxation curves. The nonlinear Maxwell–Gurevich equation was used as the deformation law. The problem was solved in the MATLAB environment. The substantiation for the choice of the neural network input and output parameters was made. An algorithm for obtaining the data for neural network training was also proposed. Neural networks were trained on theoretical stress relaxation curves constructed with the Euler method. The value of the mean square error (MSE) was used as a criterion for the performance of the training. The constructed model of the artificial neural network was tested on the experimental relaxation curves of recycled polyvinyl chloride. The quality of the experimental curve approximation was quite good and was comparable with the standard methods for processing stress relaxation curves. Unlike the standard methods, when using artificial neural networks, no preliminary data smoothing was required. It is possible to use the proposed technique for processing not only relaxation curves, but also creep curves as well as processing creep tests not only in central tension, but also in bending, torsion and shear.

Experimental Evaluation and Neural Network Prediction of Fatigue Life of E-Glass Fiber Reinforced Hybrid Composite Material

The present study focuses on investigating the impact of hybrid composite content and loading parameters on the fatigue life of E-glass fiber-reinforced composite. A mathematical model of the fatigue life was developed based on experimentation and expressed as a function of the composite content (Epoxy content (80%, 60%) wt., Novolac content (20%, 40%) wt., E-Fiber Glass content (10%, 20%, and 30%) wt.), and different loads using Adaptive Neural Fuzzy Inference Systems (ANFIS). In order to validate the ANFIS model, 30 fatigue cycle experiments were conducted. Using the experimental data sets, the ANFIS was trained with a momentum algorithm and an average absolute percentage error of 6.045% was obtained and compared to the output of the conventional Artificial Neural Network (ANN) Model. The testing accuracy was then verified with 6 extra experimental data sets and the average predicting error was 5.18%. the predicted ANFIS output data were then used to discuss the effect of the composite content and loading parameters on the fatigue life. The number and shapes of cracks observed on the failed specimens were analyzed under an optical microscope. The analysis revealed that the loading impact showed the highest influence on fatigue life for all types of enforcement and the sequence of strength was E- glass fiber, epoxy, and Novolac contents, respectively.

Symmetric Encryption using Artificial Neural Network, Binary Tree Traversal, and Interleaving Salting

We have proposed a novel way of symmetric encryption utilizing interleaving salting, Binary Tree traversal, and an Artificial Neural Network in a sequential manner. We have presented the use of an Artificial Neural Network as an invertible(reversible) mathematical function so that decryption of the encrypted output may be possible. Knowledge of the encryption pipeline, salt, and neural network weights is required for decryption. As the same set of values is required for encryption and decryption, our proposed approach is a type of symmetric-key algorithm. Each user will have a unique key. Thus if a key attributing to a particular user is compromised, the integrity of the data of the remaining users will still be maintained. Our approach can be utilized to encrypt text data such as messages, documents, and letters. The encryption process consists of interleaving salting, creation of binary tree by considering the input as its level order traversal, and passing the preorder and inorder traversal of the constructed binary tree as input to the Artificial Neural Network. The output of the Artificial Neural Network would be the encrypted data. Decryption would require determining the input of an Artificial Neural Network from the output, hence solving multiple sets of linear equations and constructing a binary tree from its preorder and inorder traversal. We have then analyzed the variation of performance with the change in the input string size. The codebase of our proposed approach is publicly available at https://github.com/Atishaysjain/Symmetric-Encryption-using-ANN-and-Binary-Trees.

Breast cancer classification by a new approach to assessing deep neural network-based uncertainty quantification methods

Deep learning-based approaches have become widespread in medical fields and have achieved profound success in recent years. Nonetheless, most of these approaches cannot provide the certainty of their output numerically, and their output often has too many confidence levels. Therefore, they cannot be efficient and may even lead to irreparable damage. Accordingly, to tackle such a challenge, the Bayesian approximation and Ensemble learning techniques are deliberated as uncertainty quantification (UQ) methods. We apply and evaluate three UQ models regarding the classification of breast tumor tissue types. These methods are Mont Carlo-dropout (MCD), Bayesian Ensemble, and MCD Ensemble. Furthermore, to increase the classification's precision and eliminate the adverse effects of the smallness of the data collection in Wisconsin Diagnostic Breast Cancer (WDBC) used in the present research, transfer learning technique and pre-trained Convolutional Neural Network (DenseNet121) are considered. Three proposed models are compared based on their ability to quantify the reliability of classification according to novel performance metrics employed to assess estimated uncertainty. Our quantitative and qualitative examinations in the study demonstrate that models show high uncertainty in misclassifications, which is vital in determining the rate of medical diagnosis risks. As a result, by using these new assessment criteria, we intend to ascertain when can trust the output of the deep neural network. Moreover, in the examination, we witness more reliability in uncertainty quantification of the Bayesian Ensemble model.

GreedyFool: Multi-factor imperceptibility and its application to designing a black-box adversarial attack

Adversarial examples are well-designed input samples that include perturbations that are imperceptible to the human eye but easily mislead the output of deep neural networks (DNNs). Existing studies synthesize adversarial examples by leveraging simple metrics to penalize perturbations that lack sufficient consideration of the human visual system (HVS), producing noticeable artifacts. To explore why these perturbations are visible,four primary factors that affect the perceptibility of human eyes are summarized in this paper. Based on this investigation, we design a multi-factor metric MulFactorLoss for measuring the perceptual loss between benign examples and adversarial examples. To test the imperceptibility of the multi-factor metric, we propose a novel black-box adversarial attack that is referred to as GreedyFool. GreedyFool applies differential evolution to evaluate the effects of perturbed pixels on the confidence of a target DNN and introduces greedy approximation to automatically generate adversarial perturbations. We conduct extensive experiments on the ImageNet and CIFRA-10 datasets and a comprehensive user study with 60 participants. The experimental results demonstrate that MulFactorLoss is a more imperceptible metric than the existing pixelwise metrics, and GreedyFool achieves a 100% success rate in a black-box manner.

Innovative Application of Artificial Intelligence in the Field of Innovation and Entrepreneurship of College Students in Internet Colleges and Universities.

With the promotion and development of artificial intelligence, the computer major has become a hot major in innovation and entrepreneurship education. It is increasingly essential to carry out a refined evaluation of creativity and venture education to achieve professional differentiation. The purpose of this paper is to study and explore the innovative use of AI in the field of innovation and entrepreneurship of college students in Internet universities. In the article, AI is first introduced. AI is inherently a novel instrument of human praxis, a tool of productivity that embodies human ability and volition. Then we analyze the innovation and venture education on the ground of the BP neural network, and finally, we conduct an empirical study on the innovation and venture on the ground of the BP neural network. Innovation and entrepreneurship refer to entrepreneurial activities based on one or several innovations in technological innovation, product innovation, brand innovation, service innovation, business model innovation, management innovation, organizational innovation, market innovation, and channel innovation. The report analyzes the influencing factors of creative venture education and designs an evaluation model of creative venture education. The test shows that the output of the BP neural network is the actual value, the real worth is reasonable, and the desired worth is also reasonable. The maximum error between the two is 1.64%. The results tend to be generally satisfactory. The BP neural network model can highly generalize students' creative venture abilities, increasing the practical application value and research scope of BP neural networks.

Deep learning framework for true amplitude imaging: Effect of conditioners and initial models

ABSTRACTWe propose a workflow to correct migration amplitudes by estimating the inverse Hessian operator weights using a neural network–based framework. We train the network such that it learns the transformation between the migration output and true amplitude reflectivity constrained by different conditioners. We analyse the network output with a velocity model and with source illumination as a conditioner. Compared to the velocity model, source illumination as a conditioner performs better because source illumination encodes the geometrical spreading information and accounts for non‐stationarity. We further use the output of the deep neural network as a starting model for accelerating the convergence of iterative least‐squares reverse time migration. Using a deep learning framework, the proposed method combines the model domain and data domain least‐squares migration approaches to recover images with interpretable amplitudes, attenuated migration artefacts, better signal‐to‐noise ratio and improved resolution. We compare the output of the proposed algorithm with conventional least‐squares output and show that the proposed workflow is more robust, especially in the areas with weak illumination.

Online Trajectory Planning with Reinforcement Learning for Pedestrian Avoidance

Planning the optimal trajectory of emergency avoidance maneuvers for highly automated vehicles is a complex task with many challenges. The algorithm needs to decrease accident risk by reducing the severity and keeping the car in a controllable state. Optimal trajectory generation considering all aspects of vehicle and environment dynamics is numerically complex, especially if the object to be avoided is moving. This paper presents a hierarchical method for the avoidance of moving objects in an autonomous vehicle, where a reinforcement learning agent is responsible for local planning, while longitudinal and lateral control is performed by the low-level model-predictive controller and Stanley controllers. In the developed architecture, the agent is responsible for the optimization. It is trained in various scenarios to provide the necessary parameters for a polynomial-based path and a velocity profile in a neural network output. The vehicle performs only the first step of the trajectory, which is redesigned repeatedly by the planner based on the new state. In the training phase, the vehicle executes the entire trajectory via low-level controllers to determine the reward value, which realizes a prediction for the future. The agent receives feedback and can further improve its performance. Finally, the proposed framework was tested in a simulation environment and was also compared to human drivers’ abilities.

Constrained and unconstrained deep image prior optimization models with automatic regularization.

Deep Image Prior (DIP) is currently among the most efficient unsupervised deep learning based methods for ill-posed inverse problems in imaging. This novel framework relies on the implicit regularization provided by representing images as the output of generative Convolutional Neural Network (CNN) architectures. So far, DIP has been shown to be an effective approach when combined with classical and novel regularizers. Unfortunately, to obtain appropriate solutions, all the models proposed up to now require an accurate estimate of the regularization parameter. To overcome this difficulty, we consider a locally adapted regularized unconstrained model whose local regularization parameters are automatically estimated for additively separable regularizers. Moreover, we propose a novel constrained formulation in analogy to Morozov’s discrepancy principle which enables the application of a broader range of regularizers. Both the unconstrained and the constrained models are solved via the proximal gradient descent-ascent method. Numerical results demonstrate the robustness with respect to image content, noise levels and hyperparameters of the proposed models on both denoising and deblurring of simulated as well as real natural and medical images.

Improved AHP Model and Neural Network for Consumer Finance Credit Risk Assessment

With the rapid expansion of the consumer financial market, the credit risk problem in borrowing has become increasingly prominent. Based on the analytic hierarchy process (AHP) and the long short-term memory (LSTM) model, this paper evaluates individual credit risk through the improved AHP and the optimized LSTM model. Firstly, the characteristic information is extracted, and the financial credit risk assessment index system structure is established. The data are input into the AHP-LSTM neural network, and the index data are fused with the AHP so as to obtain the risk level and serve as the expected output of the LSTM neural network. The results of the prewarning model after training can be used for financial credit risk assessment and prewarning. Based on LendingClub and PPDAI data sets, the experiment uses the AHP-LSTM model to classify and predict and compares it with other classification methods. Experimental results show that the performance of this method is superior to other comparison methods in both data sets, especially in the case of unbalanced data sets.

Data-Driven Coordinated Charging for Electric Vehicles With Continuous Charging Rates: A Deep Policy Gradient Approach

In this article, we consider a parking lot that manages the charging processes of its parked electric vehicles (EVs). Upon arrival, each EV requests a certain amount of energy. This request should be fulfilled before the EV’s departure. It is of critical importance to coordinate the EVs’ charging rates to smooth out the load profile of the parking lot because inappropriate charging rates can lead to sharp spikes and fluctuations on the load profile, imposing negative effects on the power grid. Meanwhile, empirical studies show that many parking lots exhibit statistical patterns on EV dynamics. For example, the bulk of EVs arrives during rush hours. Therefore, in this article, we incorporate such patterns into charging rate coordination. Although the statistical patterns can be summarized from historical data, they are difficult to be analytically modeled. As a result, we adopt a model-free deep reinforcement learning approach. We also take the latest continuous charging rate control technology into consideration. The decision variables are thus continuous and a policy gradient algorithm is needed to perform reinforcement learning. Technically, we first formulate the problem as a Markov decision process (MDP) with unknown state transition probabilities. To further derive a deep policy gradient algorithm, the challenge lies in the inconsistent and state-dependent action space of the MDP model, due to the constraint to satisfy EVs’ energy demands before their scheduled departure. To tackle the challenge, we design a customized model for neural network training by extending the action space to be consistent and state independent, and revise the reward function to penalize the neural network output if it is beyond the action space of the original MDP model. With this customized model, we then develop a deep policy gradient algorithm based on the proximal policy gradient framework. Numerical results show that our algorithm outperforms the benchmarks.

Supply chains and fake news: a novel input–output neural network approach for the US food sector

Supply chains and fake news: a novel input–output neural network approach for the US food sector

CARBON FOOTPRINT OPTIMIZED DESIGN OF SUSTAINABLE REINFORCED CONCRETE COLUMNS USING DEEP LEARNING

The objective of this paper is to present a neural network (NN) model for the sustainable structural design optimization of rectangular reinforced concrete columns under biaxial bending and axial loading to minimize the embodied carbon from concrete and steel. Using the loading combination, height, and concrete class, the model predicts the section geometry and reinforcement ratio. To train the NN, a dataset of 195 million designs was generated with the OpenSees library following Eurocode. The dataset spans six concrete classes and five different column heights. Using the estimates of the embodied carbon for concrete and steel, the designs were evaluated and filtered before training. To illustrate the performance of our model, 30 columns for different loads, heights, and concrete classes were manually designed and compared with the NN output. The results showed a 24% average reduction of embodied carbon for the NN predicted designs. In addition, the manual process took approximately six minutes per design, while it took the NN 1.2 seconds for the same task.

Application of deep learning in the detection and practical evaluation of ski resort teaching area

SummaryThe teaching land of ski resorts provides learning sites for beginners of skiing. The teaching sites of different ski resorts are different. When selecting teaching scenes, we need to focus on the safety of the site. The focus of our research is to detect more suitable teaching land in the whole ski resort and evaluate its practical indicators. In order to accurately detect the teaching area in the snow field, we propose a method based on deep learning. The algorithm innovatively optimizes the convolution network. The experiment combines convolutional neural network with recurrent neural network for training, and uses the output of convolutional neural network to train recurrent neural network, so as to continuously predict the next optimal teaching area in the sequence. The purpose of training is to optimize the detection result area. The experimental results show that the model can better detect the sites suitable for teaching in the snow field.

A Neural Network Model for Business Performance Management Based on Random Matrix Theory

This paper uses random matrix theory to construct a neural network model for business performance management. The random sample covariance matrix of the random monitoring matrix is constructed, and the maximum eigenvalue and the minimum eigenvalue of the sample covariance matrix are solved. The ratio of eigenvalues is used to construct the eigenvalue detection index and determine the eigenvalue index detection threshold algorithm to judge the abnormal state of enterprise operation. The data of 66 listed Internet finance companies are selected, normalized, and correlation tested, and the index weights of each level are obtained using hierarchical analysis to derive the expected output of the BP neural network. Finally, the constructed BP neural network performance evaluation model is used for network training and simulation analysis, in which 192 data of 48 companies in the last four years are selected as training samples and 8 companies in the last four years are used as test samples to analyze the simulation output results. Using the original data instead of the main factor to join the BP neural network model, after two systematic optimizations, the final model with high accuracy, low mean square error, and low average error was formed. When applying the newly added data for testing, an accuracy of 95.98% was achieved for the ranking prediction, and the average deviation of 0.0021 points for the score prediction, which fully reflected the feasibility and adaptability of the model.

Tooth recognition of 32 tooth types by branched single shot multibox detector and integration processing in panoramic radiographs.

Purpose: The purpose of our study was to analyze dental panoramic radiographs and contribute to dentists' diagnosis by automatically extracting the information necessary for reading them. As the initial step, we detected teeth and classified their tooth types in this study. Approach: We propose single-shot multibox detector (SSD) networks with a side branch for 1-class detection without distinguishing the tooth type and for 16-class detection (i.e., the central incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar, and third molar, distinguished by the upper and lower jaws). In addition, post-processing was conducted to integrate the results of the two networks and categorize them into 32 classes, differentiating between the left and right teeth. The proposed method was applied to 950 dental panoramic radiographs obtained at multiple facilities, including a university hospital and dental clinics. Results: The recognition performance of the SSD with a side branch was better than that of the original SSD. In addition, the detection rate was improved by the integration process. As a result, the detection rate was 99.03%, the number of false detections was 0.29 per image, and the classification rate was 96.79% for 32 tooth types. Conclusions: We propose a method for tooth recognition using object detection and post-processing. The results show the effectiveness of network branching on the recognition performance and the usefulness of post-processing for neural network output.

Mesh Denoising Based on Recurrent Neural Networks

Mesh denoising is a classical task in mesh processing. Many state-of-the-art methods are still unable to quickly and robustly denoise multifarious noisy 3D meshes, especially in the case of high noise. Recently, neural network-based models have played a leading role in natural language, audio, image, video, and 3D model processing. Inspired by these works, we propose a data-driven mesh denoising method based on recurrent neural networks, which learns the relationship between the feature descriptors and the ground-truth normals. The recurrent neural network has a feedback loop before entering the output layer. By means of the self-feedback of neurons, the output of a recurrent neural network is related not only to the current input but also to the output of the previous moments. To deal with meshes with various geometric features, we use k-means to cluster the faces of the mesh according to geometric similarity and train neural networks for each category individually in the offline learning stage. Each network model, acting similar to a normal regression function, will map the geometric feature descriptor of each facet extracted from the mesh to the denoised facet normal. Then, the denoised normals are used to calculate the new feature descriptors, which become the input of the next similar regression model. In this system, three normal regression modules are cascaded to generate the last facet normals. Lastly, the model’s vertex positions are updated according to the denoised normals. A large number of visual and numerical results have demonstrated that the proposed model outperforms the state-of-the-art methods in most cases.

Modelling of cutting force and surface roughness of ultrasonic-assisted drilling using artificial neural network

This paper presents artificial neural network modelling for the thrust force in terms of maximum and mean values and the surface roughness for drilling soda glass using ultrasonic-assisted drilling. The experimental parameters are the tool concentrations (normal and high), cutting speed, and feed rate. The feedforward architecture neural network is composed of 10 hidden layers with sigmoid function and output layer with linear function. Three models are developed for each response individually and then one model for the three outputs. The models between the neural network output and the target (experimental results) for training, validation, and test data are developed, and their coefficients of regression are reasonable for this experimental data. The suitable number of hidden layers is examined with mean square error, and it is found that it decreases with increasing the number of hidden layer. The three models are developed based on one output, and the model of the three outputs is very close and good representative for the experimental results. It is concluded that the variables can be controlled and optimized by the same conditions.