Output Of Neural Network Research Articles

Estimation of sea state parameters from ship motion responses using attention-based neural networks

On-site estimation of sea state parameters is crucial for ship navigation. Extensive research has been conducted on model-based estimation utilizing ship motion responses. Model-free approaches based on machine learning (ML) have recently gained popularity, and estimation from time-series of ship motion responses using deep learning (DL) methods has given promising results. In this study, we apply the novel, attention-based neural network (AT-NN) for estimating wave height, zero-crossing period, and relative wave direction from raw time-series data of ship pitch, heave, and roll. Despite reduced input data, it has been demonstrated that the proposed approaches by modified state-of-the-art techniques (based on convolutional neural networks (CNN) for regression, multivariate long short-term memory CNN, and sliding puzzle neural network) improved estimation MSE, MAE, and NSE by up to 86%, 66%, and 56%, respectively, compared to the best performing original methods for all sea state parameters. Furthermore, the proposed technique based on AT-NN outperformed all tested methods (original and enhanced), improving estimation MSE by 94%, MAE by 74%, and NSE by 80% when considering all sea state parameters. Finally, we proposed a novel approach for interpreting the uncertainty estimation of neural network outputs based on the Monte-Carlo dropout method to enhance the model’s trustworthiness.

Intelligent Prediction for Scattering Properties Based on Multihead Attention and Target Inherent Feature Parameter

In this communication, an artificial intelligent method based on the prevailing multi-head attention mechanism for prediction of scattering properties of 2D targets is presented. To make the predicting approach independent of the incident direction of an excitation plane wave, a kind of inherent feature parameters for a specific target are defined, and applied as the output of the artificial neural network. Two types of targets are experimented, one of which is composed of polygons and the other of smooth shapes. Numerical results show that the proposed method has satisfactory prediction accuracy and computing speed, as well as good generalization ability.

Accurate prediction and compensation of machining error for large components with time-varying characteristics combining physical model and double deep neural networks

The accurate prediction and compensation of machining error for large components are always difficult due to the complex and time-varying intrinsic interactions during material removal process. The machining error generation of large components is directly related with localized tool-workpiece deformation, which is a function of the system stiffness and the machining force. Both system stiffness and machining force pattern can hardly be recognized and determined explicitly only by physical modeling due to the time-varying characteristics of machining process. Therefore, it is important to utilize the physical model to reflect the process fundamentals, and is necessary to calibrate these two critical variables by fusing the process data. In this paper, by investigating the large shaft precision grinding process, a hybrid ‘physical model + artificial intelligence’ approach is proposed. The physical model is based on the double-integral of material removal over time and space, and also considers the deformation induced by the grinding force and the grinding wheel wear evolution. Meanwhile, the double deep neural networks are built, in which a convolution neural network (CNN) is used to obtain the instant system stiffness based on process data, while the grinding force pattern of current pass is calibrated by the instant system stiffness. A long short-term memory (LSTM) network with two sub-networks is connected to CNN to predict the system stiffness and grinding force pattern for subsequent pass. The experimental results show that the proposed method can achieve higher prediction and compensation accuracy of machining error for large shaft grinding with different profiles. Furthermore, the quantitative output of the double neural networks reveals that the system stiffness decreases first and then stabilizes, and the grinding force evolution shows diverse patterns during grinding, which can be used to describe the time-varying characteristics of the grinding system during long-cycle machining process.

Possibilities of Artificial Intelligence-Enabled Feedback Control System in Robotized Gas Metal Arc Welding

In recent years, welding feedback control systems and weld quality estimation systems have been developed with the use of artificial intelligence to increase the quality consistency of robotic welding solutions. This paper introduces the utilization of an intelligent welding system (IWS) for feedback controlling the welding process. In this study, the GMAW process is controlled by a backpropagation neural network (NN). The feedback control of the welding process is controlled by the input parameters; root face and root gap, measured by a laser triangulation sensor. The NN is trained to adapt NN output parameters; wire feed and arc voltage override of the weld power source, in order to achieve consistent weld quality. The NN is trained offline with the specific parameter window in varying weld conditions, and the testing of the system is performed on separate specimens to evaluate the performance of the system. The butt-weld case is explained starting from the experimental setup to the training process of the IWS, optimization and operating principle. Furthermore, the method to create IWS for the welding process is explained. The results show that the developed IWS can adapt to the welding conditions of the seam and feedback control the welding process to achieve consistent weld quality outcomes. The method of using NN as a welding process parameter optimization tool was successful. The results of this paper indicate that an increased number of sensors could be applied to measure and control the welding process with the developed IWS.

Forecasting Value-at-Risk Using Deep Neural Network Quantile Regression

Abstract In this article, we use a deep quantile estimator, based on neural networks and their universal approximation property to examine a non-linear association between the conditional quantiles of a dependent variable and predictors. This methodology is versatile and allows both the use of different penalty functions, as well as high dimensional covariates. We present a Monte Carlo exercise where we examine the finite sample properties of the deep quantile estimator and show that it delivers good finite sample performance. We use the deep quantile estimator to forecast value-at-risk and find significant gains over linear quantile regression alternatives and other models, which are supported by various testing schemes. Further, we consider also an alternative architecture that allows the use of mixed frequency data in neural networks. This article also contributes to the interpretability of neural network output by making comparisons between the commonly used Shapley Additive Explanation values and an alternative method based on partial derivatives.

An Ensembled Framework for Human Breast Cancer Survivability Prediction Using Deep Learning.

Breast cancer is categorized as an aggressive disease, and it is one of the leading causes of death. Accurate survival predictions for both long-term and short-term survivors, when delivered on time, can help physicians make effective treatment decisions for their patients. Therefore, there is a dire need to design an efficient and rapid computational model for breast cancer prognosis. In this study, we propose an ensemble model for breast cancer survivability prediction (EBCSP) that utilizes multi-modal data and stacks the output of multiple neural networks. Specifically, we design a convolutional neural network (CNN) for clinical modalities, a deep neural network (DNN) for copy number variations (CNV), and a long short-term memory (LSTM) architecture for gene expression modalities to effectively handle multi-dimensional data. The independent models' results are then used for binary classification (long term > 5 years and short term < 5 years) based on survivability using the random forest method. The EBCSP model's successful application outperforms models that utilize a single data modality for prediction and existing benchmarks.

Effective encoder-decoder neural network for segmentation of orbital tissue in computed tomography images of Graves' orbitopathy patients.

To propose a neural network (NN) that can effectively segment orbital tissue in computed tomography (CT) images of Graves' orbitopathy (GO) patients. We analyzed orbital CT scans from 701 GO patients diagnosed between 2010 and 2019 and devised an effective NN specializing in semantic orbital tissue segmentation in GO patients' CT images. After four conventional (Attention U-Net, DeepLab V3+, SegNet, and HarDNet-MSEG) and the proposed NN train the various manual orbital tissue segmentations, we calculated the Dice coefficient and Intersection over Union for comparison. CT images of the eyeball, four rectus muscles, the optic nerve, and the lacrimal gland tissues from all 701 patients were analyzed in this study. In the axial image with the largest eyeball area, the proposed NN achieved the best performance, with Dice coefficients of 98.2% for the eyeball, 94.1% for the optic nerve, 93.0% for the medial rectus muscle, and 91.1% for the lateral rectus muscle. The proposed NN also gave the best performance for the coronal image. Our qualitative analysis demonstrated that the proposed NN outputs provided more sophisticated orbital tissue segmentations for GO patients than the conventional NNs. We concluded that our proposed NN exhibited an improved CT image segmentation for GO patients over conventional NNs designed for semantic segmentation tasks.

RETRACTED ARTICLE: Data simulation of optimal model for numerical solution of differential equations based on deep learning and genetic algorithm

Calculus equation is an important tool for mathematical research and plays an important role in most natural science research. Since the beginning of the eighteenth century, people have gradually used differential and integral equations to solve physical problems. In general, several different aspects of differential equations in the field of mathematics are concerned and studied by most scholars. However, this paper studies and establishes the optimal model for numerical solution of differential equations through deep learning and genetic algorithm. In this paper, the solution of ordinary differential equations is solved through the use of polynomial function space, while the linear combination of simple function x and its product nx can obtain multinomial function space. The space function form of polynomial is very simple, and the operation ability is very strong. Almost all functions can be approximated, and the function space can be transformed by a simple function. Through data simulation test results, it can be found that the oscillation of neural network output is stronger and stronger with the increase in depth, that is to say, the deeper depth endows the neural network with stronger oscillation properties, so for the oscillation function, the depth neural network fitting effect is better than the shallow neural network. Therefore, by combining deep learning and genetic algorithm, this paper studies and establishes the optimal model for numerical solution of differential equations, and finds that the deep neural network can largely complete data simulation.

Tunable neural networks for CT image formation.

Optimization of CT image quality typically involves balancing variance and bias. In traditional filtered back-projection, this trade-off is controlled by the filter cutoff frequency. In model-based iterative reconstruction, the regularization strength parameter often serves the same function. Deep neural networks (DNNs) typically do not provide this tunable control over output image properties. Models are often trained to minimize the expected mean squared error, which penalizes both variance and bias in image outputs but does not offer any control over the trade-off between the two. We propose a method for controlling the output image properties of neural networks with a new loss function called weighted covariance and bias (WCB). Our proposed method uses multiple noise realizations of the input images during training to allow for separate weighting matrices for the variance and bias penalty terms. Moreover, we show that tuning these weights enables targeted penalization of specific image features with spatial frequency domain penalties. To evaluate our method, we present a simulation study using digital anthropomorphic phantoms, physical simulation of CT measurements, and image formation with various algorithms. We show that the WCB loss function offers a greater degree of control over trade-offs between variance and bias, whereas mean-squared error provides only one specific image quality configuration. We also show that WCB can be used to control specific image properties including variance, bias, spatial resolution, and the noise correlation of neural network outputs. Finally, we present a method to optimize the proposed weights for a spiculated lung nodule shape discrimination task. Our results demonstrate this new image quality can control the image properties of DNN outputs and optimize image quality for task-specific performance.

Automatic semantic segmentation of the lumbar spine: Clinical applicability in a multi-parametric and multi-center study on magnetic resonance images

Significant difficulties in medical image segmentation include the high variability of images caused by their origin (multi-center), the acquisition protocols (multi-parametric), the variability of human anatomy, illness severity, the effect of age and gender, and notable other factors. This work addresses problems associated with the automatic semantic segmentation of lumbar spine magnetic resonance images using convolutional neural networks. We aimed to assign a class label to each pixel of an image, with classes defined by radiologists corresponding to structural elements such as vertebrae, intervertebral discs, nerves, blood vessels, and other tissues. The proposed network topologies represent variants of the U-Net architecture, and we used several complementary blocks to define the variants: three types of convolutional blocks, spatial attention models, deep supervision, and multilevel feature extractor. Here, we describe the topologies and analyze the results of the neural network designs that obtained the most accurate segmentation. Several proposed designs outperform the standard U-Net used as a baseline, primarily when used in ensembles, where the outputs of multiple neural networks are combined according to different strategies.

Learning assisted column generation for model predictive control based energy management in microgrids

Model predictive control is an effective approach for microgrid energy management. However, the main downside of such method is its expensive online computational cost, which is not amenable to most practical microgrid implementations. To address this issue, we propose a deep neural network assisted column generation approach that can accelerate the solution procedure of model predictive control. In each iteration, our approach leverages different deep neural networks to predict the optimal solutions of all the subproblems in column generation, which can accelerate the computation of all the subproblems and the entire process of column generation. The pre-existing knowledge of the microgrid is also exploited to guarantee the feasibility of the neural network outputs using multi-parametric programming theory. The numerical results show that our approach leads to a reduction in computational time of approximately 50% in a medium-sized microgrid compared with the full mixed integer solution based on traditional branch and bound method, while the optimality loss is only 0.02% in terms of operating costs.

Dual stage ensemble technique for intrusion detection in cloud computing

A capability of cloud-based IDS in identifying complicated and anonymous attacks is rising in the current era. However, unwanted delays hinder the detection rate. A malicious user might utilize vast quantities of computational power. The cloud provides to perform attacks both within and without the cloud. Furthermore, there are major challenges for intrusion detection due to the ease of the cloud and also the continual restructuring and movement of cloud resources. Intruder detection, feature extraction, and data processing are all included in the novel optimization-based Intrusion Detection System (IDS) paradigm that will be presented in this study. Data normalization is used to first pre-process the input data. Then, appropriate feature extraction is carried out, including the extraction of (a) raw features, (b) statistical features, then (c) higher-order statistical features using suggested kurtosis. The detection phase is then applied to the retrieved features. A two-stage ensemble method is suggested for finding intruders in clouds. Random forest (RF), Support Vector Machine (SVM), optimal Neural Network (NN), and RNN make up the suggested ensemble technique. The RF, SVM, and Optimized NN algorithms are directly fed the collected features. The output of these classifiers is then provided to the RNN classifier (i.e.), RF output to RNN1, SVM output to RNN2, and optimized NN output to RNN3. Then, the weighted average of RNN 1, 2, and 3 is considered as the final output. A Self Adaptive Salp Swarm Optimization optimizes the weights of NN for exact detection (SA-SSO). Finally, a test is conducted to confirm the developed model’s superiority.

ANN for design and fabrication of ENi-P-nano TiO2 coatings on AH36 steel

Designing optimal ENi-P-nano TiO2 with higher surface hardness is time-consuming and expensive. Thus, this study is aims to develop the optimal optimiser-based ANN model to forcast Vickers hardness for newer process parametric combinations and find the ideal combination before manufacture. Consequently, three ANN model were devised using ADAM, AdaMax, and RMSprop optimisers with identical data set for training and testing. mean Squared Error (MSE) and R 2 values were used to compare and assess their performance. Rectified Linear Unit (ReLU) is used to provide nonlinearity to the neural network's output since the activation function affects ANN model performance. The study also highlights hyperparameter setting's impact on optimiser performance. The ADAM-based ANN model was determined as optimal design through comparing the predicted and experimental Vickers hardness for the unknown combinations using Taguchi DOE. In addition, FESEM and XRD micrographs confirmed the optimal coating's morphology and phases. This study results shown the compatibility of utilising ANN-based prediction models for ENi-P-nanoTiO2 coating design and prediction, however future research much incoporates more process parameters and their level ranges.

A Neural Network Controller Design for the Mecanum Wheel Mobile Robot

Advanced controllers are an excellent choice for the trajectory tracking problem of Wheeled Mobile Robots (WMRs). However, these controllers pose a challenge to the hardware structure of WMRs due to the controller's complex structure and the large number of calculations needed. In that context, designing a controller with a simple structure and a small number of computations but good real-time performance is necessary in order to improve the tracking accuracy for the WMRs without requiring high hardware architecture. In this work, a neural network controller with a simple structure for the trajectory-tracking of a Mecanum-Wheel Mobile robot (MWMR) based on a reference controller is proposed. A two-layer feedforward neural network is designed as a tracking controller for the robot. The neural network is trained with a sample input-output data set so that the error between the neural network output and the reference control signal of the supervisory controller is minimal. The neural network parameters are trained to update over time. The simulation results verified the effectiveness of the neural network controller, whose parameters are tuned adaptively to ensure a fast convergence to the desired Bézier trajectory.

Study of Intelligent Fire Identification System Based on Back Propagation Neural Network

In order to detect and identify fire accidents accurately and efficiently, an intelligent fire identification system based on neural network algorithm is designed, which can overcome the shortcomings of single information, complex wiring, poor adaptability, etc. The characteristic extraction of sensors is adopted in the information layer to solve the problems in multi-sensor fusion. The fire data are transmitted to the main controller through LoRa wireless module and fused by back propagation neural network, which is self-learning and adaptive. The output of neural network and fuzzy inference with other factors are used for decision criteria to improve the identification accuracy. The common combustibles and various interference sources are selected for fire tests. The result shows that the detection accuracy is up to 100% and the false alarm rate is lower than 0.1%, meanwhile, the system has the advantages of fast response and high detection efficiency.

Confidence Evaluation for Machine Learning Schemes in Vehicular Sensor Networks

In this paper, we study a cooperative perception scheme in a vehicular sensor network, attempting to fuse the semantic information provided by different sensors at multiple vehicles, so as to expand the vehicle’s perception range, eliminate blind spots, improve the ability to handle environmental interference and enhance the accuracy and robustness of the perception results. The key to guide the fusion process is the evaluation of the confidence levels of the outputs provided by various machine learning schemes implemented at individual sensors in the vehicular sensor network. We first propose an evaluation criterion termed as Environmental Sensitivity (ES), which is used to measure the sensitivity of the network to environmental changes. Based on the ES, we further evaluate the confidence of the perception output of neural networks and quantify the confidence level considering the abnormal level of the input data, the general performance of the perception algorithm, the detection performance and the ES of the network. Semantic information fusion algorithm is then developed based upon the confidence levels. Experiment results are provided to validate the proposed fusion method in various scenarios.

Bayesian Inference and Dynamic Neural Feedback Promote the Clinical Application of Intelligent Congenital Heart Disease Diagnosis

Congenital heart disease (CHD) is the leading cause of infant death. An artificial intelligence (AI)-based CHD diagnosis network (CHDNet) is an echocardiogram video-based binary classification model that judges whether echocardiogram videos contain heart defects. Existing CHDNets have shown performances comparable to or even better than medical experts, but their unreliability on cases outside of the training set has become the main bottleneck for their deployment. This is a common problem for most AI-based diagnostic approaches. Here, to overcome this challenge, we present two essential mechanisms—Bayesian inference and dynamic neural feedback—to respectively measure and improve the diagnostic reliability of AI. The former easily makes the neural network output its reliability instead of a single prediction result, while the latter is a computational neural feedback cell that allows the neural network to feed knowledge from the output layer back to the shallow layers and enables the neural network to selectively activate relevant neurons. To evaluate the effectiveness of these two mechanisms, we trained CHDNets on 4151 echocardiogram videos containing three common CHD defects and tested them on an internal test set of 1037 echocardiogram videos and an external set of 692 videos that were newly collected from other cardiovascular imaging devices. Each echocardiogram video corresponds to a unique patient and a unique visit. We demonstrate on various neural network architectures how the reliability obtained by Bayesian inference interprets and quantifies the significant performance difference between internal and external test sets of neural networks, and how the devised feedback cell helps the neural networks to maintain high accuracy and reliability, despite the input being corrupted by noise or when using an external test set.

Real time Driver’s Drowsiness Detection by Convolution Neural Network (CNN) of Deep Learning Approach

Statistics have shown that 20% of all road accidents are fatigue-related, and drowsy detection is a car safety algorithm that can alert a snoozing driver in hopes of preventing an accident. This work will propose real-time drowsiness detection; this approach is based on Convolution Neural Network (CNN) of Deep Learning. Which is aimed to implement driver’s behavior-based drowsiness detection scenario. Convolution Neural Network (CNN) for learning effective features or facial landmark input to detecting drowsiness by given an input video of driver. A common global face which is not capable enough to extracting effective facial landmarks and features, like facial movements and head gestures, which are strictly important for learning. This proposed work consists Convolution Neural Network (CNN) for attaining well-aligned facial movements and head gestures important for reliable detection. The output of neural network is integrated and feed to classifier for drowsiness detection.

The Generation of Visually Credible Adversarial Examples with Genetic Algorithms

An adversarial example is an input that a neural network misclassifies although the input differs only slightly from an input that the network classifies correctly. Adversarial examples are used to augment neural network training data, measure the vulnerability of neural networks, and provide intuitive interpretations of neural network output that humans can understand. Although adversarial examples are defined in the literature as similar to authentic input from the perspective of humans, the literature measures similarity with mathematical norms that are not scientifically correlated with human perception. Our main contributions are to construct a genetic algorithm (GA) that generates adversarial examples more similar to authentic input than do existing methods and to demonstrate with a survey that humans perceive those adversarial examples to have greater visual similarity than existing methods. The GA incorporates a neural network, and we test many parameter sets to determine which fitness function, selection operator, mutation operator, and neural network generate adversarial examples most visually similar to authentic input. We establish which mathematical norms are most correlated with human perception, which permits future research to incorporate the human perspective without testing many norms or conducting intensive surveys with human subjects. We also document a tradeoff between speed and quality in adversarial examples generated by GAs and existing methods. Although existing adversarial methods are faster, a GA provides higher-quality adversarial examples in terms of visual similarity and feasibility of adversarial examples. We apply the GA to the Modified National Institute of Standards and Technology (MNIST) and Canadian Institute for Advanced Research (CIFAR-10) datasets.

Visual Parking Occupancy Detection Using Extended Contextual Image Information via a Multi-Branch Output ConvNeXt Network.

Along with society's development, transportation has become a key factor in human daily life, increasing the number of vehicles on the streets. Consequently, the task of finding free parking slots in metropolitan areas can be dramatically challenging, increasing the chance of getting involved in an accident and the carbon footprint, and negatively affecting the driver's health. Therefore, technological resources to deal with parking management and real-time monitoring have become key players in this scenario to speed up the parking process in urban areas. This work proposes a new computer-vision-based system that detects vacant parking spaces in challenging situations using color imagery processed by a novel deep-learning algorithm. This is based on a multi-branch output neural network that maximizes the contextual image information to infer the occupancy of every parking space. Every output infers the occupancy of a specific parking slot using all the input image information, unlike existing approaches, which only use a neighborhood around every slot. This allows it to be very robust to changing illumination conditions, different camera perspectives, and mutual occlusions between parked cars. An extensive evaluation has been performed using several public datasets, proving that the proposed system outperforms existing approaches.