Output Of Neural Network Research Articles

NEURAL-NETWORK MODEL OF SOFTWARE QUALITY PREDICTION BASED ON QUALITY ATTRIBUTES

The paper proposes a neural-network model of software quality prediction based on quality attributes. The proposedmodel differs from the known models, because it provides considering the importance of each quality attribute and their interactionwithin each software quality characteristic. The artificial neural network (ANN) outputs correspond to the values of software qualitycharacteristics (functional suitability, performance efficiency, usability, reliability, compatibility, security, maintainabi lity, portability).The artificial neural network (ANN) outputs make it possible assessing the total impact of quality attributes on software qualitycharacteristics

Model-based machine learning for the recovery of lateral dose profiles of small photon fields in magnetic field

Objective. To investigate the feasibility to train artificial neural networks (NN) to recover lateral dose profiles from detector measurements in a magnetic field. Approach. A novel framework based on a mathematical convolution model has been proposed to generate measurement-less training dataset. 2D dose deposition kernels and detector lateral fluence response functions of two air-filled ionization chambers and two diode-type detectors have been simulated without magnetic field and for magnetic field B = 0.35 and 1.5 T. Using these convolution kernels, training dataset consisting pairs of dose profiles and signal profiles were computed for a total of 108 2D photon fluence profiles (80% training/20% validation). The NN were tested using three independent datasets, where the second test dataset has been obtained from simulations using realistic phase space files of clinical linear accelerator and the third test dataset was measured at a conventional linac equipped with electromagnets. Main results. The convolution kernels show magnetic field dependence due to the influence of the Lorentz force on the electron transport in the water phantom and detectors. The NN show good performance during training and validation with mean square error reaching a value of 1e-6 or smaller. The corresponding correlation coefficients R reached the value of 1 for all models indicating an excellent agreement between expected and predicted The comparisons between and using the three test datasets resulted in gamma indices (1 mm/1% global) <1 for all evaluated data points. Significance. Two verification approaches have been proposed to warrant the mathematical consistencies of the NN outputs. Besides offering a correction strategy not existed so far for relative dosimetry in a magnetic field, this work could help to raise awareness and to improve understanding on the distortion of detector’s signal profiles by a magnetic field.

Control of Variance and Bias in CT Image Processing with Variational Training of Deep Neural Networks.

Optimization of CT image quality typically involves balancing noise and bias. In filtered back-projection, this trade-off is controlled by the particular filter and cutoff frequency. In penalized-likelihood iterative reconstruction, the penalty weight serves the same function. Deep neural networks typically do not provide this tuneable control over output image properties. Models are often trained to minimize mean squared error which penalizes both variance and bias in image outputs but does not offer any control over the trade-off between the two. In this work, we propose a method for controlling the output image properties of neural networks with a new loss function called weighted covariance and bias (WCB). Our proposed method includes separate weighting parameters to control the relative importance of noise or bias reduction. Moreover, we show that tuning these weights enables targeted penalization of specific image features (e.g. spatial frequencies). To evaluate our method, we present a simulation study using digital anthropormorphic phantoms, physical simulation of non-ideal CT data, and image formation with various algorithms. We show that WCB offers a greater degree of control over trade-offs between variance and bias whereas MSE has only one configuration. We also show that WCB can be used to control specific image properties including variance, bias, spatial resolution, and the noise correlation of neural network outputs. Finally, we present a method to optimize the proposed weights for stimulus detectability. Our results demonstrate the potential for this new capability to control the image properties of DNN outputs and optimize image quality for the task-specific applications.

CSI-Based Multi-Antenna and Multi-Point Indoor Positioning Using Probability Fusion

Channel state information (CSI)-based fingerprinting via neural networks (NNs) is a promising approach to enable accurate indoor and outdoor positioning of user equipment (UE), even under challenging propagation conditions. In this paper, we propose a positioning pipeline for wireless LAN MIMO-OFDM systems which uses uplink CSI measurements obtained from one or more unsynchronized access points (APs). For each AP receiver, novel features are first extracted from the CSI that are robust to system impairments arising in real-world transceivers. These features are the inputs to a NN that extracts a probability map indicating the likelihood of a UE being at a given grid point. The NN output is then fused across multiple APs to provide a final position estimate. We provide experimental results with real-world indoor measurements under line-of-sight (LoS) and non-LoS propagation conditions for an 80MHz bandwidth IEEE 802.11ac system using a two-antenna transmit UE and two AP receivers each with four antennas. Our approach is shown to achieve centimeter-level median distance error, an order of magnitude improvement over a conventional baseline.

Handwashing quality assessment via deep learning: a modelling study for monitoring compliance and standards in hospitals and communities

BackgroundHand hygiene can be a simple, inexpensive, and effective method for preventing the spread of infectious diseases. However, a reliable and consistent method for monitoring adherence to the guidelines within and outside healthcare settings is challenging. The aim of this study was to provide an approach for monitoring handwashing compliance and quality in hospitals and communities. MethodsWe proposed a deep learning algorithm comprising three-dimensional convolutional neural networks (3D CNNs) and used 230 standard handwashing videos recorded by healthcare professionals in the hospital or at home for training and internal validation. An assessment scheme with a probability smoothing method was also proposed to optimize the neural network's output to identify the handwashing steps, measure the exact duration, and grade the standard level of recognized steps. Twenty-two videos by healthcare professionals in another hospital and 28 videos recorded by civilians in the community were used for external validation. ResultsUsing a deep learning algorithm and an assessment scheme, combined with a probability smoothing method, each handwashing step was recognized (ACC ranged from 90.64% to 98.87% in the hospital and from 87.39% to 96.71% in the community). An assessment scheme measured each step's exact duration, and the intraclass correlation coefficients were 0.98 (95% CI: 0.97–0.98) and 0.91 (95% CI: 0.88–0.93) for the total video duration in the hospital and community, respectively. Furthermore, the system assessed the quality of handwashing, similar to the expert panel (kappa = 0.79 in the hospital; kappa = 0.65 in the community). ConclusionsThis work developed an algorithm to directly assess handwashing compliance and quality from videos, which is promising for application in healthcare settings and communities to reduce pathogen transmission.

Global disparity bias in ophthalmology artificial intelligence applications

Machine learning (ML) is a branch of artificial intelligence (AI) that performs a classification, prediction and/or optimisation task. Similar to brain neurons, neural networks output a label after multiple information...

Adaptive neural network output feedback robust control of electromechanical servo system with backlash compensation and disturbance rejection

In this paper, the high-performance tracking control of electromechanical servo systems is concerned. A novel neural network state observer is designed to observe the unknown states. Compared with existing neural network observers, the proposed observer has higher observation accuracy and better robustness. The addition of a fixed-weight single-node neural network can effectively improve the approximation ability of the double-layer neural network without adding a huge amount of calculation. With the addition of the new gain adjustment terms, the observer can still achieve high observation accuracy when the neural network approximation performance is poor, and the observation error can be kept arbitrarily small. To cope with the inherent explosion of the complexity problem in the classical backstepping method in controller design, a command filter is utilized. Compared with other results, the command filtering error has also been considered, and compensating signals are designed to eliminate it. The Lyapunov function is used to show the stability of the controller. Extensive comparative simulations and experimental results verify the effectiveness and advancement of the proposed control strategy compared with other controllers.

An Artificial Neural Network Approach for Solving Inverse Kinematics Problem for an Anthropomorphic Manipulator of Robot SAR-401

The paper proposes a new design of an artificial neural network for solving the inverse kinematics problem of the anthropomorphic manipulator of robot SAR-401. To build a neural network (NN), two sets were used as input data: generalized coordinates of the manipulator and elements of a homogeneous transformation matrix obtained by solving a direct kinematics problem based on the Denavi–Hartenberg notation. According to the simulation results, the NN based on the homogeneous transformation matrix showed the best accuracy. However, the accuracy was still insufficient. To increase the accuracy, a new NN design was proposed. It consists of adding a so-called “correctional” NN, the input of which is fed the same elements of the homogeneous transformation matrix and additionally the output of the first NN. The proposed design based on the correctional NN allowed the accuracy to increase two times. The application of the developed NN approach was carried out on a computer model of the manipulator in MATLAB, on the SAR-401 robot simulator, as well as on the robot itself.

The selection of an optimal segmentation region in physiological signals

AbstractPhysiological signals are often corrupted by noisy sources. Usually, artificial intelligence algorithms analyze the whole signal, regardless of its varying quality. Instead, experienced cardiologists search for a high‐quality signal segment, where more accurate conclusions can be draw. We propose a methodology that simultaneously selects the optimal processing region of a physiological signal and determines its decoding into a state sequence of physiologically meaningful events. Our approach comprises two phases. First, the training of a neural network that then enables the estimation of the state probability distribution of a signal sample. Second, the use of the neural network output within an integer program. The latter models the problem of finding a time window by maximizing a likelihood function defined by the user. Our method was tested and validated in two types of signals, the phonocardiogram and the electrocardiogram. In phonocardiogram and electrocardiogram segmentation tasks, the system's sensitivity increased on average from 95.1% to 97.5% and from 78.9% to 83.8%, respectively, when compared to standard approaches found in the literature.

Fine-grained calibrated double-attention convolutional network for left ventricular segmentation

Objective. Left ventricular (LV) segmentation of cardiac magnetic resonance imaging (MRI) is essential for diagnosing and treating the early stage of heart diseases. In convolutional neural networks, the target information of the LV in feature maps may be lost with convolution and max-pooling, particularly at the end of systolic. Fine segmentation of ventricular contour is still a challenge, and it may cause problems with inaccurate calculation of clinical parameters (e.g. ventricular volume). In order to improve the similarity of the neural network output and the target segmentation region, in this paper, a fine-grained calibrated double-attention convolutional network (FCDA-Net) is proposed to finely segment the endocardium and epicardium from ventricular MRI. Approach. FCDA-Net takes the U-net as the backbone network, and the encoder-decoder structure incorporates a double grouped-attention module that is constructed by a fine calibration spatial attention module (fcSAM) and a fine calibration channel attention module (fcCAM). The double grouped-attention mechanism enhances the expression of information in both spatial and channelwise feature maps to achieve fine calibration. Main Results. The proposed approach is evaluated on the public MICCAI 2009 challenge dataset, and ablation experiments are conducted to demonstrate the effect of each grouped-attention module. Compared with other advanced segmentation methods, FCDA-Net can obtain better LV segmentation performance. Significance. The LV segmentation results of MRI can be used to perform more accurate quantitative analysis of many essential clinical parameters and it can play an important role in image-guided clinical surgery.

Super Resolution for Noisy Images Using Convolutional Neural Networks

The images in high resolution contain more useful information than the images in low resolution. Thus, high-resolution digital images are preferred over low-resolution images. Image super-resolution is one of the principal techniques for generating high-resolution images. The major advantages of super-resolution methods are that they are economical, independent of the image capture devices, and can be statically used. In this paper, a single-image super-resolution network model based on convolutional neural networks is proposed by combining conventional autoencoder and residual neural network approaches. A convolutional neural network-based dictionary method is used to train low-resolution input images for high-resolution images. In addition, a linear refined unit thresholds the convolutional neural network output to provide a better low-resolution image dictionary. Autoencoders aid in the removal of noise from images and the enhancement of their quality. Secondly, the residual neural network model processes it further to create a high-resolution image. The experimental results demonstrate the outstanding performance of our proposed method compared to other traditional methods. The proposed method produces clearer and more detailed high-resolution images, as they are important in real-life applications. Moreover, it has the advantage of combining convolutional neural network-based dictionary learning, autoencoder image enhancement, and noise removal. Furthermore, residual neural network training with improved preprocessing creates an efficient and versatile single-image super-resolution network.

On quadrature rules for solving Partial Differential Equations using Neural Networks

Neural Networks have been widely used to solve Partial Differential Equations. These methods require to approximate definite integrals using quadrature rules. Here, we illustrate via 1D numerical examples the quadrature problems that may arise in these applications and propose several alternatives to overcome them, namely: Monte Carlo methods, adaptive integration, polynomial approximations of the Neural Network output, and the inclusion of regularization terms in the loss. We also discuss the advantages and limitations of each proposed numerical integration scheme. We advocate the use of Monte Carlo methods for high dimensions (above 3 or 4), and adaptive integration or polynomial approximations for low dimensions (3 or below). The use of regularization terms is a mathematically elegant alternative that is valid for any spatial dimension; however, it requires certain regularity assumptions on the solution and complex mathematical analysis when dealing with sophisticated Neural Networks.

Applying self-supervised learning for semantic cloud segmentation of all-sky images

Abstract. Semantic segmentation of ground-based all-sky images (ASIs) can provide high-resolution cloud coverage information of distinct cloud types, applicable for meteorology-, climatology- and solar-energy-related applications. Since the shape and appearance of clouds is variable, and there is high similarity between cloud types, a clear classification is difficult. Therefore, most state-of-the-art methods focus on the distinction between cloudy and cloud-free pixels without taking into account the cloud type. On the other hand, cloud classification is typically determined separately at the image level, neglecting the cloud's position and only considering the prevailing cloud type. Deep neural networks have proven to be very effective and robust for segmentation tasks; however they require large training datasets to learn complex visual features. In this work, we present a self-supervised learning approach to exploit many more data than in purely supervised training and thus increase the model's performance. In the first step, we use about 300 000 ASIs in two different pretext tasks for pretraining. One of them pursues an image reconstruction approach. The other one is based on the DeepCluster model, an iterative procedure of clustering and classifying the neural network output. In the second step, our model is fine-tuned on a small labeled dataset of 770 ASIs, of which 616 are used for training and 154 for validation. For each of them, a ground truth mask was created that classifies each pixel into clear sky or a low-layer, mid-layer or high-layer cloud. To analyze the effectiveness of self-supervised pretraining, we compare our approach to randomly initialized and pretrained ImageNet weights using the same training and validation sets. Achieving 85.8 % pixel accuracy on average, our best self-supervised model outperforms the conventional approaches of random (78.3 %) and pretrained ImageNet initialization (82.1 %). The benefits become even more evident when regarding precision, recall and intersection over union (IoU) of the respective cloud classes, where the improvement is between 5 and 20 percentage points. Furthermore, we compare the performance of our best model with regards to binary segmentation with a clear-sky library (CSL) from the literature. Our model outperforms the CSL by over 7 percentage points, reaching a pixel accuracy of 95 %.

Ultrafast end-to-end protein structure prediction enables high-throughput exploration of uncharacterized proteins

Deep learning-based prediction of protein structure usually begins by constructing a multiple sequence alignment (MSA) containing homologs of the target protein. The most successful approaches combine large feature sets derived from MSAs, and considerable computational effort is spent deriving these input features. We present a method that greatly reduces the amount of preprocessing required for a target MSA, while producing main chain coordinates as a direct output of a deep neural network. The network makes use of just three recurrent networks and a stack of residual convolutional layers, making the predictor very fast to run, and easy to install and use. Our approach constructs a directly learned representation of the sequences in an MSA, starting from a one-hot encoding of the sequences. When supplemented with an approximate precision matrix, the learned representation can be used to produce structural models of comparable or greater accuracy as compared to our original DMPfold method, while requiring less than a second to produce a typical model. This level of accuracy and speed allows very large-scale three-dimensional modeling of proteins on minimal hardware, and we demonstrate this by producing models for over 1.3 million uncharacterized regions of proteins extracted from the BFD sequence clusters. After constructing an initial set of approximate models, we select a confident subset of over 30,000 models for further refinement and analysis, revealing putative novel protein folds. We also provide updated models for over 5,000 Pfam families studied in the original DMPfold paper.

Prediction of experimental thermal performance of new designed cold plate for electric vehicles’ Li-ion pouch-type battery with artificial neural network

Since liquid-based thermal management systems are usually preferred methods for battery electric vehicles and cold plates are generally preferred to circulate the coolant, studies on their design are becoming increasingly essential. Besides, it seems useful to work artificial intelligence approaches to evaluate different battery thermal management systems, as it is known that the use of artificial intelligence is increasing in many applications today. The aim of this paper is to build up an artificial neural network model due to predict average battery temperature and maximum temperature difference on the battery surface which are also the artificial neural network outputs. The model inputs are depth of discharge, coolant flow rate (0.1, 0.6 and 1.1 l/min), discharge rate (1C- 5C), coolant inlet temperature (15 , 25 and 35 °C). It is developed for a serpentine tubed cold plate, and mini channeled one which has novel design. To shorten the training time, after the optimization of the data set, a total of 270 data sets were utilized for training, validation, and test phases. In addition, the developed model predicts successfully average battery temperature and maximum temperature difference on the battery surface in the 10% error band range. Finally, the maximum margin of deviation and R values are 7.3% and 0.997%, respectively.

Bayesian neural network priors for edge-preserving inversion

&lt;p style='text-indent:20px;'&gt;We consider Bayesian inverse problems wherein the unknown state is assumed to be a function with discontinuous structure a priori. A class of prior distributions based on the output of neural networks with heavy-tailed weights is introduced, motivated by existing results concerning the infinite-width limit of such networks. We show theoretically that samples from such priors have desirable discontinuous-like properties even when the network width is finite, making them appropriate for edge-preserving inversion. Numerically we consider deconvolution problems defined on one- and two-dimensional spatial domains to illustrate the effectiveness of these priors; MAP estimation, dimension-robust MCMC sampling and ensemble-based approximations are utilized to probe the posterior distribution. The accuracy of point estimates is shown to exceed those obtained from non-heavy tailed priors, and uncertainty estimates are shown to provide more useful qualitative information.&lt;/p&gt;

Flush Air Data Sensing Based on Dimensionless Input and Output Neural Networks With Less Data

Neural networks have the ability to deal with the flush air data sensing (FADS) system of various vehicles. However, the demand for large quantities of training data limits its application. To overcome the problem, this paper develops a FADS algorithm called dimensionless input and output neural networks FADS (DIO-NNFADS) to estimate air data states. The DIO-NNFADS is utilized to approximate the aerodynamic model defined by dimensional analysis, which decouples the freestream static pressure. Thus, trained by less data from a single flight profile, the DIO-NNFADS can achieve good accuracy in the entire flight envelope, effectively reducing the training data for neural networks. The Mach number, angle of attack, angle of sideslip, and the pressure coefficients are directly output by the DIO-NNFADS. And the static pressure and dynamic pressure are solved by the equations composed of the measured pressures and pressure coefficients. The proposed FADS algorithm is verified on a simplified supersonic model through numerical simulation. Results show that the algorithm can estimate the Mach number within the relative error of 2.9&#x0025;, static pressure and dynamic pressure within the relative error of 6.2&#x0025;, and the angle of incidence within the absolute error of 0.4&#x00B0; in the entire flight envelope. Besides, the optimal size of the training data set for the DIO-NNFADS is discussed. Furthermore, the influence of port layout and selection is analyzed, and the algorithm also shows good performance for a port layout without stagnation point.

A physics-inspired neural network to solve partial differential equations - application in diffusion-induced stress.

Analyzing and predicting diffusion-induced stress are of paramount importance in understanding the structural durability of lithium- and sodium-ion batteries, which generally require solving initial-boundary value problems, involving partial differential equations (PDEs) for mechanical equilibrium and mass transport. Due to the complexity and nonlinear characteristics of the initial-boundary value problems, numerical methods, such as finite difference, finite element, spectral analysis, and so forth, have been used. In this work, we propose two whole loss functions as the sum of the residuals of the PDEs, initial conditions and boundary conditions for the problems with decoupling and coupling between diffusion and stress, respectively, and apply a physics-inspired neural network under the framework of DeepXDE to solve diffusion-induced stress in an elastic sphere in contrast to traditional numerical methods. Using time-space coordinates as inputs and displacement and the solute concentration as outputs of artificial neural networks, we solve the spatiotemporal evolution of the displacement and the solute concentration in the elastic sphere for both the decoupling and coupling problems. The numerical results from the physics-inspired neural network are validated by analytical solutions and a finite element simulation using the COMSOL package. The method developed in this work opens an approach to analyze the stress evolution in electrodes due to electrochemical cycling.

ROFT: Real-Time Optical Flow-Aided 6D Object Pose and Velocity Tracking

6D object pose tracking has been extensively studied in the robotics and computer vision communities. The most promising solutions, leveraging on deep neural networks and/or filtering and optimization, exhibit notable performance on standard benchmarks. However, to our best knowledge, these have not been tested thoroughly against fast object motions. Tracking performance in this scenario degrades significantly, especially for methods that do not achieve real-time performance and introduce non negligible delays. In this work, we introduce ROFT, a Kalman filtering approach for 6D object pose and velocity tracking from a stream of RGB-D images. By leveraging real-time optical flow, ROFT synchronizes delayed outputs of low frame rate Convolutional Neural Networks for instance segmentation and 6D object pose estimation with the RGB-D input stream to achieve fast and precise 6D object pose and velocity tracking. We test our method on a newly introduced photorealistic dataset, Fast-YCB, which comprises fast moving objects from the YCB model set, and on the dataset for object and hand pose estimation HO-3D. Results demonstrate that our approach outperforms state-of-the-art methods for 6D object pose tracking, while also providing 6D object velocity tracking. A video showing the experiments is provided as supplementary material.

A prediction model for stock market based on the integration of independent component analysis and Multi-LSTM

&lt;abstract&gt;&lt;p&gt;In this paper, we investigate the statistical behaviors of the stock market complex network. A hybrid model is proposed to predict the variations of five stock prices in the securities plate sub-network. This model integrates independent component analysis (ICA) and multivariate long short-term memory (Multi-LSTM) neural network to analyze the trading noise and improve the prediction accuracy of stock prices in the sub-network. Firstly, we apply ICA to deconstruct the original dataset and remove the independent components that represent the trading noise. Secondly, the rest of the independent components are given to Multi-LSTM neural network. Finally, prediction results are reconstructed from the outputs of the Multi-LSTM neural network and the corresponding mixing matrix. The experiment results indicate that the hybrid model outperforms the benchmark approaches, especially in terms of the stock market complex network.&lt;/p&gt;&lt;/abstract&gt;