Neural Network Learning Algorithm Research Articles

Application of deep learning neural network in predicting bone mineral density from plain X-ray radiography.

Osteoporosis is defined as a systemic disease of the bone characterized by a decrease in bone strength and deterioration of bone structure at the microscopic level, leading to bone fragility and increased risk of fracture. Bone mineral density (BMD) is the preferred method for the diagnosis of osteoporosis, and dual-energy x-ray absorptiometry (DXA) is the gold standard for diagnosing osteoporosis. Conventional radiography is more suited for the screening of osteoporosis rather than diagnosis, and osteoporosis can be detected on radiographs by experienced physicians only. This study explored the possibility of predicting BMD relative to DXA using patient radiographs. A deep learning algorithm of convolutional neural network (CNN) was used for the purpose. The method includes image segmentation, CNN learning, and a convolution-based regression model (DeepDXA) that links the isolated images of the femur bone to predict BMD value. Data were obtained in a single medical center from 2006 to 2018, with a total amount of 3472 pairs of pelvis X-ray and DXA examination within 1year. The proposed workflow successfully predicted BMD values of the femur bone with the correlation coefficient (R) of 0.85 (P < 0.001) and the accuracy of 0.88 for prediction osteoporosis, a finding that could be reliably ready for further clinical use. When suspicious osteoporosis is seen on plain films using the deep learning method we developed, further referral to DXA for the definite diagnosis of osteoporosis is indicated.

In situ porosity intelligent classification of selective laser melting based on coaxial monitoring and image processing

Selective laser melting (SLM) has shown unique advantages in fabricating metal components. However, the part quality still largely suffered from the porosity defects that are not easily detected and eliminated. In this work, the objective is to realize the porosity classification based on high-speed melt pool images. A coaxial high-speed in situ monitoring system was first developed to capture the melt pool images during the multi-track and multi-layer printing process. Then, a novel image and data processing method was proposed to extract the critical and high-level melt pool features data. Three intelligent machine learning algorithms of back propagation neural network (BPNN), support vector machine (SVM), and deep belief network (DBN) were finally developed to match the features data with porosity modes. Results show that it is feasible and effective for the proposed method to realize porosity classification during the SLM process, which can provide a potential to reduce porosity defects.

Non-Intrusive Load Monitoring Based on Unsupervised Optimization Enhanced Neural Network Deep Learning

Non-intrusive load monitoring has broad application prospects because of its low implementation cost and little interference to energy users, which has been highly expected in the industrial field recently due to the development of learning algorithms. Targeting at the investigation of practical and reliable load monitoring in field implementations, a non-intrusive load disaggregation approach based on an enhanced neural network learning algorithm is proposed in this article. The presented appliance monitoring approach establishes the neural network model following the supervised learning strategy at first and then utilizes the unsupervised learning based optimization to enhance the flexibility and adaptability for diverse scenarios, leading to the improvement of disaggregation performance. By verifications on the REDD public dataset, the proposed approach is demonstrated to be with good performance in non-intrusive load monitoring. In addition to the accuracy enhancement, the proposed approach is also with good scalability, which is efficient in recognizing the newly added appliance.

Personalised predictive modelling with brain-inspired spiking neural networks of longitudinal MRI neuroimaging data and the case study of dementia

Background:Longitudinal neuroimaging provides spatiotemporal brain data (STBD) measurement that can be utilised to understand dynamic changes in brain structure and/or function underpinning cognitive activities. Making sense of such highly interactive information is challenging, given that the features manifest intricate temporal, causal relations between the spatially distributed neural sources in the brain. Methods:The current paper argues for the advancement of deep learning algorithms in brain-inspired spiking neural networks (SNN), capable of modelling structural data across time (longitudinal measurement) and space (anatomical components). The paper proposes a methodology and a computational architecture based on SNN for building personalised predictive models from longitudinal brain data to accurately detect, understand, and predict the dynamics of an individual’s functional brain state. The methodology includes finding clusters of similar data to each individual, data interpolation, deep learning in a 3-dimensional brain-template structured SNN model, classification and prediction of individual outcome, visualisation of structural brain changes related to the predicted outcomes, interpretation of results, and individual and group predictive marker discovery. Results:To demonstrate the functionality of the proposed methodology, the paper presents experimental results on a longitudinal magnetic resonance imaging (MRI) dataset derived from 175 older adults of the internationally recognised community-based cohort Sydney Memory and Ageing Study (MAS) spanning 6 years of follow-up. Significance:The models were able to accurately classify and predict 2 years ahead of cognitive decline, such as mild cognitive impairment (MCI) and dementia with 95% and 91% accuracy, respectively. The proposed methodology also offers a 3-dimensional visualisation of the MRI models reflecting the dynamic patterns of regional changes in white matter hyperintensity (WMH) and brain volume over 6 years. Conclusion:The method is efficient for personalised predictive modelling on a wide range of neuroimaging longitudinal data, including also demographic, genetic, and clinical data. As a case study, it resulted in finding predictive markers for MCI and dementia as dynamic brain patterns using MRI data.

A data-driven T2 relaxation analysis approach for myelin water imaging: Spectrum analysis for multiple exponentials via experimental condition oriented simulation (SAME-ECOS).

The decomposition of multi-exponential decay data into a T2 spectrum poses substantial challenges for conventional fitting algorithms, including non-negative least squares (NNLS). Based on a combination of the resolution limit constraint and machine learning neural network algorithm, a data-driven and highly tailorable analysis method named spectrum analysis for multiple exponentials via experimental condition oriented simulation (SAME-ECOS) was proposed. The theory of SAME-ECOS was derived. Then, a paradigm was presented to demonstrate the SAME-ECOS workflow, consisting of a series of calculation, simulation, and model training operations. The performance of the trained SAME-ECOS model was evaluated using simulations and six in vivo brain datasets. The code is available at https://github.com/hanwencat/SAME-ECOS. Using NNLS as the baseline, SAME-ECOS achieved over 15% higher overall cosine similarity scores in producing the T2 spectrum, and more than 10% lower mean absolute error in calculating the myelin water fraction (MWF), as well as demonstrated better robustness to noise in the simulation tests. Applying to in vivo data, MWF from SAME-ECOS and NNLS was highly correlated among all study participants. However, a distinct separation of the myelin water peak and the intra/extra-cellular water peak was only observed in the mean T2 spectra determined using SAME-ECOS. In terms of data processing speed, SAME-ECOS is approximately 30 times faster than NNLS, achieving a whole-brain analysis in 3 min. Compared with NNLS, the SAME-ECOS method yields much more reliable T2 spectra in a dramatically shorter time, increasing the feasibility of multi-component T2 decay analysis in clinical settings.

Handwritten Documents Validation Using Pattern Recognition and Transfer Learning

Handwritten documents in an Enterprise Resource Planning (ERP) system can come from different sources and usually have different designs, sizes, and subjects (i.e. bills, checks, invoices, etc.). Given these documents were filled manually, they have to be inspected to detect various kinds of issues (missing signature or stamp, missing name, etc.) before being saved in the ERP system or processed by an OCR engine. In this paper, the authors present a transfer learning approach to detect issues in scanned handwritten documents, using an award-winning deep convolutional neural network (InceptionV3) and different machine learning algorithms such as Logistic Regression (LR), Support Vector Machine (SVM) and Naive Bayes (NB). The experiment shows that the combination of InceptionV3 and LR got an accuracy of 91.8% for missing stamp detection. This can allow using this approach in an ERP system as an automatic verification procedure in a document processing flow.

Analysis on frosting of heat exchanger and numerical simulation of heat transfer characteristics using BP neural network learning algorithm.

The study is aimed at the frosting problem of the air source heat pump in the low temperature and high humidity environment, which reduces the service life of the system. First, the frosting characteristics at the evaporator side of the air source heat pump system are analyzed. Then, a new defrost technology is proposed, and dimensional theory and neural network are combined to predict the transfer performance of the new system. Finally, an adaptive network control algorithm is proposed to predict the frosting amount. This algorithm optimizes the traditional neural network algorithm control process, and it is more flexible, objective, and reliable in the selection of the hidden layer, the acquisition of the optimal function, and the selection of the corresponding learning rate. Through model performance, regression analysis, and heat transfer characteristics simulation, the effectiveness of this method is further confirmed. It is found that, the new air source heat pump defrost system can provide auxiliary heat, effectively regulating the temperature and humidity. The mean square error is 0.019827, and the heat pump can operate efficiently under frosting conditions. The defrost system is easy to operate, and facilitates manufactures designing for different regions under different conditions. This research provides reference for energy conservation, emission reduction, and sustainable economic development.

Extreme events in globally coupled chaotic maps

Understanding and predicting uncertain things are the central themes of scientific evolution. Human beings revolve around these fears of uncertainties concerning various aspects like a global pandemic, health, finances, to name but a few. Dealing with this unavoidable part of life is far tougher due to the chaotic nature of these unpredictable activities. In the present article, we consider a global network of identical chaotic maps, which splits into two different clusters, despite the interaction between all nodes are uniform. The stability analysis of the spatially homogeneous chaotic solutions provides a critical coupling strength, before which we anticipate such partial synchronization. The distance between these two chaotic synchronized populations often deviates more than eight times of standard deviation from its long-term average. The probability density function of these highly deviated values fits well with the generalized extreme value distribution. Meanwhile, the distribution of recurrence time intervals between extreme events resembles the Weibull distribution. The existing literature helps us to characterize such events as extreme events using the significant height. These extremely high fluctuations are less frequent in terms of their occurrence. We determine numerically a range of coupling strength for these extremely large but recurrent events. On-off intermittency is the responsible mechanism underlying the formation of such extreme events. Besides understanding the generation of such extreme events and their statistical signature, we furnish forecasting these events using the powerful deep learning algorithms of an artificial recurrent neural network. This long short-term memory (LSTM) can offer handy one-step forecasting of these chaotic intermittent bursts. We also ensure the robustness of this forecasting model with two hundred hidden cells in each LSTM layer.

Bright galaxy sample in the Kilo-Degree Survey Data Release 4

We present a bright galaxy sample with accurate and precise photometric redshifts (photo-zs), selected using ugriZYJHKs photometry from the Kilo-Degree Survey (KiDS) Data Release 4. The highly pure and complete dataset is flux-limited at r &lt; 20 mag, covers ∼1000 deg2, and contains about 1 million galaxies after artifact masking. We exploit the overlap with Galaxy And Mass Assembly spectroscopy as calibration to determine photo-zs with the supervised machine learning neural network algorithm implemented in the ANNz2 software. The photo-zs have a mean error of |⟨δz⟩|∼5 × 10−4 and low scatter (scaled mean absolute deviation of ∼0.018(1 + z)); they are both practically independent of the r-band magnitude and photo-z at 0.05 &lt; zphot &lt; 0.5. Combined with the 9-band photometry, these allow us to estimate robust absolute magnitudes and stellar masses for the full sample. As a demonstration of the usefulness of these data, we split the dataset into red and blue galaxies, used them as lenses, and measured the weak gravitational lensing signal around them for five stellar mass bins. We fit a halo model to these high-precision measurements to constrain the stellar-mass–halo-mass relations for blue and red galaxies. We find that for high stellar mass (M⋆ &gt; 5 × 1011 M⊙), the red galaxies occupy dark matter halos that are much more massive than those occupied by blue galaxies with the same stellar mass.

Ensemble Learning Based on Policy Optimization Neural Networks for Capability Assessment.

Capability assessment plays a crucial role in the demonstration and construction of equipment. To improve the accuracy and stability of capability assessment, we study the neural network learning algorithms in the field of capability assessment and index sensitivity. Aiming at the problem of overfitting and parameter optimization in neural network learning, the paper proposes an improved machine learning algorithm—the Ensemble Learning Based on Policy Optimization Neural Networks (ELPONN) with the policy optimization and ensemble learning. This algorithm presents an optimized neural network learning algorithm through different strategies evolution, and builds an ensemble learning model of multi-intelligent algorithms to assess the capability and analyze the sensitivity of the indexes. Through the assessment of capabilities, the algorithm effectively avoids parameter optimization from entering the minimum point in performance to improve the accuracy of equipment capability assessment, which is significantly better than previous neural network assessment methods. The experimental results show that the mean relative error is 4.10%, which is better than BP, GABP, and early stopping. The ELPONN algorithm has better accuracy and stability performance, and meets the requirements of capability assessment.

Predicting heat release properties of flammable fiber-polymer laminates using artificial neural networks

Heat release rate is an important fire reaction property used to quantify the flammability of composite materials in fire. In this study, an artificial neural network (ANN) model was developed to predict the heat release properties of composites. The ANN model was trained using 10,419 data points for heat release rate extracted from the results of cone calorimetry tests performed on 14 sets of composite laminates. Two machine learning algorithms of Multiple Linear Regression (MLR) and Bayesian regularized artificial neural network with Gaussian prior (BRANNGP) are compared. The composites used to demonstrate the predictive accuracy of the ANN model were phenolic-based laminates containing different types and amounts of flame retardant additives. The BRANNGP model is capable of predicting the heat release rate-time curve, peak heat release value and total heat release of the composites. In addition, the BRANNGP model with outlier-elimination strategy can estimate with good accuracy the complex non-linear relationship between heat release rate and heat flux exposure time without considering the mechanistic interactions between the input and output parameters.

RETRACTED ARTICLE: Mountain rainfall estimation and online English teaching evaluation based on RBF neural network

In order to solve the problem of the main parameters setting of traditional RBF neural network learning algorithm and the lack of local information based on space, this paper proposes a particle swarm hybrid optimization RBF neural network prediction algorithm based on genetic operation, which solves the early problems of PSO on the basis of the cross and mutation fusion of PSO and GA advantages. At the same time, we analyze the limitations of using the mountain rainfall simulation process and taking the mountain rainfall estimation as the model input, and put forward the idea of using the radar estimation technology from the perspective of spatial rainfall, and carrying out the mountain rainfall evaluation and simulation according to the spatial rainfall. The application of flood forecasting focuses on the research of mountain rainfall estimation simulation method based on radar precipitation estimation technology, and it is also the combination of mountain rainfall simulation process. Finally, this paper evaluates and analyzes online English teaching, and the new network environment has become the carrier of College English learning. In the new network environment, students will form the awareness of reading, listening, and speaking English, and conduct conscious online English learning guidance and enhance their language ability to find their own understanding and perception of English. In this paper, we reveal the problems and shortcomings of the current college English education evaluation in China, put forward various new intelligence evaluation theories, and try to explore the practical functions of several education evaluation systems in education, so as to better use the means of education evaluation to improve the efficiency of College English teaching. According to the research of the RBF neural network, it is applied to rainfall estimation and teaching evaluation, which can effectively improve rainfall estimation and teaching evaluation.

Effective implementation of machine learning algorithms using 3D colour texture feature for traffic sign detection for smart cities

AbstractIn recent past, emerging technology, such as AI has revolutionized progress and advancement of traffic management solutions in context of smart cities. This paper describes a novel approach of using 3D colour‐texture based feature for image detection on public datasets of Chinese traffic sign research database (TSRD) and Mapillary image database. The implementation of 3D colour texture feature for traffic sign detection is evaluated using artificial neural network and multiple other machine learning algorithms as classifier for image detection purposes. Both datasets used in our experiments are considered among most diverse traffic signage datasets globally with annotations of almost all classes. Image datasets have been publicly made available to researchers for academic purposes. For classification of traffic sign images on Chinese TSRD and Mapillary datasets, the result outcome from back‐propagation neural network (NN) classifier outperforms all other and produced best results and under multiple ML algorithms, support vector machines (SVM) cubic has best results. For classification of traffic sign images on Mapillary dataset, the result outcome from rational quadratic Gaussian process regression has best results. Also, the comparisons of results with other similar novel works have also been discussed. The proposed approach outperformed the previous experiments as observed through comparative analysis performed on both datasets. The conclusion of the research work highlights that the image detection performance is noticeably improved by using the combined 3D colour‐texture feature based proposed approach.

Prediction of the Electromagnetic Shielding Effectiveness of Metal Grid Using Neural Network Algorithm

With the advantages of simple structure, high electromagnetic (EM) shielding effectiveness (SE), wide shielding bandwidth and adjustable transmittance, the metal grid is a popular tool in the field of electromagnetic shielding. In this work, the deep learning algorithm of artificial neural network was used to predict the EM SE of the metal grid. After certain number of trainings, both the SE spectrum and the minimum SE value in a given frequency band can be predicted accurately and effectively. On the other hand, for a target SE, the design parameters of the metal grid structure can be immediately predicted. The results show that the relative mean square deviation of the predicted SE curve is about 5%, the relative error of the minimum SE value of the curve is less than 5%, and the average relative error of the minimum SE value in the predicted frequency band is 11.5%.

Prediction and Value of Ultrasound Image in Diagnosis of Fetal Central Nervous System Malformation under Deep Learning Algorithm

This study was to explore the application of deep learning neural network (DLNN) algorithms to identify and optimize the ultrasound image so as to analyze the effect and value in diagnosis of fetal central nervous system malformation (CNSM). 63 pregnant women who were gated in the hospital were suspected of being fetal CNSM and were selected as the research objects. The ultrasound images were reserved in duplicate, and one group was defined as the control group without any processing, and images in the experimental group were processed with the convolutional neural network (CNN) algorithm to identify and optimize. The ultrasound examination results and the pathological test results before, during, and after the pregnancy were observed and compared. The results showed that the test results in the experimental group were closer to the postpartum ultrasound and the results of the pathological result, but the results in both groups showed no statistical difference in contrast to the postpartum results in terms of similarity ( P &gt; 0.05 ). In the same pregnancy stage, the ultrasound examination results of the experimental group were higher than those in the control group, and the contrast was statistically significant ( P &lt; 0.05 ); in the different pregnancy stages, the ultrasound examination results in the second trimester were more close to the postpartum examination results, showing statistically obvious difference ( P &lt; 0.05 ). In conclusion, ultrasonic image based on deep learning was higher in CNSM inspection; and ultrasonic technology had to be improved for the examination in different pregnancy stages, and the accuracy of the examination results is improved. However, the amount of data in this study was too small, so the representative was not high enough, which would be improved.

A deep learning based decision support system for diagnosis of Temporomandibular joint disorder

Temporomandibular Joint sounds are a very common disorder in the general population. Temporomandibular Disorder (TMD) is any discomfort related to Temporomandibular Joint (TMJ). In this paper, a novel decision support system based on deep learning and neural network algorithms for diagnosis of Temporomandibular Joint disorder is introduced. A non-invasive device is designed for the recording of TMJ sounds. An interface is developed that will facilitate the dentist to operate on the recorded audio data. The collected data consist of the patient's left and right Temporomandibular Joint sound, ambient noise sound, the patient's clinical data, the dentist's notes about the patient, diagnosis, and treatment. Then signal processing, artificial neural network and deep learning algorithms are used to classify these measurements, and thus, the method that decides about the patient's condition is developed. Frequency, statistical and deep learning-based methods are compared in terms of classification success. The results show that the classification success of the classification method based on deep learning is consistently over 94.5% and it is more successful than the previous two methods. The proposed system can give the physician an idea about the effectiveness of the treatment methods applied to the patient in order to treat joint sounds which are among the important symptoms of TMD.

Neuromorphic advancements architecture design and its implementations technique

Neuromorphic Architectures (NA) is hardware network systems which, designed on the principles of neural functions. The network systems are inspired from biological neural networks. Each node or neuron in the artificial Neural networks (ANN) are connected to each other using a synapse. Similar to the biological brains, the connection will be controlled with the amplitude of the connection between nodes, which termed as synaptic weights. Unlike in the conventional architecture, in ANNs consists of huge quantity of extremely organized dealing out elements operational in union to resolve the real world problems. NA is considered as the main soft-computing knowledge and has been widely researched. It is applied during last decades for the computational model. This paper basically focuses on the NA and neural networks and implementation. Neural network and machine learning algorithms are used by data classification in NA. This data will be provided a number of of the modern advancement, including super-computer, and single device implementations, approaches dependent on spiking and non-spiking neuron. Machine learning hardware devices are used to utilization of memristive device.

Automatic COVID-19 pneumonia diagnosis from x-ray lung image: A Deep Feature and Machine Learning Solution

Coronavirus disease 2019 was announced after unidentified pneumonia was discovered in Wuhan, China, and quickly spread around the world (COVID-19). This outbreak has claimed the lives of so many people. It has a long-term negative impact on public health. The goal of this study is to develop an intelligent computer-aided system that can detect positive COVID-19 cases automatically, which can help with daily medical problems. The proposed system is based on the convolution neural network (CNN) architecture and can automatically expose discriminative features on chest X-ray images due to its convolution with rich filter families and weight-sharing characteristics. As a deep feature extractor, the CNN model SqueezeNet was used. The extracted deep discriminative features were fed machine Decision Tree, Random Forest, Neural Network (NN), Naive Bayes, Logistic Regression, and k-nearest neighbor learning algorithms. As a result, the NN classifier with an accuracy of 97.24 per cent, a sensitivity of 0.9724, a specificity of 0.9858, and an F-score of 0.972 provided the most effective results. The high detection performance obtained in this study demonstrates the utility of deep CNN features and an NN classifier approach for detecting COVID-19 cases in CXR images. With the current resources, this would be hugely beneficial in speeding up disease diagnosis.

The ensemble approach in comparison with the diverse feature selection techniques for estimating NPPs parameters using the different learning algorithms of the feed-forward neural network

Several reasons such as no free lunch theorem indicate that there is not a universal Feature selection (FS) technique that outperforms other ones. Moreover, some approaches such as using synthetic dataset, in presence of large number of FS techniques, are very tedious and time consuming task. In this study to tackle the issue of dependency of estimation accuracy on the selected FS technique, a methodology based on the heterogeneous ensemble is proposed. The performance of the major learning algorithms of neural network (i.e. the FFNN-BR, the FFNN-LM) in combination with the diverse FS techniques (i.e. the NCA, the F-test, the Kendall's tau, the Pearson, the Spearman, and the Relief) and different combination techniques of the heterogeneous ensemble (i.e. the Min, the Median, the Arithmetic mean, and the Geometric mean) are considered. The target parameters/transients of Bushehr nuclear power plant (BNPP) are examined as the case study. The results show that the Min combination technique gives the more accurate estimation. Therefore, if the number of FS techniques is m and the number of learning algorithms is n, by the heterogeneous ensemble, the search space for acceptable estimation of the target parameters may be reduced from n × m to n × 1. The proposed methodology gives a simple and practical approach for more reliable and more accurate estimation of the target parameters compared to the methods such as the use of synthetic dataset or trial and error methods.

Evaluation of ASR in concrete using acoustic emission and deep learning

Alkali-silica reaction (ASR) is one of main damages causes in concrete structures such as nuclear power plants which may endanger structural serviceability and integrity. Acoustic emission (AE) is a passive nondestructive method for structural health monitoring. It is very sensitive and has the capability of monitoring structures continuously. This method may be an alternative for early damage detection in concrete nuclear structures affected by ASR. The innovation of this paper lies in the implementation of deep learning algorithms to evaluate the ASR progress. ASR was monitored by AE in a concrete specimen, which was cast with reactive coarse aggregates and reinforced by steel rebars. The AE signals recorded during the experiment were filtered and divided into two classes. Two deep learning algorithms of convolutional neural network (CNN) and stacked autoencoder were employed to classify the AE signals into the corresponding classes. The model based on CNN resulted in a classifier with higher accuracy than the model based on the autoencoder network.