Neural Network Learning Methods Research Articles

Deep representation-based transfer learning for deep neural networks

In recent years, deep neural networks (DNNs) have become the de facto models for practically all visual tasks and most temporal analysis tasks due to the abundance of available labeled data and advances in computational resources. Deep transfer learning, a learning paradigm that investigates how deep neural networks can leverage knowledge from other domains, has been successful in many practical applications. Most deep transfer learning methods rely on directly available source domain data, especially instances-based and mapping-based methods. However, the source domain data required for transfer cannot be provided in some cases because of personal privacy or copyright implications. In this regard, we propose a deep representation-based transfer learning method for deep neural networks to deal with transfer scenarios without source domain data. The proposed method induces the target deep neural network to learn the similarity structure between source deep representations by minimizing the representational similarity discrepancy between the networks, thereby transferring the domain knowledge from the source network to the target network. In addition, the proposed method combines the boosting-based deep representation transfer method to more fully utilize the source domain knowledge, so that the target network learns in both the reweighted source deep representation and target deep representation to further improve the generalization ability. The proposed method is applied to image classification and time series prediction tasks and achieves better transfer performance than other similar methods, which shows that the method has good practicability and generality in practical applications.

Research on Chlorophyll-a Concentration Retrieval Based on BP Neural Network Model—Case Study of Dianshan Lake, China

The Chlorophyll-a (Chl-a) concentration is an important indicator of water environmental conditions; thus, the simultaneous monitoring of large-area water bodies can be realized through the remote sensing-based retrieval of the Chl-a concentrations. The back propagation (BP) neural network learning method has been widely used for the remote sensing retrieval of water quality in first and second-class water bodies. However, many Chl-a concentration measurements must be used as learning samples with this method, which is constrained by the number of samples, due to the limited time and resources available for simultaneous measurements. In this paper, we conduct correlation analysis between the Chl-a concentration data measured at Dianshan Lake in 2020 and 2021 and synchronized Landat-8 data. Through analysis and study of the radiative transfer model and the retrieval method, a BP neural network retrieval model based on multi-phase Chl-a concentration data is proposed, which allows for the realization of remote sensing-based Chl-a monitoring in third-class water bodies. An analysis of spatiotemporal distribution characteristics was performed, and the method was compared with other constructed models. The research results indicate that the retrieval performance of the proposed BP neural network model is better than that of models constructed using multiple regression analysis and curve estimation analysis approaches, with a coefficient of determination of 0.86 and an average relative error of 19.48%. The spatial and temporal Chl-a distribution over Dianshan Lake was uneven, with high concentrations close to human production and low concentrations in the open areas of the lake. During the period from 2020 to 2021, the Chl-a concentration showed a significant upward trend. These research findings provide reference for monitoring the water environment in Dianshan Lake.

Survey of brain tumor segmentation with deep neural networks

A brain tumour is a serious condition that, if not diagnosed and treated early on, can lead to death. Researchers have proposed a variety of traditional and recently developed deep learning based segmentation and classification approaches for determining the condition of the tumor. Deep learning is found to be efficient and robust for classification and segmentation as it detects the fine-to-coarse information about the tumors. The main component of deep learning is layered neural network architecture popularly known as convolutional neural network. Distinct information from brain images can be acquired and analysed depending on the architecture. In order to achieve high segmentation and classification accuracy, more research is required in this area. In this paper presents a review of state-of-the-art deep learning methods for brain tumor segmentation and deep learning neural networks, clearly highlighting their building blocks and different strategies. Finally, this article implying about present status on segmentation and classification of tumor-based image processing through deep learning models.

Bayesian Depth-Wise Convolutional Neural Network Design for Brain Tumor MRI Classification.

In recent years, deep learning has been applied to many medical imaging fields, including medical image processing, bioinformatics, medical image classification, segmentation, and prediction tasks. Computer-aided detection systems have been widely adopted in brain tumor classification, prediction, detection, diagnosis, and segmentation tasks. This work proposes a novel model that combines the Bayesian algorithm with depth-wise separable convolutions for accurate classification and predictions of brain tumors. We combine Bayesian modeling learning and Convolutional Neural Network learning methods for accurate prediction results to provide the radiologists the means to classify the Magnetic Resonance Imaging (MRI) images rapidly. After thorough experimental analysis, our proposed model outperforms other state-of-the-art models in terms of validation accuracy, training accuracy, F1-score, recall, and precision. Our model obtained high performances of 99.03% training accuracy and 94.32% validation accuracy, F1-score, precision, and recall values of 0.94, 0.95, and 0.94, respectively. To the best of our knowledge, the proposed work is the first neural network model that combines the hybrid effect of depth-wise separable convolutions with the Bayesian algorithm using encoders.

The Analysis of Plants Recognition Based on Deep Learning and Artificial Neural Network

Abstract: Classification and identification of plants are helpful for people to effectively understand and safeguard plants. The leaves of plants are the most important identifying organs. With the development of artificial intelligence and machine vision technology, plants leaf recognition technology based on image analysis is used to ameliorate the knowledge of plant classification and protection. Deep learning is the condensation of deep neural network learning method and belongs to neural network structure. The main content of this paper is to extract plant leaf features and identify plant species based on image analysis.

Classification of physiological disorders in apples fruit using a hybrid model based on convolutional neural network and machine learning methods

Classification of physiological disorders in apples fruit using a hybrid model based on convolutional neural network and machine learning methods

Unveiling the pole structure of S-matrix using deep learning

Particle scattering is a powerful tool to unveil the nature of various subatomic phenomena. The key quantity is the scattering amplitude whose analytic structure carries the information of the quantum states. In this work, we demonstrate our first step attempt to extract the pole configuration of inelastic scatterings using the deep learning method. Among various problems, motivated by the recent new hadron phenomena, we develop a curriculum learning method of deep neural network to analyze coupled channel scattering problems. We show how effectively the method works to extract the pole configuration associated with resonances in the πN scattering.

Fully Elman Neural Network: A Novel Deep Recurrent Neural Network Optimized by an Improved Harris Hawks Algorithm for Classification of Pulmonary Arterial Wedge Pressure.

Heart failure (HF) is one of the most prevalent life-threatening cardiovascular diseases in which 6.5 million people are suffering in the USA and more than 23 million worldwide. Mechanical circulatory support of HF patients can be achieved by implanting a left ventricular assist device (LVAD) into HF patients as a bridge to transplant, recovery or destination therapy and can be controlled by measurement of normal and abnormal pulmonary arterial wedge pressure (PAWP). While there are no commercial long-term implantable pressure sensors to measure PAWP, real-time non-invasive estimation of abnormal and normal PAWP becomes vital. In this work, first an improved Harris Hawks optimizer algorithm called HHO+ is presented and tested on 24 unimodal and multimodal benchmark functions. Second, a novel fully Elman neural network (FENN) is proposed to improve the classification performance. Finally, four novel 18-layer deep learning methods of convolutional neural networks (CNNs) with multi-layer perceptron (CNN-MLP), CNN with Elman neural networks (CNN-ENN), CNN with fully Elman neural networks (CNN-FENN), and CNN with fully Elman neural networks optimized by HHO+ algorithm (CNN-FENN-HHO+) for classification of abnormal and normal PAWP using estimated HVAD pump flow were developed and compared. The estimated pump flow was derived by a non-invasive method embedded into the commercial HVAD controller. The proposed methods are evaluated on an imbalanced clinical dataset using 5-fold cross-validation. The proposed CNN-FENN-HHO+ method outperforms the proposed CNN-MLP, CNN-ENN and CNN-FENN methods and improved the classification performance metrics across 5-fold cross-validation with an average sensitivity of 79%, accuracy of 78% and specificity of 76%. The proposed methods can reduce the likelihood of hazardous events like pulmonary congestion and ventricular suction for HF patients and notify identified abnormal cases to the hospital, clinician and cardiologist for emergency action, which can diminish the mortality rate of patients with HF.

Wind turbine load estimation using machine learning and transfer learning

Machine learning method has always been popular to solve wind turbine related problems at a data level. However, with the limitation of the availability of relevant data, transfer learning has gained increasing attention. In this study, traditional machine learning method of artificial neural networks (ANN), together with parameter-based transfer learning method has been used to estimate wind turbine load. First, ANN load model was built for DTU 10MW wind turbine as well as NREL 5MW wind turbine. Then, parameter-based transfer learning has been applied to the above-mentioned models to estimate load for a different turbine type or two mixed turbine types. Results indicate that ANN method provides good estimation on wind turbine fatigue load. For DTU 10MW ANN model, the trend of accuracy becomes steady as the number of input samples increases and 1500 samples is deemed as the optimal number of samples for training DTU 10MW. In addition, with transfer learning, it was succeeded in building NREL 5MW model with corresponding DTU 10MW pretrained model but failed in establishing mixed dataset model neither with DTU 10MW nor with NREL 5MW pretrained model.

Optimized Extreme Learning Machine

Abstract: Extreme Learning Machine (ELM) is a learning method for single-hidden layer feedforward neural network (SLFN) training. The ELM strategy speeds up learning by generating input weights and biases for hidden nodes at random rather than modifying network parameters, making it much faster than the standard gradient-based approach. In this project, an ELM optimized by Hybrid Particle Swarm Optimization approach is presented to optimize the input weights and hidden biases for ELM. We will analyze and obtain results for benchmark datasets. The Optimized Extreme Learning Machine algorithm's output is compared to publicly available data. Later we will compare different algorithms and check which one gives better output metrics. Keywords: ELM, SLFN, PSO, Gradient-based approach, Optimization

Comparison of Feedforward Perceptron Network with LSTM for Solar Cell Radiation Prediction

Intermittency of electrical power in developing countries, as well as some European countries such as Turkey, can be eluded by taking advantage of solar energy. Correct prediction of solar radiation constitutes a very important step to take advantage of PV solar panels. We propose an experimental study to predict the amount of solar radiation using a classical artificial neural network (ANN) and deep learning methods. PV panel and solar radiation data were collected at Duzce University in Turkey. Moreover, we included meteorological data collected from the Meteorological Ministry of Turkey in Duzce. Data were collected on a daily basis with a 5-min interval. Data were cleaned and preprocessed to train long-short-term memory (LSTM) and ANN models to predict the solar radiation amount of one day ahead. Models were evaluated using coefficient of determination (R2), mean square error (MSE), root mean squared error (RMSE), mean absolute error (MAE), and mean biased error (MBE). LSTM outperformed ANN with R2, MSE, RMSE, MAE, and MBE of 0.93, 0.008, 0.089, 0.17, and 0.09, respectively. Moreover, we compared our results with two similar studies in the literature. The proposed study paves the way for utilizing renewable energy by leveraging the usage of PV panels.

Physics-informed Karhunen-Loéve and neural network approximations for solving inverse differential equation problems

We present the PI-CKL-NN method for parameter estimation in differential equation (DE) models given sparse measurements of the parameters and states. In the proposed approach, the space- or time-dependent parameters are approximated by Karhunen-Loéve (KL) expansions that are conditioned on the parameters' measurements, and the states are approximated by deep neural networks (DNNs). The unknown weights in the KL expansions and DNNs are found by minimizing the cost function that enforces the measurements of the states and the DE constraint. Regularization is achieved by adding the l2 norm of the conditional KL coefficients into the loss function. Our approach assumes that the parameter fields are correlated in space or time and enforces the statistical knowledge (the mean and the covariance function) in addition to the DE constraints and measurements as opposed to the physics-informed neural network (PINN) and other similar physics-informed machine learning methods where only DE constraints and data are used for parameter estimation. We use the PI-CKL-NN method for parameter estimation in an ordinary differential equation with an unknown time-dependent parameter and the one- and two-dimensional partial differential diffusion equations with unknown space-dependent diffusion coefficients. We also demonstrate that PI-CKL-NN is more accurate than the PINN method, especially when the observations of the parameters are very sparse.

Making the hospital smart: using a deep long short-term memory model to predict hospital performance metrics

PurposeAbundant studies of outpatient visits apply traditional recurrent neural network (RNN) approaches; more recent methods, such as the deep long short-term memory (DLSTM) model, have yet to be implemented in efforts to forecast key hospital data. Therefore, the current study aims to reports on an application of the DLSTM model to forecast multiple streams of healthcare data.Design/methodology/approachAs the most advanced machine learning (ML) method, static and dynamic DLSTM models aim to forecast time-series data, such as daily patient visits. With a comparative analysis conducted in a high-level, urban Chinese hospital, this study tests the proposed DLSTM model against several widely used time-series analyses as reference models.FindingsThe empirical results show that the static DLSTM approach outperforms seasonal autoregressive integrated moving averages (SARIMA), single and multiple RNN, deep gated recurrent units (DGRU), traditional long short-term memory (LSTM) and dynamic DLSTM, with smaller mean absolute, root mean square, mean absolute percentage and root mean square percentage errors (RMSPE). In particular, static DLSTM outperforms all other models for predicting daily patient visits, the number of daily medical examinations and prescriptions.Practical implicationsWith these results, hospitals can achieve more precise predictions of outpatient visits, medical examinations and prescriptions, which can inform hospitals' construction plans and increase the efficiency with which the hospitals manage relevant information.Originality/valueTo address a persistent gap in smart hospital and ML literature, this study offers evidence of the best forecasting models with a comparative analysis. The study extends predictive methods for forecasting patient visits, medical examinations and prescriptions and advances insights into smart hospitals by testing a state-of-the-art, deep learning neural network method.

Nuclear mass based on the multi-task learning neural network method

Nuclear mass based on the multi-task learning neural network method

Analysis of the Effectiveness of Various Machine Learning, Artificial Neural Network and Deep Learning Methods in Detecting Fraudulent Credit Card Transactions

A credit card is an important financial tool that has emerged in parallel with the developments in technology from the past to the present and has become an indispensable part of human life. The credit card has many advantages that can be listed as facilitating online shopping, providing installments in purchases, and preventing cash dependence. This is why the rate of use of credit cards worldwide is increasing day by day. On the other hand, there are some risks of the credit cards highlighted by security concerns. The fraudsters who access the identity and credit card information of the consumers through different means use it to shop online without the consumer’s knowledge and gain an unfair advantage. Therefore, it is crucial to eliminate this security vulnerability that the fraudsters exploit and to develop an effective solution to the customer victimization experienced by e-commerce companies due to the fraudulent credit card transactions. With this motivation, the performance of the methods from different research fields was examined to explore the solution space in detail in terms of the problem at hand within the scope of this study. For this purpose, three machine learning algorithms (K-Nearest Neighbor, Naive Bayes, Support Vector Machine), two artificial neural network algorithms (Binary Classifier, Autoencoder), and two deep learning algorithms (Deep Autoencoder and Deep Neural Network Classifier) were implemented. The effectiveness of the algorithms in question was tested with a famous dataset widely used in the literature. Experimental results showed that the Deep Neural Network Classifier outperformed the other algorithms used in this study and the best study ever reported in the literature in detecting fraudulent credit card transactions when accuracy and AUROC performance criteria were taken into account.

Adaptive Learning Neural Network Method for Solving Time-Fractional Diffusion Equations.

A neural network method for solving fractional diffusion equations is presented in this letter. An adaptive gradient descent method is proposed to minimize energy functions. Due to the memory effects of the fractional calculus, the gradient of energy function becomes much more complicated, and we suggest a simplified method. Numerical examples with one-layer and two-layer neurons show the effectiveness of the method.

Construction of the Social Network Information Dissemination Index System Based on CNNs

The information dissemination index system is an effective way to measure the dissemination of public opinion events in social networks. Due to the complexity, variability, and asymmetry of information, the construction of traditional information dissemination index systems demands excessive reliance on manual intervention, has large deviations, and is applied in a limited range. Such shortcomings cannot meet the requirements of constructing an objective, comprehensive, and highly credible index system. Therefore, we propose a method of constructing a multilevel and multigranular information dissemination index system with complex perspectives. In addition, we use the deep learning method of the convolutional neural network to extract the rich convolution features of public opinion events in the information dissemination process. Then, we train the weight, and it forms the corresponding weight of the information dissemination index systems. The experimental results prove that the method we use is superior to other methods and has better performance on the data set of a specific field.

Determination of viability and vigor of naturally-aged rice seeds using hyperspectral imaging with machine learning

Viability and vigor of rice seeds are related to the yield. The existing seed viability and vigor detection methods cannot meet the demand for precise planting, and a method that can quickly and non-destructively predict the vigor of rice seeds is needed. In this study, near-infrared hyperspectral imaging was used to determine the viability and vigor of naturally-aged rice seeds. Standard germination test was conducted to determine the reference values of the viability and vigor. Convolutional neural network (CNN) and conventional machine learning methods (support vector machine (SVM) and logistic regression (LR)) were built using full range spectra and characteristic wavelengths selected by principal component analysis (PCA) to predict the viability and vigor of different varieties of rice seeds under natural aging conditions. The overall results showed that deep learning methods and conventional machine learning methods could predict the viability and vigor of different varieties of rice seeds well, and the accuracy of most models was over 85%. Models using full spectra and the characteristic wavelengths showed close results. Models on all varieties performed closely to those on single variety. This study provided an effective method for fast, non-destructive and efficient prediction of rice seed viability and vigor.

Short-Term Prediction of PM2.5 Using LSTM Deep Learning Methods

This paper implements deep learning methods of recurrent neural networks and short-term memory models. Two kinds of time-series data were used: air pollutant factors, such as O3, SO2, and CO2 from 2017 to 2019, and meteorological factors such as temperature, humidity, wind direction, and wind speed. A trained model was used to predict air pollution within an eight-hour period. Correlation analysis was applied using Pearson and Spearman correlation coefficients. The KNN method was used to fill in the missing values to improve the generated model’s accuracy. The average absolute error percentage value was used in the experiments to evaluate the model’s performance. LSTM had the lowest RMSE value at 1.9 than the other models from the experiments. CNN had a significant RMSE value at 3.5, followed by Bi-LSTM at 2.5 and Bi-GRU at 2.7. In comparison, the RNN was slightly higher than LSTM at a 2.4 value.

Comparative Study of Convolutional Neural Network and Conventional Machine Learning Methods for Landslide Susceptibility Mapping

Landslide susceptibility mapping (LSM) is a useful tool to estimate the probability of landslide occurrence, providing a scientific basis for natural hazards prevention, land use planning, and economic development in landslide-prone areas. To date, a large number of machine learning methods have been applied to LSM, and recently the advanced convolutional neural network (CNN) has been gradually adopted to enhance the prediction accuracy of LSM. The objective of this study is to introduce a CNN-based model in LSM and systematically compare its overall performance with the conventional machine learning models of random forest, logistic regression, and support vector machine. Herein, we selected Zhangzha Town in Sichuan Province, China, and Lantau Island in Hong Kong, China, as the study areas. Each landslide inventory and corresponding predisposing factors were stacked to form spatial datasets for LSM. The receiver operating characteristic analysis, area under the curve (AUC), and several statistical metrics, such as accuracy, root mean square error, Kappa coefficient, sensitivity, and specificity, were used to evaluate the performance of the models. Finally, the trained models were calculated, and the landslide susceptibility zones were mapped. Results suggest that both CNN and conventional machine learning-based models have a satisfactory performance. The CNN-based model exhibits an excellent prediction capability and achieves the highest performance but also significantly reduces the salt-of-pepper effect, which indicates its great potential for application to LSM.