Types Of Deep Learning Methods Research Articles

Should we really use graph neural networks for transcriptomic prediction?

The recent development of deep learning methods have undoubtedly led to great improvement in various machine learning tasks, especially in prediction tasks. This type of methods have also been adapted to answer various problems in bioinformatics, including automatic genome annotation, artificial genome generation or phenotype prediction. In particular, a specific type of deep learning method, called graph neural network (GNN) has repeatedly been reported as a good candidate to predict phenotypes from gene expression because its ability to embed information on gene regulation or co-expression through the use of a gene network. However, up to date, no complete and reproducible benchmark has ever been performed to analyze the trade-off between cost and benefit of this approach compared to more standard (and simpler) machine learning methods. In this article, we provide such a benchmark, based on clear and comparable policies to evaluate the different methods on several datasets. Our conclusion is that GNN rarely provides a real improvement in prediction performance, especially when compared to the computation effort required by the methods. Our findings on a limited but controlled simulated dataset shows that this could be explained by the limited quality or predictive power of the input biological gene network itself.

Machine Learning Approach for New COVID-19 Cases Using Recurrent Neural Networks and Long-Short Term Memory

This research aims to predict new COVID-19 cases in Bandung, Indonesia. The system implemented two types of deep learning methods to predict this. They were the recurrent neural networks (RNN) and long-short-term memory (LSTM) algorithms. The data used in this study were the numbers of confirmed COVID-19 cases in Bandung from March 2020 to December 2020. Pre-processing of the data was carried out, namely data splitting and scaling, to get optimal results. During model training, the hyperparameter tuning stage was carried out on the sequence length and the number of layers. The results showed that RNN gave a better performance. The test used the RMSE, MAE, and R2 evaluation methods, with the best numbers being 0.66975075, 0.47075, 0.29616625, and 0.7644 on the test data.

An efficient framework for solving forward and inverse problems of nonlinear partial differential equations via enhanced physics-informed neural network based on adaptive learning

In recent years, the advancement of deep learning has led to the utilization of related technologies to enhance the efficiency and accuracy of scientific computing. Physics-Informed Neural Networks (PINNs) are a type of deep learning method applied to scientific computing, widely used to solve various partial differential equations (PDEs), demonstrating tremendous potential. This study improved upon original PINNs and applied them to forward and inverse problems in the nonlinear science field. In addition to incorporating the constraints of PDEs, the improved PINNs added constraints on gradient information, which further enhanced the physical constraints. Moreover, an adaptive learning method was used to update the weight coefficients of the loss function and dynamically adjust the weight proportion of each constraint term. In the experiment, the improved PINNs were used to numerically simulate localized waves and two-dimensional lid-driven cavity flow described by partial differential equations. Meanwhile, we critically evaluate the accuracy of the prediction results. Furthermore, the improved PINNs were utilized to solve the inverse problems of nonlinear PDEs, where the results showed that even with noisy data, the unknown parameters could be discovered satisfactorily. The study results indicated that the improved PINNs were significantly superior to original PINNs, with shorter training time, increased accuracy in prediction results, and greater potential for application.

Pathological prediction of heart disease based on deep width learning algorithm

As the technology has become more and more advance and massive data has been produced and pre-processed became useful information to become big data so come out a lot of technology like data mining, neural network, deep learning, etc. Each technology has its advantages, this paper put forward the width of the depth of the learning algorithm is to the advantage of two types of deep learning methods This new method is combining of Flexible Neural Tree(FNT) and Broad Learning System(BLS) to become a new deep width network. After several times of experiments, we can confirm that this new hybrid method can increase the accuracy, sensitivity, and specificity of Heart disease prediction. Therefore, having higher accuracy of prediction for heart disease means we can notice more people before having heart disease and start to prevent it and in the end, we can prevent loss of life and economy.

TSCMDL: Multimodal Deep Learning Framework for Classifying Tree Species Using Fusion of 2-D and 3-D Features

Accurate tree species information is a prerequisite for forest resource management. Combining light detection and ranging (LiDAR) and image data is one main method of tree species classification. Traditional machinelearningmethods rely on expert knowledge to calculatea large number of feature parameters.Deep learning technology can directly use the original image and pointclouddata to classify tree species. However, data with different patterns require the use of different types of deeplearningmethods. In this study, a multimodal deeplearningframework (TSCMDL) that fuses 2D and 3D features was constructed and then used to combine data from multiple sources for tree species classification. This framework uses an improved version of the PointMLP model as its backbone network and uses ResNet50 and PointMLP networks to extract the image features and pointcloudfeatures, respectively. The proposed framework was tested using UAV LiDAR data and RGB orthophotos. The results showed that the accuracy of the tree species classification using the TSCMDL framework was 98.52%, which was 4.02% higher than that based on pointcloudfeatures only. In addition, when the same hyperparameters were used for training the model, the efficiency of the model training was not significantly lower than for models based on pointcloudfeatures only. The proposed multimodal deeplearningframework extracts features directly from the original data and integrates them effectively, thus avoiding manual feature screening and achieving more accurate classification. The feature extraction network used in the TSCMDL framework can be replaced by other suitable frameworks and has strong application potential.

A fast inversion approach for the identification of highly transient surface heat flux based on the generative adversarial network

The efficient identification of surface heat flux during highly transient pool boiling process is an essential task for engineering production and manufacture, which can be mathematically defined as nonlinear inverse heat transfer problems (IHTP) of Neumann boundary conditions. Conventional regularization-based inversion approaches for these types of problems are computationally expensive since packs of nonlinear partial differential equations (PDE) constrained optimization problems in complex computational domains need to be solved. In this paper, an operator-learning method is proposed for the efficient solution of complex three-dimensional (3D) transient nonlinear IHTP. FluxNet, a generated adversarial network composed of a discrete wavelet transform (DWT) layer, a “generator” and a “discriminator”, is constructed to significantly improve the computational efficiency, reaching sub-second for the first time. The DWT layer is capable of extracting detailed information of the temperature distribution while the “generator” approximates the inverse operator of the governing PDE system with its convolution kernels. A minimax optimization strategy is adopted through adversarial training between the “generator” and the “discriminator” to encourage deeper understanding of data distribution. Compared with other types of deep learning methods, the proposed method shows its superiority when dealing with highly transient experimental data and micro-nano porous structured geometry. Based on the developed sub-second light-weight numerical solver with operator learning, the development of advanced real-time soft sensor techniques for a class of 3D transient IHTP in practical engineering applications is possible.

Deep Learning in the State of Charge Estimation for Li-Ion Batteries of Electric Vehicles: A Review

As one of the critical state parameters of the battery management system, the state of charge (SOC) of lithium batteries can provide an essential reference for battery safety management, charge/discharge control, and the energy management of electric vehicles (EVs). To analyze the application of deep learning in electric vehicles’ power battery SOC estimation, this study reviewed the technical process, common public datasets, and the neural networks used, as well as the structural characteristics and advantages and disadvantages of lithium battery SOC estimation in deep learning methods. First, the specific technical processes of the deep learning method for SOC estimation were analyzed, including data collection, data preprocessing, feature engineering, model training, and model evaluation. Second, the current commonly and publicly used lithium battery dataset was summarized. Then, the input variables, data sets, errors, and advantages and disadvantages of three types of deep learning methods were obtained using the structure of the neural network used for training as the classification criterion; further, the selection of the deep learning structure for SOC estimation was discussed. Finally, the challenges and future development directions of lithium battery SOC estimation using the deep learning method were explained. Over all, this review provides insights into deep learning for EVs’ Li-ion battery SOC estimation in the future.

Deep Learning for Smart Healthcare-A Survey on Brain Tumor Detection from Medical Imaging.

Advances in technology have been able to affect all aspects of human life. For example, the use of technology in medicine has made significant contributions to human society. In this article, we focus on technology assistance for one of the most common and deadly diseases to exist, which is brain tumors. Every year, many people die due to brain tumors; based on “braintumor” website estimation in the U.S., about 700,000 people have primary brain tumors, and about 85,000 people are added to this estimation every year. To solve this problem, artificial intelligence has come to the aid of medicine and humans. Magnetic resonance imaging (MRI) is the most common method to diagnose brain tumors. Additionally, MRI is commonly used in medical imaging and image processing to diagnose dissimilarity in different parts of the body. In this study, we conducted a comprehensive review on the existing efforts for applying different types of deep learning methods on the MRI data and determined the existing challenges in the domain followed by potential future directions. One of the branches of deep learning that has been very successful in processing medical images is CNN. Therefore, in this survey, various architectures of CNN were reviewed with a focus on the processing of medical images, especially brain MRI images.

Deep Learning on Graphs: A Survey

Deep learning has been shown to be successful in a number of domains, ranging from acoustics, images, to natural language processing. However, applying deep learning to the ubiquitous graph data is non-trivial because of the unique characteristics of graphs. Recently, substantial research efforts have been devoted to applying deep learning methods to graphs, resulting in beneficial advances in graph analysis techniques. In this survey, we comprehensively review the different types of deep learning methods on graphs. We divide the existing methods into five categories based on their model architectures and training strategies: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods. We then provide a comprehensive overview of these methods in a systematic manner mainly by following their development history. We also analyze the differences and compositions of different methods. Finally, we briefly outline the applications in which they have been used and discuss potential future research directions.

Deep Convolutional Neural Network with RNNs for Complex Activity Recognition Using Wrist-Worn Wearable Sensor Data

Sensor-based human activity recognition (S-HAR) has become an important and high-impact topic of research within human-centered computing. In the last decade, successful applications of S-HAR have been presented through fruitful academic research and industrial applications, including for healthcare monitoring, smart home controlling, and daily sport tracking. However, the growing requirements of many current applications for recognizing complex human activities (CHA) have begun to attract the attention of the HAR research field when compared with simple human activities (SHA). S-HAR has shown that deep learning (DL), a type of machine learning based on complicated artificial neural networks, has a significant degree of recognition efficiency. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are two different types of DL methods that have been successfully applied to the S-HAR challenge in recent years. In this paper, we focused on four RNN-based DL models (LSTMs, BiLSTMs, GRUs, and BiGRUs) that performed complex activity recognition tasks. The efficiency of four hybrid DL models that combine convolutional layers with the efficient RNN-based models was also studied. Experimental studies on the UTwente dataset demonstrated that the suggested hybrid RNN-based models achieved a high level of recognition performance along with a variety of performance indicators, including accuracy, F1-score, and confusion matrix. The experimental results show that the hybrid DL model called CNN-BiGRU outperformed the other DL models with a high accuracy of 98.89% when using only complex activity data. Moreover, the CNN-BiGRU model also achieved the highest recognition performance in other scenarios (99.44% by using only simple activity data and 98.78% with a combination of simple and complex activities).

Deep learning-based tree species mapping in a highly diverse tropical urban setting

Spatially explicit information on urban tree species distribution is crucial for green infrastructure management in cities. This information is usually acquired with ground-based surveys, which are time-consuming and usually cover limited spatial extents. The combination of machine learning algorithms and remote sensing images has been hailed as a promising way to map tree species over broad areas. Recently, convolutional neural networks (CNNs), a type of deep learning method, have achieved outstanding results for tree species discrimination in various remote sensing data types. However, there is a lack of studies using CNN-based methods to produce tree species composition maps, particularly for tropical urban settings. Here, we propose a multi-task CNN to map tree species in a highly diverse neighborhood in Rio de Janeiro, Brazil. Our network architecture takes an aerial photograph (RGB bands and pixel size = 0.15 m) and delivers two outputs: a semantically segmented image and a distance map transform. In the former, all pixel positions are labeled, while in the latter, each pixel position contains the Euclidean distance to the crown boundary. We developed a post-processing approach that combines the two outputs, and we classified nine and five tree species with an average F1-score of 79.3 ± 8.6% and 87.6 ± 4.4%, respectively. Moreover, our post-processing approach produced a realistic tree species composition map by labeling only pixels of the target species with high class membership probabilities. Our results show the potential of CNNs and aerial photographs to map tree species in highly diverse tropical urban settings, providing valuable insights for urban forest management and green spaces planning.

Measurement and prediction of tunnelling-induced ground settlement in karst region by using expanding deep learning method

This paper presents the measurement and prediction of the tunnelling-induced surface response in karst ground, Guangzhou, China. A predictive method of ground settlement is proposed named as the expanding deep learning method. This method kinetically uses the expanding tunnelling data to predict ground settlement in real time. Four types of deep learning methods are compared, including artificial neural network (ANN), long short-term memory neural networks (LSTM), gated recurrent unit neural networks (GRU), and 1d convolutional neural networks (Conv1d). Based on static Pearson correlation coefficient, a kinetic correlation analysis method is proposed to evaluate the variable significance of input data on the ground settlement. The effect of cemented karst caves and variable geological conditions are then analysed. The results indicate that the expanding Conv1d model precisely predict the tunnelling-induced ground settlement. The kinetic correlation analysis can reflect the variable influence of geological condition and tunnelling operation parameters on the ground settlement.

River water temperature forecasting using a deep learning method

Accurate water temperature forecasting is essential for understanding thermal regimes of rivers in the context of climate change and anthropogenic disturbances, such as dam construction. Machine-learning models proffer an empirically based approach to predicting water temperatures with a high degree of accuracy. This study explores the potential of long short-term neural network (LSTM), a type of deep learning method, to forecast daily river water temperatures and quantify temporal variations in thermal regime induced by changes in climate and by dam construction. The performance of LSTM is compared with that of several other models using daily water-temperature data for nine river gauges around the world. In a detailed analysis, the models are evaluated for the Yichang gauge on the Yangtze River to reconstruct the natural thermal conditions and help to assess daily water temperature variations induced by operation of the Three Gorges Reservoir (TGR). The collective results show that LSTM outperforms other methods for predicting mean daily water temperature in rivers, capturing accurately mean daily variations in thermal regime. The construction of the TGR strongly influenced water temperature variations at Yichang, producing the strongest cooling effect from mid-April to mid-May and the greatest warming effect in late December and early January. These marked effects are most prominent at the highest water levels in the TGR. The enhanced predictive capabilities of the LSTM model provide a powerful tool for water temperature forecasting and ecological management of rivers in the Anthropocene.

A Review of Plant Phenotypic Image Recognition Technology Based on Deep Learning

Plant phenotypic image recognition (PPIR) is an important branch of smart agriculture. In recent years, deep learning has achieved significant breakthroughs in image recognition. Consequently, PPIR technology that is based on deep learning is becoming increasingly popular. First, this paper introduces the development and application of PPIR technology, followed by its classification and analysis. Second, it presents the theory of four types of deep learning methods and their applications in PPIR. These methods include the convolutional neural network, deep belief network, recurrent neural network, and stacked autoencoder, and they are applied to identify plant species, diagnose plant diseases, etc. Finally, the difficulties and challenges of deep learning in PPIR are discussed.

Hybrid fuzzy integrated convolutional neural network (HFICNN) for similarity feature recognition problem in abnormal netflow detection

This paper presents a hybrid fuzzy integrated convolutional neural network (HFICNN) to deal with the similarity feature recognition problem in abnormal netflow detection, which is that different labels with the same netflow eigenvalues are not well disposed by traditional machine learning and deep learning methods. The HFICNN model can convert the same feature individuals with different labels to different feature individuals with different labels by constructing an integration method. Our method recognizes the actual environment and operational complexity, and converts the unique occurrence time feature performance as a new feature for feature recognition. HFICNN integrates time-dependent records into a new record to identify a feature, and converts the integrated records to the original label. The integration process plays a good role and effect in the entire process. HFICNN is realized with the aid of function mapping label redefinition (FMLR), adaptive feature integration convolutional neurons (AFICNs), fuzzy classification (FC), and inverse function mapping integrated reduction (IFMIR). The ICMPv6-based DDoS attacks dataset of a new-generation network is tested, and experimental results show that HFICNN performs better than 10 types of traditional machine learning and two types of deep learning methods on the similarity feature recognition problem, and the HFICNN model is reliable and effective.

Deep Learning on Three-Dimensional Multiscale Data for Next-Hour Tornado Prediction

Abstract This paper describes the development of convolutional neural networks (CNN), a type of deep-learning method, to predict next-hour tornado occurrence. Predictors are a storm-centered radar image and a proximity sounding from the Rapid Refresh model. Radar images come from the Multiyear Reanalysis of Remotely Sensed Storms (MYRORSS) and Gridded NEXRAD WSR-88D Radar dataset (GridRad), both of which are multiradar composites. We train separate CNNs on MYRORSS and GridRad data, present an experiment to optimize the CNN settings, and evaluate the chosen CNNs on independent testing data. Both models achieve an area under the receiver-operating-characteristic curve (AUC) well above 0.9, which is considered to be excellent performance. The GridRad model achieves a critical success index (CSI) of 0.31, and the MYRORSS model achieves a CSI of 0.17. The difference is due primarily to event frequency (percentage of storms that are tornadic in the next hour), which is 3.52% for GridRad but only 0.24% for MYRORSS. The best CNN predictions (true positives and negatives) occur for strongly rotating tornadic supercells and weak nontornadic cells in mesoscale convective systems, respectively. The worst predictions (false positives and negatives) occur for strongly rotating nontornadic supercells and tornadic cells in quasi-linear convective systems, respectively. The performance of our CNNs is comparable to an operational machine-learning system for severe weather prediction, which suggests that they would be useful for real-time forecasting.

Deep Learning Approaches Applied to Remote Sensing Datasets for Road Extraction: A State-Of-The-Art Review

One of the most challenging research subjects in remote sensing is feature extraction, such as road features, from remote sensing images. Such an extraction influences multiple scenes, including map updating, traffic management, emergency tasks, road monitoring, and others. Therefore, a systematic review of deep learning techniques applied to common remote sensing benchmarks for road extraction is conducted in this study. The research is conducted based on four main types of deep learning methods, namely, the GANs model, deconvolutional networks, FCNs, and patch-based CNNs models. We also compare these various deep learning models applied to remote sensing datasets to show which method performs well in extracting road parts from high-resolution remote sensing images. Moreover, we describe future research directions and research gaps. Results indicate that the largest reported performance record is related to the deconvolutional nets applied to remote sensing images, and the F1 score metric of the generative adversarial network model, DenseNet method, and FCN-32 applied to UAV and Google Earth images are high: 96.08%, 95.72%, and 94.59%, respectively.

A Hybrid Deep Learning Model to Forecast Particulate Matter Concentration Levels in Seoul, South Korea

Both long- and short-term exposure to high concentrations of airborne particulate matter (PM) severely affect human health. Many countries now regulate PM concentrations. Early-warning systems based on PM concentration levels are urgently required to allow countermeasures to reduce harm and loss. Previous studies sought to establish accurate, efficient predictive models. Many machine-learning methods are used for air pollution forecasting. The long short-term memory and gated recurrent unit methods, typical deep-learning methods, reliably predict PM levels with some limitations. In this paper, the authors proposed novel hybrid models to combine the strength of two types of deep learning methods. Moreover, the authors compare hybrid deep-learning methods (convolutional neural network (CNN)—long short-term memory (LSTM) and CNN—gated recurrent unit (GRU)) with several stand-alone methods (LSTM, GRU) in terms of predicting PM concentrations in 39 stations in Seoul. Hourly air pollution data and meteorological data from January 2015 to December 2018 was used for these training models. The results of the experiment confirmed that the proposed prediction model could predict the PM concentrations for the next 7 days. Hybrid models outperformed single models in five areas selected randomly with the lowest root mean square error (RMSE) and mean absolute error (MAE) values for both PM10 and PM2.5. The error rate for PM10 prediction in Gangnam with RMSE is 1.688, and MAE is 1.161. For hybrid models, the CNN–GRU better-predicted PM10 for all stations selected, while the CNN–LSTM model performed better on predicting PM2.5.

Mapping Landslides on EO Data: Performance of Deep Learning Models vs. Traditional Machine Learning Models

Mapping landslides using automated methods is a challenging task, which is still largely done using human efforts. Today, the availability of high-resolution EO data products is increasing exponentially, and one of the targets is to exploit this data source for the rapid generation of landslide inventory. Conventional methods like pixel-based and object-based machine learning strategies have been studied extensively in the last decade. In addition, recent advances in CNN (convolutional neural network), a type of deep-learning method, has been widely successful in extracting information from images and have outperformed other conventional learning methods. In the last few years, there have been only a few attempts to adapt CNN for landslide mapping. In this study, we introduce a modified U-Net model for semantic segmentation of landslides at a regional scale from EO data using ResNet34 blocks for feature extraction. We also compare this with conventional pixel-based and object-based methods. The experiment was done in Douglas County, a study area selected in the south of Portland in Oregon, USA, and landslide inventory extracted from SLIDO (Statewide Landslide Information Database of Oregon) was considered as the ground truth. Landslide mapping is an imbalanced learning problem with very limited availability of training data. Our network was trained on a combination of focal Tversky loss and cross-entropy loss functions using augmented image tiles sampled from a selected training area. The deep-learning method was observed to have a better performance than the conventional methods with an MCC (Matthews correlation coefficient) score of 0.495 and a POD (probability of detection) rate of 0.72 .

Speed Distribution Prediction of Freight Vehicles on Mountainous Freeway Using Deep Learning Methods

Driving speed is one of the most critical indicators in safety evaluation and network monitoring in freight transportation. Speed prediction model serves as the most efficient method to obtain the data of driving speed. Current speed prediction models mostly focus on operating speed, which is hard to reveal the overall condition of driving speed on the road section. Meanwhile, the models were mostly developed based on the regression method, which is inconsistent with natural driving process. Recurrent neural network (RNN) is a distinctive type of deep learning method to capture the temporary dependency in behavioral research. The aim of this paper is to apply the deep learning method to predict the general condition of driving speed in consideration of the road geometry and the temporal evolutions. 3D mobile mapping was applied to obtain road geometry information with high precision, and driving simulation experiment was then conducted with the help of the road geometry data. Driving speed was characterized by the bimodal Gauss mixture model. RNN and its variants including long short-term memory (LSTM) and RNN and gated recurrent units (GRUs) were utilized to predict speed distribution in a spatial-temporal dimension with KL divergence being the loss function. The result proved the applicability of the model in speed distribution prediction of freight vehicles, while LSTM holds the best performance with the length of input sequence being 400 m. The result can be related to the threshold of drivers’ information processing on mountainous freeway. Multiple linear regression models were constructed to be a contrast with the LSTM model, and the results showed that LSTM was superior to regression models in terms of the model accuracy and interpretability of the driving process and the formation of vehicle speed. This study may help to understand speed change behavior of freight vehicles on mountainous freeways, while providing the feasible method for safety evaluation or network efficiency analysis.