Deep Belief Network Algorithm Research Articles

The possibility of the combination of OCT and fundus images for improving the diagnostic accuracy of deep learning for age-related macular degeneration: a preliminary experiment.

Recently, researchers have built new deep learning (DL) models using a single image modality to diagnose age-related macular degeneration (AMD). Retinal fundus and optical coherence tomography (OCT) images in clinical settings are the most important modalities investigating AMD. Whether concomitant use of fundus and OCT data in DL technique is beneficial has not been so clearly identified. This experimental analysis used OCT and fundus image data of postmortems from the Project Macula. The DL based on OCT, fundus, and combination of OCT and fundus were invented to diagnose AMD. These models consisted of pre-trained VGG-19 and transfer learning using random forest. Following the data augmentation and training process, the DL using OCT alone showed diagnostic efficiency with area under the curve (AUC) of 0.906 (95% confidence interval, 0.891-0.921) and 82.6% (81.0-84.3%) accuracy rate. The DL using fundus alone exhibited AUC of 0.914 (0.900-0.928) and 83.5% (81.8-85.0%) accuracy rate. Combined usage of the fundus with OCT increased the diagnostic power with AUC of 0.969 (0.956-0.979) and 90.5% (89.2-91.8%) accuracy rate. The Delong test showed that the DL using both OCT and fundus data outperformed the DL using OCT alone (P value < 0.001) and fundus image alone (P value < 0.001). This multimodal random forest model showed even better performance than a restricted Boltzmann machine (P value = 0.002) and deep belief network algorithms (P value = 0.042). According to Duncan's multiple range test, the multimodal methods significantly improved the performance obtained by the single-modal methods. In this preliminary study, a multimodal DL algorithm based on the combination of OCT and fundus image raised the diagnostic accuracy compared to this data alone. Future diagnostic DL needs to adopt the multimodal process to combine various types of imaging for a more precise AMD diagnosis. Graphical abstract The basic architectural structure of the tested multimodal deep learning model based on pre-trained deep convolutional neural network and random forest using the combination of OCT and fundus image.

Accelerating the image processing by the optimization strategy for deep learning algorithm DBN

In recent years, image processing especially for remote sensing technology has developed rapidly. In the field of remote sensing, the efficiency of processing remote sensing images has been a research hotspot in this field. However, the remote sensing data has some problems when processing by a distributed framework, such as Spark, and the key problems to improve execution efficiency are data skew and data reused. Therefore, in this paper, a parallel acceleration strategy based on a typical deep learning algorithm, deep belief network (DBN), is proposed to improve the execution efficiency of the DBN algorithm in Spark. First, the re-partition algorithm based on the tag set is proposed to the relief data skew problem. Second, the cache replacement algorithm on the basis of characteristics is proposed to automatic cache the frequently used resilient distributed dataset (RDD). By caching RDD, the re-computation time of frequently reused RDD is reduced, which lead to the decrease of total computation time of the job. The numerical and analysis verify the effectiveness of the strategy.

Daily Urban Water Demand Forecasting Based on Chaotic Theory and Continuous Deep Belief Neural Network

The prediction of daily water demands is a crucial part of the effective functioning of the water supply system. This work proposed that a continuous deep belief neural network (CDBNN) model based on the chaotic theory should be implemented to predict the daily water demand time series in Zhuzhou, China. CDBNN should initially be used to predict the urban water demand time series. First, the power spectrum and the largest Lyapunov exponent is used to determine the chaotic characteristic of the daily water demand time series. Second, C–C method is utilized to reconstruct the water demand time series’ phase space. Lastly, the forecasting model should be produced with the continuous deep belief network and neural network algorithms implemented for feature learning and regression, respectively, and the CDBNN input established by the best embedding dimension of the reconstructed phase space. The proposed method is contrasted with the support vector regression, generalized regression neural networks and feed forward neural networks, and they are accepted with the identical dataset. The predictive performance of the models is examined using normalized root-mean-square error (NRMSE), correlation coefficient (COR), and mean absolute percentage error (MAPE). The results suggest that the hybrid model has the smallest NRMSE and MAPE values, and the largest COR.

Classification of sudden deafness prognosis based on deep learning method

Objective:This study aimed to develop predictive models for sudden sensorineural hearing loss through deep belief network (DBN) and explore whether the model performances differ when adopting different outcome criteria. Method: 228 potential predictors involving the clinical characteristics, audio logical data, and serological parameters out of 1 220 hospitalized SSHL patients who were admitted from June 2008 to December 2015 were analyzed retrospectively. The hearing data of sudden deafness were classified into two or four categories based on Chinese criteria and Siegel criteria, which were used to develop the DBN models. The area under the receiver operating characteristic curve (ROC-AUC) and accuracy were used to compare the predictive performance of different models. Result: The DBN model developed for predicting the dichotomized outcomes had better performance than that of the fourcategory outcomes. When the iteration number reached 500 times, DBN model constructed for prediction of dichotomized outcomes based on Siegel's criteria had demonstrated the best performance with an accuracy of 76.25% and an AUC of 0.81. According to indices from first layer weights, DBN gave a rank of top 10 sensitive features for hearing outcome prediction focusing on indicators regarding coagulation, demographics and pre-treatment hearing levels independent of the outcome assessment criteria. Conclusion: DBN provides a robust outcome prediction ability in SSHL datasets with rich and complex variables, especially when utilized to predict dichotomized outcomes based on the Siegel criteria. In addition, this advanced deep learning technique can automatically extract valuable predictors, which is consistent with those that had been verified in previous studies by traditional statistical methods. This study provides further evidence for extending the use of DBN algorithm to the field of developing prediction or classification models for other otological diseases in the future..

Research on EEG recognition based on improved-common spatial patterns and deep-belief network algorithm

Using electroencephalogram (EEG) signals to control intelligent wheelchairs is one of the new control methods for controlling intelligent wheelchairs. The most serious problem in the control process is recognition and classification of EEG signals, especially a variety of EEG signals. To improve the accuracy of EEG classification, improved-common spatial patterns and a deep-belief network algorithm are proposed and used for recognition and classification of EEG signals. A variety of different EEG signals were collected by an Emotiv EPOC+ EEG collector, and the characteristic signals were extracted by improved-common spatial patterns and identified and classified by a deep-belief network algorithm. Simulation results show that the classification accuracy of improved-common spatial patterns and the deep-belief network algorithm is higher than that of the traditional classification method, and a research perspective is provided for the classification of various EEG signals in the future.

A new deep belief network based on RBM with glial chains

A deep belief network (DBN) is an unsupervised learning method that is widely used to build hierarchical structures and learn feature representations based on unlabeled data; however, it is limited in feature acquisition because no interconnections occur between neurons in the same layer. Moreover, information shared between different neurons may be ignored, so deep correlative features of the data cannot be recognized. Inspired by the functions of glial cells in the neural network of the human brain, this paper proposes a variation of DBN based on Restricted Boltzmann Machines (RBMs) with glial chains. Additionally, an improved greedy layer-wise learning algorithm for the new DBN is applied to enhance learning accuracy and extract more information from the data. Furthermore, to ensure the glial effect is maintained, the glial chain is always contained in the supervised fine-tuning process. The experimental results based on benchmark image datasets show that the proposed DBN model can acquire more abstract features and achieve a lower mean squared error (MSE) than traditional DBN and other learning algorithms.

Deep learning with 3D-second order difference plot on respiratory sounds

Abstract The second order difference plot (SODP) is a nonlinear signal analysis method that visualizes two consecutive data points for many types of biomedical signals. The proposed method is based on analysing quantization of 3D-space which is originated using three consecutive data points in signal. The obtained 3D-SODP space was segmented into 3–10 spaces using octants, spheres and cuboid polyhedrons of which centroids are at the origin. Lung sound is an indispensable tool for respiratory and cardiac diseases. The study is focused on classifying the lung sounds from at risk level and the interior level of chronic obstructive pulmonary disease (COPD). The COPD is one of the most deadliest and common respiratory diseases which come into existence as a consequence of smoking. The smokers for a few years are qualified as at risk level of COPD (COPD-0). The 12 channels of lung sounds from the RespiratoryDatabase@TR were utilized in the analysis of the proposed 3D-SODP quantization method. The lung sounds are auscultated synchronously from posterior and anterior sides of subjects using two digital stethoscopes by a pulmonologist clinician in Antakya State Hospital, Turkey. Deep Belief Networks (DBN) algorithm was preferred in the classification stage. It has a greedy layer-wise pre-training which is based on restricted Boltzmann machines and optimizes the pre-trained weights using supervised iterations. The proposed DBN model had 2 hidden layers with 270 and 580 neurons, respectively. The conjunction usage of 3D-SODP quantization features with the DBN separated the lung sounds from different levels of COPD with high classification performance rates of 95.84%, 93.34% and 93.65% for accuracy, sensitivity and specificity, respectively. The results indicate that the 3D-SODP quantization on respiratory sounds has ability to diagnose the levels of the COPD using the deep learning model. Especially, the octant-based quantization is effective on lung sounds with high generalization capability using a small number of feature set dimension.

Traffic flow prediction model based on deep belief network and genetic algorithm

Traffic flow prediction plays an indispensable role in the intelligent transportation system. The effectiveness of traffic control and management relies heavily on the prediction accuracy. The authors propose a model based on deep belief networks (DBNs) to predict the traffic flow. Moreover, they use Fletcher–Reeves conjugate gradient algorithm to optimise the fine-tuning of model's parameters. Since the traffic flow has various features at different times such as weekday, weekend, daytime and night-time, the hyper-parameters of the model should adapt to the time. Therefore, they employ the genetic algorithm to find the optimal hyper-parameters of DBN models for different times. The dataset from Caltrans Performance Measurement System was used to evaluate the performance of their models. The experimental results demonstrate that the proposed model achieved better performance in different times.

Modeling and predicting remanufacturing time of equipment using deep belief networks

Predicting optimal remanufacturing time is a key point for improving the quality performance and economic effectiveness of multi-life cycle of remanufactured equipment, and knowing the manufacturing cost is a critical point for accurate prediction of optimal remanufacturing time. On the basis of understanding equipment multi-life cycle and cost composition, this paper investigated the correlation between optimal remanufacturing time and remanufacturing cost. Due to the randomness and nondeterminacy of remanufacturing cost, the limitation of available data sample, the insufficiency of shallow learning algorithm, we proposed deep belief networks (DBN) prediction model to improve the prediction accuracy of optimal remanufacturing time. In such model, the planning cost, disposal service cost, remanent value, annual cost increment, parts remanufacturing ratio, parts replacement ratio and technical indexes were adopted as inputs, while optimal remanufacturing time was regarded as output. The proposed algorithm can automatically extract more abstract and more expressive characteristics from sample, thus realizing complex nonlinear mapping between input and output data. In this research, DBN algorithm was used to estimate the training samples and test samples of mechanical transmission, and the predicting error of DBN algorithm was only 27% of the BP training network. The results of our prediction experiments verified the feasibility and effectiveness of the proposed model algorithm method.

Pedestrian detection for traffic safety based on Accumulate Binary Haar features and improved deep belief network algorithm

ABSTRACTIn order to improve traffic safety and protect pedestrians, an improved and efficient pedestrian detection method for auto driver assistance systems is proposed. Firstly, an improved Accumulate Binary Haar (ABH) feature extraction algorithm is proposed. In this novel feature, Haar features keep only the ordinal relationship named by binary Haar features. Then, the feature brings in the idea of a Local Binary Pattern (LBP), assembling several neighboring binary Haar features to improve discriminating power and reduce the effect of illumination. Next, a pedestrian classification method based on an improved deep belief network (DBN) classification algorithm is proposed. An improved method of input is constructed using a Restricted Bolzmann Machine (RBM) with T distribution function visible layer nodes, which can convert information on pedestrian features to a Bernoulli distribution, and the Bernoulli distribution can then be used for recognition. In addition, a middle layer of the RBM structure is created, which achieves data transfer between the hidden layer structure and keeps the key information. Finally, the cost-sensitive Support Vector Machine (SVM) classifier is used for the output of the classifier, which could address the class-imbalance problem. Extensive experiments show that the improved DBN pedestrian detection method is better than other shallow classic algorithms, and the proposed method is effective and sufficiently feasible for pedestrian detection in complex urban environments.

Incremental extreme learning machine based on deep feature embedded

Extreme learning machine (ELM) algorithm is used to train Single-hidden Layer Feed forward Neural Networks. And Deep Belief Network (DBN) is based on Restricted Boltzmann Machine (RBM). The conventional DBN algorithm has some insufficiencies, i.e., Contrastive Divergence (CD) Algorithm is not an ideal approximation method to Maximum Likelihood Estimation. And bad parameters selected in RBM algorithm will produce a bad initialization in DBN model so that we will spend more training time and get a low classification accuracy. To solve the problems above, we summarize the features of extreme learning machine and deep belief networks, and then propose Incremental extreme learning machine based on Deep Feature Embedded algorithm which combines the deep feature extracting ability of Deep Learning Networks with the feature mapping ability of extreme learning machine. Firstly, we introduce Manifold Regularization to our model to attenuate the complexity of probability distribution. Secondly, we introduce the semi-restricted Boltzmann machine (SRBM) to our algorithm, and build a deep belief network based on SRBM. Thirdly, we introduce the thought of incremental feature mapping in ELM to the classifier of DBN model. Finally, we show validity of the algorithm by experiments.

Deep Belief Network를 이용한 뇌파의 음성 상상 모음 분류

In this paper, we found the usefulness of the deep belief network (DBN) in the fields of brain-computer interface (BCI), especially in relation to imagined speech. In recent years, the growth of interest in the BCI field has led to the development of a number of useful applications, such as robot control, game interfaces, exoskeleton limbs, and so on. However, while imagined speech, which could be used for communication or military purpose devices, is one of the most exciting BCI applications, there are some problems in implementing the system. In the previous paper, we already handled some of the issues of imagined speech when using the International Phonetic Alphabet (IPA), although it required complementation for multi class classification problems. In view of this point, this paper could provide a suitable solution for vowel classification for imagined speech. We used the DBN algorithm, which is known as a deep learning algorithm for multi-class vowel classification, and selected four vowel pronunciations:, /a/, /i/, /o/, /u/ from IPA. For the experiment, we obtained the required 32 channel raw electroencephalogram (EEG) data from three male subjects, and electrodes were placed on the scalp of the frontal lobe and both temporal lobes which are related to thinking and verbal function. Eigenvalues of the covariance matrix of the EEG data were used as the feature vector of each vowel. In the analysis, we provided the classification results of the back propagation artificial neural network (BP-ANN) for making a comparison with DBN. As a result, the classification results from the BP-ANN were 52.04%, and the DBN was 87.96%. This means the DBN showed 35.92% better classification results in multi class imagined speech classification. In addition, the DBN spent much less time in whole computation time. In conclusion, the DBN algorithm is efficient in BCI system implementation.