Discriminative Deep Belief Network Research Articles

A Pseudo-Satellite Fingerprint Localization Method Based on Discriminative Deep Belief Networks

Pseudo-satellite technology has excellent compatibility with the BDS satellite navigation system in terms of signal systems. It can serve as a stable and reliable positioning signal source in signal-blocking environments. User terminals can achieve continuous high-precision positioning both indoors and outdoors without any modification to the navigation module. As a result, pseudo-satellite indoor positioning has gradually emerged as a research hotspot in the field. However, due to the complex and variable indoor radio propagation environment, signal propagation is interfered with by noise, multipath, non-line-of-sight (NLOS) propagation, etc. The geometric relation-based localization algorithm cannot be applied in indoor non-line-of-sight environments. Therefore, this paper proposes a pseudo-satellite fingerprint localization method based on the discriminative deep belief networks (DDBNs). The method acquires the model parameters of pseudo-satellite multi-carrier noise density signal strength in non-line-of-sight indoor spaces through a greedy unsupervised learning method and gradient descent-supervised learning method. It establishes a mapping relationship between the implied features of the pseudo-satellite multi-carrier noise density signal strength and indoor location, enabling pseudo-satellite fingerprint matching localization in indoor non-line-of-sight environments. In this paper, the performance of the positioning algorithm is verified in dynamic and static scenarios through numerous experiments in a laboratory environment. Compared to the commonly used localization algorithms based on fingerprint library matching, the results demonstrate that, in indoor non-line-of-sight test conditions, the system’s 2D static positioning has a maximum error of less than 0.24 m, an RMSE better than 0.12 m, and a 2σ (95.4%) positioning error better than 0.19 m. For 2D dynamic positioning, the maximum error is less than 0.36 m, the average error is 0.23 m, and the 2σ positioning error is better than 0.26 m. These results effectively tackle the challenge of pseudo-satellite indoor positioning in non-line-of-sight environments.

Detection of microcalcification clusters regions in mammograms combining discriminative deep belief networks

In order to overcome the shortcomings of high false positive rate and poor generalization in the detection of microcalcification clusters regions, this paper proposes a method combining discriminative deep belief networks (DDBNs) to automatically and quickly locate the regions of microcalcification clusters in mammograms. Firstly, the breast region was extracted and enhanced, and the enhanced breast region was segmented to overlapped sub-blocks. Then the sub-block was subjected to wavelet filtering. After that, DDBNs model for breast sub-block feature extraction and classification was constructed, and the pre-trained DDBNs was converted to deep neural networks (DNN) using a softmax classifier, and the network is fine-tuned by back propagation. Finally, the undetected mammogram was inputted to complete the location of suspicious lesions. By experimentally verifying 105 mammograms with microcalcifications from the Digital Database for Screening Mammography (DDSM), the method obtained a true positive rate of 99.45% and a false positive rate of 1.89%, and it only took about 16 s to detect a 2 888 × 4 680 image. The experimental results showed that the algorithm of this paper effectively reduced the false positive rate while ensuring a high positive rate. The detection of calcification clusters was highly consistent with expert marks, which provides a new research idea for the automatic detection of microcalcification clusters area in mammograms.

Cyber Intrusion Prevention for Large-Scale Semi-Supervised Deep Learning Based on Local and Non-Local Regularization

The cyber intrusion prevention model represents a new means of cyber protection with intelligent defense capability. It can not only detect intrusion behavior but also respond to such behavior in a timely manner. This study applies deep learning theory and semi-supervised clustering to cyber intrusion prevention technology. Deep learning based on deep structures represents the current development trend of neural networks. Semi-supervised learning uses a large amount of unlabeled cyber traffic data and a small amount of labeled cyber traffic data to achieve cyber intrusion prevention with a low recognition error rate. Discriminative deep belief network (DDBN)-based cyber defense technology has emerged as a research hotspot in the field of cyber intrusion prevention owing to its low error rate. This paper proposes a cyber intrusion prevention technology using DDBN for large-scale semi-supervised deep learning based on local and non-local regularization to overcome the problem of high classification error rates of the cyber intrusion prevention model. Through comparisons with the cyber intrusion prevention results of the Hopfield, support vector machine (SVM), generative adversarial network (GAN), and deep belief network-random forest (DBN-RFS) classifiers, the proposed DDBN model is shown to have the lowest error rate. Thus, the proposed approach can improve the performance of the cyber intrusion prevention system. The training and testing error rates of the exponent loss function with local and non-local regularization (exponent with LNR) are lower than those of the exponent, square, and hinge loss functions. The experimental results show that the running time decreases as the number of hidden layers increases, especially with 6144 and 4096 hidden layer nodes.

Automatic classification method of arrhythmia based on discriminative deep belief networks

Existing arrhythmia classification methods usually use manual selection of electrocardiogram (ECG) signal features, so that the feature selection is subjective, and the feature extraction is complex, leaving the classification accuracy usually affected. Based on this situation, a new method of arrhythmia automatic classification based on discriminative deep belief networks (DDBNs) is proposed. The morphological features of heart beat signals are automatically extracted from the constructed generative restricted Boltzmann machine (GRBM), then the discriminative restricted Boltzmann machine (DRBM) with feature learning and classification ability is introduced, and arrhythmia classification is performed according to the extracted morphological features and RR interval features. In order to further improve the classification performance of DDBNs, DDBNs are converted to deep neural network (DNN) using the Softmax regression layer for supervised classification in this paper, and the network is fine-tuned by backpropagation. Finally, the Massachusetts Institute of Technology and Beth Israel Hospital Arrhythmia Database (MIT-BIH AR) is used for experimental verification. For training sets and test sets with consistent data sources, the overall classification accuracy of the method is up to 99.84% ± 0.04%. For training sets and test sets with inconsistent data sources, a small number of training sets are extended by the active learning (AL) method, and the overall classification accuracy of the method is up to 99.31% ± 0.23%. The experimental results show the effectiveness of the method in arrhythmia automatic feature extraction and classification. It provides a new solution for the automatic extraction of ECG signal features and classification for deep learning.

Discriminative Autoencoder for Feature Extraction: Application to Character Recognition

Conventionally, autoencoders are unsupervised representation learning tools. In this work, we propose a novel discriminative autoencoder. Use of supervised discriminative learning ensures that the learned representation is robust to variations commonly encountered in image datasets. Using the basic discriminating autoencoder as a unit, we build a stacked architecture aimed at extracting relevant representation from the training data. The efficiency of our feature extraction algorithm ensures a high classification accuracy with even simple classification schemes like KNN (K-nearest neighbor). We demonstrate the superiority of our model for representation learning by conducting experiments on standard datasets for character/image recognition and subsequent comparison with existing supervised deep architectures like class sparse stacked autoencoder and discriminative deep belief network.

Discriminative Deep Belief Network for Indoor Environment Classification Using Global Visual Features

Indoor environment classification, also known as indoor environment recognition, is a highly appreciated perceptual ability in mobile robots. In this paper, we present a novel approach which is centered on biologically inspired methods for recognition and representation of indoor environments. First, global visual features are extracted by using the GIST descriptor, and then we use the subsequent features for training the discriminative deep belief network (DDBN) classifier. DDBN employs a new deep architecture which is based on restricted Boltzmann machines (RBMs) and the joint density model. The back-propagation technique is used over the entire classifier to fine-tune the weights for an optimum classification. The acquired experimental results validate our approach as it performs well both in the real-world and in synthetic datasets and outperforms the Convolution Neural Networks (ConvNets) in terms of computational efficiency.

Deep learning framework for early detection of intrusion in Virtual Environment

Today's business enterprise adapts cloud-based services as its architectural design. Intelligence technique incorporated into the architecture gives massive tangible and intangible benefits in terms of performance and reliability. Such cloud-based business architecture faces many threats towards its availability. DDoS attack is the most prominent threat as its impact is more in the virtual resource-based cloud infrastructure. Therefore, there is a need for a business intelligence-based framework to detect early the attack by monitoring the virtual network traffic. The proposed intelligence framework uses a deep learning framework, continuous discriminative-deep belief network (CD-DBN). CD-DBN dynamically captures attack patterns from the network data, analyses the data and detects the intrusion to the cloud. The observed result shows that the earlier detection approach guarantees the availability of cloud services to the legitimate users and enhances the cloud resource usage.

Discriminative Deep Belief Networks with Ant Colony Optimization for Health Status Assessment of Machine

On-line health status monitoring, a key part of prognostics and health management, provides various benefits, such as preventing unexpected failure and improving safety and reliability. In this paper, a data-driven approach for health status assessment is presented. A novel method based on discriminative deep belief networks (DDBN) and ant colony optimization (ACO) is used to predict health status of machine. DDBN is a new paradigm that utilizes a deep architecture to combine the advantages of deep belief networks and discriminative ability of back-propagation strategy. DDBN works through a greedy layer-by-layer training with multiple stacked restricted Boltzmann machines, which preserves information well when embedding features from high-dimensional space to low-dimensional space. However, selecting the parameters of DDBN is quite challenging. To address the problem, ACO is introduced to DDBN in this paper. By optimization, the structure of DDBN model is determined automatically without prior knowledge and the performance is enhanced. To evaluate the proposed approach, two case studies were carried out, which shows that it can achieve a good result. The performance of this model is also compared with support vector machine. It is concluded that the proposed method is very promising in the field of prognostics.

Imposing Class-Wise Feature Similarity in Stacked Autoencoders by Nuclear Norm Regularization

This work proposes a new formulation for supervised stacked autoencoder. We argue that features from the same class should be similar to each other and hence linearly dependent. This means that, when stacked as columns, the feature matrix for each class will be rank deficient (low-rank). We impose this constraint into the stacked autoencoder formulation in the form of nuclear norm penalties on class-wise feature matrices at each level. The nuclear norm penalty is the convex surrogate of rank, and promotes a low-rank solution as desired by our proposal. Owing to the nuclear norm penalties, our formulation is non-smooth; hence cannot be solved using gradient descent based techniques like backpropagation directly. Moreover we learn the stacked autoencoder in one go, without the usual pre-training followed by fine-tuning regime. Both the ends (non-smooth cost function and single stage training for all the layers simultaneously) are met by employing the variable splitting followed by augmented Lagrangian method of alternating directions. Two sets of experiments have been carried out. The first set is on a variety of benchmark datasets. Our method excels over other deep learning models compared against—class sparse stacked autoencoder, deep belief network and discriminative deep belief network. The second experiment is on the brain computer classification problem; we find that our method outperforms prior deep learning based solutions utilized for this task.

Discriminative deep belief networks for visual data classification

Visual data classification using insufficient labeled data is a well-known hard problem. Semi-supervise learning, which attempts to exploit the unlabeled data in additional to the labeled ones, has attracted much attention in recent years. This paper proposes a novel semi-supervised classifier called discriminative deep belief networks (DDBN). DDBN utilizes a new deep architecture to integrate the abstraction ability of deep belief nets (DBN) and discriminative ability of backpropagation strategy. For unsupervised learning, DDBN inherits the advantage of DBN, which preserves the information well from high-dimensional features space to low-dimensional embedding. For supervised learning, through a well designed objective function, the backpropagation strategy directly optimizes the classification results in training dataset by refining the parameter space. Moreover, we apply DDBN to visual data classification task and observe an important fact that the learning ability of deep architecture is seriously underrated in real-world applications, especially in visual data analysis. The comparative experiments on standard datasets of different types and different scales demonstrate that the proposed algorithm outperforms both representative semi-supervised classifiers and existing deep learning techniques. For visual dataset, we can further improve the DDBN performance with much larger and deeper architecture.