Bernoulli Restricted Boltzmann Machine Research Articles

Research on a Feature Enhancement Extraction Method for Underwater Targets Based on Deep Autoencoder Networks

The low-frequency line spectrum of the radiated noise signals of hydroacoustic targets contains features describing the intrinsic properties of the target that make the target susceptible to exposure. In order to extract the line spectral features of underwater acoustic targets, a method combining image processing and a deep autoencoder network (DAE) is proposed in this paper to enhance the low-frequency weak line spectrum of underwater targets in an extremely low signal-to-noise ratio environment based on the measured data of large underwater vehicles. A Gauss–Bernoulli restricted Boltzmann machine (G–BRBM) for real-value signal processing was designed and programmed by introducing a greedy algorithm. On this basis, the encoding and decoding mechanism of the DAE network was used to eliminate interference from environmental noise. The weak line spectrum features were effectively enhanced and extracted under an extremely low signal-to-noise ratio of 10–300 Hz, after which the reconstruction results of the line spectrum features were obtained. Data from large underwater vehicles detected by far-field sonar arrays were processed and the results show that the method proposed in this paper was able to adaptively enhance the line spectrum in a data-driven manner. The DAE method was able to achieve more than double the extractable line spectral density in the frequency band of 10–300 Hz. Compared with the traditional feature enhancement extraction method, the DAE method has certain advantages for the extraction of weak line spectra.

Monitoring and diagnosis of complex production process based on free energy of Gaussian–Bernoulli restricted Boltzmann machine

Monitoring and diagnosis of complex production process based on free energy of Gaussian–Bernoulli restricted Boltzmann machine

Channel Estimation for UAV Communication Systems Using Deep Neural Networks

Channel modeling of unmanned aerial vehicles (UAVs) from wireless communications has gained great interest for rapid deployment in wireless communication. The UAV channel has its own distinctive characteristics compared to satellite and cellular networks. Many proposed techniques consider and formulate the channel modeling of UAVs as a classification problem, where the key is to extract the discriminative features of the UAV wireless signal. For this issue, we propose a framework of multiple Gaussian–Bernoulli restricted Boltzmann machines (GBRBM) for dimension reduction and pre-training utilization incorporated with an autoencoder-based deep neural network. The developed system used UAV measurements of a town’s already existing commercial cellular network for training and validation. To evaluate the proposed approach, we run ray-tracing simulations in the program Remcom Wireless InSite at a distinct frequency of 28 GHz and used them for training and validation. The results demonstrate that the proposed method is accurate in channel acquisition for various UAV flying scenarios and outperforms the conventional DNNs.

Bluetooth Indoor Localization With Gaussian–Bernoulli Restricted Boltzmann Machine Plus Liquid State Machine

The location information of sensors and devices plays an important role in Internet of Things (IoT). Low cost and energy efficient Bluetooth Low Energy (BLE)-based localization solutions are ideal for extensive use in IoT applications. The main challenge is to combat the signal fluctuation in non-line-of-sight (NLOS) propagation. Most Machine Learning (ML) algorithms available in the literature to address this issue belong to classification and batch learning. This article proposes a novel online learning algorithm for BLE localization based on Gaussian–Bernoulli Restricted Boltzmann Machine (GBRBM) plus Liquid State Machine (LSM), which learns the training data one-by-one. Unsupervised GBRBM is able to extract sound patterns of fluctuated RSS inputs, and LSM manages to map the patterns to real-time position estimation. Extensive experimental results demonstrate the superiority of the proposed method over other state-of-the-art batch learning methods in terms of localization accuracy and complexity.

An efficiency-enhanced deep learning model for citywide crowd flows prediction

The crowd flows prediction plays an important role in urban planning management and urban public safety. Accuracy is a challenge for predicting the flow of crowds in a region. On the one hand, crowd flow is influenced by many factors such as holidays and weather. On the other hand, sample data about crowd flows are generally high-dimensional, which not only has a negative impact on the prediction accuracy but also increases computational complexity. In this paper, an efficiency-enhanced model is constructed for predicting citywide crowd flows based on multi-source data using deep learning techniques. Specifically, a data reconstruction mechanism is built with Bernoulli restricted Boltzmann machine (BRBM), for the purpose of reducing the dimension of sample data. A collaborative prediction mechanism is introduced to improve the prediction accuracy of crowd flows, in which a spatio-temporal data oriented prediction model is constructed based on bottleneck residual network that can reduce the effectively computational complexity of model training, and an auxiliary prediction to further optimize the prediction accuracy based on the fully-connected network. The proposed method is evaluated by using two open datasets. The experimental results show that our method can significantly improve the prediction accuracy and reduce the training time of the prediction model, compared with other methods.

Deep Learning-Based Trajectory Tracking Control forUnmanned Surface Vehicle

Trajectory tracking control based on waypoint behavior is a promising way for unmanned surface vehicle (USV) to achieve autonomous navigation. This study is aimed at the guidance progress in the kinematics; the artificial intelligence method of deep learning is adopted to improve the trajectory tracking level of USV. First, two deep neural network (DNN) models are constructed to evaluate navigation effects and to estimate guidance law parameters in real time, respectively. We then pretrain the DNN using a Gaussian–Bernoulli restricted Boltzmann machine to further improve the accuracy of predicting navigation effect. Finally, two DNNs are connected in parallel with the control loop of USV to provide predictive supervision and auxiliary decision making for traditional control methods. This kind of parallel way conforms to the ship manipulation of habit. Furthermore, we develop a new application on the basis of Mission Oriented Operating Suite Interval Programming named “pDeepLearning.” It can predict the navigation effect online by DNN and adjust the guidance law parameters according to the effect level. The experimental results show that, compared with the original waypoint behavior of USV, the prediction model proposed in this study reduces the trajectory tracking error by 19.0% and increases the waypoint behavior effect level.

Bus travel time prediction based on deep belief network with back-propagation

In an intelligent transportation system, accurate bus information is vital for passengers to schedule their departure time and make reasonable route choice. In this paper, an improved deep belief network (DBN) is proposed to predict the bus travel time. By using Gaussian–Bernoulli restricted Boltzmann machines to construct a DBN, we update the classical DBN to model continuous data. In addition, a back-propagation (BP) neural network is further applied to improve the performance. Based on the real traffic data collected in Shenyang, China, several experiments are conducted to validate the technique. Comparison with typical forecasting methods such as k-nearest neighbor algorithm (k-NN), artificial neural network (ANN), support vector machine (SVM) and random forests (RFs) shows that the proposed method is applicable to the prediction of bus travel time and works better than traditional methods.

Real-Time Fault Detection for IIoT Facilities Using GBRBM-Based DNN

Fault detection is a fundamental requirement for Industrial Internet of Things (IIoT), such as the process industry. This article first reviews the recent studies focusing on applying the fault detection techniques to the IIoT networks. However, we find that numerous studies focus on the resource utilization and workload allocation. The fault detection toward IIoT facilities is still in its immature stage because the existing approaches are not accurate enough for the stringent fault detection in IIoT networks. To this end, we present a novel algorithm, named Gaussian Bernoulli restricted Boltzmann machines (GBRBMs)-based deep neural network (DNN), to transform the fault detection into a classification problem. The real trace-driven experiments show that the proposed scheme outperforms other baseline machine learning methods. We anticipate that this article can inspire blooming studies on the related topics of smart IIoT networks.

Improved Gaussian–Bernoulli restricted Boltzmann machine for learning discriminative representations

Restricted Boltzmann machines (RBMs) have received considerable research interest in recent years because of their capability to discover latent representations in an unsupervised manner. The standard RBM is only suitable for processing binary-valued data. To address this limitation, the Gaussian–Bernoulli RBM (GRBM) has been designed to model real-valued data, particularly images. A GRBM that seeks to map real-valued data nonlinearly into a latent representation space is typically trained by contrastive divergence learning. However, most existing GRBM-based models neglect the inherent interpoint affinity information from the original data that can be used to enhance the expression ability of the model. In this study, a novel interpoint-affinity-based GRBM (abGRBM) is proposed to learn discriminative representations in the hidden layer. By incorporating the interpoint affinity information into the training process of the GRBM, the proposed model can not only utilize the GRBM’s powerful latent representation learning capabilities for real-valued data, it can also transform the original data into another space with improved separability. We prove the availability of our model using several image datasets of Microsoft Research Asia Multimedia for unsupervised clustering and supervised classification tasks. The experimental results show the superior performance of the GRBM in discovering discriminative representations and demonstrate the effectiveness of the affinity information.

The Market Generator

We propose to use a special type of generative neural networks - a Restricted Boltzmann Machine (RBM) - to build a powerful generator of synthetic market data that can replicate the probability distribution of the original market data. An RBM constructed with stochastic binary activation units in both the hidden and the visible layers (Bernoulli RBM) can learn complex dependence structures while avoiding overfitting. In this paper we consider an efficient data transformation and sampling approach that allows us to operate Bernoulli RBM on real-valued data sets and control the degree of autocorrelation and non-stationarity in the generated time series.

A Parallel Gaussian–Bernoulli Restricted Boltzmann Machine for Mining Area Classification With Hyperspectral Imagery

In this paper, a novel feature extraction method is proposed for hyperspectral image classification using a Gaussian–Bernoulli restricted Boltzmann machine (GBRBM) in parallel. The proposed approach employs several GBRBMs with different hidden layers to extract deep features from hyperspectral images, which are nonlinear and local invariant. Based on the learned deep features, a logistic regression layer is trained for classification. The proposed approaches are carried out on two public hyperspectral datasets: Pavia University dataset and Salinas dataset, and a new dataset obtained by HySpex imaging spectrometer in the mining area in Xuzhou. The obtained results reveal that the proposed approach offers superior performance compared to traditional classifiers. The advantage of the proposed GBRBM is that it can extract deep features in an unsupervised way and reduce the prediction time by using GPU. In particular, the classification results of the mining area provide valuable suggestions to improve environmental protection.

From Class-Specific to Class-Mixture: Cascaded Feature Representations via Restricted Boltzmann Machine Learning

In this paper, we propose two kinds of feature extracting frameworks that can extract cascaded class-specific and class-mixture features, respectively, by taking the restricted Boltzmann machine (RBM) as the basic building blocks; we further call them as a CS-RBM and CM-RBM feature extractor. The discriminations of features from both CS-RBM and CM-RBM are verified better than the class-independent (traditional) RBM (CI-RBM) feature extractor. As one mini-batch samples are randomly selected from all classes during the training phase of the traditional RBM, which can make that the above mini-batch data contain easy-confusing samples from different categories. Therefore, the features from CI-RBM are difficult to distinguish these samples from the confused categories. CS-RBM and CM-RBM can overcome the above sample confusing problem efficiently and effectively. To cope with the real-valued input samples, we further extend the binary RBM to Gaussian–Bernoulli RBM (GBRBM), leading to the CS-GBRBM (CM-GBRBM) feature extracting framework. Experiments on binary datasets, i.e., MNIST and USPS, scene image dataset (Scene-15), and object image dataset (Coil-100), well verify the above facts and show the competitive results.

Learning framework of multimodal Gaussian–Bernoulli RBM handling real-value input data

Abstract The conventional Gaussian–Bernoulli restricted Boltzmann machine (GBRBM), which is a RBM model for processing real-valued data, presumes single Gaussian distribution for learning real numbers. However, a single distribution is not able to effectively reflect complex data in many cases of real applications. In order to overcome this limitation, Gaussian mixture model (GMM) based RBM is proposed. As a learning mechanism for the proposed model, an energy function handling multi-modal distribution is provided. Then, a memetic algorithm (MA) was applied in order to train the proposed framework more accurately in real-valued input data. In order to show the effectiveness of the proposed framework, the method is applied to image reconstructions. The experiments show that the proposed framework provides more valid results than the other RBM based models in reconstruction error. Through the experiment results, it is concluded that the proposed framework is able to apply real-valued input data extensively and reduce difficulties of learning parameters by capturing the characteristics of real-value input data using GMM.

POL-SAR Image Classification Based on Wishart DBN and Local Spatial Information

Inspired by a popular deep neural network, i.e., deep belief network (DBN), a novel method for polarimetric synthetic aperture radar (POL-SAR) image classification is proposed in this paper. For the particularity of POL-SAR data, a new type of restricted Boltzmann machine (RBM) is specially defined, which we name the Wishart–Bernoulli RBM (WBRBM), and is used to form a deep network named as Wishart DBN (W-DBN). Numerous unlabeled POL-SAR pixels are made full use of in the modeling of POL-SAR pixels by W-DBN. In addition, the coherency matrix is used directly to represent a POL-SAR pixel without any manual feature extraction, which is simple and time saving. Local spatial information, together with the confusion matrix, is used in this paper to clean the preliminary classification result obtained by the method based on W-DBN. Making full use of the prior knowledge of POL-SAR data and local spatial information, the proposed method overcomes shortcomings of traditional methods, in which they are sensitive to extracted features and slow to execute. The experiments, tested on three POL-SAR data sets, show that the proposed method produces better results and is much faster than traditional methods.

Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units

The restricted Boltzmann machine (RBM) is an essential constituent of deep learning, but it is hard to train by using maximum likelihood (ML) learning, which minimizes the Kullback–Leibler (KL) divergence. Instead, contrastive divergence (CD) learning has been developed as an approximation of ML learning and widely used in practice. To clarify the performance of CD learning, in this paper, we analytically derive the fixed points where ML and CDn learning rules converge in two types of RBMs: one with Gaussian visible and Gaussian hidden units and the other with Gaussian visible and Bernoulli hidden units. In addition, we analyze the stability of the fixed points. As a result, we find that the stable points of CDn learning rule coincide with those of ML learning rule in a Gaussian–Gaussian RBM. We also reveal that larger principal components of the input data are extracted at the stable points. Moreover, in a Gaussian–Bernoulli RBM, we find that both ML and CDn learning can extract independent components at one of stable points. Our analysis demonstrates that the same feature components as those extracted by ML learning are extracted simply by performing CD1 learning. Expanding this study should elucidate the specific solutions obtained by CD learning in other types of RBMs or in deep networks.