Layers Of Restricted Boltzmann Machine Research Articles

Deep belief network based statistical feature learning for fingerprint liveness detection

Fingerprint recognition systems are vulnerable to impersonation by fake or spoof fingerprints. Fingerprint liveness detection is a step to ensure whether a scanned fingerprint is live or fake prior to a recognition step. This paper presents a fingerprint liveness detection method based on a deep belief network (DBN). A DBN with multiple layers of restricted Boltzmann machine is used to learn features from a set of live and fake fingerprints and also to detect the liveness. The proposed method is a systematic application of a deep learning technique, and does not require specific domain expertise regarding fake fingerprints or recognition systems. The proposed method provides accurate detection of the liveness with various sensor datasets collected for the international fingerprint liveness detection competition.

A Multilayer Improved RBM Network Based Image Compression Method in Wireless Sensor Networks

The processing capacity and power of nodes in a Wireless Sensor Network (WSN) are limited. And most image compression algorithms in WSN are subject to random image content changes or have low image qualities after the images are decoded. Therefore, an image compression method based on multilayer Restricted Boltzmann Machine (RBM) network is proposed in this paper. The alternative iteration algorithm is also applied in RBM to optimize the training process. The proposed image compression method is compared with a region of interest (ROI) compression method in simulations. Under the same compression ratio, the qualities of reconstructed images are better than that of ROI. When the number of hidden units in top RBM layer is 8, the peak signal-to-noise ratio (PSNR) of the multilayer RBM network compression method is 74.2141, and it is much higher than that of ROI which is 60.2093. The multilayer RBM based image compression method has better compression performance and can effectively reduce the energy consumption during image transmission in WSN.

V4 Neural Network Model for Shape-Based Feature Extraction and Object Discrimination

Visual area V4 plays an important role in the neural mechanism of shape recognition. V4 neurons exhibit selectivity for the orientation and curvature of boundary fragments. In this paper, we propose a novel neural network model of V4 for shape-based feature extraction and other vision tasks. The low-level layers of the model consist of computational units simulating simple cells and complex cells in the primary visual cortex. These layers extract preliminary visual features including edges and orientations. The V4 computational units calculate the entropy of the extracted features as a measure of visual saliency. The features around salient points are then selected and encoded with a layer of restricted Boltzmann machine to generate an intermediate representation of object shapes. The model is evaluated in shape distinction, feature detection, feature matching, and object discrimination experiments. The results demonstrate that this model generates discriminative local representation of object shapes. It shows a successful attempt to construct a computation model of visual object recognition in the brain.

Object recognition in remote sensing images using sparse deep belief networks

Object recognition has been one of the hottest issues in the field of remote sensing image analysis. In this letter, a new pixel-wise learning method based on deep belief networks (DBNs) for object recognition is proposed. The method is divided into two stages, the unsupervised pre-training stage and the supervised fine-tuning stage. Given a training set of images, a pixel-wise unsupervised feature learning algorithm is utilized to train a mixed structural sparse restricted Boltzmann machine (RBM). After that, the outputs of this RBM are put into the next RBM as inputs. By stacking several layers of RBM, the deep generative model of DBNs is built. At the fine-tuning stage, a supervised layer is attached to the top of the DBN and labels of the data are put into this layer. The whole network is then trained using the back-propagation (BP) algorithm with sparse penalty. Finally, the deep model generates good joint distribution of images and their labels. Comparative experiments are conducted on our dataset acquired by QuickBird with 60 cm resolution and the recognition results demonstrate the accuracy and efficiency of our proposed method.

Analysis of Different Sparsity Methods in Constrained RBM for Sparse Representation in Cognitive Robotic Perception

Cognitive robotic systems nowadays are intensively involving learning algorithms to achieve highly adaptive and intelligent behaviors, including actuation, sensing, perception and adaptive control. Deep learning has emerged as an effective approach in image-based robotic perception and actions. Towards cognitive robotic perception based on deep learning, this paper focuses the Constrained Restricted Boltzmann Machine (RBM) on visual images for sparse feature representation. Inspired by sparse coding, the sparse constraints are performed on the hidden layer of RBM to obtain sparse and effective feature representation from perceived visual images. The RBM with Sparse Constraint (RBMSC) is proposed with a generalized optimization problem, where the constraints are applied on the probability density of hidden units directly to obtain more sparse representation. This paper presents three novel RBM variants, namely L1-RBM, L2-RBM, and L1/2-RBM constrained by L1-norm, L2-norm, and L1/2-norm on RBM, respectively. A Deep Belief Network with two hidden layers is built for comparison between each RBM variants. The experiments on MNIST database (Mixed National Institute of Standards and Technology database) show that the L1/2-RBM can obtain more sparse representation than RBM, L1-RBM, L2-RBM, and Sparse-RBM (SRBM) in terms of sparseness metric. For further verification, the proposed methods are still tested on MNIST Variations dataset. The recognition results from perceived images in MNIST and MNIST Variations demonstrate that our proposed constrained RBM variants are feasible for object cognitive and perception, and the proposed L1/2-RBM and L1-RBM outperforms RBM and SRBM in terms of object recognition.

Time series forecasting using a deep belief network with restricted Boltzmann machines

Multi-layer perceptron (MLP) and other artificial neural networks (ANNs) have been widely applied to time series forecasting since 1980s. However, for some problems such as initialization and local optima existing in applications, the improvement of ANNs is, and still will be the most interesting study for not only time series forecasting but also other intelligent computing fields. In this study, we propose a method for time series prediction using Hinton and Salakhutdinov׳s deep belief nets (DBN) which are probabilistic generative neural network composed by multiple layers of restricted Boltzmann machine (RBM). We use a 3-layer deep network of RBMs to capture the feature of input space of time series data, and after pretraining of RBMs using their energy functions, gradient descent training, i.e., back-propagation learning algorithm is used for fine-tuning connection weights between “visible layers” and “hidden layers” of RBMs. To decide the sizes of neural networks and the learning rates, Kennedy and Eberhart׳s particle swarm optimization (PSO) is adopted during the training processes. Furthermore, “trend removal”, a preprocessing to the original data, is also approached in the forecasting experiment using CATS benchmark data. Additionally, approximating and short-term prediction of chaotic time series such as Lorenz chaos and logistic map were also applied by the proposed method.

Chinese Relation Extraction Based on Deep Belief Nets

PDF HTML阅读 XML下载 导出引用 引用提醒 基于 Deep Belief Nets 的中文名实体关系抽取 DOI: 10.3724/SP.J.1001.2012.04181 作者: 作者单位: 作者简介: 通讯作者: 中图分类号: 基金项目: 国家自然科学基金(61073130); 国家高技术研究发展计划(863)(2011AA01A207) Chinese Relation Extraction Based on Deep Belief Nets Author: Affiliation: Fund Project: 摘要 | 图/表 | 访问统计 | 参考文献 | 相似文献 | 引证文献 | 资源附件 | 文章评论 摘要:关系抽取是信息抽取的一项子任务,用以识别文本中实体之间的语义关系.提出一种利用 DBN(deepbelief nets)模型进行基于特征的实体关系抽取方法,该模型是由多层无监督的 RBM(restricted Boltzmann machine)网络和一层有监督的 BP(back-propagation)网络组成的神经网络分类器. RBM 网络以确保特征向量映射达到最优,最后一层 BP 网络分类 RBM 网络的输出特征向量,从而训练实体关系分类器.在 ACE04 语料上进行的相关测试,一方面证明了字特征比词特征更适用于中文关系抽取任务;另一方面设计了 3 组不同的实验,分别使用正确的实体类别信息、通过实体类型分类器得到实体类型信息和不使用实体类型信息,用以比较实体类型信息对关系抽取效果的影响.实验结果表明,DBN 非常适用于基于高维空间特征的信息抽取任务,获得的效果比 SVM 和反向传播网络更好. Abstract:Relation extraction is a fundamental task in information extraction, which is to identify the semanticrelationships between two entities in the text. In this paper, deep belief nets (DBN), which is a classifier of acombination of several unsupervised learning networks, named RBM (restricted Boltzmann machine) and asupervised learning network named BP (back-propagation), is presented to detect and classify the relationshipsamong Chinese name entities. The RBM layers maintain as much information as possible when feature vectors aretransferred to next layer. The BP layer is trained to classify the features generated by the last RBM layer. Theexperiments are conducted on the Automatic Content Extraction 2004 dataset. This paper proves that acharacter-based feature is more suitable for Chinese relation extraction than a word-based feature. In addition, thepaper also performs a set of experiments to assess the Chinese relation extraction on different assumptions of anentity categorization feature. These experiments showed the comparison among models with correct entity types andimperfect entity type classified by DBN and without entity type. The results show that DBN is a successfulapproach in the high-dimensional-feature-space information extraction task. It outperforms state-of-the-art learningmodels such as SVM and back-propagation networks. 参考文献 相似文献 引证文献

Tractable Multivariate Binary Density Estimation and the Restricted Boltzmann Forest

We investigate the problem of estimating the density function of multivariate binary data. In particular, we focus on models for which computing the estimated probability of any data point is tractable. In such a setting, previous work has mostly concentrated on mixture modeling approaches. We argue that for the problem of tractable density estimation, the restricted Boltzmann machine (RBM) provides a competitive framework for multivariate binary density modeling. With this in mind, we also generalize the RBM framework and present the restricted Boltzmann forest (RBForest), which replaces the binary variables in the hidden layer of RBMs with groups of tree-structured binary variables. This extension allows us to obtain models that have more modeling capacity but remain tractable. In experiments on several data sets, we demonstrate the competitiveness of this approach and study some of its properties.