Performance Of Restricted Boltzmann Machine Research Articles

Effect of stochastic activation function on reconstruction performance of restricted Boltzmann machines with stochastic magnetic tunnel junctions

Stochastic Magnetic Tunnel Junctions (SMTJs) emerge as a promising candidate for neuromorphic computing. The inherent stochasticity of SMTJs makes them ideal for implementing stochastic synapses or neurons in neuromorphic computing. However, the stochasticity of SMTJs may impair the performance of neuromorphic systems. In this study, we conduct a systematic examination of the influence of three stochastic effects (shift, change of slope, and broadening) on the sigmoid activation function. We further explore the implications of these effects on the reconstruction performance of Restricted Boltzmann Machines (RBMs). We find that the trainability of RBMs is robust against the three stochastic effects. However, reconstruction error is strongly related to the three stochastic effects in SMTJs-based RBMs. Significant reconstruction error is found when the stochastic effect is strong. Last, we identify the correlation of the reconstruction error with each stochastic factor. Our results might help develop more robust neuromorphic systems based on SMTJs.

Anomaly detection speed-up by quantum restricted Boltzmann machines

Quantum machine learning promises to revolutionize traditional machine learning by efficiently addressing hard tasks for classical computation. While claims of quantum speed-up have been announced for gate-based quantum computers and photon-based boson samplers, demonstration of an advantage by adiabatic quantum annealers (AQAs) is open. Here we quantify the computational cost and the performance of restricted Boltzmann machines (RBMs), a widely investigated machine learning model, by classical and quantum annealing. Despite the lower computational complexity of the quantum RBM being lost due to physical implementation overheads, a quantum speed-up may arise as a reduction by orders of magnitude of the computational time. By employing real-world cybersecurity datasets, we observe that the negative phase on sufficiently challenging tasks is computed up to 64 times faster by AQAs during the exploitation phase. Therefore, although a quantum speed-up highly depends on the problem’s characteristics, it emerges in existing hardware on real-world data.

Measuring the Impact of Accurate Feature Selection on the Performance of RBM in Comparison to State of the Art Machine Learning Algorithms

The amassed growth in the size of data, caused by the advancement of technologies and the use of internet of things to collect and transmit data, resulted in the creation of large volumes of data and an increasing variety of data types that need to be processed at very high speeds so that we can extract meaningful information from these massive volumes of unstructured data. The process of mining this data is very challenging since a lot of the data suffers from the problem of high dimensionality. The quandary of high dimensionality represents a great challenge that can be controlled through the process of feature selection. Feature selection is a complex task with multiple layers of difficulty. To be able to grasp and realize the impediments associated with high dimensional data a more and in-depth understanding of feature selection is required. In this study, we examine the effect of appropriate feature selection during the classification process of anomaly network intrusion detection systems. We test its effect on the performance of Restricted Boltzmann Machines and compare its performance to conventional machine learning algorithms. We establish that when certain features that are representative of the model are to be selected the change in the accuracy was always less than 3% across all algorithms. This verifies that the accurate selection of the important features when building a model can have a significant impact on the accuracy level of the classifiers. We also confirmed in this study that the performance of the Restricted Boltzmann Machines can outperform or at least is comparable to other well-known machine learning algorithms. Extracting those important features can be very useful when trying to build a model with datasets with a lot of features.

Approximating power of machine-learning ansatz for quantum many-body states

An artificial neural network (ANN) with the restricted Boltzmann machine (RBM) architecture was recently proposed as a versatile variational quantum many-body wave function. In this work we provide physical insights into the performance of this ansatz. We uncover the connection between the structure of the RBM and perturbation series, which explains the excellent precision achieved by the RBM ansatz in certain simple models, demonstrated in G. Carleo and M. Troyer [Science 355, 602 (2017)]. Based on this relation, we improve the numerical algorithm to achieve a better performance of the RBM in cases where local minima complicate the convergence to the global one. Furthermore, we study the performance of a sparse RBM for approximating ground states of random, translationally invariant models in one dimension, as well as random matrix-product states. We find that the error in approximating such states exhibits a broad distribution and shows a positive correlation with the entanglement properties of the targeted state.

Prevalence and Diagnosis of Neurological Disorders Using Different Deep Learning Techniques: AMeta-Analysis.

This paper dispenses an exhaustive review on deep learning techniques used in the prognosis of eight different neuropsychiatric and neurological disorders such as stroke, alzheimer, parkinson's, epilepsy, autism, migraine, cerebral palsy, and multiple sclerosis. These diseases are critical, life-threatening and in most of the cases may lead to other precarious human disorders. Deep learning techniques are emerging soft computing technique which has been lucratively used to unravel different real-life problems such as pattern recognition (Face, Emotion, and Speech), traffic management, drug discovery, disease diagnosis, and network intrusion detection. This study confers the discipline, frameworks, and methodologies used by different deep learning techniques to diagnose different human neurological disorders. Here, one hundred and thirty-six different articles related to neurological and neuropsychiatric disorders diagnosed using different deep learning techniques are studied. The morbidity and mortality rate of major neuropsychiatric and neurological disorders has also been delineated. The performance and publication trend of different deep learning techniques employed in the investigation of these diseases has been examined and analyzed. Different performance metrics like accuracy, specificity, and sensitivity have also been examined. The research implication, challenges and the future directions related to the study have also been highlighted. Eventually, the research breaches are identified and it iswitnessed that there is more scope in the diagnosis of migraine, cerebral palsy and stroke using different deep learning models. Likewise, there is a potential opportunity to use and explore the performance of Restricted Boltzmann Machine, Deep Boltzmann Machine and Deep Belief Network for diagnosis of different human neuropsychiatric and neurological disorders.

An evaluation of the performance of Restricted Boltzmann Machines as a model for anomaly network intrusion detection

The continuous increase in the number of attacks on computer networks has raised serious concerns regarding the importance of establishing a methodology that can learn and adapt to new and novel attacks, such a model should be able to act or react to such threats within a timely manner, so that measures are undertaken to counter any potential breaches within the network. Training a model to distinguish between normal and anomalous network behavior is a difficult task due to the high dimensionality of the network traffic data. One of the key requirements of a successful Anomaly Network Intrusion Detection Systems (A-NIDS) is the ability to recognize new patterns of attacks that it has never before seen. This objective can be achieved through incorporating machine leaning techniques in the learning model of the A-NIDS. In this study, we demonstrate the use of a powerful machine learning technique called the Restricted Boltzmann Machine (RBM) to distinguish between normal and anomalous NetFlow traffic. We evaluate our approach through testing it on the newly renowned Information Security Center of Excellence (ISCX) dataset. Our results indicate that RBMs can be trained successfully to classify normal and anomalous NetFlow traffic. Unlike previous studies, we employ measures of true positives and negatives along with the accuracy to test the effectiveness of RBM as a classifier for A-NIDS. We also utilize the usage of a balanced set to reduce any biases that appear during the RBM training.

Enhancing performance of restricted Boltzmann machines via log-sum regularization

Restricted Boltzmann machines (RBMs) are often used as building blocks to construct a deep belief network. By optimizing several RBMs, the deep networks can be trained quickly to achieve good performance on the tasks of interest. To further improve the performance of data representation, many researches focus on incorporating sparsity into RBMs. In this paper, we propose a novel sparse RBM model, referred to as LogSumRBM. Instead of constraining the expected activation of every hidden unit to the same low level of sparsity as done in [27], we explicitly encourage the hidden units to be sparse through adding a log-sum norm constraint on the totality of the hidden units’ activation probabilities. In this approach, we do not need to keep the “firing rate” of each hidden unit at a certain level that is set beforehand, and therefore the level of sparsity corresponding to each hidden unit can be automatically learnt based on the task at hand. Some experiments conducted on several image data sets of different scales show that LogSumRBM learns sparser and more discriminative representations compared with the related state-of-the-art models, and stacking two LogSumRBMs learns more significant features which mimic computations in the cortical hierarchy. Meanwhile, LogSumRBM can also be used to pre-train deep networks, and achieve better classification performance.