Knowledge Representation Using Probabilistic Model and Reconstruction-Based Algorithms

Abstract

Representation learning is a prominent area in the machine learning due to the underlying fact that the performance of machine learning methods is dependent on a surmount extent on the choice of data representation. In this chapter, a primary focus is given to the different learning and probabilistic models. Probabilistic graphical models (PGMs), like directed graphical models (DGMs), alternatively called Bayesian networks (BNs) and undirected graphical models (UGMs), alternatively called Markov random fields (MRFs), are discussed. As over many years, knowledge graphs are convened for a varied area, these can be reutilized in PGMs. PGMs help assess the model structure and furthermore aid in providing solutions to tasks like inference or learning in accordance with estimating parameters of probability functions. This chapter throws light on restricted Boltzmann machines (RBMs) that have had a high impact on the fields of unsupervised feature learning and deep learning. They are used exclusively for dimensionality reduction, classification, regression, collaborative filtering, feature learning, and topic modeling as they are a special class of Boltzmann machines and are restricted in terms of the connections between the visible and the hidden units. Some of the advancements in defining generalizations of the RBM that better capture real-valued data along with various approaches to modeling real-valued observations within the RBM framework are further explored. In training various probabilistic models, parameters are typically adapted to maximize the likelihood of the training data and hence RBM parameter estimation is also discussed. Auto encoders that are an unsupervised learning technique, just like Principal Components Analysis (PCA) technique, mostly known as a dimensionality reduction technique, which broadly falls in the category of regularized, sparse, and denoising, are discussed. The chapter ends by evaluating and monitoring the performance of all the learning algorithms and models by weighing theory usability for any given task.