Topic Hierarchies for Knowledge Capitalization using Hierarchical Dirichlet Processes in Big Data Context

Abstract

Intelligent Technologies and research results from the field of knowledge management, have steadily and progressively improved knowledge quality, over the course of the last decades, especially in the current industrial context. The companies consider the knowledge as an important strategic resource for innovation. This paper focus on the problem of learning topic hierarchies from knowledge. The aim targeted is to respond to knowledge capitalization issues in big data context, by proposing a Knowledge capitalization system as an adaptive intelligent technique. This system acts on top of a big data platform, and runs on large scale globally to constitute a robust intelligent knowledge capitalization paradigm, with a clear separation of concerns. The architecture considers a batch processing as a preparation stage which starts by extracting hidden topics, by means of the HLDA in order to handle the complexity of multi knowledge domains, and to keep the semantic relations between knowledge entities. The hierarchical mechanism gives an effective and flexible way to store and analyses the knowledge. As a result, the time responding is obtained of high quality, and with the best precision, in comparison with the systems which uses LSA and LDA approach as a preparation stage.