The structure of word co-occurrence network for microblogs

Abstract

The study of structure and dynamics of complex networks is seeking attention of academic researchers and practitioners in recent years. Although Word Co-occurrence Networks (WCN) have been studied for different languages, yet there is the need to study the structure of WCN for microblogs due to the presence of ill-formed and unstructured data. In this research article, existing WCN based applications have been explored and microblog WCN have been analysed for multiple key parameters to uncover the hidden patterns. The key parameters studied for microblogs WCN are scale-free property, small world feature, hierarchical organization, assortativity and spectral analysis. The twitter FSD dataset has been used for experimental results and evaluation. Different mathematical, statistical and graphical interpretations proved that the microblog WCN are different from the WCN of traditional well-formed text. The robustness of the key parameters of microblogs WCN have been explored for keyphrase extraction from domain specific set of microblogs. The baseline methods used for comparisons are TextRank, TopicRank, and NErank. Extensive experiments over standard public dataset proved that the proposed keyphrase extraction technique outperforms the existing techniques in terms of precision, recall, F-measure, and ROUGE scores.