Feature Extraction of Dialogue Text Based on Big Data and Machine Learning

Abstract

In this article, a dialogue text feature extraction model based on big data and machine learning is constructed, which transforms the high-dimensional space of text features into the low-dimensional space that is easy to process, so that the best feature words can be selected to represent the document set. Tests show that in most cases, the classification accuracy of this model is higher than 88%, and the recall rate is higher than 85%, thus achieving the goal of higher classification accuracy with less computation. When extracting the features of dialogue texts, there is no need for preprocessing, just count the data such as lexical composition, sentence length and sentence-to-sentence relationship of the target text, and make linear analysis to obtain key indicators and weights. Based on this, the classification model can achieve good results, thus effectively reducing the workload and computation of text classification.