Exploring Themes and Patterns in American Literature Using Text Analysis Techniques

Abstract

Abstract This study leverages web crawling techniques within the realm of text mining to collect text data from American literary sources. Given the presence of substantial extraneous information in the data, preprocessing is essential to refine the content. Once processed, the data is fed into a Long Short-Term Memory (LSTM) network to generate text vectors that encapsulate characteristics of American literature. Building on this, the Latent Dirichlet Allocation (LDA) topic model extracts salient features from these vectors. An illustrative analysis of American literature is conducted using these methods. Our findings indicate that a significant portion of novels, which mirror the societal life themes prevalent in the United States, aligned with the ideological currents of their time. Notably, these constituted a substantial majority—62.18%—of all American literary works during the period under review. This research not only facilitates the wider dissemination and exchange of American literary works but also contributes to elevating American literature to a new echelon of global recognition.