The impact of differences in text segmentation on the automated quantitative evaluation of song-lyrics.

Friederike Tegge,Katharina Parry,Diego Raphael Amancio

doi:10.1371/journal.pone.0241979

Abstract

The text-evaluation application Coh-Metrix and natural language processing rely on the sentence for text segmentation and analysis and frequently detect sentence limits by means of punctuation. Problems arise when target texts such as pop song lyrics do not follow formal standards of written text composition and lack punctuation in the original. In such cases it is common for human transcribers to prepare texts for analysis, often following unspecified or at least unreported rules of text normalization and relying potentially on an assumed shared understanding of the sentence as a text-structural unit. This study investigated whether the use of different transcribers to insert typographical symbols into song lyrics during the pre-processing of textual data can result in significant differences in sentence delineation. Results indicate that different transcribers (following commonly agreed-upon rules of punctuation based on their extensive experience with language and writing as language professionals) can produce differences in sentence segmentation. This has implications for the analysis results for at least some Coh-Metrix measures and highlights the problem of transcription, with potential consequences for quantification at and above sentence level. It is argued that when analyzing non-traditional written texts or transcripts of spoken language it is not possible to assume uniform text interpretation and segmentation during pre-processing. It is advisable to provide clear rules for text normalization at the pre-processing stage, and to make these explicit in documentation and publication.

Highlights

Automated evaluation of language and text is becoming increasingly sophisticated, with a range of programmes available to examine text at the word level, sentence level, and even discourse level
The goal of this study is to understand the potential impact of text normalisation at the pre-processing stage, when it comes to the identification of sentence boundaries, on the results of a Coh-Metrix analysis of English pop song lyrics
As song lyrics are frequently published in a format that requires text normalisation at the preprocessing, we addressed the following questions before proceeding with the wider investigation: 1. Can the use of different transcribers to insert punctuation into pop song lyrics result in significant differences in sentence segmentation?

Summary

Introduction

Automated evaluation of language and text is becoming increasingly sophisticated, with a range of programmes available to examine text at the word level, sentence level, and even discourse level. Coh-Metrix is an application “at the forefront of these technologies” [1], offering a broad range of measures of local and global, lexical and textual discourse features. Its usefulness is demonstrated by its popularity with applications ranging from research on schizophrenia [2] to higher education [3]. While Coh-Metrix has most often been used for the analysis of printed text, it can, according to its creators, be used to analyse spoken discourse and non-traditional written texts.

Objectives

Methods

Results

Discussion

Conclusion