An Alternative Approach to Measuring Reliability of Transcription in Children’s Speech Samples: Extending the Concept of Near Functional Equivalence

Abstract

Aim: To explore a novel and efficient way of calculating transcription reliability of connected speech data using the concept of near functional equivalence. Using this approach, differences between two transcribed phonemes that are nearly phonetically equivalent are disregarded if both reflect two plausible and acceptable pronunciations for the word produced. Method: The study used transcriptions of connected speech samples from 63 five-year-olds who participated in a large-scale population study. Each recording was phonetically transcribed by two speech and language therapists. Two independent researchers then examined agreement ­between the two sets of transcripts, marking differences in vowels, consonants and diacritics and identifying segments which represented near functional equivalence. Results: Overall percentage agreement between the transcripts was 77%. One quarter of the differences between the two transcripts were identified as showing near functional equivalence. When this category was excluded, the transcripts showed 82% reliability. Conclusions: This study demonstrates the issues to consider when calculating transcription reliability. Other methods are often time-intensive and may highlight differences between transcribed units which are audibly very similar and would be negligible in ordinary conversation. Inclusion of the concept of “near functional equivalence” can result in higher reliability scores for transcription, without loss of rigour.