Accuracy of the language environment analysis (LENA) speech processing system for detecting communicative vocalizations of young children

Abstract

Automated audio processing systems, such as the Language Environment Analysis (LENA) system, are useful tools for understanding developmental language behaviors for clinical and basic research purposes. However, it is still unclear how accurate they may be in comparison to the traditional gold standard of evaluation by trained human listeners. In our study, human coders identified starts and ends of communicative vocalizations of children and adults from sampled audio in day-long LENA recordings of 23 families with a child with variable hearing status; accuracy of LENA was then determined for each recording by comparing LENA and human-derived labels for 100-ms frames of sampled audio. Preliminary analysis suggests that LENA accurately identified communicative vocalizations of the target child wearing the device as being produced by that target child 65% of the time (35% error); accuracy ranged from 49%—79% across recordings. When any child vocalization was correctly identified, LENA accurately distinguished whether this belonged to the target child or another child 75% of the time (25% error); accuracy, however, ranged from 7%—96%. These accuracy levels suggest caution is needed in applying popular speech processing systems like LENA to clinical and scientific questions in absence of additional validation measures.