Classification of head movements during multi-person conversation

Abstract

Hearing-aid processing strategies could be greatly improved by real-time knowledge of the users’ listening intentions, information that may be encoded by characteristic head movements during conversational turn-takingwith multiple talkers, and measured by accelerometers in the hearing devices. First, however, it must be determined to what degree head movements are stereotypic across listeners, or alternatively, unique to each individual, and whether an automatic classifier can identify specific movements. Three cohorts of three young, normal-hearing individuals participated in a semi-structured conversation for 50 minutes. Participants wore hearing aids with accelerometers and were video-recorded. Video recordings were then used to track head movements using zFace (Jeni et al. 2017) and manually annotate head movement and communication activities. Accelerometer data was segmented into 1-s windows and labeled by the annotated activity identified in that window. Features based on temporal and frequency characteristics were used to train multi-class models with across-subject or within-subject design. Significantly better than chance classification was observed for all activities using both model designs, with higher classification accuracy for the models trained on the individuals’ own-data. In summary, head movement behaviors can be classified with short-scale time resolution, and individual differences are a key factor for accurate classification.