Quantifying the Quality of Parent-Child Interaction Through Machine-Learning Based Audio and Video Analysis: Towards a Vision of AI-assisted Coaching Support for Social Workers

Abstract

Attachment is the emotional bonding between a child and a caregiver. Whether or not there is a secure attachment in early childhood has a profound life-long impact on the child. In recent years, attachment-based interventions have been developed and implemented, especially with families from low socioeconomic backgrounds. One important aspect of the program is to assess the quality of parent-child interactions through audio/video recorded at home while parent-child dyads were engaged in semi-structured interaction tasks, such as “three-bag-assessment.” The current practice relies on human coders to rate the videos, which is a time-consuming process. Using a dataset of 220 video recordings of parent-child dyads collected at home as part of an attachment-based intervention program, we prototype a machine learning approach based on human body keypoints extracted from the posture analysis tool OpenPose and voice activity features derived from audio recordings. The results show that there are potential values in using machine learning to improve the coding efficiency of parent-child interactions. When further developed and improved, this kind of model may contribute to a new vision of AI-assisted parenting coaching support to make evidence-based interventions accessible and affordable at a large scale to children and families.