Running a double-blind true social experiment with a goal oriented adaptive AI-based conversational agent in educational research

Abstract

This study introduces an innovative AI-facilitated interview-like survey system generating a combination of qualitative and quantitative data insights for higher education research. We employed a goal oriented adaptive AI-based Conversational Agent (AICA) which collected data directly from 1223 participants globally and ran a double-blind true social experiment online. During interviews, the AI established strong rapport with the participants, offering them personalized guidance while fostering comfort, ownership, and commitment to the study. In this entirely automated experiment, we empirically tested 8 hypotheses related to students' university selection. The results confirmed 5 of these hypotheses while refuting 3 factors previously identified in the literature. The study showcases the potential of AICAs to efficiently collect and analyse data from substantial sample sizes in real-time, fostering a streamlined and harmonious research process producing results that are not only statistically reliable and bias-free but also broadly generalizable.