Designing Hybrid Human–AI Orchestration Tools for Individual and Collaborative Activities: A Technology Probe Study

Abstract

Combining individual and collaborative learning is common, but dynamic combinations (which happen as-the-need arises, rather than in pre-planned ways, and may happen on an individual basis) are rare. This work reports findings from a technology probe study exploring alternative designs for classroom co-orchestration support for dynamically transitioning between individual and collaborative learning. The study involved 1) a technology-probe classroom study in an authentic, AI-supported classroom to understand teachers' and students' needs for co-orchestration support over dynamic transitions; and 2) workshops and interviews with students and teachers to get informed feedback about their lived experiences. 118 students and three teachers from a middle school in the US experienced a pairing policy – student, teacher and, AI-controlled pairing policy – (i.e., identifying students needing help and potential helpers) for switching from individual to a peer tutoring activity. This work aims to answer the following questions: 1) How did students and teachers react to these pairing policies?; and 2) What are students' and teachers' desires for sharing control over the orchestration of dynamic transitions? Findings suggest the need for a form of hybrid control between students, teachers, and AI systems over transitions, as well as for adaptivity and adaptability for different classroom characteristics, teachers, and students' prior knowledge.