Choosing the Task Allocator: Effect on Performance and Satisfaction in Human-Agent Team

Abstract

Ad hoc human-agent teams, where team members interact without prior experience with teammates and only for a limited number of interactions, will be commonplace in dynamic environments with opportunity windows for collaboration between diverse groups. We study the efficacy of virtual ad-hoc teams, consisting of a human and an agent, collaborating to complete tasks in each of a few episodes. To maximize team potential, the relative expertise of team members must be measured and utilized in allocating tasks. As team members are not initially aware of each other's task competence and as humans often cannot accurately estimate their competencies, adapting allocation over the episodes is critical to team performance. Human team member satisfaction with allocations is also critical to determining team viability. We therefore use both these criteria to measure the effectiveness of task allocation procedures with varying degree of flexibility and human teammate control: (a) alternating, (b) performance adaptive, (c) agent-guided, (d) human-selected. We report on the relative strengths of these allocation procedures based on results from experiments with MTurk workers.