Intuiting machine failures

Abstract

A key ingredient of effective collaborations is knowing the strengths and weaknesses of one’s collaborators. But what if one’s collaborator is a machine-classification system, of the sort increasingly appearing in semi-autonomous vehicles, radiologists’ offices, and other contexts in which visual processing is automated? Such systems have both remarkable strengths (including superhuman performance at certain classification tasks) and striking weaknesses (including susceptibility to bizarre misclassifications); can naive subjects intuit when such systems will succeed or fail? Here, five experiments (N=900) investigate whether humans can anticipate when machines will misclassify natural images. E1 showed subjects two natural images: one which reliably elicits misclassifications from multiple state-of-the-art Convolutional Neural Networks, and another image which reliably elicits correct classifications. We found that subjects could predict which image was misclassified on a majority of trials. E2 and E3 showed that subjects are sensitive to the nature of such misclassifications; subjects’ performance was better when they were told what the misclassification was (but not which image received it), and worse when the label shown was from another, randomly-chosen category. Crucially, in E4, we asked subjects to either (a) choose which image they thought was misclassified, or (b) choose the image that is the worst example of its category. While both instructions resulted in subjects choosing misclassified images above chance, subjects who were instructed to identify misclassifications performed better. In other words, humans appreciate the cues that mislead machines, beyond simply considering the prototypicality of an image. Lastly, E5 explored more naturalistic settings. Here, instead of an NAFC choice, subjects identified potential misclassifications from a stream of individual images; even in this setting, subjects were remarkably successful in anticipating machine errors. Thus, humans can anticipate when and how their machine collaborators succeed and fail, a skill that may be of great value to human-machine teams.