A Learning Effect by Presenting Machine Prediction as a Reference Answer in Self-correction

Abstract

Can people learn from machines behavior in microtask based crowdsourcing? Can we train the machines as our mentor even without domain expertise? In this paper, we investigate how the task results improve concerning quality during and after presenting machine prediction as a reference answer in self-correction. Four reference types were examined in the experiment; Correct, Random, Machine prediction trained by correct answers, and that trained by human answers. Learning effects were observed only in presenting machine prediction, although those accuracy rates were far from correct (100%). Moreover, there were no learning effects in Correct and Random. This suggests the following hypothesis: Since machine learners make some models for the problem, it is easier for humans to interpret the outputs of machine learners than the results without via them; it is more difficult to interpret not only random answers but also the correct answers in a case where the perfect interpretation of the problem is difficult. Furthermore, some workers answered with higher accuracy rate than machines in the post-test. Therefore, this strategy can be expected to be useful for bootstrapping solutions in the situation where unknown problems occur without expertise or at a low cost.