Scaling Up Content Analysis: Crowd-Coding Text Units

Abstract

This article describes a new Crowd Content Analysis Assembly Line and enabling software allowing researchers to complete large-scale content analysis projects with the help of citizen scientists and/or crowd workers. The key innovation enabling this new text analysis workflow is the TUA, the Text Unit of/for Analysis. TUAs are the subset of words in a document that refer to a single case of a single unit of analysis under study. Once a research team has identified the TUAs in a corpus, detailed hand-coding work can be performed by internet users who experience the work as a series of reading comprehension requiring them to highlight the text they use to justify their answers. By breaking down content analysis into these cognitively simpler tasks that scale to a larger workforce, the approach can reduce the duration of large-scale content analysis projects by a factor of six. The article explains how databases produced by this text analysis approach are created faster, and are richer, more reliable, larger, and more transparent than those generated using previous methods and technology. The integration of the approach with automated approaches is also discussed.