An introduction to CIABATTA: Increasing accessibility to corpus creation for non-experts

Abstract

This methodological report introduces CIABATTA (Corpus In A Box: Automated Tools, Tutorials, & Advising), a suite of tools that provides novice corpus compilers with a starting point to develop new corpora. These tools are a distillation of guiding principles and templates for corpus building developed by the Corpus & Repository of Writing (CROW) team during the compilation of the CROW corpus. We start by discussing the best practices and ethical issues that need to be considered when compiling a corpus. We then focus on the steps of corpus data processing, introducing two tools that support corpus building. The first tool, the Corpus Text Processor, is used to convert, encode, and standardize texts in the corpus. The second, the deidentifying tool, replaces identifying information with special annotation tags in corpus files, such as [name] and [place]. Both can be downloaded from the CIABATTA wiki and do not require programming experience. In the final part of this paper, we focus on customizable Python scripts developed for CIABATTA that aid in the processing of metadata. We believe that CIABATTA can aid those without corpus building expertise to build corpora for research purposes, thus expanding the use of corpora to a wider group of researchers.