A corpus-based error typology: towards a more objective approach to measuring quality in localization

Abstract

This paper examines current localization quality assessment through a product-based analysis of error typologies and the role they play in producing reliable and valid measurements. In order to provide a more objective foundation for localization quality, the methodology used is based on a monolingual comparable corpus of original and localized Spanish corporate websites. The contrastive analysis of this corpus highlighted recurrent inadequacies that are difficult to control with current evaluation metrics and, as a result, a comprehensive error categorization for localization is presented. It incorporates certain aspects, such as pragmatic errors, that are absent in current testing instruments used in the industry. In order to observe whether this typology can provide a more holistic and valid approach, it is applied to a case study that measures the errors in a localized Spanish website from one of the largest US companies. The results shed some light onto the shortcomings of current Quality Analysis practices and illustrate the benefits of adopting a bottom-up corpus approach in quality assessment. The wider implications for testing practices in localization are also described.