Performance of an online translation tool when applied to patient educational material

Abstract

Language barriers may prevent clinicians from tailoring patient educational material to the needs of individuals with limited English proficiency. Online translation tools could fill this gap, but their accuracy is unknown. We evaluated the accuracy of an online translation tool for patient educational material. We selected 45 sentences from a pamphlet available in both English and Spanish, and translated it into Spanish using GoogleTranslate™ (GT). Three bilingual Spanish speakers then performed a blinded evaluation on these 45 sentences, comparing GT-translated sentences to those translated professionally, along four domains: fluency (grammatical correctness), adequacy (information preservation), meaning (connotation maintenance), and severity (perceived dangerousness of an error if present). In addition, evaluators indicated whether they had a preference for either the GT-translated or professionally translated sentences. The GT-translated sentences had significantly lower fluency scores compared to the professional translation (3.4 vs. 4.7, P < 0.001), but similar adequacy (4.2 vs. 4.5, P = 0.19) and meaning (4.5 vs. 4.8, P = 0.29) scores. The GT-translated sentences were more likely to have any error (39% vs. 22%, P = 0.05), but not statistically more likely to have a severe error (4% vs. 2%, P = 0.61). Evaluators preferred the professional translation for complex sentences, but not for simple ones. When applied to patient educational material, GT performed comparably to professional human translation in terms of preserving information and meaning, though it was slightly worse in preserving grammar. In situations where professional human translations are unavailable or impractical, online translation may someday fill an important niche.