Application of the Cloze Procedure to Evaluate Comprehension and Demonstrate Rewriting of Pharmacy Educational Materials

Abstract

Written materials are commonly used to communicate pharmacy-relevant information to patients. However, they are often composed at a level that limits comprehension, mitigating a well-intended effect. To (1) use the cloze procedure (a test designed to assess reading comprehension) to evaluate an individual's understanding of a pharmacy-relevant educational pamphlet; (2) compare results of the cloze procedure with the reading comprehension component of the Short Test of Functional Health Literacy in Adults (S-TOFHLA); and (3) use results to demonstrate rewriting of the educational pamphlet. The cloze procedure was applied to a pharmacy-relevant educational pamphlet describing safe medication practices. A total of 162 subjects were recruited from university faculty, staff, and students; a local adult literacy center; and community senior centers. Subjects completed a background interview, the S-TOFHLA, and cloze procedure for the pharmacy-relevant educational pamphlet. S-TOFHLA and cloze procedure scores were described and compared. Cloze procedure responses were used to demonstrate revision of the pamphlet. Of the 154 subjects analyzed, mean +/- SD age was 56.5 +/- 20.4 years. Subjects were predominantly white (93.5%), female (71.4%), and college graduates (42.2%). Mean score on the S-TOFHLA was 92.1%. A majority (95.5%, 147/154) of subjects demonstrated adequate functional health literacy. In contrast, mean score on the cloze procedure was 53.3%. Internal consistencies of the S-TOFHLA and the cloze procedure were 0.92 and 0.90, respectively. Scores on the cloze procedure and the S-TOFHLA were highly correlated (r = 0.71, p < 0.001). Performance on the cloze procedure indicated that 55.2% of subjects required supplemental teaching. In this highly educated, health-literate sample, a majority did not understand the pharmacy-relevant educational pamphlet despite adequate performance on a standard measure of health literacy. The cloze procedure can be used to assess comprehension of educational materials, solicit feedback from intended users, and guide the revision of educational materials.