Statistical Approach to Word Stem Completion Test

Abstract

We appreciate the interest of Drs. Kerssens and Falzetti in our report.1 Our study was closely modeled on the previous work of Dr. Kerssens et al.,2–4 who published the following instructions (with minor variations): “In the inclusion part, patients were asked to complete the stems, if possible, with a word presented during surgery, or, otherwise, with the first word that comes to mind. [In the exclusion part of the test], patients were asked, if possible, not to use presented words for stem completion but to use any other word they could think of.”2On our audio compact disks, we issued the following instructions about completing the word stems: In the inclusion part, we asked the patients to “write down a word you remember hearing during your surgery or the first word that comes to mind.” In the exclusion part, we asked patients to “write down a word you did not hear during your surgery.” We believe that memory would be encouraged in both conditions, because to choose a word that had not been heard during surgery, patients would need to think about which words had been heard. Nevertheless, we accept that the subtle difference between our instructions and previous instructions may have resulted in a different base rate and encourage future researchers to use the exactly comparable instructions.2–4 Finally, as emphasized by Dr. Kerssens, although our conclusion about which type of memory may be responsible for our result is subject to debate, our conclusion that Bispectral Index values greater than 50 may be associated with memory is still correct.Dr. Falzetti is correct that, mathematically, there are six ways of presenting two out of four lists (i.e., 1 + 2, 1 + 3, 1 + 4, 2 + 3, 2 + 4, and 3 + 4). However, to use all of the lists would complicate the procedure unnecessarily, and so we followed the lead of Lubke et al.2,3 and used only four of the combinations (i.e., 1 + 2, 1 + 4, 2 + 3, and 3 + 4), so that we had a different set for each condition (i.e., inclusion target, inclusion distracter, exclusion target, and exclusion distracter). Because we wanted all of the words to appear in each condition, we ended up with four groups. The items on each list were presented in a random order to each individual, thereby taking care of list order effects. One could make a case of counterbalancing all six combinations of lists, but that would only be necessary if the combination of lists is expected to have an effect (which it is not).Therefore, we advise that if researchers wish to create their own language-specific lists, they should read the original articles on list construction,5 create a large set of comparable words, run a pilot study in a representative and sufficiently large sample using all words, and select those items that perform uniformly (in our case, word stem completion base rates of 33%), to avoid using highly unusual or very common words in their lists.The authors thank Chantal Kerssens, Ph.D. (Assistant Professor, Department of Anesthesiology, Emory University School of Medicine, Atlanta, Georgia), for her advice with regard to Dr. Falzetti's letter.