Expectation and experience of ‘nonspecific’ (whole person) feelings elicited by acupuncture: Content validity of a set of questionnaires

Abstract

BackgroundTwo 32-item questionnaires designed to assess expectation and experience of the relatively nonspecific feelings (whether bodily, emotional or mental) that may arise in response to acupuncture-type interventions have been developed and tested on acupuncture and other complementary health practitioners and students (N = 204). ObjectivesTo conduct a survey to assess content validity (CV) for the questionnaires, following two methods popularised by Lawshe and Lynn, and to compare the results. A secondary objective was to reduce questionnaire length from 32 to 20 items. MethodsTwo surveys (Phase 1, 48 items; Phase 2, 18 items) were conducted among 20 experienced acupuncture practitioners and researchers, who scored items as either ‘essential’, ‘useful but not essential’, or ‘not necessary’. Lawshe's and Lynn's methods were applied to the returned data, the former using Wilson's recent modification, the latter with ‘useful but not essential’ rescored as either ‘essential’ or ‘not necessary’. Some subgroup analysis was carried out. In addition, Randolph's multi-rater kappa was used to assess inter-rater reliability as a further measure of content validity. ResultsThere were 17 usable responses in Phase 1, 13 in Phase 2. In both surveys, women considered more items as ‘essential’ than men, but fewer as ‘not necessary’, and vice versa (p < 0.001). Lawshe's content validity ratio (CVR) was significant for only 3 items, and the resulting content validity index (CVI) therefore meaningless. When ‘useful’ was rescored as ‘essential’ and Lynn's Scale CVI (S-CVI) calculated as the average of the CVI only for those items considered appropriate for retention (S-CVIAV-UA), acceptable values were found for subsets of the full list of items, although only for the women respondents. Inter-rater reliability was low for most items, but on the basis of the content validity analysis, together with respondent reservations about particular items and analysis of actual questionnaire usage, a 20-item list was created and circulated to the survey respondents for further feedback. DiscussionThis shorter scale is now being tested on electroacupuncture training courses, and may be useful in determining whether different feelings are elicited by different types of acupuncture, other CAM modalities, or even other, unrelated activities. ConclusionsA modified content validity assessment method was found to give more usable results than conventional methods. With this new approach, 32-item questionnaires on expectation and experience of nonspecific feelings were shortened to more manageable 20-item instruments. In general, CV was greater among women than among men.