A review of research on the development of instruments for therapeutic efficacy evaluation of traditional Chinese medicine

Abstract

Due to the differences between Chinese and Western cultures, Chinese version of foreign research instruments may not be totally applicable for use in evaluating the therapeutic effect of traditional Chinese medicine (TCM). Great efforts have been made by Chinese researchers to develop scales for evaluation of the therapeutic effects of TCM. This study aims to understand the current situation of research in the development of evaluation instrument in TCM. Database searches of Chinese Biomedical Literature Database, China Academic Journal Network Publishing Database and Chinese Scientific Journals Database were undertaken to identify published studies with the purpose of developing instruments in assessing the effectiveness of TCM, including papers regarding the construction of conceptual framework of instrument, item generation and selection and the evaluation of measurement properties. A total of 60 pieces of literature involving 36 instruments were included. The first article on the development of each of the instruments was published between 2005 and 2011 and the instruments were used in many kinds of medical conditions, including cardiocerebrovascular, respiratory, digestive and infectious diseases. The number of items ranged from 10 to 52. Of the 36 instruments, 13 (36.1%) defined the hypothesized concepts measured by the instrument, 30 (83.3%) reported the domains of the questionnaires before measurement property testing and all of them were multidomain. Of 32 studies regarding item selection and the instrument's property evaluation, 14 (43.8%) articles reported the administration mode, 24 (75%) reported response option types, and 10 (31.5%) provided scoring algorithm for the scale, but none of these 32 studies specified the recall period. In 29 studies aiming at testing instrument's measurement property, 28 articles tested the Cronbach's α coefficient of the full scale and/or subscales, and retest reliability was also detected in 15 studies. Twenty-seven studies evaluated the construct validity by exploratory factor analysis and among them there were two studies applying confirmatory factor analysis. Content validity, responsiveness and feasibility of instruments were assessed in 11, 16 and 16 studies, respectively. This study shows that in recent years many instruments have been developed in an attempt to evaluate the therapeutic effects of TCM, but some problems still exist in their practical implementation, including negligence in outlining the hypothesized concepts of the TCM instruments and in the reporting of instrument's content validity such as administration mode, scoring and recall period. Some instrument attributes and testing methods were misunderstood and/or misused. Revision of instruments is rarely carried out, though the development of an instrument is an iterative process. Researchers should have a thorough understanding of the general procedure and steps before starting to develop an instrument.