The psychometric properties of four quality of life instruments used in cardiovascular populations.

Abstract

Quality of life (QOL) is a complex concept that has become a prominent concern to practitioners caring for those with cardiovascular disease. There are several reasons for the growing emphasis of QOL measures in clinical practice. Prior to World War II, the goal of healthcare generally was palliative in nature. With the explosion of biotechnology and the advent of critical care units, the primary goal of treatment became survival and prolongation of life. 1 Research and clinical practice were aimed at preserving life at all costs and these goals were largely successful. However, technological advances and the resultant high cost of healthcare have led society to reassess the value of the quantity of life and embrace the value of quality of life. As Edlund and Tancredi 2 noted, a patient can be biologically “alive” while at the same time being socially “dead.” Contemporary goals of treatment have expanded to include reducing the severity of symptoms, improving functional status, and improving general QOL. 3 As the population ages and healthcare evolves from an emphasis on acute care to an emphasis on chronic care, the assessment of QOL will become even more important. The purpose of this article is to examine the psychometric properties of 4 QOL instruments used in studies of patients with cardiovascular disease. REVIEW OF RELEVANT LITERATURE There is general agreement that QOL is a multidimensional concept but little agreement as to what items are subsumed in the concept. 4 This is not surprising, as values are socially defined by people living in a dynamic world. Five broad conceptualizations of QOL have been identified in a review of cardiovascular research. The concepts include social usefulness, happiness/affect, satisfaction, achievement of personal goals, and normal life. 5 The variety of definitions used by researchers has made comparisons between populations and between instruments problematic. However, lack of a uniform and explicit definition does not preclude researchers from gathering valid and useful QOL data. It merely serves as a reminder that the concept is implicit, subject to interpretation, and very subjective. 6 Several authors have provided conceptual frameworks for the assessment of QOL. These models have identified several distinct domains, which in sum represent QOL. A model proposed by Ferrans 7 includes health and functioning, psychological/spiritual, social and economic, and family domains (see Table 1). Ferrell et al 8 created a model that includes the dimensions of physical well-being and symptoms, psychological well-being, social well-being, and spiritual well-being (Table 2). These models are quite similar and provide mutual validation as they were developed simultaneously and independently. 7Table 1: FERRANS 7 CONCEPTUAL MODEL FOR QUALITY OF LIFETable 2: FERRELL et al 8 CONCEPTUAL MODEL FOR QUALITY OF LIFEFinally, Wenger et al 9 presented a conceptual framework for the measurement of QOL in clinical trials of cardiovascular therapies. The model includes the domains of functional capacity (including social and emotional), perceptions, and symptoms (Table 3). Health status, economic status, emotional capacity, and life satisfaction are assessed demonstrating that this model also approximates the models created by Ferrans 7 and Ferrell et al. 8 Even though the model by Wenger et al 9 was designed for use in cardiovascular patients, it could be applied to other populations with chronic illness or in trials of therapeutic interventions. The assessment of QOL is further complicated by the following questions: Why explore it? Who should evaluate it? How can it be measured?Table 3: WENGER et al 9 CONCEPTUAL MODEL FOR QUALITY OF LIFE IN CARDIOVASCULAR THERAPIESThe Value of Quality of Life Assessments There are numerous benefits to exploring QOL. According to Stewart, 10 QOL research has descriptive, comparative, predictive, and theoretical benefits. Measures of QOL can provide valuable descriptors of the experiences of a specific population of patients, such as those with cardiovascular (CV) disease. These measures may be used to compare different groups or subgroups including patients with angina, myocardial infarction (MI), congestive heart failure, cardiac interventions, or bypass surgery. Quality of life research may help predict the course of disease, the process of recovery, the usefulness of therapeutic interventions, or the need for specific services. Finally, research into QOL is likely to contribute to the development of new theories of illness, illness behavior, and responses to treatment. Who Should Evaluate Quality of Life? The determination of QOL requires a value judgment. Intuitively, the individual most suited to critiquing a person’s QOL is that individual. Hunt 11 observed that though it is desirable to base QOL assessments on the patient’s views, this often is not reflected in the content of the instruments. In certain instances, the evaluation of QOL may be made by a proxy such as the patient’s physician, nurse, or significant others. However, discrepancies have often surfaced between patient and caregiver’s judgments. 12 There are times when the use of proxy measures serves a useful purpose. The determination of public health policy or the allocation of limited resources are examples. A situation in which the patient is unable or unwilling to participate in an evaluation of their QOL is another example. However, the evaluation of QOL of an individual is subjective in nature, dependent on the individual’s values, and is best evaluated by the patient. 13 Measurement Issues The choice of domains to include, the utilization of generic or disease-specific tools, the use of subjective or objective measures, and repeated measures all are salient issues in the measurement of QOL in patients with CV disease. Support for the reliability, validity, and sensitivity of the tool is a prerequisite for choosing the most suitable QOL instrument for the population under study. This type of evidence gives some assurance that the results are reproducible, that the instrument measures what it is purported to measure, and is sensitive enough to detect temporal changes in QOL. METHODS In this article, we examine the psychometric properties of 4 QOL instruments used in CV populations. The goal was to choose both generic and CV disease-specific instruments for review. An additional goal was to select instruments that had been tested extensively and to select newer instruments that had been tested on a limited basis. The Medline and CINAHL databases were searched for QOL studies used in patients with CV diagnoses and reviews of specific instruments used in CV populations. Psychometric Properties of Quality of Life Measures For the purposes of this review, psychometric properties of the 4 instruments evaluated include estimations of validity, reliability, and responsiveness. Three types of validity were assessed including content, criterion, and construct related. Four methods for evaluating reliability were examined and included test-retest, alternative form, split-halves technique, and internal consistency. Finally, estimations of responsiveness to change were reported for applicable studies. Validity Validity is an estimation of the extent to which a measurement tool reflects what it intends to measure. 14 Concluding that an instrument is valid is not an “all or nothing” proposition but rather an accumulation of evidence over repeated measurements. Three types of validity have been identified to support the overall validity of the measure. Content validity is supported if the items in a tool sample the full range of content under study. Although the endorsement of content validity is subjective, support for a specific instrument may be accomplished through review of the literature, comparison with existing standards, interviews with targeted populations, and expert opinions. 15 Criterion validity is of concern to the investigator if the subject’s performance on the measure is to be correlated with performance on some other variable. For example, a test on the proper use of sublingual nitroglycerin would be validated if the patient were found to use nitroglycerin properly as determined by pill counts and notations of anginal episodes in a diary (the criteria). Construct validity provides the link between theory and the conceptualizations in the instrument. 16 Simply stated, the instrument must validly measure what it purports to measure. Construct validity can be supported in two ways. Convergent validity is assumed if two independent methods lead to similar findings. Divergent validity is assumed if group differences are evident on a measure. For instance, patients with congestive heart failure would be expected to score significantly lower than a control group on an instrument that measures functional status. If this is indeed the case, then there is support for the validity of the instrument because the scores differed across divergent groups in predicted ways. Reliability A measure is composed of two elements: a true score, or the “real” score on the item, and the measurement error caused by random disturbances. 17 A reliable measure is one that is consistent and stable over time or over repeated measures. Reliability is a necessary but not a sufficient component of validity. There are 4 methods for assessing the reliability of an instrument including test-retest, alternative form, split-halves, and internal consistency. 18 The test-retest is appropriate for measuring characteristics that are stable over time. Several statistics have been identified to assess test-retest reliability and are calculated by comparing scores measured at baseline with a subsequent test. The Pearson product moment correlation has been commonly used but does not account for variability in results that are attributable to systematic as opposed to random differences in repeated measures. 16 The interclass correlation coefficient, which reflects both random and systematic differences, is the preferred method of quantifying test-retest reliability. 19 The alternative form method requires that a parallel form of the test be constructed, which may be unrealistic for certain measures. It may be difficult, if not impossible, to construct another form of the same test especially if the content is limited or every item is required for testing. The split-halves technique can be considered a substitute for an alternative form approach and a form of internal consistency reliability. It requires that the test be divided into two equal sections. Typically, odd and even items are divided into two groups. The scores from each half are then correlated and the Spearman-Brown formula is applied to estimate the reliability of the test as a whole. 20 The internal consistency (Cronbach’s alpha) method assesses how well the items on an instrument fit together conceptually. An instrument is said to be internally consistent if all of its subparts are measuring the same attribute. Coefficient alpha estimates reliability by computing an average inter-item correlation between all items on the test. An alpha of 0.70 is acceptable for research studies. According to Nunnally and Bernstein, 21 this method of reliability testing should be applied to all measurement instruments even if other reliability estimates are used. Responsiveness (Sensitivity) An instrument is said to be responsive if it is able to detect small but important changes in the construct being measured, within subjects over time. 22 The term “sensitivity” has been used interchangeably with responsiveness; however, is avoided by some authors because of confusion with its other clinical and epidemiological meanings. 23 Several methods for computing responsiveness have been proposed. A commonly applied strategy is to run paired t tests on pre- and posttest scores for a treatment with known efficacy. Significant differences would indicate that the instrument is responsive to change in clinical status. In addition, comparison of t scores from multiple instruments would demonstrate which is most responsive. Calculation of effect size can also be used to assess the responsiveness of an instrument. Effect size is computed by dividing the difference in mean scores (pre- to posttest) by the SD of baseline scores. 24 Instruments The instruments that were selected for evaluation include three disease-specific and one generic tool. All instruments have been tested in a variety of cardiovascular populations. The instruments evaluated are the Quality of Life Index-Cardiac Version III (QLI-III), 25 the Seattle Angina Questionnaire (S), 26 the Quality of Life After Acute Myocardial Infarction (QLMI) Questionnaire, 27 and the 36-item Short Form Health Survey (SF-36). 28 Quality of Life Index-Cardiac Version III The QLI-III is a comprehensive instrument specifically designed to measure generic QOL with an added disease component. 25 It is a self-administered instrument composed of 64 items in two parts. Part I measures satisfaction with 4 different domains of life including health and functioning, social and economic, psychological/spiritual, and family (Table 4). Part II measures the importance of these same 4 domains to the subject. Items are scored on a 6-point scale. For Part I, the scale ranges from “very satisfied” (6) to “very dissatisfied (1). For Part II, the scale ranges from “very important” (6) to “very unimportant” (1). An overall QOL score is obtained by weighting each response in Part I (satisfaction) with its response in Part II (importance).Table 4: QUALITY OF LIFE INSTRUMENTS AND SUBSCALESThe generic QLI was designed for use in healthy populations as well as those with illnesses. The QLI-III contains 6 items specifically related to cardiac disease. The instrument is intended to measure life in specific domains so that healthcare providers may identify problem areas, examine practices, and develop therapeutic interventions. 29 The items in the generic version of the QLI were generated from an extensive review of the literature. The instrument initially was tested on healthy subjects and dialysis patients. The QLI-III has been used in patients experiencing angioplasty and coronary bypass surgery, antiarrhythmic therapy and implantable defibrillators, cardiac rehabilitation, and stable angina. 30–34 Seattle Angina Questionnaire The SAQ is a 19-item self-administered questionnaire that measures 5 clinical dimensions of health in patients with coronary heart disease. 26 The dimensions are physical limitations, anginal stability, anginal frequency, treatment satisfaction, and disease perception (Table 4). The instrument takes approximately 5 minutes to complete. Subscales are scored on a 5- or 6-point ordinal scale. Summary scores are tabulated for each subscale and transformed to a 1-to-100 scale by subtracting the lowest possible scale score, dividing by the scale range, and multiplying by 100. According to the authors, no summary score is tabulated as each scale measures a unique aspect of coronary artery disease (CAD). The questionnaire was originally tested on 4 groups of patients. The first two groups include those undergoing treadmill testing or coronary angioplasty. The third and fourth groups included those who self-reported CAD and those diagnosed with stable CAD. The SAQ has subsequently been tested in patients undergoing cardiac surgery, laser revascularization for angina, angioplasty and stent placement, cardiac rehabilitation, and in women with CAD. 35–39 The authors concluded that the tool is useful as an outcome measure in cardiovascular research. The Quality of Life After Myocardial Infarction The QLMI was constructed to measure health-related QOL after MI and was designed to be administered by the interviewer. The 26-item questionnaire consists of two dimensions, Limitations and Emotions, and can be completed in less than 10 minutes. 27 Symptoms and restrictions come under the Limitations heading and confidence, self-esteem, and emotions fall under the Emotions heading (Table 4). Responses are scored on a 7-point scale. The items contained in the QLMI were chosen based on the contents of generic and disease-specific QOL instruments and literature review of problems experienced by patients after acute MI. An instrument containing 97 items was given to a sample of 60 patients. The patients were asked to identify areas that were problematic for them. Twenty-six items identified most frequently are contained in the final instrument. The tool initially was tested in a randomized trial of conventional care versus a program of physical conditioning and behavioral counseling in 201 patients after MI. 27 Psychometric data on the QLMI were reported for this sample in a subsequent article by the authors. 40 A modified, self-administered version of the QLMI has been used in a study of secondary prevention after suspected MI. 41,42 The QLMI was further revised and tested in a study of the effects of counseling for patients with acute MI or angina. 43 The revised instrument was called the QLMI-2 The Medical Outcomes Study 36-Item Short Form Health Survey The SF-36 was developed to survey health status and satisfaction with care as part of the Medical Outcomes Study. 44 The tool contains 36 items, takes about 10 minutes to complete, and measures 8 dimensions of health including: Physical limitations due to health problems; Social limitations due to physical or emotional problems; Role limitations due to physical problems; Bodily pain; Mental health; Role limitations due to emotional problems; Vitality; and General health perceptions. The final item assesses change in health over the past year and is scored separately (Table 4). A score for each of the 8 domains is obtained by summing unweighted scores on each item and transforming to a score from 0 to 100. A summary score for the instrument is not computed. Higher scores indicate better health. Items were chosen to represent the content of the full-length Medical Outcomes Study form. The SF-36 was specifically designed for use in diverse groups to facilitate comparisons of the relative burden of different diseases and the benefits of different treatment. 28 The SF-36 has been evaluated with cardiac patients including those undergoing angioplasty and medical treatment for angina, echocardiogram for chest pain, angina pectoris, MI, cardiac rehabilitation, and aortic valve replacement. 45–50 RESULTS All published studies that reported psychometric properties were evaluated for each of the four instruments. These data are presented separately for each instrument. Detailed test results are given in Tables 5 to 8.Table 5: PSYCHOMETRIC DATA FOR THE QUALITY OF LIFE INDEX (QLI)Table 6: PSYCHOMETRIC DATA FOR THE SEATTLE ANGINA QUESTIONNAIRE (SAQ)Table 7: PSYCHOMETRIC DATA FROM THE QUALITY OF LIFE AFTER MYOCARDIAL INFARCTION INSTRUMENT (QLMI)Table 8: PSYCHOMETRIC DATA FROM THE SHORT FORM-36 (SF-36)Table 8: PSYCHOMETRIC DATA FROM THE SHORT FORM-36 (SF-36) (continued)Quality of Life Index and QLI-III Ferrans and Powers 25 presented the results from a study of 88 graduate students and 37 dialysis patients using the original version of the QLI (see Table 5 for all data related to the QLI). Content validity was supported by review of the literature and patient reports of the issues relating to QOL and the effects of hemodialysis on QOL. Criterion validity also was supported in this study. A satisfaction with life question was used as a criterion with which to compare scores on the QLI. Campbell’s definition of QOL as an individual’s overall satisfaction with life was the rationale for the use of this criterion. 51 Results indicated a high degree of overlap between the QLI and the life satisfaction question. Test-retest reliability was supported by significant correlations for both graduate students and dialysis patients. A minimum 2-week interval between test administrations was maintained. These data indicate that the QLI is a stable measure. Internal consistency reliability was supported by high Cronbach’s alpha statistics. In a study of 349 hemodialysis patients using the generic version of the QLI, Ferrans and Powers 29 demonstrated support for construct validity via factor analysis. Four factors were identified, labeled as health and functioning, social and economic, psychological/spiritual, and family domains. The highest correlation between factors was 0.60, indicating that the factors were not redundant. Construct validity also was supported by using the contrasted groups method. Subjects were divided into two groups based on income (< or ≥ $10,000 per year). Significant differences were found between groups only in the social and economic subscales with the higher income group reporting higher QOL. Convergent validity of the QLI was supported by a high correlation between the overall QLI and the score on the life satisfaction assessment (0.77). The individual subscale scores on the QLI were also positively correlated to the life satisfaction assessment. Support for internal consistency reliability was demonstrated by computing Cronbach’s alphas for the entire instrument (0.93) and for each subscale. All scores fell within the acceptable range (≥ 0.70). Dougherty et al 34 compared the S, SF-36, and QLI-III in a comparison of anti-anginal medications for use in stable angina. Convergent validity was supported by a comparison of the scores on the three instruments. The health and functioning domain of the QLI-III was highly correlated with the physical limitation subscale of the SAQ and with 3 subscales of the SF-36 including the physical component, general health, and physical functioning. In addition, the psychological/spiritual domains of the QLI were correlated with 5 scales of the SF-36 including mental health, emotional role function, social functioning, general health, and mental component. Cronbach’s alpha analysis was reported on each domain as well as the total score of the QLI-III and were all in the acceptable range indicating that the instrument demonstrated internal consistency reliability. Test-retest reliability was calculated and demonstrated that the instrument was stable from baseline to 2 weeks. The QLI-III was responsive to improvements in anginal status as demonstrated by significant increases in the health and functioning, social and economic, and total QLI-III scores after 3 months. The final 3 studies demonstrated support for the internal consistency reliability of the QLI-III with Cronbach’s alpha statistics all within the adequate range. The studies included an examination of QOL before and after coronary artery bypass surgery and coronary angioplasty, in a study of patients undergoing cardiac rehabilitation, and in a study of QOL in a sample of 66 survivors of life-threatening arrhythmias. 52–54 Seattle Angina Questionnaire Three studies were found that addressed psychometric properties of the S. (See Table 6 for all data related to the SAQ.) The authors of the instrument examined the responsiveness of the SAQ and the SF-36 in two groups of patients: 45 undergoing coronary angioplasty and 130 with stable CAD. 55 Psychometric data were generated individually for each subscale. Responsiveness to change in status was examined in patients 3 months after successful angioplasty. Patients showed significant improvement in 4 of the 5 scales of the SAQ. Only treatment satisfaction was unchanged. There was a significant improvement in only 1 of the 8 subscales in the SF-36. Bodily pain improved after angioplasty. The responsiveness statistics were considerably higher in the SAQ scales than in the SF-36 scale indicating that the SAQ is a more sensitive measure of change in clinical status. Responsiveness was evaluated in patients who originally presented with stable angina. At 3-month follow-up there were significant differences in all scales of the SAQ in patients who reported either an improvement or a deterioration in their condition. Those who reported improvement in their condition demonstrated a corresponding increase in SAQ scores. Those who reported a deterioration in their condition showed a decrease in SAQ scores. Another study by the authors reported psychometric properties of the SAQ in 4 different populations of cardiovascular patients. 26 The questionnaire was administered to 70 patients undergoing stress testing, 58 undergoing angioplasty, 160 with stable CAD, and 84 with self-reported CAD. Support for concurrent validity was obtained by comparing scores on the anginal stability scale to the presence or absence of unstable angina at the time of angioplasty. Anginal stability scale scores were significantly lower in patients with unstable angina. Criterion validity was supported by examining correlations between the SAQ and several other criteria. Treadmill test duration (the criterion variable) was correlated with scores on the SAQ physical limitation scale, as well as the Duke Activity Status Index, the SF-36, the Specific Activity Scale, and the Canadian Cardiovascular Society classification of angina. These instruments are measures of functional status. Correlations between exercise duration and the SAQ, Duke Activity Status Index, and Specific Activity Scale scores were significant. Criterion validity of the anginal frequency scale was supported by a correlation between the SAQ scores and the number of nitroglycerin refills (the criterion) in 1 year’s time in the cohort of patients with stable CAD. Treatment satisfaction scale scores were correlated to the American Board of Internal Medicine’s Patient Satisfaction Questionnaire (the criterion) in the cohort of patients with self-reported CAD. Finally, the disease perception scale scores, secured from the patients with stable angina and those undergoing angioplasty, were significantly correlated with the general health perceptions scale of the SF-36. Reliability of the SAQ was supported by retesting the stable CAD group at 3 months. There were no significant differences between pre- and posttest scores as would be expected in a group of clinically stable patients. Support for the responsiveness of the SAQ was demonstrated by a comparison of baseline and 3-month follow-up scores in the cohort undergoing successful angioplasty. Scores on all scales except treatment satisfaction were significantly higher at the 3-month follow-up point. Responsiveness also was evaluated in the group of patients with initially stable angina. At follow-up, all scales except treatment satisfaction showed significant improvement in the patients who reported improvement in their condition. Conversely, all scales except treatment satisfaction decreased significantly in the group reporting a deterioration in their condition. Psychometric data for the SAQ were reported in a study by Dougherty et al that sampled patients with stable angina. 34 Convergent validity of the physical limitation component of the SAQ was supported by correlations between the physical limitation dimension of the SAQ and both the physical functioning and physical component scales of the SF-36. The disease perception subscale of the SAQ also was positively correlated with 4 subscales of the SF-36, including general health, vitality, physical component, and mental component. These results offer support for the construct validity of the specified dimensions of both the SAQ and the SF-36. Test-retest reliability over a 2-week interval was evaluated by computing intraclass correlation coefficients for the physical limitation, angina frequency, angina stability, treatment satisfaction, and disease perception subscales. All correlations were statistically significant. The angina stability subscale showed a lower correlation than the other subscales but this would be expected to vary over time. Internal consistency was measured on the physical limitation, angina frequency, treatment satisfaction, and disease perception subscales by means of Cronbach’s alpha. With the exception of disease perception (α = 0.66), reliabilities were greater than the α ≥ 0.70 standard. Finally, the SAQ was found to be sensitive to changes in clinical status as measured by change scores from baseline to 3 months. There were significant improvement in scores on angina stability, angina frequency, treatment satisfaction, and disease perception. There was no change in the physical limitation scale scores. Quality of Life After Myocardial Infarction Only two studies were identified that reported psychometric properties for the QLMI. The first was a study conducted by the authors assessing QOL in a randomized trial comparing a cardiac rehabilitation program to conventional care. 40 Six other tools, including the Quality of Well-Being questionnaire, the Time Trade-Off, the Katz Questionnaire, the Beck Inventory, the Profile of Mood States (tension and depression versions), and the Spielberger State-Trait Anxiety Inventory (state and trait forms) were administered along with the QLMI. All data related to the QLMI can been seen in Table 7. The authors reported two types of construct validity, which they label discriminative and evaluative. Discriminative validity is supported if an instrument is able to measure differences between subjects at a point in time. Support for the validity of an evaluative instrument would occur if significant differences were detected longitudinally, within subjects. Both discriminative and evaluative validity are forms of construct validity. Discriminative construct validity was supported by comparing 4 dimensions of the QLMI (symptoms were excluded) to 5 emotional function measures. Correlations ranged from 0.28 to 0.70. Fifteen of 20 correlations were 0.49 or greater indicating that the QLMI is a valid measure of emotional status. Correlations between the 5 dimensions of the QLMI and 4 other functional and utility measur