Summary and Conclusions From the Academy??s Sixth State-of-the-Science Conference on Lower Limb Prosthetic Outcome Measures

Abstract

GOALS OF THE STATE-OF-THE-SCIENCE CONFERENCE NO. 6 (SSC6) This State-of-the-Science Conference was convened to examine the body of scientific knowledge related to lower limb prosthetic (LLP) outcome measures. The meeting was held September 7 through 9, 2005, in Chicago, Illinois, and funding for this conference was provided by the American Academy of Orthotists and Prosthetists through a grant from the US Department of Education. Experts in the field were invited to offer their insight and expertise in summarizing the evidence and identifying key areas for future research. It was the goal of the conference to examine, debate, and answer these questions: What validated instruments are available in English to measure global lower limb prosthetic outcomes? What do these instruments attempt to measure? What are the relative strengths and weaknesses of current instruments? What are the barriers to clinical application of these tools? In light of the literature review and the panel's discussion, what are the primary future research priorities? To address questions 1 and 2, a comprehensive written review of the literature was completed by Elizabeth Condie, PT, FCSP, and colleagues. Copies of the relevant articles selected for review were made available to the participants before the meeting. In preparation for the conference, participants were asked to review the literature applicable to questions 3 and 4. Selected participants were asked to present their experience with specific tools (question three), whereas other participants were asked to provide certain perspectives on the clinical application of outcome measurement tools (question 4). REVIEW OF PRESENTATIONS (QUESTIONS 1 THROUGH 4) There is a great deal of variability in terminology in the field of outcome measures. To ensure that all participants were using the same terminology in discussing LLP outcomes, two individuals presented on terminology. Christiane Gauthier-Gagnon, MS, recommended the WHO standards of terminology for disability. It was conveyed that it is important to understand the influence of personal factors and environmental factors related to body function and structure, activities, and participation when describing and constructing measurement tools. Measurement tools can evaluate either the actual level of performance or perceived performance and can also be generic or population specific. Kathryn Roach, PhD, PT, continued the discussion on terminology related to health outcomes by noting that various measurements of outcomes can be recorded. The quality of the information provided by an outcome measure depends in part on psychometric properties such as the level of measurement (e.g., type of data such as categories and scores), reliability (differences between scores are due to real differences), validity (measuring the intended point of interest), and responsiveness to change (can a measure respond to an intervention?). The group collectively agreed that this terminology is often misused and suggested two references1,2 for those interested in conducting outcome measurements in the field of LLP. The literature review presented by Elizabeth Condie, PT, FCSP, provided a critical appraisal of the general information, practicality, reliability, validity, scaling, and bias of 28 key articles published in the last 10 years. One conclusion to be drawn from the literature is that there is currently no gold standard or consensus opinion on the most appropriate measure in LLP. A multitude of measures is available, yet there is an absence of guidance on how to select the most appropriate tool and some flawed measures are still applied. Also, many papers are difficult to read, use much jargon, and present a great deal of complex statistical information that may at times be misinterpreted. Inconsistency in definitions, terminology, and interpretations of psychometric tests was also found throughout the reviewed papers. After systematic review of the literature, it was concluded by the reviewers and agreed on by the group that this inconsistent and incorrect use of terminology and overuse of complex language were barriers to the implementation of outcome measures into routine clinical practice. Various developers and users of specific outcome measurement tools in LLP were asked to discuss strengths and weaknesses or any specific applications or limitations with application of the outcome measurement tool. Robert Gailey, PhD, PT, presented his experience in the development of the Amputee Mobility Predictor (AMP),3 designed to assist clinical decision-making as a predictor instrument to determine the amputee's ability for ambulation. The AMP has the ability to be administered without (AMPnoPro) and with (AMPPro) the use of a prosthesis. Therefore, it can be administered before the initial fitting of the first prosthesis and can also assess an amputee who has used a prosthesis for years. The AMP is a valid measure of function as it pertains to an amputee's ability to ambulate. Not only can the AMP differentiate among Medicare Functional Classification Levels or K-Levels, it is strongly related to other measures of function in the amputee such as the 6-minute walk. Therefore, clinicians have an objective functional measurement tool to support their clinical judgment. The AMP is relatively easy to administer in 15 minutes or less, with a simple scoring system, requiring very little equipment or space. Three self-report tools were discussed. Christiane Gauthier-Gagnon, MS, spoke on the application of the Prosthetic Profile of the Amputee (PPA) and Locomotor Capabilities Index (LCI),4 a subset of the PPA. David Boone, CP, LP, MPH, PhD, discussed the implementation of the Prosthesis Evaluation Questionnaire (PEQ),5 and Allen Heinemann, PhD, described the development of the Orthotics and Prosthetics User Survey (OPUS).6 The PPA is a tool that includes measurements in the domains of physical conditions, prostheses, prosthetic capabilities (when including the LCI), environment, leisure, and demographic characteristics. The LCI, which can be administered independently, is the only portion that results in a total score. This tool is currently in use in the research field, graduate studies, and in clinical practice. The PEQ is a self-administered test with nine scales, each with a single underlying concept. Each subscale has been validated for independent use, so the entire evaluation does not need to be administered. The PEQ is effective in evaluating progress in a test-retest situation; however, it has not shown the ability to detect differences between various components (e.g., a comparison of different prosthetic foot designs). The OPUS is a self-reported or practitioner-administered survey used to evaluate lower extremity function, quality of life, satisfaction with service, and satisfaction with both orthotics and prosthetic devices. It can be completed quickly (5 to 10 minutes), and the subscales can be used independently. Biomechanical measures of performance were also discussed, including the Step Activity Monitor and Quantitative Gait Analysis, for evaluation of performance. Selected conclusions from the previous State-of-the-Science Conference on prosthetic ankle-foot mechanisms were also reviewed. In David Boone's second presentation, he described the data collected using the Step Activity Monitor. This measurement tool quantitatively records the number of steps taken by an individual over a period up to 2 months. In the LLP population, this device can be used to assess prosthesis use. Results can be interpreted through the steps per day and also the distribution of the steps throughout the day, providing insight into real-world use of the prosthesis. Steven Gard, PhD, provided a description of the measurements available in gait analysis. Gait analysis can be used for documentation of rehabilitation progress over time and may sometimes be useful for evaluation of pathological conditions, but it cannot currently be used for diagnosis of prosthetic problems or assist with prosthetic prescription. Gait analysis provides objective measures that can be used to confirm visual analysis and record changes over time. To evaluate gait analysis data, it is necessary to control for variations in ambulation speed, and it is important that data be collected in the same manner (e.g., consistent marker placement). Although significant quantities of data can be recorded, it is currently difficult to use gait analysis to evaluate prosthetic locomotor function. The overview report of SSC5, delivered by Brian Hafner, PhD, found that the tools available to measure prosthetic ankle-foot systems in human subject testing could be divided into biomechanical studies and perceptive/functional analyses, but noted that few consistent statistical findings could be documented in the reported literature. Various alternative assessment tools, including descriptive dialogues, functional assessment questionnaires, and numerical rating scales were described. Key recommendations from this prior State-of-the-Science conference included evaluation of components in situations other than level ground walking, identification of assessment tools sensitive to prosthetic devices, and development of tools for clinicians to use to assess patient needs. One primary conclusion that emerged from the discussion regarding the strengths and weaknesses of outcome measures in LLP was that although there are many tools that have been validated in one situation, this does not mean they can be used in any situation. The tool must be validated for the specific application (question to be answered and population to be studied). An article on how to choose an outcome measurement tool using an evidence-based approach (not limited to the area of LLP) was recommended during the discussion.7 Before developing new prosthetic outcome tools, the group recommended first investigating current tools, both in our field and in others, and how they could be applied to LLP in a validated and reliable manner, with a clear definition of the outcome to be evaluated. As an example, there has been substantial investigation into how falls impact the geriatric population, but these results have not been confirmed in the lower limb amputee population. In addition to the presentations on the strengths and weaknesses of the various outcome measurement tools, presentations were given on clinical barriers to outcome measurement. Two surgeon perspectives were offered by Michael Pinzur, MD, and Douglas Smith, MD. Paul Pasquina, MD, discussed the unique opportunities for gathering outcome data that have been available at Walter Reed Army Medical Center. Further clinical perspectives were provided by Jack Uellendahl, CPO, on the clinician's perspective, and by Deanna Fish, CPO, on concerns from the payer's perspective that affect the prosthetic profession. Dr. Pinzur outlined his perspective on outcome measure use in prosthetics. The surgeon is primarily interested in subjective and objective measures relating to function, whereas the patient is often more interested in communicating their subjective personal impressions. For example, although a 49-year-old tennis player will have significantly different expectations of success than a 70-year-old geriatric diabetic patient, the surgeon might feel that if they can both walk and go back to work and community activities, then their outcomes or results were equal from a medical perspective. The PEQ was recommended as a tool for comparing very specific and clearly defined populations (e.g., unilateral, dysvascular, transtibial amputees, 1 month after fitting using the prosthesis). However, the PEQ did not correlate with functional tests, perhaps due to the previously mentioned differences in expectations and perceptions of function between the amputee and the surgeon. Dr. Smith offered additional insights from the surgical perspective. He believes the number-one issue for individuals who use a lower limb prosthesis is socket fit and socket comfort. Unfortunately, there currently are no standardized, well-accepted methods to adequately measure socket fit and comfort either for research or for clinical use. Five additional points were discussed: There exists much more day-to-day variation in functional activity, walking ability, walking symmetry, and forces than we think. We do not know when a person has actually become accommodated to a new prosthesis or a new prosthetic component. The functional consequences of the medical comorbidities are likely to change more during the research trial than the variable we are trying to measure. Subjects may become overwhelmed with research forms when using multiple questionnaires (question overload) thus impacting the accuracy of the data. We are limited in prosthetic research trial size by a) the limited number of available research subjects who meet enrollment criteria, b) the extended duration of these trials, c) the time commitment required of the research subjects, and d) the financial expense of research protocols involving prosthetic devices that by necessity involve customized socket fabrication, fitting and alignment. Dr. Pasquina presented an overview of outcome assessment of the combat amputee undertaken at Walter Reed Army Medical Center. The rehabilitation process of those returning from the conflict in Iraq offers unique opportunities for study. This population has been predominantly treated in two locations, and evaluations have been quite thorough. The group is young (average age in the 20s and 30s), making it potentially possible to track a large population over an extended period. Specific tools used to assess this population include those to measure mobility (AMP,3 6-minute walk test,8 Timed Up and Go,9 balance, and gait analysis), quality of life (SF-3610 at admission and discharge), and function (Functional Independence Measure11 at admission and discharge). This group of amputees has a high level of motivation that is distinctly different from many other prosthetic populations. Motivation is a potentially important influence on outcomes that cannot be measured at this time. Ms. Fish discussed the influence of the payer on prosthetic prescription. She reported that the prosthetic community has failed to produce the necessary research to identify and quantify the benefits of specific prosthetic designs and components. These limitations include a lack of outcomes research, randomized controlled trials, and publications in peer-reviewed journals. Lack of consistent terminology increases the difficulties in justifying and receiving financial approval for new components (e.g., standard vs. enhanced). To better justify new componentry, it is important to show not only how this technology can improve function or quality of life, but also how optimal care can reduce the need for more expensive healthcare overall. Mr. Uellendahl highlighted barriers to the routine clinical application of outcome measurement tools, noting that prosthetists are generally good at subjectively assessing patient satisfaction with comfort, function, and appearance. For outcome measurement tools to gain widespread clinical acceptance, these tests must be quick and easy to set up, administer, and analyze. Prosthetists might need training in administering any new outcome measurement tools. If prosthetists are to use quality-of-life measures, they should be provided guidance on how to manage issues that might arise related to depression, pain, medication, and so forth, that would be considered outside their current scope of practice. In summary, the group agreed the principle clinical barriers to implementing outcome measures on a routine basis are the demand on clinicians' time (having the time to apply the tool); recruiting an adequate population for studies (especially long-term studies where resources are tight and patient attrition due to disease or death is likely); potential differences between the patient's and clinician's goals (such as restoring walking function versus cosmetic appearance); the technology required to perform the test (equipment, software, and so forth); finances (funding opportunities); concerns about poorly constructed research studies; concern about justifying the effort required (how do we show it is financially worthwhile to spend money and time on outcome measures unless we can convince clinicians they can get a return on that investment from improved patient care or consistent insurer payment?). Finally, it was suggested that a national multicenter databank of outcome results to pool the available information within our relatively small field would help demonstrate, with greater power and significance than would otherwise be possible, differences among components, various levels of activity, and similar clinical concerns. RESULTS OF QUESTION 5 Overall, five Research Priorities and their order of importance were agreed on by the participants. The primary future research questions were then categorized into these five priority areas: 1) Health Care Economic Impact; 2) Rehabilitation Guidance Tools; 3) Activity Limitation/Performance Measures; 4) Pain and Comfort; and 5) Quality of Life. Health Care Economic Impact was agreed to be the most important category, with Rehabilitation Guidance Tools second. After determining the list of five research priorities, the group discussed the current outcome measurement tools and categorized them according to priority (Table 1).12–24 Based on the discussion of the group, the Barthel test25 is not recommended as an instrument for outcome measures in LLP and was therefore not included in the review.Table 1: Review of existing outcome measurement tools related to lower limb prostheticsAfter reviewing the existing tools, the primary research questions relative to each of the priority categories were determined: Health Care Economic Impact: How does patient management and prosthesis prescription affect health care costs over time? Rehabilitation Guidance Tools: What existing tools can be used to predict outcomes? AMP Others What are the clinical indicators that can maximize prosthetic rehabilitation? Can existing tools be used to measure premorbid function? Activity Limitation/Performance Measures: What measurements are responsive to changes in __________? Improvements Deficits Physical therapy interactions Componentry Time Health status Population groups Pain and Comfort: How can socket comfort and performance be evaluated and measured? How does prosthetic use correlate to secondary conditions or pain, including but not limited to: Back pain? Degenerative joint disease? Diabetic foot complications? Residual limb pain? Phantom pain? Hip pain? Osteoporosis? Quality of Life (QOL): How do QOL measures correlate to performance measures? What constructs in existing QOL tools are addressed in various prosthetic populations? SUMMARY In summary, the primary future research questions were categorized into: Health Care Economic Impact Rehabilitation Guidance Tools Activity Limitation/Performance Measures Pain and Comfort Quality of Life The group stressed that overlap among the five priority categories is to be expected, and in many cases, encouraged. For example, pain and comfort will inevitably affect an amputee's ability to wear a prosthesis and go to work, which will have a direct impact on the economics of the situation. Therefore, tools that quantify socket fit and comfort may show an improved economic impact to the payer. Also, if the person has less pain, it is likely that they will be more active, have improved performance, and report an improved quality of life. It was strongly recommended that existing outcome tools in other fields that might be applicable to prosthetics be identified and investigated. For instance, studies have been done in other fields on such relevant topics as falls, health care economics, over-use injury, and chronic disability. Due to the importance given by the group to Health Care Economic Impact, it was recommended that this be the topic for a future State-of-the-Science meeting.26,27