Western Canada Waiting List Project Research Articles

Development and Reliability of an Appropriateness and Prioritization Instrument for Eye Care Practice: A Modified Delphi Process

ABSTRACTPurpose: To modify the Western Canada Waiting List Project (WCWLP) cataract prioritization criteria into measurement instruments for assessing appropriateness and prioritization emphasizing health-related quality of life (HRQoL) and examining reliability.Methods: A modified Delphi process was conducted using two rounds of online surveys and a face-to-face meeting. A panel of eye care experts rated the relevancy of criteria on a 5-point scale. Patient narratives highlighted the patient experience. A G-theory framework was used to assess inter- and intra-rater agreement using the criteria.Results: Nine Ophthalmologists, three Optometrists, and one General Practitioner participated in the modified Delphi process. Consensus to include/exclude was reached on all criteria. Seventeen criteria were included and 16 were excluded. The most significant changes were related to the categorization of comorbidities and expansion of HRQoL questions. The overall reliability was good (ϕ = 0.852).Conclusion: The WCWLP was modified to include appropriateness and to better reflect HRQoL. Having achieved consensus and demonstrated reliability of the criteria, the next step is to estimate weights for criteria and to validate against patient health outcomes data.

Comparison of 3 systems for assigning priority to patients on waiting lists for cataract extraction

Objective: As the demand for cataract surgery grows, patients in some health care systems are assigned to a waiting list. Several explicit priority tools have been developed. We compared 3 of these: the IRYSS-Cataract Priority Score (ICPS), Catalan Agency for Health Technology Assessment and Research cataract priority system (CCPS), and Western Canada Waiting List project for cataract surgery (WCWL). Design: Prospective cohort study. Participants: A total of 1723 consecutive patients awaiting cataract surgery at 5 hospitals. Methods: The ICPS, CCPS, and WCWL were applied to these patients after the collection of data. The 3 prioritization systems were compared using correlation methods, 95% limits of agreement, and the kappa coefficient. Results: Means and standard deviations for the different prioritization systems were 60.9 (22.2) for the ICPS, 35.7 (20.4) forthe CCPS, and 25.7 (21.5) forthe WCWL. Pearson's correlation coefficients were 0.56 between the ICPS and the CCPS, 0.62 between the WCWL and CCPS, and 0.71 between the ICPS and WCWL. The kappa value among them ranged from 0.13 to 0.40, and the intraclass correlation coefficients ranged from 0.31 to 0.55. Conclusions: These 3 prioritization tools showed acceptable correlations but assigned significantly different point scores to similar scenarios. This may have consequences when using these tools for managing a waiting list for cataract extraction.

The Western Canada Waiting List Project: development of a priority referral score for hip and knee arthroplasty

Recognizing the concerns about long waiting times to see some specialists in Canada, and the burden this places on both primary care and specialist clinicians, the Western Canada Waiting List (WCWL) undertook the Primary Care Project. The goal was to develop a valid, reliable, standardized prioritization tool for use by primary care providers in making referrals to specialists. WCWL is a 20-partner collaboration committed to addressing long waiting times to access scheduled health care services. A previously developed prioritization tool for hip and knee replacement was adapted for use by family doctors, based on expert feedback from a clinical panel of primary care providers and from orthopaedic surgeons. Rater assessments of standardized paper cases were used to generate weights for criteria items in the Priority Referral Score (PRS). Intraclass correlations (ICCs) were calculated to assess reproducibility, and weights were estimated using a mixed-effects model. The weights and criteria items were modified following feedback of these results to the panel. The resulting PRS was reliability-tested with a different set of standardized case descriptions. One item was removed from the Hip and Knee Surgery tool and two items more pertinent to family medicine (mobility and medications) were added. The resulting eight-item PRS had a test-retest ICC of 0.84. The mean intrarater ICC was 0.79. An eight-item priority-setting tool has been developed to assist in queuing patients in order of urgency when they are referred to an orthopaedic surgeon for possible hip or knee arthroplasty. The tool had excellent inter- and intrarater reliability and was seen to have face validity by a panel of primary care providers who advised on the project.

NIKE: a new clinical tool for establishing levels of indications for cataract surgery

The purpose of this study was to construct a new clinical tool for establishing levels of indications for cataract surgery, and to validate this tool. Teams from nine eye clinics reached an agreement about the need to develop a clinical tool for setting levels of indications for cataract surgery and about the items that should be included in the tool. The tool was to be called 'NIKE' (Nationell Indikationsmodell för Kataraktextraktion). The Canadian Cataract Priority Criteria Tool served as a model for the NIKE tool, which was modified for Swedish conditions. Items included in the tool were visual acuity of both eyes, patients' perceived difficulties in day-to-day life, cataract symptoms, the ability to live independently, and medical/ophthalmic reasons for surgery. The tool was validated and tested in 343 cataract surgery patients. Validity, stability and reliability were tested and the outcome of surgery was studied in relation to the indication setting. Four indication groups (IGs) were suggested. The group with the greatest indications for surgery was named group 1 and that with the lowest, group 4. Validity was proved to be good. Surgery had the greatest impact on the group with the highest indications for surgery. Test-retest reliability test and interexaminer tests of indication settings showed statistically significant intraclass correlations (intraclass correlation coefficients [ICCs] 0.526 and 0.923, respectively). A new clinical tool for indication setting in cataract surgery is presented. This tool, the NIKE, takes into account both visual acuity and the patient's perceived problems in day-to-day life because of cataract. The tool seems to be stable and reliable and neutral towards different examiners.

Are there better ways to determine wait times?

Recently, the Western Canada Waiting List Project (WCWLP) and the Wait Time Alliance (WTA) independently released benchmark wait times for various medical procedures. Both groups used a panel of experts who reviewed survey data from clinicians, patients and the general public as well as literature

Patient and physician perspectives of maximum acceptable waiting times for cataract surgery

Background: Lengthy waiting times for cataract surgery are an important issue in countries with publicly funded health care systems. To improve the fairness, timeliness, and certainty of waiting-time management, the Western Canada Waiting List Project has developed priority criteria scores (PCSs) related to urgency and linked to maximum acceptable waiting times (MAWTs). The purpose of our study was to compare patient and physician perspectives of MAWT for different levels of urgency. A second aim was to assess the determinants of patient and surgeon perspectives on MAWT. Methods: Ophthalmologists assessed consecutive patients waitlisted for cataract surgery. Data included a MAWT, a visual analogue scale of urgency (VAS urgency), and the cataract PCS. Patients were mailed questionnaires to assess their perspectives of MAWT and VAS urgency. They were also sent a measure of visual function called the Visual Function Assessment. We used hierarchical linear regression to assess the determinants of MAWT. Results: The mean age of the 213 patients was 73.9 years; 56.8% were female and 71.8% were booked for first eye surgery. Physician-rated MAWT was significantly longer than patient-rated MAWT (mean 15.1 vs. 9.9 weeks). Median physician MAWTs ranged from 12 (most urgent) to 20 (least urgent) weeks, and patient MAWTs, from 4 to 8 weeks. A 3-step hierarchical linear regression model showed that, after adjusting for age and sex, the priority criteria added significantly to the surgeon model ( R 2 change = 0.22). Significant predictors were ocular comorbidity, impairment in visual function, and ability to work or live independently or care for dependents. After the addition of VAS urgency, the final model explained 42% of the variance in surgeon MAWT. Significant predictors were age-related macular degeneration and VAS urgency. A 4-step hierarchical regression model for patient MAWT showed that after step 2, sex and visual acuity in the nonsurgery eye were significant predictors. The final model accounted for 11% of the variance in patient MAWT. Significant predictors were sex (males had lower MAWT) and VAS urgency. Interpretation: Patient and physician views on MAWT differ, yet both are critical to a fair process for developing standardized waiting times related to levels of urgency. Results from this study provide initial inputs to the formulation of benchmark waiting times for different levels of the cataract PCS.

Determinants of patient and surgeon perspectives on maximum acceptable waiting times for hip and knee arthroplasty

Lengthy waiting times for hip and knee arthroplasty have raised concerns about equitable and timely access to care. The Western Canada Waiting List project has developed priority criteria scores linked to maximum acceptable waiting times (MAWT) for different levels of priority. Our study purpose was to assess the determinants of patient- and surgeon-rated MAWT, and to test whether the anticipated waiting time has an independent influence after adjusting for age, sex and patient urgency. A second aim was to compare MAWT, waiting time and anticipated waiting time for different levels of urgency assessed using the priority criteria score. Orthopaedic surgeons assessed 233 consecutive patients waiting for arthroplasty in terms of their urgency (assessed using the priority criteria score and a visual analogue scale), MAWT and anticipated waiting time. Patient data included urgency (assessed by a visual analogue scale), MAWT and the Western Ontario McMaster Osteoarthritis index. We used hierarchical linear regression to test the models. After adjusting for age and sex, urgency (assessed by priority criteria score and visual analogue scale) and anticipated waiting time accounted for 40% of the variance in surgeon MAWT. The patient model accounted for 30% of the variance in patient MAWT. Older patients preferred signficantly shorter MAWTs (P <0.05). Anticipated waiting time added significantly to both the surgeon and patient MAWT models (R(2) change 0.11 and 0.07, respectively). Actual waiting time was weakly correlated with urgency assessed using the priority criteria score (r = -0.25, P <0.0001). Patients' and surgeons' views are critical to a fair process of establishing MAWT for elective procedures. Anticipated waiting time may influence the perspectives on MAWT and must be considered in their interpretation.

The Prioritization of Patients on Waiting Lists for Cataract Surgery: Validation of the Western Canada Waiting List Project Cataract Priority Criteria Tool

Purpose: To assess the validity of the Cataract Priority Criteria Score (PCS), developed by the Western Canada Waiting List (WCWL) Project to determine patient prioritization for cataract surgery. Methods: Ophthalmologists assessed consecutive patients with the PCS and a visual analogue scale of urgency (VAS Urgency). Patients were mailed questionnaires pre- and post-surgery. Outcome measures were the Visual Function Assessment (VFA), EuroQol (EQ-5D), and best-corrected visual acuity. Results: The sample of 253 patients was 58% female (mean age, 73.7 years); 166 completed pre-and post-surgery VFA. The correlation of the PCS and VAS Urgency was 0.65 (p = 0.000). Adjusting for age, first or second eye surgery, and post-operative complication, the PCS predicted improvement in the VFA and visual acuity (p < .05). Conclusions: These data provide some evidence to support the convergent and predictive validity of the PCS. Multiple patient outcomes should be used in the evaluation of the validity of priority scores. Steering Committee of the Western Canada Waiting List Project*

Prioritization of patients on waiting lists for hip and knee replacement: Validation of a priority criteria tool

This study tested the reliability and validity of the Western Canada Waiting List Project priority criteria score (PCS) for prioritizing patients waiting for hip and knee arthroplasty. Sixteen orthopedic surgeons assessed 233 consecutive patients at consultation for hip or knee arthroplasty. Measures included the PCS, a visual analogue scale of urgency (VAS urgency), and maximum acceptable waiting time (MAWT). Patients completed a VAS urgency, an MAWT, the Western Ontario McMaster Osteoarthritis Index (WOMAC), and the EQ-5D. Using correlational analysis, convergent and discriminant validity was assessed between similar constructs in the priority criteria and WOMAC. Median MAWTs were determined for five levels of urgency based on PCS percentiles. Internal consistency reliability was assessed with Cronbach's alpha. The sample of 233 patients (62 percent female) ranged in age from 18 to 89 years (mean, 66.3 years). A total of 45 percent were booked for hip and 55 percent for knee arthroplasty. Correlations were strong between the PCS and surgeon VAS urgency (r = .79) and weaker between patient and surgeon measures of VAS urgency (r = .24) and MAWT (r = .44). Correlation coefficients between similar constructs in the priority criteria and WOMAC ranged from 0.24 to 0.32 and were higher than those measuring dissimilar constructs. For decreasing levels of urgency, the median MAWT ranged from 10 to 12 weeks for surgeons and 4 to 12 weeks for patients. Cronbach's alpha was 0.79. Results support the validity of the PCS as a measure of surgeon-rated urgency. Patients might be ranked differently with different prioritization measures.

Toward Standard Definitions for Waiting Times

There are no standard or universally accepted definitions of waiting times for a broad range of health services and procedures. The Western Canada Waiting List Project, like other similar projects, has recognized the need to establish such standard definitions to improve the accuracy and comparability of waiting time information across procedures and jurisdictions and of information provided to patients. This article proposes standard definitions of waiting times for surgery and magnetic resonance imaging.

Setting priorities on waiting lists: point-count systems as linear models.

The Western Canada Waiting List Project (WCWL) is a federally funded initiative designed to develop tools for managing waiting lists. The principal tools developed by WCWL are point-count measures that assess the severity of patients' conditions and the extent of benefit expected from wait-listed services. Points are assigned according to the severity of patients' symptoms and clinical findings. Points on each factor are added and the total score is considered indicative of relative clinical urgency. Such point-count measures function as linear models from a statistical perspective. This paper describes the relevance of this functional relationship for the development and validation of priority criteria.

Developing cataract surgery priority criteria: results from the Western Canada Waiting List Project

Background: The Western Canada Waiting List Project (WCWL) is a federally funded partnership of 19 organizations, including medical associations, health authorities, ministries of health and research organizations, that was created to develop tools to assist in the management of waiting lists. The WCWL cataract surgery panel, one of five panels constituted under this project, developed and tested a set of standardized clinical criteria for prioritizing among patients awaiting cataract surgery.Methods: The cataract surgery panel was composed of seven academic and community ophthalmologists, two family physicians, an optometrist, a health care services researcher and a health information specialist. The panel met three times between October 1999 and June 2000. The priority criteria were applied to 563 patients in the four western provinces. Regression analysis was used to determine the set of criteria weights that collectively best predicted clinicians' overall ratings of urgency. Interrater and test-retest reliability were assessed, based on clinicians' ratings of videotaped interviews with six hypothetical patients.Results: The resulting criteria accounted for about one-third of the observed variance in clinicians' ratings of overall clinical urgency (R2 = 31.20. The panel modified the original criteria and weights based on empirical results and clinical judgement. Reliability of the revised criteria items was mixed, with the lowest reliability observed for items dealing with glare and social role.Interpretation: Participating clinicians considered the criteria easy to use and reasonably reflective of expert surgical judgement regarding clinical urgency. Further development and testing of the tool appear warranted.

Lining Up for Children's Mental Health Services: A Tool for Prioritizing Waiting Lists

The Western Canada Waiting List Project (WCWL) is a federally funded partnership of 19 health-related organizations that was created to develop tools to manage waiting lists for five types of health services. The children's mental health (CMH) panel developed and tested a set of standardized clinical criteria for setting priorities among patients awaiting CMH services. The criteria were applied to 817 patients by 92 mental health professionals in three western provinces. Regression analysis was used to determine the set of criteria weights that collectively best predicted clinicians' global urgency ratings. To assess reliability, raters used the criteria to score six standardized "paper cases." The criteria accounted for about 40% of the observed variance in overall urgency ratings (R2 = 41.7%). The panel modified the criteria on the basis of the initial empirical work. Reliability assessment of the revised tool indicated that half of the items had excellent or fair/good interrater agreement; test-retest reliability was good. Priority criteria were able to capture clinicians' judgments of relative urgency in the CMH setting. A number of operational challenges remain with the use of priority criteria for scheduling CMH services. Further development and testing of the tool appear warranted.

Commentary: Creating Order Out of Chaos: A Response to “Lining Up for Children's Mental Health Services”

One cannot help but initially react to Smith et al., 2002 Smith D Hadorn DC the Steering Committee of the Western Canada Waiting List Project Lining up for children's mental health services: a tool for prioritizing waiting lists. J Am Acad Child Adolesc Psychiatry. 2002; 41: 367-376 Abstract Full Text Full Text PDF PubMed Scopus (41) Google Scholar paper on a method for prioritizing waiting lists by deploring the situations that create such lists in the first place.