National Patient-reported Outcome Measures Research Articles

Proceedings of the 6th National Patient Reported Outcome Measures (PROMs) Annual UK Research Virtual Conference, Bridgend, Wales 2022

Proceedings of the 6th National Patient Reported Outcome Measures (PROMs) Annual UK Research Virtual Conference, Bridgend, Wales 2022

Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme

ObjectiveOver 160 000 participants per year complete the 12-item Oxford Hip and Knee Scores (OHS/OKS) as part of the NHS England Patient-Reported Outcome Measures (PROMs) programme. We used a modern...

A matched comparison of the patient-reported outcome measures of 38,716 total and unicompartmental knee replacements: an analysis of linked data from the National Joint Registry of England, Northern Ireland and Isle of Man and England’s National PROM collection programme

Background and purpose — The surgical treatment options for severe knee osteoarthritis are unicompartmental (UKR) and total knee replacement (TKR). For patients, functional outcomes are more important than revision rate. We compared the patient-reported outcome measures (PROMs) of both implant types using a large PROMs dataset. Patients and methods — We analysed a propensity-matched comparison of 38,716 knee replacements (19,358 UKRs and 19,358 TKRs) enrolled in the National Joint Registry and the English National PROM collection programme. Subgroup analyses were performed in different age groups. Results — 6-month postoperative Oxford Knee Score (OKS) for UKR and TKR were 38 (SD 9.4) and 36 (SD 9.4) respectively. A higher proportion of UKRs had an excellent OKS (≥ 41) compared with TKR (47% vs 36%) and a lower proportion of poor OKS (< 27) scores (13% vs. 16%). The 6-month OKS was higher in all age groups for UKR compared with TKR, with the difference increasing in older age groups. The mean 6-month EQ-5D score was 0.78 (SD 0.25) and 0.75 (SD 0.25) respectively. The improvement in EQ-5D resulting from surgery was higher for UKR than TKR both overall and in all age groups. All comparisons were statistically significant (p < 0.05). Interpretation — UKR had a greater proportion of excellent OKS scores and lower proportion of poor scores than TKR. Additionally, the quality of life was higher for UKR compared with TKR. These factors should be balanced against the higher revision rate for UKR when choosing which procedure to perform.

Outcome-based commissioning of knee arthroplasty in the NHS: system error in a national monitoring programme and the unintended consequences on achieving the Best Practice Tariff.

Our unit was identified as a negative outlier in the national patient-reported outcome measures (PROMs) programme, which has significant funding implications. As a centre that carries out a high volume of unicompartmental knee arthroplasty (UKA), our objectives were: 1) to determine whether the PROMs programme included primary UKA when calculating the gain in Oxford Knee Score (OKS); and 2) to determine the impact of excluding primary UKA on calculated OKS gains for primary knee arthroplasty. National PROMs data from England (2012 to 2016) were analyzed. Inclusion of UKA cases in the national PROMs programme was determined using clinical codes. Local OKS gain was calculated for UKA and TKA and compared with the published PROMs results for 2012/13. Use of the recommended codes for primary UKA excluded 99.6% of UKAs locally and 97% nationally from PROMs programme analysis. Inclusion of primary UKAs in PROMs analysis improved our OKS gain from 15.1 (below average) to 16.3 (above average) for 2012/13 for primary knee arthroplasty. Exclusion of UKA patients from the PROMs programme is a nationwide issue that potentially introduces bias when comparing OKS gain between centres. Where commissioning decisions are based on routinely collected data, it is imperative that the underlying methodology is appropriate to generate valid results.

PP054 The All Wales Patient Reported Outcome Measures (PROMs), Patient Reported Experience Measures (PREMs) and Effectiveness Program

INTRODUCTION:Prudent health care aims to do the minimum needed to achieve the greatest patient benefit. This aim relies on the availability of evidence on the safety and efficacy of interventions to support decision making. The principles of prudent healthcare support co-production, whereby service users contribute to service provision. Collection of patient reported data is becoming more widespread, however use of this data to inform decision making is limited.METHODS:A national patient reported outcome measures (PROMs) program has been formed supported by the Welsh Government, Welsh Health Boards and the NHS Wales Informatics Service. An electronic platform has been developed to facilitate collection of PROMs and patient reported experience measures (PREMs) from patients treated in secondary care. We collected baseline PROMs where possible and invited patients to submit PROMs and PREMs post-treatment. Data collected included EuroQuol five dimensions questionnaire (EQ5D), co-morbidities, body mass index (BMI), smoking history and alcohol intake. Disease specific tools were used where available and responses linked to clinical data. Individual level data will be available during clinic consultations, and collated data analyzed on national and health board levels to assess clinical effectiveness. The platform is currently being piloted in several sites across Wales.RESULTS:Initial baseline pilot data from hip replacement patients found that over 55 percent of responders were classed as overweight or obese, with over 80 percent carrying out less than the national guidelines for exercise.The baseline scores for hip patients were; EQ-5D Index (Mean .29, median .29, range (-.59 -1)), EuroQol-visual analogue scales (EQ-VAS) (Mean 57.8, median 60, range (0:100)), and Oxford Hip Score (Mean 14.9, median 14, range (0:48)).When compared to baseline scores collected by NHS England in 2015/16 (1), the average EQ5D Index and Oxford Hip Score collected in Wales was lower than that in England (p&lt; .05).CONCLUSIONS:The program will provide a large dataset from patients across all of Wales with data on numerous chronic and acute conditions. The data collected will facilitate service improvements and will inform decision making as part of the prudent healthcare agenda.

Waiting time prioritisation: Evidence from England

A number of OECD countries have introduced waiting time prioritisation policies which give explicit priority to severely ill patients with high marginal disutility of waiting. There is however little empirical evidence on how patients are actually prioritised. We exploit a unique opportunity to investigate this issue using a large national dataset with accurate measures of severity on nearly 400,000 patients. We link data from a national patient-reported outcome measures survey to administrative data on all patients waiting for a publicly funded hip and knee replacement in England during the years 2009–14. We find that patients suffering the most severe pain and immobility have shorter waits than those suffering the least, by about 24% for hip replacement and 11% for knee replacement, and that the association is approximately linear. These differentials are more closely associated with pain than immobility, and are larger in hospitals with longer average waiting times. These result suggests that doctors prioritise patients according to severity even when no formal prioritisation policy is in place and average waiting times are short.

The Outcomes and Experiences Questionnaire: development and validation.

BackgroundThis report presents evidence regarding the development and validation of a new questionnaire, the Outcomes and Experiences Questionnaire (OEQ). The rationale for the questionnaire is to bring together into one short instrument questions about two distinct domains – patients’ reports of the outcomes of their care and how they experience care.MethodsThe OEQ was developed from literature reviews, iterative drafting and discussion within the research group and cognitive testing with a sample of patients who had a hospital experience. Two validation studies were carried out with an eleven item OEQ. The goals of the studies were to examine response rates and to test specific hypotheses of how OEQ should relate to other variables normally collected in the two studies. In the first study, the OEQ was added to the follow-up questionnaires for patients (n=490) receiving surgery for hip or knee replacement or varicose vein procedures participating in the national Patient Reported Outcome Measures (PROMs) program permitting the analysis of the OEQ against change scores for the measures obtained before and after surgery. In the second study the OEQ was included in a sample of patients (n=586) who had been selected to receive the National Health Service (NHS) inpatient survey from three contrasting hospital trusts.ResultsResults from study one provided consistent and substantial evidence of construct validity of OEQ particularly for those receiving hip or knee replacement. The OEQ sub-scales behaved differently and as predicted against other PROMs variables. Again hypotheses of how the two sub-scales regarding outcomes and experiences would relate to the existing domains of patient experience in the inpatient survey were broadly confirmed in study two.ConclusionThe report provides encouraging evidence of the OEQ’s capacity to assess distinct reports from patients about outcomes and experiences of care within a single short questionnaire.

Relationship between patient-reported outcomes of elective surgery and hospital and consultant volume.

Our aim was to analyze the relationship for 3 elective operations between outcome [patient-reported outcome measures (PROMs) for functional status, health-related quality of life, and postoperative complications] and both hospital and consultant volume. Hospitals (NHS and independent) and consultants undertaking at least 10 NHS-funded procedures during 2011/2012 were included (230 hospitals for hip and knee replacement, 257 for hernia repair; 978 consultants for hip replacement, 1172 for knee replacement, and 1288 for hernia repair). Outcomes (disease-specific and generic PROMs, patient-reported complications) were available from the NHS National PROMs Programme for 2009/2010 to 2011/2012. Relationship between case-mix adjusted outcomes and volume investigated using multilevel modeling. There was considerable variation in hospital volumes (about 10-fold) and consultant volumes (about 5-fold). No significant association was observed between hospital volume and outcome for all 3 procedures. For consultant volume, there was no significant association for knee replacement or hernia repair. However, outcomes were statistically significantly better for hip replacement, although the effect was of little clinical significance: an additional 10 cases was associated with a higher Oxford Hip Score (0.06), higher EQ-5D score (0.001), and lower odds ratio of complications (0.992). There are unlikely to be any benefits to patients from centralization of elective surgery into higher volume hospitals as regards the effectiveness of surgery or the avoidance of minor complications. There is some evidence that very low volume consultants achieve poorer outcomes than higher volume colleagues but the difference is slight and of little or no clinical significance.

Estimating recruitment rates for routine use of patient reported outcome measures and the impact on provider comparisons

BackgroundThe routine use of patient reported outcome measures (PROMs) aims to compare providers as regards the clinical need of their patients and their outcome. Simple methods of estimating recruitment rates based on aggregated data may be inaccurate. Our objectives were to: use patient-level linked data to evaluate these estimates; produce revised estimates of national and providers’ recruitment rates; and explore whether or not recruitment bias exists.MethodsCase study based on patients who were eligible to participate in the English National PROMs Programme for elective surgery (hip and knee replacement, groin hernia repair, varicose vein surgery) using data from pre-operative questionnaires and Hospital Episode Statistics. Data were linked to determine: the eligibility for including operations; eligibility of date of surgery; duplicate questionnaires; cancelled operations; correct assignment to provider. Influence of patient characteristics on recruitment rates were investigated.ResultsNational recruitment rates based on aggregated data over-estimated the true rate because of the inclusion of ineligible operations (from 1.9% - 7.0% depending on operation) and operations being cancelled (1.9% - 3.6%). Estimates of national recruitment rates using inclusion criteria based on patient-level linked data were lower than those based on simple methods (eg hip replacement was 73% rather than 78%).Estimates of provider’s recruitment rates based on aggregated data were also adversely affected by attributing patients to the wrong provider (2.4% - 4.9%). Use of linked data eliminated all estimates of over 100% recruitment, though providers still showed a wide range of rates.While the principal determinant of recruitment rates was the provider, some patients’ socio-demographic characteristics had an influence on non-recruitment: non-white (Adjusted Odds Ratio 1.25-1.67, depending on operation); most deprived socio-economic group (OR 1.11-1.23); aged over 75 years (OR 1.28-1.79). However, there was no statistically significant association between providers’ recruitment rates and patients’ pre-operative clinical need.ConclusionsAccurate recruitment rates require the use of linked data to establish consistent inclusion criteria for numerators and denominators. Non-recruitment will bias comparisons of providers’ pre-operative case-mix and may bias comparisons of outcomes if unmeasured confounders are not evenly distributed between providers. It is important, therefore, to strive for high recruitment rates.

Impact on hospital performance of introducing routine patient reported outcome measures in surgery

To determine the impact of introducing patient reported outcome measures (PROMs) on the selection of patients for surgery and on outcome for four elective operations (hip replacement, knee replacement, varicose vein surgery and groin hernia repair). Patient-level data from the National PROMs programme for England from April 2009 to March 2012 were analysed to determine the extent of change in mean preoperative scores and mean adjusted postoperative scores using disease-specific and generic PROMs assessed using standardized effect sizes (SESs). Variation between providers was determined by intra-class correlation coefficients for each time period. Consistency in outlier ratings was tested using weighted Kappa statistics. There was little apparent impact. Preoperative severity increased slightly for two procedures only: varicose vein surgery (SES disease-specific PROM 0.10; generic PROM -0.07) and to a lesser extent for hip replacement (SES disease-specific PROM -0.03). There was little inter-provider variation and it did not change significantly over time. There were also slight improvements in outcomes for hip and knee replacement (SES for disease-specific and generic PROMs 0.03) though not for hernia repair and a slight worsening for varicose vein surgery. The extent of variation in performance between providers was unchanged. The proportion of providers deemed to be outliers did not change over time. There was only moderate consistency in those providers deemed to be outliers for hip and knee replacement (Kappa 0.31-0.47) and it was even weaker for the other two procedures. Although 35% of providers of hip replacement were outliers in at least one year, only 6% were consistently outliers. Such inconsistency may be partly due to regression to the mean. The minimal impact that the routine use and feedback of PROMs had on provider behaviour during the initial years suggests that more attention needs to be paid to how results are communicated and to the provision of advice as to what action may be taken.

Application of patient-reported outcome measures (PROMs) data to estimate cost-effectiveness of hernia surgery in England

To demonstrate potential uses of nationally collected patient-reported outcome measures (PROMs) data to estimate cost-effectiveness of hernia surgery. Cost-utility model populated with national PROMs, National Reference Cost and Hospital Episodes Statistics data. Hospitals in England that provided elective inguinal hernia repair surgery for NHS patients between 1 April 2009 and 31 March 2010. Patients >18 years undergoing NHS-funded elective hernia surgery in English hospitals who completed PROMs questionnaires. Change in quality-adjusted life year (QALY) following surgery; cost per QALY of surgery by acute provider hospital; health gain and cost per QALY by surgery type received (laparoscopic or open hernia repair). The casemix-adjusted, discounted (at 3.5%) and degraded (over 25 years) mean change in QALYs following elective hernia repair surgery is 0.826 (95% CI, 0.793-0.859) compared to a counterfactual of no treatment. Patients undergoing laparoscopic surgery show a significantly greater gain in health-related quality of life (EQ-5D index change, 0.0915; 95% CI, 0.0850-0.0979) with an estimated gain of 0.923 QALYS (95% CI, 0.859-0.988) compared to those having open repair (EQ-5D index change, 0.0806; 95% CI, 0.0771-0.0841) at 0.817 QALYS (95% CI, 0.782-0.852). The average cost of hernia surgery in England is £1554, representing a mean cost per QALY of £1881. The mean cost of laparoscopic and open hernia surgery is equivocal (£1421 vs. £1426 respectively) but laparoscopies appear to offer higher cost-utility at £1540 per QALY, compared to £1746 per QALY for open surgery. Routine PROMs data derived from NHS patients could be usefully analyzed to estimate health outcomes and cost-effectiveness of interventions to inform decision-making. This analysis suggests elective hernia surgery offers value-for-money, and laparoscopic repair is more clinically effective and generates higher cost-utility than open surgery.

The Association Between Body Mass Index and the Outcomes of Total Knee Arthroplasty

In the United Kingdom, organizations involved in health-care commissioning have recently introduced legislation limiting access to total knee arthroplasty through the introduction of arbitrary thresholds unsupported by the literature and based on body mass index. This study aimed to establish the relationship between body mass index and patient-reported specific and general outcomes on total knee arthroplasty. Using national patient-reported outcome measures (PROMs) linked to the National Joint Registry, we identified 13,673 primary total knee arthroplasties performed for the treatment of osteoarthritis. The PROMs project involves the collection of condition-specific and general health outcomes before and at six months following total knee arthroplasty. The relationships between body mass index and the Oxford Knee Score, EuroQol 5D index, and EuroQol 5D Visual Analogue Scale were assessed with use of scatterplots and linear regression. The improvement in these measures was compared for three distinct groups based on body mass index (Group I [15 to 24.9 kg/m(2)], Group II [25 to 39.9 kg/m(2)], and Group III [40 to 60 kg/m(2)]) with use of multiple regression analysis to adjust for differences in age, sex, American Society of Anesthesiologists grade, general health rating, and number of comorbidities. The preoperative and postoperative patient-reported outcome measures declined to a similar extent with increasing body mass index. The gradient of the linear regression equation relating to the change in scores was positive in all cases, indicating that there was a tendency for scores to improve to a greater extent as body mass index increased. After adjustment, the changes in patient-reported outcome measures in Group I and Group III were equivalent for the Oxford Knee Score (mean difference, 0.5 point [95% confidence interval, -0.5 to 1.5 points]; p = 0.78), the EuroQol 5D index (mean difference, 0.014 point [95% confidence interval, -0.021 to 0.048 point]; p = 1.00), and the EuroQol 5D Visual Analogue Scale (mean difference, 1.9 points [95% confidence interval, -0.4 to 4.1 points]; p = 0.13). Wound complications were significantly higher (p < 0.001) at a rate of 17% (168 of 1018 patients) in Group III compared with 9% (121 of 1292 patients) in Group I. The improvements in patient-reported outcome measures experienced by patients were similar, irrespective of body mass index. Health policy should be based on the overall improvements in function and general health gained through surgery. Obese patients should not be excluded from the benefit of total knee arthroplasty, given that their overall improvements were equivalent to those of patients with a lower body mass index.

The Non Arthroplasty Hip Register (NAHR)

Dear Sir, These are exciting times in the orthopaedic world with the profession taking the lead in collecting data to define the natural history of disease processes and the success of our surgical interventions. Initiatives include the National Joint Registry (NJR), National Hip Fracture Database (NHFD), the collection of national Patient Reported Outcome Measures (PROMS) data, a national register for spinal surgery through the British Association of Spinal Surgery (BASS), and a planned soft-tissue knee surgery register (Knee Ligament Registry). With the formation of all these national registers the UK may indeed be entering a golden …

The myClinicalOutcomes website: providing real-time, patient-level PROMs data

The national patient reported outcome measures (PROMs) programme has been under way since April 2009, yet the true value of capturing and utilising these metrics has yet to be realised. The current system needs to evolve and should deliver real-time, patient-level benefits for more procedures, bringing long-term monitoring into routine clinical practice. The myClinicalOutcomes website is a low-cost, straightforward web-based system that could improve the current situation.